Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00006401 IMPLEMENTATION COMPLETION AND RESULTS REPORT (Loan No. 8540-DO) ON A LOAN IN THE AMOUNT OF US$50 MILLION TO THE DOMINICAN REPUBLIC FOR THE SUPPORT TO THE NATIONAL EDUCATION PACT PROJECT December 22, 2023 Education Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Dec 15, 2023}) Currency Unit = Dominican Peso 56.80 = US$1 US$ 0.018 = SDR 1 FISCAL YEAR July 1 – June 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Michael Kerf Regional Director: Jaime Saavedra Chanduvi Practice Manager: Emanuela Di Gropello Task Team Leader(s): Katia Herrera Sosa ICR Main Contributor: Wendy De Leon ABBREVIATIONS AND ACRONYMS CAIPI Early Childhood Development Centers (Centro de Atención Integral a la Primera Infancia) CPS Country Partnership Strategy DLI Disbursement Link Indicators DR Dominican Republic ECE Early Childhood Education EFA Economic and Financial Analysis GDP Gross Domestic Product IDEICE Instituto Dominicano de Evaluación e Investigación de la Calidad Educativa IFR Interim Financial Report INAFOCAM National Institute of Education and Training of Teachers ( Instituto Nacional de Formación y Capacitación del Magisterio) INAIPI National Institute of Early Childhood Care ( Instituto Nacional de Atención Integral a la Primera Infancia) IPF Investment Project Financing IRI Intermediate Results Indicator ISFODOSU Salomé Ureña Teacher Training Institute (Instituto Superior de Formación Docente Salomé Ureña) ISR Implementation Status and Results Report LAC Latin America and Caribbean MINERD Ministry of Education of the Dominican Republic (Ministerio de Educación de la República Dominicana) MoF Ministry of Finance NEP National Education Pact OCI Office of International Cooperation (Oficina de Cooperación Internacional) PAD Project Appraisal Document PDO Project Development Objective PDV Present Discounted Value PIU Project Implementation Unit POA Operating Annual Plan (Plan Operativo Anual) QEC Quisqueya Early Childhood Development Program (Quisqueya Empieza Contigo) TERCE Third Regional Comparative and Explanatory Study (Tercer Estudio Regional Comparativo y Explicativo) TOC Theory of Change TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 14 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 15 C. EFFICIENCY ........................................................................................................................... 21 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................23 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 23 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 25 A. KEY FACTORS DURING PREPARATION ................................................................................... 25 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 26 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ... 26 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 26 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 27 C. BANK PERFORMANCE ........................................................................................................... 28 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 46 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 49 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 50 ANNEX 5. BORROWER’S IMPLEMENTATION AND COMPLETION REPORT ............................... 59 ANNEX 6. RATINGS FROM ISRS............................................................................................. 67 ANNEX 7. REFERENCES ......................................................................................................... 68 The World Bank Support to the National Education Pact Project (P146831) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P146831 Support to the National Education Pact Project Country Financing Instrument Dominican Republic Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency The Dominican Republic Ministerio de Educacion de la Republica Dominicana Project Development Objective (PDO) Original PDO The objective of the Project is to improve the Borrower’s capacity to: (i) recruit and train primary and secondary school teachers, (ii) assess student learning in primary and secondary education, (iii) evaluate the quality of service provided by Public Early Childhood Development Centers, and (iv) enhance the process for decentralizing public school management.  Page 1 of 69 The World Bank Support to the National Education Pact Project (P146831) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 50,000,000 47,500,000 47,500,000 IBRD-85400 100,000,000 0 0 IBRD-89130 Total 150,000,000 47,500,000 47,500,000 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 150,000,000 47,500,000 47,500,000 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 30-Sep-2015 09-Dec-2016 31-Jul-2018 30-Jun-2020 30-Jun-2023 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 22-Aug-2022 40.00 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Jan-2016 Moderately Satisfactory Moderately Satisfactory 0 Page 2 of 69 The World Bank Support to the National Education Pact Project (P146831) 02 25-Feb-2016 Moderately Satisfactory Moderately Satisfactory 0 03 07-Jul-2016 Moderately Satisfactory Moderately Satisfactory 0 04 20-Dec-2016 Satisfactory Satisfactory 0 05 28-Jun-2017 Satisfactory Satisfactory 15.00 06 30-Dec-2017 Satisfactory Satisfactory 24.00 07 21-Jun-2018 Satisfactory Satisfactory 24.00 08 18-Sep-2018 Satisfactory Satisfactory 24.00 09 02-May-2019 Satisfactory Satisfactory 34.00 10 25-Dec-2019 Satisfactory Moderately Satisfactory 34.00 11 30-Jun-2020 Satisfactory Moderately Satisfactory 36.00 12 18-Dec-2020 Satisfactory Moderately Satisfactory 36.00 13 13-Jun-2021 Satisfactory Moderately Satisfactory 36.00 14 21-Dec-2021 Moderately Satisfactory Moderately Satisfactory 36.00 15 17-Jul-2022 Satisfactory Satisfactory 40.00 16 28-Mar-2023 Satisfactory Satisfactory 44.88 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Early Childhood Education 10 Public Administration - Education 13 Primary Education 39 Secondary Education 38 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 69 The World Bank Support to the National Education Pact Project (P146831) Human Development and Gender 100 Education 100 Access to Education 50 Education Financing 50 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Sophie Sirtaine Michel Kerf Director: Claudia Maria Costin Jaime Saavedra Chanduvi Practice Manager: Reema Nayar Emanuela Di Gropello Task Team Leader(s): Juan Baron Katia Marina Herrera Sosa Wendy Elizabeth De Leon ICR Contributing Author: Samayoa Page 4 of 69 The World Bank Support to the National Education Pact Project (P146831) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country context at appraisal 1. The project was appraised in 2015 after a sustained period of economic growth in the Dominican Republic (DR), which translated into a reduction in poverty and inequality. In the decade between 1995 and 2004, the DR Gross Domestic Product (GDP) grew at an average rate of 4.6 percent. In the following decade (2005–2014), it accelerated to 5.6 percent, which was among the highest growth rates in the Latin America and Caribbean (LAC) region. This sustained economic growth, in turn, led to a significant reduction in the poverty rate from 48.2 percent in 2005 to 34.8 percent in 2014.1 Additionally, the Gini coefficient, which measures income inequality on a scale from 0 to 1, fell from 0.5 in 2005 to 0.44 in 2014.2 Recognizing that an inadequately educated workforce limits the country’s future growth potential ,3 education was included in the National Development Strategy 2010-2030. 2. The Government’s National Development Strategy for 2010–2030 aimed to establish a culture of planning and policy actions with a long-term vision for overcoming poverty and increasing welfare. The strategy includes four main axes: (i) democratic social state of law; (ii) society with equal rights and opportunities; (iii) sustainable and competitive economy; and (iv) environmentally sustainable production and consumption. Human development was included as the second of the four axes in its goal towards guaranteeing equal rights and opportunities in access to health, housing, and basic services.4 In addition to quality education for all, education targets comprised of strengthening the quality of training for pre-service and in-service teachers,5 teacher evaluation,6 and strengthening community participation in building quality education,7 among others. Education Sector Context 3. At the time of appraisal, the Government of DR was aware of the importance of education for developing a well-trained and skilled workforce to meet the needs of a modern economy. After years of underinvestment, DR demonstrated its firm commitment to education as a national priority by raising the education sector budget from 2.2 percent of GDP in 2012 to an average of 3.7 percent in 2014.8 While the sector had made progress in achieving universal primary education, secondary access, attendance, and quality remained important issues. 1 This was calculated using the World Bank LAC Equity Lab’s $6.85-a-day (2017 PPP) poverty line. The global poverty lines of $2.15, $3.65, and $6.85 reflect the typical national poverty lines of low-income, lower-middle-income, and upper-middle-income countries in 2017 prices. 2 World Development Indicators (2023). 3 World Economic Forum. 2014. The Global Competitiveness Report, 2014-2015. Geneva: World Economic Forum. 4 Law 1-12. National Development Strategy 2010–2030. Dominican Republic, 2012. 5 Target 2.1.1.4. 6 Target 2.1.1.6. 7 Target 2.1.1.13. 8 Ministry of Education, retrieved October 26, 2023, from https://www.ministeriodeeducacion.gob.do/transparencia/media/presupuesto/gastos-en-educacion/49o-gastos-del-ministerio- de-educacion-2004-2022pdf.pdf Page 5 of 69 The World Bank Support to the National Education Pact Project (P146831) 4. At appraisal (2013-2014), the education system in Dominican Republic enrolled 89 percent of all primary school aged children, mostly in public schools. 9 The student enrollment in the Dominican education system for the 2013–2014 school year was 1,645,422 students, of which 75 percent or 1,241,212 were in the public sector. At the secondary level, net enrollment was around 67 percent. On average, the school attendance rates for the 3 to 5, 6 to 11 and 14 to 17 age groups were 49 percent, 94 and 83 percent, respectively. 5. In addition to limited access to education10, poor learning outcomes were the main challenge to the education system in Dominican Republic. Results from the 2013 Third Regional Comparative and Explanatory Study (Tercer Estudio Regional Comparativo y Explicativo , TERCE) revealed that reading and mathematics scores of third and sixth graders were low in comparison with other countries in the region. 11 Approximately 74 percent of third graders were at the lowest level in reading and 85 percent in mathematics. Among sixth graders, 38 percent and 80 percent of students were in at the lowest level in reading and mathematics, respectively. Even more dire, 81 percent (2013) of 10-year-olds could not read and understand a simple text by the end of primary school. This is higher than both the average for the region (55%) and the average for the country’s income group (38%). 6. To improve learning outcomes, the DR needed to build the foundations that would allow the sector to develop and implement quality-enhancing public education policies. As a first step, four main underlying challenges that significantly constrained the performance of the education system were identified.12 These included the Ministry of Education’s (Ministerio de Educación de la República Dominicana , MINERD) capacity to: • Recruit and train teachers according to quality standards. MINERD’s capacity to recruit and train teachers impacts the quality of its teaching force, and ultimately, the quality of instruction in the classroom. A survey of mathematics teachers conducted by the Dominican Institute of Research and Evaluation (Instituto Dominicano de Evaluación e Investigación de la Calidad Educativa , IDEICE) revealed that only 45 percent of teachers were able to correctly answer more than half of the questions related to content they were supposed to be teaching.13 This was a result of several weaknesses including: the low capacity to attract high-achieving candidates into teaching; the absence of teacher standards; an insufficiently transparent teacher selection and recruitment system; low quality pre-service and in-service training; lack of data on teacher performance; and lack of incentives to further teachers’ professional development.14 • Assess student learning in basic education and use and share the resulting information with relevant actors in the system (i.e., teacher training institutions, schools, parents) for decision making and accountability. Although student national examinations were administered at the end of basic education (grade 8) and intermediate education (grade 12), data was not used to identify learning gaps and challenges, or to design corrective actions. Additionally, the timing of the first student examination at the end of basic education (grade 8), meant that in many cases it was too late to address the fundamental weaknesses in math and reading instruction. 9 Net enrollment rate, World Development Indicators. 10 Mainly in preschool and secondary levels. 11 UNESCO, TERCE, 2013. Fifteen countries participated in the regional assessment: Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Guatemala, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Dominican Republic, and Uruguay. 12 Project Appraisal Document for the National Education Pact Project. Report No. PAD 850, 2015. 13 IDEICE. 2013. Dominio de los contenidos matemáticos por parte del docente, como factor del éxito o fracaso escolar en el primer ciclo del nivel básico en República Dominicana. Santo Domingo: Instituto Dominicano de Evaluación e Investigación de la Calidad Educativa (IDEICE). 14 Programa de Promoción de la Reforma Educativa en América Latina y el Caribe. 2010. Page 6 of 69 The World Bank Support to the National Education Pact Project (P146831) • Evaluate and ensure good quality of service provided during preschool with the aim of enhancing school readiness of children entering basic education. The lack of quality standards for early childhood education (ECE) centers limited MINERD’s capacity to regulate and evaluate the quality of their services and resulted in low school readiness among students, particularly those coming from disadvantaged backgrounds. • Decentralize the school management system. At the school level, evidence suggests that strengthened school management committees (SMCs) and increased social participation have increased efficiency in schools as well as use of resources in a way that is most useful for the school given the local needs.15 However, at appraisal there were no clear mechanisms for strengthening school autonomy or increasing accountability to either the community or MINERD. Although the existing systems and regulations ensured careful management of resources, their specificity and rigidity 16 limited the number of established school management committees (SMCs) and existing committees were not fully functional. 7. The government showed commitment to tackling these challenges by drafting a broad reform of the education sector and signing the Pacto Educativo (National Education Pact, NEP) in April 2014. The NEP included an agreement to reform the teaching profession by: (i) revising pre-service and in-service training programs; (ii) implementing an induction exam; (iii) adopting a culture of evaluation across all sectors and actors in the education system; (iv) improving the quality of early education; and (v) modernizing management with a focus on decentralization to increase the system’s efficiency. The NEP received broad endorsement from independent agencies, universities, teachers’ unions, and civil society and is valid until 2030. Its strategies are aligned with the National Development Strategy 2010–2030 and the Ten- Year Education Plan. 8. The strategies of the National Education Pact (NEP) were also aligned with the National Development Strategy 2010–2030 and the Ten-Year Education Plan. The National Development Strategy 2010–2030 guides how the national and local public sector manages regulation, promotion and production of goods and services. It also focuses on creating conditions that encourage synergy between public and private sectors to achieve the nation's long-term vision and the strategy’s specific objectives and goals. It postulates that the construction of a society where equal rights and opportunities effectively exist demands the achievement of seven general objectives, the first of which is "quality education for all". Rationale for Bank Support 9. Given the complexity of the above listed challenges, the government of Dominican Republic determined that support from the World Bank would be of significant value to the education reform proposed in the NEP. MINERD and the Ministry of Finance (MoF) requested technical expertise and financial support from the WB to develop the NEP’s reforms to the education system, which targeted the improvement of school performance and, ultimately, student learning. The financial support was designed as a full results-based operation in which all financing was disbursed linked to indicator targets17 using the Investment Project Financing (IPF) instrument; which the Bank had substantial experience with in the region. The DLI’s have precise verification protocols and procedures. At the time of preparation, there were no other Bank projects in the education sector in the Dominican Republic, and Bank projects had been absent in the sector for a number of years in DR. 15 Bruns, Filmer, and Patrinos, 2011. 16 Administrative bottlenecks do not allow SMCs to receive all the transfers they are due, and they are unable to spend them as needed. 17 This allows disbursements to be made when a set of tangible and verifiable results are achieved, rather than inputs (expenses), to improve project performance. Page 7 of 69 The World Bank Support to the National Education Pact Project (P146831) Theory of Change (Results Chain) 10. At the time of appraisal and project design, the theory of change (TOC) presented in figure 1 was included in the project appraisal document (PAD).18 The TOC assumed that the Ministry would know how to implement and conduct the activities, and that these would indeed need to take place for the outputs and outcomes to be achieved. Ultimately, the longer-term outcomes, in particular student learning, was expected to improve beyond the project’s closing. The TOC remained unchanged during the implementation of the project. The project was restructured twice to extend the closing date, but no design or implementation changes were incorporated during the restructurings. 11. The TOC rests on the following critical building blocks supporting the achievement of the outcomes: a. High-performing education systems require high-quality teachers. Despite an increase in the length of teachers’ pre-service training, the quality of teaching had not improved in the Dominican Republic. For teacher training to be effective, the quality and not just the quantity of training must be enhanced.19 Standards should provide the basis on which teachers are selected, trained, certified, evaluated, and promoted. In addition to improving the quality of teachers, the effective application of standards to teacher selection renders the hiring process more transparent and meritocratic. National competency teacher tests, such as screening exams prior to hiring, are yet another means of increasing the preparation and knowledge of incoming teachers. b. To improve student learning outcomes, developing a strong system for assessing student learning is vital. The implementation of national student assessments, particularly in early grades, 20 plays a key role in building an effective education system. Research reveals that the appropriate types of assessment activities and the right use of the data generated by those activities contribute to both better learning and better-informed policy decisions.21 Such a system allows education actors, including teachers and policy makers, to promptly intervene in response to poor outcomes. c. The low access to and the poor quality of ECE make foundational learning difficult and contribute to the global learning crisis. High quality ECE help children develop the cognitive and socioemotional skills that will help them succeed both in school and later in their life. Standards need to be established and an evaluation system for early childhood services is needed to ensure compliance and drive towards ECE improvement.22 Effective early childhood programs teach children self-regulation, executive functioning skills, and basic learning skills that will help them acquire literacy and numeracy when they enter school. d. Decentralizing the school management system contributes to higher-performing schools. There is growing evidence that at least some of the characteristics of education decentralization reforms that focus on school autonomy, as opposed to municipal or regional autonomy, contribute to higher school performance. Decentralization to subregional governments has been shown to yield some educational benefits by allowing greater innovation and greater flexibility to adapt resource allocation to local prices, although more research is required.23 18 World Bank, 2015. 19 Bruns and Luque, 2015. 20 Grades 1, 2 and 3 of primary education. 21 Clarke, 2012. 22 Bendini and Devercelli, 2022. World Bank. 23 Winker and Gershberg, 2000. World Bank. Page 8 of 69 The World Bank Support to the National Education Pact Project (P146831) Figure 1: Theory of Change Page 9 of 69 The World Bank Support to the National Education Pact Project (P146831) Project Development Objectives (PDOs) 12. The project development objective (PDO) was to improve the borrower’s capacity to: (i) recruit and train primary and secondary school teachers; (ii) assess student learning in primary and secondary education; (iii) evaluate the quality of service provided by public early childhood development centers; and (iv) enhance the process for decentralizing public school management. 24 The PDO did not change during the implementation period. Key Expected Outcomes and Outcome Indicators 13. The project captured the four main objectives in the PDO statement: • Objective 1. Recruit and train primary and secondary school teachers. The achievement of objective 1 is measured using PDO indicator 1 (teacher positions staffed through the revised teacher entrance exam) 25 and PDO indicator 2 (positions to study teaching degrees at Salomé Ureña Teacher Training Institute [Instituto Superior de Formación Docente Salomé Ureña , ISFODOSU] filled through revised entry exam). 26 • Objective 2. Assess student learning in primary and secondary education. The achievement of objective 2 is measured using PDO indicator 3 (student learning outcomes for the first cycle of primary education analyzed and disseminated).27 • Objective 3. Evaluate the quality of service provided by public early childhood development centers. The achievement of objective 3 is measured using PDO indicator 4 (percentage of Quisqueya Early Childhood Development Program [Quisqueya Empieza Contigo, QEC] care centers that have been evaluated against quality standards and their results analyzed).28 The QEC centers are the Early Childhood Care Centers (Centro de Atención Integral a la Primera Infancia, CAIPI).29 • Objective 4. Enhance the process for decentralizing public school management. The achievement of objective 4 is measured using PDO indicator 5 (increasing percentage of fully functional school management committees [Juntas de Centro, SMCs] implementing their school development plans). 30 24 As stated in the original Loan Agreement dated October 8, 2015, and the PAD. 25 “Teacher positions” refers to positions for teachers or directors hired by MINERD. Teacher entrance exam refers to the Concurso de Oposición. “Revised” means aligned with professional standards for teachers. “Staffed” means hiring teachers for available positions. 26 “Positions to study teaching degrees at ISFODOCU” refers to students that ISFODOSU could accept to study a teaching degree. “ Entry exam” refers to an assessment of potential candidates for those positions that at least assesses content knowledge in key areas. “Revised entry exam” refers to an exam that has incorporated technical improvements recommended by an expert on student assessment.” 27 “Analyzed” means reporting and publishing results by area of learning. “First cycle of primary” refers to the first three grades of primary education. 28 The assessment must be in accordance with the guidelines established in the strategic plan for INAIPI, and for the last year should be verified by an independent agency. 29 The Government’s program Quisqueya Starts with You (Quisqueya Empieza Contigo, QEC) is under the National Institute of Comprehensive Care for Early Childhood (Instituto Nacional de Atención Integral a la Primera Infancia , INAIPI). INAIPI regulates the provision of ECD services under QEC for children 0-4 through CAIPIs. 30 Fully functional is understood as SMCs that are legally constituted, with an improvement plan, with a bank account, and receiving transfers from MINERD to achieve the objectives for which they were created. Implementing means that the SMCs spend the money according to the improvement plan and with the participation of members in the decision-making process. Page 10 of 69 The World Bank Support to the National Education Pact Project (P146831) Components 14. The project comprised of four components, each of which corresponds to one main objective of the project. The description of each of the four components, along with their original values at appraisal, the revised values at closing, and the actual values effectively disbursed were as follows: 15. Component 1. Improving capacity to recruit and train primary and secondary school teachers (at appraisal US$25.375 million, assigned at closing US$25.375 million, actual executed US$22.875 million). • PDO 1: Teacher recruitment. Activities included: (i) developing teacher standards to serve as reference in the recruitment process, (ii) designing and implementing a competitive entrance exam for public school teachers that was aligned with the new standards, (iii) launching an online teacher recruitment system, and (iv) creating a web-enabled management information system. • PDO 2: Pre-service training. The subcomponent focused on the pre-service training carried out by the ISFODOSU. Activities included: (i) revision of the standardized entry exam used to select students for teaching degree programs; (ii) revision of ISFODOSU teaching degree curricula and competency profiles; (iii) creation of specialized teaching degrees for secondary school teachers; (iv) implementation of a quality assurance and monitoring system for ISFODOSU; (v) implementation of a scholarship program to attract high-performing secondary students to ISFODOSU pre-service degrees; (vi) implementation of a revised remedial program for admitted students with skill gaps; and (vii) revision of career structure and promotion schemes of ISFODOSU teachers. Other activities included: • Teaching career framework. Activities included: (i) design and adoption of guidelines for the teaching career, (ii) revised teacher evaluation system based on international best practices, and (iii) a certification system. • In-service training. Activities included: (i) development and piloting of in-service training programs for current teachers through the National Institute of Education and Training of Teachers ( Instituto Nacional de Formación y Capacitación del Magisterio, INAFOCAM), (ii) revision of guidelines and the evaluation of existing in-service programs, (iii) institutional capacity strengthening of the INAFOCAM evaluation office, and (iv) an impact evaluation of INAFOCAM in-service programs. 16. Component 2. Improving capacity to assess student learning in primary and secondary education (at appraisal US$13 million, assigned at closing US$13 million, actual executed US$13 million). • PDO 3: Assess student learning. Activities of this component included: (i) development and implementation of a national assessment in early grades; (ii) development and implementation of a national strategy for pre-tertiary student assessments, (iii) participation in international assessments, (iv) technical improvement of national student assessments, and (v) design and implementation of a system for the dissemination of assessment results to improve the use of this data. 17. Component 3. Improving capacity to evaluate the quality of service provided by public early childhood development centers (at appraisal US$5 million, assigned at closing US$5 million, actual executed US$5 million). • PDO 4: Evaluate the quality of early childhood development centers. Activities of this component included: (i) strengthening of the National Institute of Early Childhood Care (Instituto Nacional de Atención Integral a la Page 11 of 69 The World Bank Support to the National Education Pact Project (P146831) Primera Infancia, INAIPI) by developing a strategic plan and quality standards for public early childhood development centers31 and personnel, and (ii) development and implementation of an information and communications strategy to disseminate the standards of quality service delivery within the public early childhood development centers, and to promote public understanding of the QEC program and improve parenting practices. 18. Component 4. Improving capacity to enhance the process for decentralizing public school management (at appraisal US$6.5 million, assigned at closing US$6.5 million, actual executed US$6.5 million). • PDO 5: Decentralizing public school management. Increasing percentage of fully functional school management committees (Juntas de Centro, SMCs) implementing their school development plans. This component supported the decentralization of MINERD’s functions through: (i) increase the number of school management committees (SMCs) in public schools, (ii) the review of regulations and processes,32 (iii) implementation of a functional expenditure system for SMCs, (iv) improvement of SMC training for developing school improvement plans and annual work plans, (v) improvement of financial and procurement practices at the school level, and (vi) revision of school auditing guidelines and procedures. Financing 19. The loan instrument was a US$50 million Investment Project Financing (IPF) implemented using a results- based approach with Disbursements-Linked Indicators (DLIs), which capture measurable results. Disbursements were made against eligible expenditures (Eligible Expenditure Program, EEP) incurred under the National Education Pact . Table 1: Indicators linked to disbursements by year of completion 2015 2016 2017 2018 2019 2020 2021 2022 Improving capacity to recruit and train primary and secondary school teachers (US$25.375 M) DLI 1 (PDO 2): DLI 5: DLI 8: DLI 12: DLI 17: ISFODOSU Standardized Professional ISFODOSU INAFOCAM has started and revised Teacher offers two new and ISFODOSU implementing its entrance test Standards degrees for prepare their own professional is applied to all have been secondary annual development applicants for adopted education financial report plan for its studies at and teacher using the educators ISFODOSU disseminati training Integrated (US$2,375) (US$2.5M) on has (Scalable) Financial begun. (US$2.5M) Management (US$2.5M) System (SIGEF) (Scalable) (US$3.0M) 31 The name of the centers changed from QEC to CAIPIs during the implementation of the Project, however both refer to the same Early Childhood Development Centers. 32 Revision of the ordinances and resolutions regulating SMCs to provide more flexibility to the SMCs in terms of allowable expenses and streamlining financial flows. Page 12 of 69 The World Bank Support to the National Education Pact Project (P146831) DLI 2: MINERD DLI 13: 20% of DLI 18: DLI 9: Design of DLI 6 (PDO 1): administers public-school INAFOCAM the professional MINERD applies the teachers are makes career and the Opposition Opposition evaluated modifications to evaluation of Contest aligned Competition through the continuing teachers (based with Professional to hire new teaching training on standards) is Teaching teachers and programs adopted by Standards (US$2.5M) professional according to the MINERD (US$2.5M) evaluation results of (US$2.5M) system evaluations (US$2.5M) and/or impact evaluations (US$2.5M) Improving the capacity to evaluate school learning in primary and secondary education (US$13.0 M) DLI 3: MINERD DLI 7: DLI 10: DLI 14 (PDO 3): DLI 19: The staff applies PISA Implementa MINERD pilots MINERD of MINERD 2015 tion of the the administers a National (US$3.0M) Student dissemination student Evaluation Office Assessment component of evaluation for has received Strategy has the student the first cycle training in begun assessment of primary sampling, (US$2.5M) and revises the education element design, dissemination (US$2.5M) database component management based on the and statistical results of the software pilot (US$2.5M) (US$2.5M) Improving the capacity to evaluate the quality of services provided by Public Early Childhood Centers (US$5.0 M) DLI 15: The DLI 20: 50% of DLI 20 (PDO 4): pilot Early 50% of Early evaluation of Childhood Childhood the Early Development Development Childhood Centers are Centers are Development operational operational and Centers has and are evaluated been carried evaluated according to out according to quality standards (US$2.5M) quality (Scalable) standards (US$1.0M) (Scalable) (US$1.5M) Improve the capacity to strengthen the decentralization process of public-school management (US$6.5 M) DLI 4: 250 DLI 11: MINERD DLI 16 (PDO 5): additional adopts new 1,750 School School Boards functional Boards are fully receive spending functional transfers guidelines for (Scalable) (Scalable) School Boards (US$2.5M) (US$2.0M) (US$2.0M) Page 13 of 69 The World Bank Support to the National Education Pact Project (P146831) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 20. The project was restructured two times to extend the closing date by a total of three years. The project was approved by the Bank’s Board of Executive Directors on September 30, 2015. It was declared effective 14 months after approval, on December 9, 2016. The delay between approval and effectiveness is common among projects in the Dominican Republic, since Congress’ approval is needed. The first restructuring was approved on December 10, 2019; and the second on August 22, 2022. The first extension was for an additional 27 months (from June 30, 2020, to September 30, 2022). At this time, additional financing was requested for $100 million. This extension was intended to account for the initial 14-month delay and to meet the additional financing requirements. Ultimately, though the additional financing was not approved by Congress. The extension of project closing date allowed for increased effectiveness in implementation. The second extension was for an additional 9 months (from September 30, 2022, to June 30, 2023) to give time for the completion of PDO 5 (increasing percentage of fully functional school management committees -Juntas de Centro), which had been impacted by delays due to the COVID-19 pandemic. Revised PDO and Outcome Indicators 21. The PDO was not revised during the lifetime of the project. 22. As a result of the extension of the closing date, the end target dates for all indicators were updated twice. In the first restructuring, the closing date was changed to September 30, 2022; and in the second restructuring, to June 30, 2023. The additional financing included three new PDO indicators and several new IRI ’s to measure new achievements, as well as adding new gender results indicators. However, since the additional financing was not approved by Congress, the original loan agreement, targets, and indicators remained in place. Revised Components 23. The components and activities remained the same throughout the life of the project. However, at the time of the second restructuring, it was determined that the sum of US$2.5 million (5 percent of the loan amount) allocated for Disbursement Linked Indicator (DLI) 9 (adoption of the teaching career and evaluation system based on standards established by MINERD) would likely not be disbursed. The activities under DLI 9 were not part of the detailed action plan shared by the government, although at the design stage the team was hoping to incentivize important policy actions required to improve teacher’s careers and ultimately learning. Changes to DLIs however, would have included an addendum to the Loan Agreement, which would have required congressional approval, in accordance with the country’s laws and regulations. At the time of the second restructuring, given that there were only nine months left before the closing of the project, the decision was made to leave the amount of this DLI undisbursed. 33 Other project costs remained the same throughout the implementation period (Annex 3). Rationale for Changes and Their Implication on the Original Theory of Change 24. The changes from the restructurings affected only the closing date and did not have an impact on the theory of change. 33 See Restructuring Paper dated September 30, 2015 (Report No. RES45715). Page 14 of 69 The World Bank Support to the National Education Pact Project (P146831) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 25. The relevance of the project’s objectives, which supported the National Education Plan, is rated high, as they were strongly aligned with the Country Partnership Strategy and the Country Partnership Framework (CPF) at preparation and at closure. At preparation, the project contributed to the Country Development Goal of Quality Education for All under Results Area 5 “Strengthening Social Service Delivery” of the Bank’s Country Partnership Strategy FY15-18 (Report No. 89551-DO) by supporting the Government to improve quality of teaching through better teaching policies and improve quality of learning through the implementation of a robust student assessment system. In addition, the operation was to contribute to twin goals of boosting shared prosperity as well as to other Bank priorities (including gender) and the Education Sector Strategy 2020: Learning for All. At closing, the project was aligned with objective 2.1 of the FY22–FY26 Country Partnership Framework (CPF), which focused on the enhancement of quality of education and training. The project objective was intended to support effective, long-term solutions to the persistent education system challenges with the goal of providing all Dominicans with the foundational skills needed to become lifelong learners and enhance their productivity was fully aligned with the main priorities identified by the 2018 World Development Report to increase student learning. Moreover, the staffing of teachers through the revised entrance exam based on standards represents a milestone towards a more objective, merit-based, and transparent recruitment system, where all applicants have equal chances to be hired, ensuring that the hired teachers are those with the best qualifications. 26. The project’s objectives and outcomes in improving the quality of education continue to be highly relevant at closure for the current administration and, in particular, for MINERD’s strategic plan. The five central axes of the new institutional strategic plan of MINERD 2021–2024 include: (i) enhancing quality of education, (ii) improving competencies and welfare of teaching staff, (iii) strengthening the leadership and regulation of the education system, (iv) increasing social participation and active citizenship, and (v) strengthening internal administrative processes and management. The project addressed the first four priorities. The project was fully aligned with the National Education Pact. The NEP and its strategies are aligned with the National Development Strategy 2010–2030 and the Ten-Year Education Plan. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome PDO Objective 1: Recruit and train primary and secondary school teachers Rating: High 27. The project’s objective to improve the borrower’s capacity to recruit and train primary and secondary school teachers is rated high. The achievement of this objective was measured by PDO indicators 1 (teacher positions staffed through the revised teacher entrance exam) and PDO indicator 2 (positions to study teaching degrees at ISFODOSU filled through revised entry exam). The baseline was “no” and the end target “yes” for both indicators. Both indicators were Page 15 of 69 The World Bank Support to the National Education Pact Project (P146831) fully achieved and therefore given ratings of high. PDO Indicator 1: Teacher positions staffed through the revised teacher entrance exam (Yes/No) 28. At appraisal, one of the challenges in improving learning outcomes was the low quality of teaching, which was partly due to a lack of quality standards in the recruitment process. Professional standards for teachers were designed and adopted to serve as a reference in the recruitment process (Intermediate Results Indicator [IRI] 5). The administration explicitly defined what they expected of teachers by creating professional standards and passing regulations that describe teacher competencies in terms of personal and professional development. The “Professional and Performance Standards for Teaching Certification and Development of the Teaching Career” was developed by the Vice Ministry of Certification within MINERD and approved by the National Council of Education in 2014.34 The approval of the professional standards established a national consensus on (i) the expectations of teachers in terms of their knowledge and their performance in the classroom and the work environment, and (ii) the attitudes and values that define pedagogical excellence.35 The country also passed regulations which included the competencies expected of teachers. The standards for teachers were disseminated (DLI 5). 29. The entrance exam for teacher recruitment was aligned with the above standards of the teacher profession (IRI 6). MINERD started administering a competitive entrance exam to hire teachers in 2015 (DLI 2 and IRI 6). It then aligned the exam with the professional standards for teachers (DLI 6). In 2021, an evaluation consultant was hired under the project to ensure the exams were aligned with professional standards. The reports indicated that the tests were adequately aligned with the standards and recommendations provided for improving the formulation of questions for greater clarity.36 30. Teaching positions were filled through the revised teacher entrance exam (PDO indicator 1). On April 16, 2021, MINERD launched a teacher competitive entrance exam aligned with professional standard process to fill teaching vacancies. After evaluating applicants, the selection process was completed in the beginning of November 2021. In February of 2022, MINERD announced that 24,300 candidates had been selected to fill available positions as teachers, principals, and counselors. PDO indicator 1 was met in 2022 when positions were staffed through the revised entrance exam. IRI 9 measures the number of teacher positions staffed through an entrance exam, including those staffed before the exam was aligned with standards. In combination with 15,793 vacancies that were filled in 2015, there was a total of 40,093 positions that was filled through the teacher entrance exam, achieving more than double of the original target for IRI 9 (19,000 positions). The Ministry website37 has the openings, locations, and the steps for the selection process. The teacher entrance exam process has been institutionalized and mainstreamed. Currently, the entrance exam for 2023 is taking place and the Ministry is expecting this process to continue in the years to come. PDO Indicator 2: Positions to study teaching degrees at ISFODOSU filled through revised entry exam (Yes/No) 34 Resolution Number 17. 35 They identify, describe, and establish the essential competencies (human, technical and ethical) that Dominican teachers must possess, as a guarantee that the country has in its classrooms and educational centers a staff capable of ensuring the achievement of high levels of learning (quantitatively and qualitatively) by the students. 36 MINERD prepared a report documenting the design and implementation of the competitive examination for pre-primary, primary, and secondary math teachers. The consultant also revised additional items supporting sustainability that were being piloted to be used in future tests. MINERD had additional support from a consulting firm in the development of an item bank. 37 https://www.ministeriodeeducacion.gob.do/sobre-nosotros/areas-institucionales/concurso-oposicion-docente-2023 Page 16 of 69 The World Bank Support to the National Education Pact Project (P146831) 31. A revised standardized entrance examination was applied to all applicants of the Salomé Ureña Teacher Training Institute (Instituto Superior de Formación Docente Salomé Ureña, ISFODOSU) beginning 2016 (PDO indicator 2 and DLI 1). The project focused on the pre-service training carried out by ISFODOSU, the second largest pre-service teacher training institution (graduating 20 percent of teachers every year). By 2017, the admission process for schools of education became more selective and students were now required to take the Orientation and Academic Measurement Test (Prueba de Orientación y Medición Académica, POMA)38 as a requirement for admission to a university degree program. This improved the quality of candidates for the teaching profession, as students who participated in the academic leveling programs for the admission test tended to test higher than other applicants. 32. In 2021 ISFODOSU continued to implement the revised exam to select teaching degree candidates (IRI 1). A total of 26,686 applicants were offered places for teaching degrees at ISFODOSU after passing the exam, almost five times the original target of 5,600 positions (IRI 2). ISFODOSU also improved the selection process of the existing scholarship program to target high-achieving secondary school graduates.39 ISFODOSU continues to implement the revised exam to select teaching degrees and is planning to continue implementing it going forward. 33. Additionally, the regulatory framework for pre-service teacher training was strengthened . The Regulation for Teacher Training Quality in the Dominican Republic, approved in 2015, (i) outlined teacher competencies, (ii) established a mandatory entry test for candidates seeking teacher training (POMA Test) and introduced an internationally recognized standardized test for university admissions designed by College Board (Prueba de Aptitud Académica, PAA), (iii) proposed a minimum of four days of face-to-face classroom time at the university (previously only 1 to 2 days per week), and (iv) included practice in schools as part of the training curriculum. Although it was not included as part of the original project activities, the regulation was supported with technical assistance by the World Bank. The regulation includes a special thank you from the Reform Commission to the World Bank for the continued technical assistance and collaboration. Moreover, students who attended universities that complied with the new regulations received scholarships, so they did not need to work. Furthermore, future secondary school teachers were trained in their respective subjects (mathematics, science, etc.), with a focus on pedagogy. 34. As a result of strengthened quality at entry and quality of the training itself, the education system improved its pool of teacher candidates. The system now administers a rigorous competitive selection exam.40 A greater percentage of ISFODOSU students pass the teacher recruitment exam. At the beginning of the Project, 70 percent of applicants passed the examination and at the time of Project closing, 95 percent of applicants passed the test. 35. Additionally, the National Institute of Education and Training of Teachers ( Instituto Nacional de Formación y Capacitación del Magisterio, INAFOCAM) piloted improved in-service training courses and evaluated the impact of existing programs (IRI 8 and DLI 18). It revised and adjusted teacher training programs for better focus and strengthened the institutional capacity of the INAFOCAM evaluation office. 36. The development of a teaching career framework (DLI 9) became secondary for the government with the onset of the COVID-19 pandemic, but there was substantial progress on that front. The development of the teaching career framework was a high-risk, high-reward activity and was included under DLI 9 to provide an incentive to the government to substantially advance in the professionalization of the teacher profession. Although activities under DLI 9 did move forward during the project, they were not completed. Nonetheless, the project assisted ISFODOSU in designing a 38 Regulation 09-2015, issued by the MESCyT. 39 The scholarship program was financed with national funds and supported by technical assistance from the World Bank. 40 In 2015, only 28 percent of approximately 37,000 candidates who applied to be public school teachers were selected. By contrast, in 2006, 100 percent of those who applied to be teachers were successful (see Educa, 2015). Page 17 of 69 The World Bank Support to the National Education Pact Project (P146831) professional development plan for its educators including a revised career structure and promotion scheme (IRI 4 and DLI 17). This plan is currently pending the approval of the ISFODOSU Board. A review of the Regulations for the Teaching Career by the Ministry of Public Administration was also carried out. The revised regulations were presented to the Academic Council in September 2022 and to the Board of Directors in October 2022. Furthermore, ISFODOSU began implementing its own teaching career framework in September 2017 (IRI 4). 37. In 2021, teachers at all levels of education began receiving two evaluations per year. The evaluations consist of a self-evaluation, a supervisor evaluation, and a student-teacher evaluation. The results of these three assessments correspond to different qualifications, which, in turn, determine if teachers receive a bonus. DLI 13 was achieved, with evidence that at least 20 percent of public-school teachers were evaluated using the new teacher evaluation system. 38. To sum up, the borrower’s capacity to recruit and train teachers was improved by: • Creating and adopting professional standards for teachers, promoting higher quality of teachers, and endorsing a regulation that describes the expected competencies. • Enhancing the selection process of teaching degree candidates, with a revised entry exam. ISFODOSU continues to implement the revised exam to select teaching degree candidates. Additionally, the regulatory framework for pre-service teacher training was strengthened. • Implementing a more rigorous teacher recruitment process, including a revised entrance exam. MINERD will continue to use the revised entrance tests developed by the Bank as the tools for future evaluations and, considering that all teachers must pass the entrance exam, this activity has the potential to improve teaching and learning on a large-scale basis. The implementation of the revised examination has been mainstreamed. PDO Objective 2: Assess student learning in primary and secondary education. Rating: High 39. The project’s second objective was to improve the borrower’s capacity to assess student learning in primary and secondary education. Achievement of this objective was measured by PDO indicator 3 (student learning outcomes for the first cycle of primary education analyzed and disseminated). The baseline was “no”, and the end target was “yes”. The target was achieved and surpassed, with a final measurement of “yes”. In addition to the evaluations implemented for the first cycle of primary education -grade 3- (DLI 14), the project also implemented evaluations at the secondary school level (grade 6 and grade 9). Since the project fully achieved its objective and surpassed its original scope of evaluating the first cycle of primary (grades 1 to 3), to include secondary, it is rated as high. 40. Assessment activities were successfully implemented and institutionalized in MINERD. The regulatory framework was approved to support sustainability of evaluation implementation. MINERD implemented national diagnostic evaluations for grade 3 in 2017 and reports with the learning results were generated and disseminated at the regional, district, and school levels. Diagnostic evaluations were also carried out and disseminated for grades 6 and 9 in 2018 and 2019, respectively. Government Ordenanza Number 1-2016 was approved and included an article that established census evaluations for all 3rd, 6th, and 9th graders, representing the last grades of each education cycle.41 Starting 2017, one grade was evaluated per year, and all three grades had their first round of evaluations before the onset 41 https://www.ministeriodeeducacion.gob.do/docs/direccion-general-de-evaluacion-y-control-de-la-calidad-educativa/uLRy- ordenanza-n01-2016pdf.pdf Page 18 of 69 The World Bank Support to the National Education Pact Project (P146831) of COVID-19 pandemic. In addition, a census evaluation of grade 3 using DR’s own assessment took place in 2023. 41. With these project activities, MINERD increased its capacity to assess student learning. The project activities were supported by training and South-to-South learning to ensure the strengthening of MINERD’s ability to carry out the evaluations in the future. Workshops and trainings were carried out with the Ministry of Education of Colombia on the development of instrument items for student assessment. Trainings were also carried out on topics including sampling, database management, evidence-based methodologies, psychometric analysis, data analysis and dissemination. In 2022, MINERD prioritized two key activities related to the learning measurement agenda: (i) application of sample-based national student learning assessments of grades 3, 6, and 9 to estimate learning losses following the pandemic, and (ii) design of a student learning monitoring system for schools. A de-facto important change introduced in the application of the evaluations was a shift away from the “high stakes” evaluations that were previously given to 8th graders which kept some students from advancing and contributed to greater school dropout. 42. The project also aided the Dominican education system with the participation in and dissemination of international assessments (IRI 12). A total of 254 schools and 7,878 students participated in the Programme for International Student Assessment (PISA) in May 2022 (DLI 3). The results were published in December of 2023. The latest PISA results showed statistically significant improvements in mathematics, reading, and science for the DR. The project also developed a strategy for using and disseminating results data. DR has expressed its commitment to continue participation both in ERCE and in PISA and to the dissemination of results and on using those results for better education interventions. PDO Objective 3: Evaluate the quality of service provided by public early childhood development centers. Rating: High 43. The project’s third objective was to improve the borrower’s capacity to evaluate the quality of service provided by public early childhood development centers. The achievement of objective 3 was to be measured using PDO indicator 4 (percentage of QEC care centers- called CAIPIs- that have been evaluated against quality standards and their results analyzed). The baseline was 0 percent, and the end target was 50 percent. The target of evaluating 50 percent of QEC care centers against quality standards and analyzing their results was achieved, with a final measurement of 50 percent (PDO 4 and DLI 20). Since the Project fully achieved its objective, it is rated as High. 44. The Instituto Dominicano de Evaluacion e Investigacion de la Calidad Educativa (IDEICE) and INAIPI evaluated the teaching quality of 30 CAIPI or the equivalent of 50 percent of all ECE centers in the country. The CAIPIs were evaluated using TEACH-ECE, a classroom observation tool that explores what goes on in ECE classrooms vis-a-vis the quality of teaching practices that nurture children’s cognitive and language skills, socioemotional competencies, and executive functioning abilities for children 3-6 years of age42. IDECIE and INAIPI staff were trained and certified to use the TEACH-ECE tool (IRI 14).43 After the training, the staff evaluated the preschool teachers’ pedagogical practices and prepared results reports, focusing on the areas where pedagogical practices needed to be strengthened. The observations included around 60 auxiliary teaching agents and 750 children. The implementation of the assessments was, however, interrupted by the COVID-19 pandemic as it was difficult to get the analysis from the professionals who were required to 42 TEACH-ECE is a free classroom observation tool for early childhood education for children ages 3-6 that explores what goes on in the classroom and the quality of ECE teaching practices. The tool allows countries to monitor and improve teaching quality. Videos were also recorded on the correct use of the tool. 43 Other activities include: INAIPIs strategic plan was implemented, including quality assurance standards; a communication strategy about the program was developed; and a discussion table on the results of the last TEACH ECE 2022 evaluation was formed. Page 19 of 69 The World Bank Support to the National Education Pact Project (P146831) process the observations form the TEACH-ECE. Despite the challenges, after COVID, the analysis of the data collected through the instruments was analyzed and the project objective was met. 45. The evaluation of ECE centers encouraged the discussion of the importance of measuring standards at the ECE level. In April 2022, more than 40 decision makers and staff from MINERD, INAIPI, and IDEICE gathered to discuss the application of the TEACHE-ECE tool over the preceding three years and expressed its value to accurately measure pedagogical practices and promote improvement in classroom practices. 46. The project successfully improved the capacity to evaluate early childhood development centers through the implementation of the TEACH-ECE tool. This tool provided a detailed assessment of the quality of teaching practices in these centers. The evaluations covered a significant number of centers and involved comprehensive training for staff to effectively use the assessment tool. Currently, the World Bank team is working with the Ministry of education to develop a tool to assess the student outcomes and continue improving the quality of the early childhood development centers. PDO Objective 4: Enhance the process for decentralizing public school management. Rating: High 47. The project’s fourth objective was to improve the borrower’s capacity to enhance the process for decentralizing public school management. Achievement of this objective was measured by PDO indicator 5 (increasing percentage of fully functional school management committees (Juntas de Centro, SMC) implementing their school development plans). A “fully functional” SMC: (i) is legally constituted, (ii) has an improvement plan and bank account, and (iii) receives transfers from MINERD. Implementing school development plan translates to Implementing SMCs spending the money according to the improvement plan with the participation of all members of the SMCs in the decision making. The baseline was 0 percentage points (pp) and the end target was 10 pp. The target was surpassed, with a final 28 pp increase in SMCs implementing their school development plans. Since the project surpassed the original target, it is rated as high. 48. By the closure of the project, out of 7,687 schools which could have SMCs, 7,276 schools had formed legally constituted SMCs, of which 2,525 were had received resources from MINERD. An audit was carried out during 2023 by IDEICE to evaluate the number of SMCs that were fully functional and used school resources according to their POA (Operating Annual Plan). As a starting point for the audit, IDEICE took the database provided by the Directorate of Decentralization with the 2,524 that were legally constituted, had bank accounts, and had received funds from the MINERD. A sample of 304 SMCS was randomly selected from this database to verify the accomplishment of DLI 16. For this purpose, SMCs were interviewed and POA were collected from them. The findings of this audit showed that 91 percent of the sample prepared a POA, and 94 percent spent most of the funds according to the POA. Around 85 percent of the sample selected prepared a POA and also spent their resources according to their POA. By applying this percentage to the total number of SMCs that received resources from MINERD, around 2,145 SMCs would be considered fully functional. Therefore, DLI 16 target of 1,750 fully functional school management committees was achieved 44. 49. A review of the regulation on school-based management was also supported by the project. A new regulation for SMCs was approved in 2018 (Ordenanza No. 02-2018, which modifies Ordenanza No. 2-2008), and included new manuals for SMCs. A new regulation from 2019 (Ordenanza No. 02-2019) redefined how funds for SMCs were managed and allowed for greater flexibility in school spending, while maintaining transparency and efficiency. This was a welcome development since the rigidity of spending with the previous legal framework was a great constraint for schools. In addition, a communication and training campaign was developed to inform SMC members about their roles and 44 In addition to the external audit, staff from the OCI financial team from MINERD carried out a separate review in a sample of schools, which also concluded that the objective was met. Page 20 of 69 The World Bank Support to the National Education Pact Project (P146831) responsibilities. This included training on: (i) how to form SMCs, (ii) resource management, and (iii) financial accountability. 50. The project moved decentralization forward, which proved to be crucial for the reopening of schools during the COVID-19 pandemic. Student support programs were administered with the support of SMCs. In addition, during the pandemic, principals were authorized to use decentralized funds for basic services such as food rations. The need to provide technological resources to students to allow distance learning during school closures was also important and SMCs played a role in making this happen. 51. The project excelled in enhancing decentralized school management by increasing the number of fully functional School Management Committees (SMCs). These committees played a pivotal role in managing school resources and implementing school development plans. The project also redefined regulatory frameworks to allow more flexibility in school spending and promoted the training of SMC members in financial accountability and resource management. In addition to the planned activities, the project also made significant progress in strengthening decentralized management of the education system. This was achieved by: (i) re-defining the roles and responsibilities of district and regional directors and establishing the first competitive recruitment for these roles, and (ii) improving training for in-service school principals. Justification of Overall Efficacy Rating 52. The overall efficacy of the PDO is High based on the high ratings of the four Subobjectives. As described above, the rating is justified considering the achievement of PDOs, related DLIs and IRIs. The overall achievement of project objectives is summarized in Table 2. Table 2: Summary Assessment of Overall Efficacy Overall Efficacy High (4) Objective 1 High (PDO 1 and 4 Objective 2 PDO 2) 4 Objective 3 High (PDO 3) 4 Objective 4 High (PDO 4) 4 High (PDO 5) C. EFFICIENCY Assessment of Efficiency and Rating 53. The project’s efficiency is rated substantial, considering high rating of e economic and financial efficiency s (Ref: Economic and Financial analysis (ETA) Annex 4), and modest rating of project implementation efficiency. 54. The economic and financial efficiency of the project is rated high, with an Internal Rate of Return (IRR) between 41.6 percent and 61.8 percent with benefits at project closing exceeding the forecasts at appraisal. The targets reached under the project had a direct impact on the capacity of MINERD better manage the education system, improving both Page 21 of 69 The World Bank Support to the National Education Pact Project (P146831) education and labor market outcomes. This impact was measured by: (i) the higher probability of school completion, which increases lifetime earnings and improves employment opportunities and (ii) the higher labor productivity growth (using labor earnings as a proxy for labor productivity). 55. A Present Discounted Value (PDV) approach was used to estimate the benefits and costs of schooling over the lifetime of a student at project appraisal and at the project’s closing . Table 3 presents a summary of these analyses undertaken for the EFA. The secondary school completion rates in 2018 were found to be higher than those forecasted in the PAD.45 The project interventions also yielded expected economic incremental benefits of RD$105,767 (in 2020 RD$, or roughly US$1,923) per final beneficiary student in terms of adult labor earnings, which is 12 percent higher than forecasted in the PAD (RD$94,795 in 2020 RD$). Based on this result, the IRR for the project was estimated to be between 41.6 percent and 61.8 percent. Table 3 presents the results of the sensitivity analysis, where some parameters of the model vary to verify how gains change. Even when reducing the expected labor earnings growth rate or reducing the effect on secondary completion rate, or reducing both, the result is smaller, but remains positive. Table 3: PDV Estimates of the Expected Individual Economic Gains from the Project PDV of expected PDV of expected PDV of expected Case incremental economic incremental economic incremental economic benefits (in 2020 RD$) costs (in 2020 RD$) benefits (in 2020 RD$) PAD Closing PAD Closing PAD Closing Base 225,542 274,152 130,747 168,385 94,795 105,767 Reduced adult labor earnings growth rates 174,993 272,889 130,747 168,385 44,247 104,504 Reduced project effect on the probability of secondary school completion 184,063 237,351 128,696 168,295 55,367 69,056 Reduced adult labor earnings growth rates + probability of secondary school completion 134,253 236,094 128,696 168,295 5,557 67,799 Source: Authors’ calculations. 56. The efficiency of implementation is rated modest. There was a trade-off between efficacy and efficiency of the project. All PDO level indicators were achieved and 19 out of 20 DLIs disbursed. However, the project needed a three- year extension to accommodate the 14-month effectiveness delay, proposed Additional Financing, and delays related to 45 For instance, we observe that the secondary completion rate for urban females was 46 pp higher than expected and 34 pp higher for rural females. Page 22 of 69 The World Bank Support to the National Education Pact Project (P146831) the COVID-19 global pandemic. 46 Even though the additional financing did not go through, the extension was required to make up for the implementation delay and ended up providing a more comfortable, and less rushed, timeline for the project, that allowed to fulfill the ambitious objectives under a difficult pandemic. Other than implementation delays, there were no other changes to the project, including financing amounts of project components. The modest rating is broadly consistent with the ratings included in the Implementation Status and Results Reports (ISRs) for implementation progress where 10 of the 16 ISRs rated the implementation progress as moderately satisfactory (Annex 6). Table 4: Summary of Overall Efficiency Overall Efficiency Substantial (3) EFA analysis High (4) Implementation Modest (2) D. JUSTIFICATION OF OVERALL OUTCOME RATING 57. The overall outcome of the project is rated satisfactory, in alignment with the ratings given to the relevance of the project at closing, its efficacy during the implementation period and its efficiency. Relevance at closing is rated High, efficacy is rated High, and efficiency is rated Substantial. The satisfactory rating of this project considers the minor shortcomings of the project, particularly regarding its implementation efficiency. Table 5: Summary of Overall Outcome Rating Relevance (at closing) High Efficacy High Efficiency (at closing) Substantial Overall Outcome Satisfactory Note: *Highly Unsatisfactory (1); Unsatisfactory (2); Moderately Unsatisfactory (3); Moderately Satisfactory (4); Satisfactory (5); and Highly Satisfactory (6). E. OTHER OUTCOMES AND IMPACTS Gender 58. The project exhibited positive results in addressing gender equity. The project disaggregated by gender several intermediate indicators to measure progress towards closing of gender gaps. The proportion of female beneficiaries surpass the targets in all three indicators that were disaggregated: 1) Number of positions to study teaching degrees at ISFODOSU filled through revised entrance exam; 2) Number of positions staffed through the teacher entrance exam; 3) Direct number of project beneficiaries benefiting from enhanced decentralized school management. For instance, 46 As Appendix G of the ICR 2021 Guidelines indicate: “In general, if no additional outcomes are enabled by a time extension or cost overrun, then adverse impact on efficiency is likely indicated; if an extension of more than one year is required to achieve the initially expected outcomes, inefficiency is suggested.” Page 23 of 69 The World Bank Support to the National Education Pact Project (P146831) seventy-nine percent of the places for teaching degrees at ISFODOSU were filled by women with the institution’s revised entrance exam. The staffing of teaching positions through the revised professional entrance exam mirrored this trend with 79 percent of the 40,093 vacancies filled by females, which was a considerable increase from the baseline of 6,000. Of the total number of teachers, women's participation increased by two percentage points from the time of project approval to 2021, from seventy-five percent to seventy-seven percent. The project also had an impact in terms of its beneficiaries. Out of the 524,476 project beneficiaries from enhanced decentralized school management, 58 percent were female, emphasizing the project's commitment to supporting and empowering female students and teachers. 59. Additional efforts were made to combat violence in schools, with a focus on gender violence and gender stereotypes. In 2018 and 2019, the Bank team used a grant from the Nordic Trust Fund to support MINERD in developing and delivering a workshop for teachers and other stakeholders to develop skills in combating violence in schools, with a specific focus on gender stereotypes and gender-based violence. The team built on this experience to help MINERD integrate the methodology into the in-service teacher training, utilizing a small grant from the State and Peacebuilding Fund. Institutional Strengthening 60. The project strengthened MINERD’s institutional capacity. Many of the achievements under the project will continue to have a significant impact after its closure. One of the primary indications of this is seen in the effort to enhance the borrower's capacity to assess student learning across primary and secondary education levels. The project’s influence led to the integration of the evaluations into the regulatory framework, ensuring the use of these assessments in the long run. The project supported workshops and trainings focusing on sampling, data management, and evidence-based methodologies, to strengthen the capacity of MINERD staff to carry out assessments. MINERD now conducts census assessments in 3rd grade and is continuing its participation in ERCE and PISA international assessments. Analysis of the data from the assessments is used for policymaking. In addition, MINERD is extending the use of assessments to other grades to keep track of learning and inform the Ministry’s policies. In addition, the project also strengthened the overall schooling system and the accountability mechanisms. The project's emphasis on decentralizing public school management through the successful establishment of school management committees, led to a more participatory and accountable framework for school administration that is now fully functional. Finally, the teacher entrance exam, as well as the exam to select teaching degrees are processes that have been internalized by the Ministry and continue to take place. We expect that both entrance exams will continue to take place in the years to come, as the change of culture favoring meritocracy has strengthened over the years. Poverty Reduction and Shared Prosperity 61. By focusing on improving the quality of education from early childhood to secondary school, the project laid the groundwork for equipping students with the necessary skills and knowledge to pursue higher-paying jobs, boosting their human capital, and escaping the cycle of poverty. Furthermore, the decentralization of school management has enabled more localized decision-making to ensure resources are directed to areas of utmost need and that the challenges faced by individual communities are addressed through the process. The skills and knowledge developed through formal education thus provided significant contributions to individual labor productivity. 62. Regarding shared prosperity, the increased emphasis on female participation in teaching roles has also promoted a more inclusive growth model, reduced gender disparities and ensured that the benefits of economic Page 24 of 69 The World Bank Support to the National Education Pact Project (P146831) growth are more equally distributed. Over time, this foundation of quality education and equal opportunity is crucial for human capital formation, specifically to reduce socio-economic inequalities for sustainable shared prosperity. Other Unintended Outcomes and Impacts 63. In addition to its success achieving its targeted objectives, the project had other positive outcomes and impacts. The broadened scope of student evaluations to include grades beyond the original target of the first primary cycle is a notable outcome that could potentially offer deeper insights into educational attainment at different levels. Additionally, the shift away from high-stakes evaluations in grade 8, although not originally envisioned at project design given the likely political implications, is nonetheless a significant achievement. This shift is likely to improve student motivation and reduce dropout rates. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 64. The preparation phase of the project was rooted in a comprehensive understanding of the educational landscape and a commitment to bringing about transformative change. Central to this was the government's strategy to embrace a reform of the education sector, culminating in the signing of the National Education Pact (NEP) in April 2014, after resources to the sector were increased to almost 4% of GDP. The NEP garnered broad endorsement from diverse sectors of society including universities, teachers' unions, independent agencies, and civil society. It proposed significant reformative measures including: (i) an overhaul of both pre-service and in-service training programs complemented by the introduction of a professional entrance exam; (ii) holistic adoption of a culture grounded in the evaluation of all stakeholders in the education ecosystem; (iii) an emphasis on improving the quality of early education, and (iv) efforts to modernize educational management with an emphasis on decentralization for greater efficiency. The fact that there was broad support of these initiatives among key stakeholders ensured that the project was established on a shared vision for the education sector. 65. The time lapse between project approval and effectiveness created a lag in project advancement. The project was approved on September 30, 2015, but only became effective over a year later on December 9, 2016. Due to the delays in effectiveness, some results were retroactively accepted.47 The one-year delay between the approval and effectiveness of the project follows the time frame typically necessary for projects to become effective in the country. 47 Retroactive financing was allowed up to a maximum of US$10 million (20 percent of the project) under the project, conditional on i) achievement of DLIs, and ii) eligible expenditures under the eligible expenditure program made one year before the loan signing date. Page 25 of 69 The World Bank Support to the National Education Pact Project (P146831) B. KEY FACTORS DURING IMPLEMENTATION 66. The Additional Financing was prepared and approved but never became effective due to approval delays and the emergency brought on by COVID-19. The Additional Financing (AF, P167815) in the amount of US$100 million was approved by the World Bank Board of Executive Directors on December 13, 2018. The AF Loan Agreement was signed by the Dominican Republic’s Ministry of Finance in December 2019. At the COVID-19 outbreak in March 2020, the government prioritized emergency responses to alleviate the impact of the pandemic. As a result, congressional approval of the AF Loan Agreement was not obtained before the 18-month deadline for declaration of effectiveness and the AF Loan Agreement was terminated on June 7, 2020. 67. When the new administration took office in August 2020, after a complex political transition, it confirmed its support of the project’s objectives and requested the reinstatement of the AF Loan Agreement as the proposed activities were consistent with its new policies, priorities, and COVID-19 recovery strategy. On March 5, 2021, the Bank agreed to proceed with an exceptional reinstatement of the AF Loan Agreement and established September 30, 2021, as the new effectiveness deadline. However, the government requested a four-month extension of the effectiveness deadline from September 30, 2021, to January 31, 2022, due to the significant increase in the number of requests for congressional approval to support multiple COVID-19 crisis responses, the increasing political polarization, and the lengthy scrutiny of loan agreements. While the government reiterated its support of the learning recovery agenda and implementation of the proposed project activities, congressional approval of the AF was delayed and the Loan Agreement was terminated on January 31, 2022, after failing to meet the revised effectiveness deadline. 68. The COVID-19 pandemic posed significant challenges and interruptions to the project. The political transition in August 2020 occurred in the middle of the COVID-19 crisis, which hit the country very hard. The new administration had to deal with the establishment of a new government team in parallel to organizing health, economic, and social responses to the COVID-19 crisis. This slowed progress in the implementation of some of the project activities. The COVID-19 crisis required MINERD, Office for International Cooperation (OCI), and other decentralized agencies to focus efforts on the development of education inputs for the provision of distance learning to students. In addition, the COVID-19 crisis also reduced the workday of the project’s staff in MINERD, OCI, and other decentralized agencies. It also required a share of the staff to work from home for part of the week. The reduced workday and Internet connectivity restrictions, in turn reducing the pace of implementation of some of the project activities. School closures due to the pandemic also required the adaptation of student assessments. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 69. The main objectives were clearly stated in the PDO statement. These included improving the borrower’s capacity to: (i) recruit and train primary and secondary school teachers, (ii) assess student learning in primary and secondary education, (iii) evaluate the quality of service provided by public early childhood development centers, and (iv) enhance the process for decentralizing public school management. The PDO indicators and intermediate results indicators were well-aligned with the objectives and encompassed all outcomes in the PDO statement (Annex 1 –results framework). Page 26 of 69 The World Bank Support to the National Education Pact Project (P146831) Baselines and targets were established for all indicators. The DLIs were also aligned with the PDO and the methodology for verification was clear. The theory of change established clear links between activities, outputs, and outcomes. 70. The results framework remained the same during the implementation of the project, reflecting its robust design. The framework includes 5 PDO level indicators and 17 intermediate results indicators. 71. By using a results-based system of management, monitoring and evaluation was built into the design of the Project. Disbursements required evidence of compliance with the DLI’s that were established. The Government presented evidence on the completion to evaluate the achievement of each DLI. M&E Implementation 72. Project monitoring reports were created every six months using regular administrative data from the Ministry of Education, which allowed the World Bank team to monitor the project’s progress. The main PDOs were reported every six months in the ISRs. The data for the indicators and activities were also monitored during each Bank mission, ensuring the project was on track. Training sessions for MINERD staff were carried out, mainly for objective 2 and 3 to ensure adequate data collection. The project’s ISRs maintained a satisfactory M&E rating (Annex 6). M&E Utilization 73. The use of M&E data was adequate to enable the project to track progress. M&E information was monitored and updated by the project implementation unit (PIU), allowing the Bank team to review the progress of the project towards achieving its objectives. The project design included an impact evaluation of INAFOCAM in-service programs, and adjustments were incorporated to improve two of the programs. 74. The development of student assessments filled an existing information gap regarding learning in primary education. Reports were generated for different audiences, marking a milestone for decision making and taking actions for improvement. Justification of Overall Rating of Quality of M&E 75. The overall quality of the M&E was substantial. As noted above, a strength of the project was its M&E system’s design. In terms of implementation, the generated data was not only adequate for tracking the progress of the project but was also sufficient to assess the achievement of the objectives and test the links in the results chain. The results of the impact evaluation of INAFOCAM in-service programs allowed for improving the two programs, thus justifying the utilization of M&E. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 76. Procurement. This project was disbursed through DLIs, which are tied to achieved targets and eligible expenditures. The only procurement conducted by the project was the selection of the audit firm. The procurement performance rating was satisfactory throughout the project life. 77. Financial Management. Regular audits were conducted to identify potential issues and maintain the financial soundness of the project. The financial management (FM) arrangements of the project generally maintained moderately Page 27 of 69 The World Bank Support to the National Education Pact Project (P146831) satisfactory ratings in the ISRs throughout implementation. In May 2021, the rating was downgraded to moderately unsatisfactory, where it remained for almost one year because of an overdue audit report from 2019 and because interim financial reports (IFRs) had not been generated or submitted to the Bank until December 2021. The project Financial Management rating went back to MS in July 2022 and maintained that rating until closing date. C. BANK PERFORMANCE Quality at Entry 78. The World Bank performance for quality at entry is rated satisfactory. The project design took into consideration lessons learned from other Bank projects.48 The WB team ensured that the design was realistic in its expectations, which translated into a good project design, emphasizing a commitment to enhancing the quality of learning rather than an outright promise to improve learning outcomes. The Team appropriately appraised the operation at entry by ensuring it was aligned with the DRs education challenges and medium-term sector priorities and by designing an intervention with MINERD that was aligned with the National Education Pact. Thus, the project had government ownership. The objectives were relevant, and the results framework and results chain were clear and aligned with one another. Quality of Supervision The quality of World Bank supervision is rated satisfactory. 79. The World Bank team engaged with MINERD and OCI teams throughout the project lifetime particularly when implementation was progressing slowly. During the first years of the project the team adapted to circumstances to improve the project through the restructuring, which even if the additional financing was not approved, it did provide an extension to make up for the implementation delay. In the two years after the COVID-19 pandemic, the implementation progress remained at moderately satisfactory (ISR 10 to ISR 14), returning to satisfactory shortly thereafter. The ISR’s noted that, “the Bank's team has been proactively engaging with MINERD and OCI teams through bi-weekly meetings to follow up on Project implementation. This combined with the lifting of some of the social distancing restrictions has helped the pace of Project implementation with respect to previous semester”. As indicated in ISR 14 and 15, the Bank's team increased the frequency of the meetings to weekly follow-up meetings for technical issues and project implementation. The Bank team proactivity promptly identifying challenges and proposing measures to address them allowed for all project activities to move forward and be satisfactorily completed. [this only describes the latter part of project implementation. What about 2016-2019. 80. Supervision missions were undertaken every six months, and ISRs were timely and complete. During the pandemic, some missions were cancelled. However, during 2022, three missions were carried out. The first one was an implementation support mission and took place on April to follow up on project activities. The second one was a technical mission, which took place that same month to accompany the start of the implementation of the third pilot of an adaptive learning platform and to continue discussions related to project activities. The third mission was another technical mission that was carried out in June and was focused on policy discussions to learn about MINERD's strategies for the 48 The project design was informed by lessons learned from other Bank projects, namely the three phases of the Performance and Accountability of Social Sectors (PASS) Development Policy Loan (P116972), and the DR Early Childhood Development Project (P079566). The project also incorporated experiences from other DLI-based projects. As per experiences in countries like Jamaica, Pakistan, Peru and Moldova, a DLI-based design would provide flexibility to the Government in achievement of their targets and align interests around national results, while simplifying fiduciary procedures. Page 28 of 69 The World Bank Support to the National Education Pact Project (P146831) implementation of the new educational model. This showed the Bank’s commitment to have the project on track to meet its objectives by closing, which indeed happened. Justification of Overall Rating of Bank Performance 81. The overall rating of World Bank performance is rated satisfactory considering Bank’s satisfactory performance at entry and supervision. In the face of challenges during the implementation period, the Bank team proactively addressed the issues in close collaboration with the government to be able to achieve project objectives. The Borrower Report rated the World Bank performance as highly satisfactory (Annex 5). D. RISK TO DEVELOPMENT OUTCOME 82. The rating for all risk categories using the Systematic Operations Risk-Rating Tool was low to moderate at closing. At the time of approval, the ratings were, in general, moderate to substantial, showing that the risk ratings, in particular technical design and stakeholders had decreased by closing. The Bank team worked closely with government counterparts to smoothen technical difficulties that ultimately ended in the government moving forward and ultimately mainstreaming project activities. Many activities of the project were also accompanied by changes in regulatory framework and a much more technically experienced Ministry, which help to ensure the continuity of activities (see section II.B for further details). 83. At the time of closing, capacity has been created by the Borrower to maintain the activities that were supported by the project. Capacity was effectively built through the project by enhancing teacher recruitment and training standards, adopting professional competencies, and implementing rigorous entrance exams. The project also significantly advanced student learning assessment by exceeding its primary objectives, incorporating evaluations across various educational levels, and institutionalizing these assessments within the Ministry of Education (MINERD). Additionally, the capacity for evaluating Early Childhood Development centers was improved through the implementation of the TEACH- ECE tool. Key to this success was the emphasis on decentralized school management, which not only facilitated the management of resources during the COVID-19 pandemic but also strengthened School Management Committees (SMCs) in financial accountability and resource management. This collectively built a sustainable framework for maintaining and enhancing educational practices after the project closing. V. LESSONS AND RECOMMENDATIONS 84. While there were many important lessons learned during the project, the following are considered most relevant: a. Aligning a project with medium or long-term sectoral policies provides a solid foundation to ensure continuity. The project was fully aligned with the National Education Pact, which was signed in 2014. The NEP and its strategies are aligned with the National Development Strategy 2010–2030 and the Ten-Year Education Plan. Further, the NEP had a broad endorsement from various stakeholders including independent agencies, universities, teachers’ unions, and civil society. The solidity of its framework ensured the continuation of the project despite a change in government in 2020 and Page 29 of 69 The World Bank Support to the National Education Pact Project (P146831) a change in education ministers in 2016 (right before approval of the project), 2019, 2020, and 2022 (during its implementation). b. Proposed structural reforms, though not all of which were completed during project lifetime, should be pursued to maintain momentum of the mandated transformative changes. Even if the final target is not realized, the process often brings improvements that serve to enhance the overall education landscape. The teaching profession is the backbone of any education system, and attempts to elevate its standards, provide comprehensive training, or streamline recruitment processes can lead to profound and lasting impacts on the quality of education. While the project may not have achieved all its objectives regarding the teaching career framework, it advanced the basis for moving this topic forward. c. The development of strategic collaboration between government agencies facilitates meeting project objectives while strengthening institutional capacity. For the fulfillment of the indicators of component 3, INAIPI, a relatively new institution, was supported with the collaboration of IDEICE, an institution that has demonstrated its strengths in terms of evaluation. IDEICE staff worked hand in hand with INAIPI staff. This allowed not only to have solid processes of evaluation of the quality of INAIPI's services, but also the staff of this institution also benefited by acquiring knowledge of how to evaluate their processes. d. World Bank projects depend on many actors for their successful implementation. The additional financing process that the project went through and finally fell through due to change of priorities in the congress of the Dominican Republic, shows that World Bank projects often operate in political environments with competing and urgent demands, like the COVI-19 pandemic. Sector loans are only one instrument that are at government’s discretion and agreements with governments may never materialize. Potential changes done to projects, like extending dates to accommodate for additional financing, can end up negatively affecting the projects—as in this case the efficiency rating—if the changes are not approved by other actors on which projects depend on. e. Empowering a decentralized and inclusive (including SMCs) financial decision-making process can lead to more efficient and relevant allocations , directly benefiting the school community. However, with this autonomy comes the added responsibility of ensuring financial integrity and transparency. It is important to provide SMCs with comprehensive training on financial practices and on the use of Annual Operations Plans. By equipping them with the necessary skills and knowledge, SMCs can effectively allocate money towards the students’ and school’s best interests. f. The use of DLIs was the right instrument for regulatory reforms. When the project objectives are aligned with the government objectives, activities do not require a significant amount of resources by the government, or they relate to regulatory changes and their implementation, DLIs provide important incentives to have these done. This is because Ministries have support (and pressure) for their reforms from other government ministries, including finance. The use of DLIs for project with such activities is recommended for future operations in the region. . Page 30 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improve the Borrower's capacity to recruit and train primary and secondary school teacher Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teacher positions staffed Yes/No No Yes No Yes through the revised teacher entrance exam 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): See Section II.B. for more details. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Positions to study teaching Yes/No No Yes No Yes degrees at ISFODOSU filled through revised entry exam 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Page 31 of 69 The World Bank Support to the National Education Pact Project (P146831) See Section II.B. for more details. Objective/Outcome: Improve the Borrower's capacity to assess student learning in primary and secondary education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Student learning outcomes Yes/No No Yes No Yes for the first cycle of primary education analyzed and 30-Jun-2014 30-Jun-2020 30-Jun-2023 disseminated Comments (achievements against targets): See Section II.B. for more details. Objective/Outcome: Improve Borrower's capacity to evaluate quality of service provided by Public Early Child. Dev. Cen. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of QEC care Percentage 0.00 50.00 0.00 50.00 centers that have been evaluated against quality 30-Jun-2014 30-Jun-2020 30-Jun-2023 standards and their results analyzed Comments (achievements against targets): Page 32 of 69 The World Bank Support to the National Education Pact Project (P146831) See Section II.B. for more details. Objective/Outcome: Improve the Borrower's capacity to enhance the process for decentralizing public school management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increasing percentage of fully Text 0 percentage points 10 percentage points NA 28 percentage points functional school management committees 30-Jun-2014 30-Jun-2020 30-Jun-2023 (Juntas de Centro) implementing their school development plans Comments (achievements against targets): See Section II.B. for more details. The percentage of schools that are implementing their projects would be determined through a school sample. An audit would be carried out by an agent external to MINERD, and the estimation of the percentage would be based on a random sample of the SMCs that are fully functional (denominator). By fully functional it is understood the SMCs that are legally constituted, with an improvement plan, with a bank account, and receiving transfers from MINERD. By implementing it is meant that SMCs spend the money according to the improvement plan and with the participation of all members of the SMCs in the decision making. Guidelines for evaluating successful implementation of improvement plans would be jointly designed by MINERD and the Bank, and they would be made clear in the terms of reference of the agent that would verify the achievement. The size of the sample and the actual random draw of the sample would be done by an expert in sampling. Units of measure for this indicator are percentage points and it is cumulative. Page 33 of 69 The World Bank Support to the National Education Pact Project (P146831) A.2 Intermediate Results Indicators Component: Improving capacity to recruit and train primary and secondary school teachers Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design and implementation Text Revised entrance ISFODOSU revised NA ISFODOSU revised of ISFODOSU revised examination designed. entrance examination entrance examination entrance examination is applied to all its is applied to all its applicants. applicants. 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): The entrance examination was revised at the beginning of the Project, and is applied to all applicants since the year 2021. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of positions to study Number 0.00 5,600.00 0.00 26,686.00 teaching degrees at ISFODOSU filled through 30-Jun-2014 30-Jun-2020 30-Jun-2023 revised entry exam Percentage of Female Percentage 0.00 65.00 0.00 79.00 Comments (achievements against targets): Page 34 of 69 The World Bank Support to the National Education Pact Project (P146831) The actual number of positions filled was almost five times de original target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Implementation of Text Not started ISFODOSU offers at NA ISFODOSU offers at ISFODOSU secondary-level least two secondary least two secondary teaching degrees level teaching degrees level teaching degrees and advances on and advances on implementation of implementation of curricula. curricula. 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion ISFODOSU has started the Text Not started ISFODOSU has started NA ISFODOSU has implementation of its own the implementation of prepared its new teaching career its own professional professional development plan for development plan. its educators (DLI 17) 30-Jun-2014 30-Jun-2020 30-Jun-2023 Page 35 of 69 The World Bank Support to the National Education Pact Project (P146831) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design and adoption of Text Design and adoption Professional standards NA Professional standards professional standards for completed by MINERD for teachers adopted for teachers adopted teachers and disseminated and disseminated 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design and implementation Text MINERD applies MINERD applies NA MINERD applies of competitive entrance competitive entrance entrance exam entrance exam exam for teachers exam to hire teachers (Concurso de (Concurso de (Concurso de Oposición) aligned Oposición) aligned Oposición) with professional with professional standards for teachers standards for teachers 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Page 36 of 69 The World Bank Support to the National Education Pact Project (P146831) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design and adoption of Text Career proposal At least 40% of NA Not achieved teaching career proposal, drafted teachers have been and implementation of evaluated using the teacher evaluation system revised teacher evaluation system, which is aligned with professional standards for teachers 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design and adjustment of Text Not started INAFOCAM adjusts NA INAFOCAM adjusts in- INAFOCAM in-service inservice training service training training programs programs based on programs based on results from results from assessments and/or assessments and/or impact evaluations impact evaluations (DLI 18) 30-Jun-2014 30-Jun-2020 30-Jun-2023 Page 37 of 69 The World Bank Support to the National Education Pact Project (P146831) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teacher positions Number 6,000.00 19,000.00 0.00 40,126.00 staffed through the teacher entrance exam 30-Jun-2014 30-Jun-2023 30-Jun-2023 Percentage of Female Percentage 0.00 80.00 79.00 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Implementation of SIGEF in Text Planning for INAFOCAM and NA INAFOCAM and MINERD decentralized implementation in ISFODOSU’s annual ISFODOSU’s annual agencies both institutions financial reports are financial reports are prepared using SIGEF prepared using SIGEF (DLI 12) 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Page 38 of 69 The World Bank Support to the National Education Pact Project (P146831) Component: Improving capacity to assess student learning in primary and secondary education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Student learning outcomes Text Not started Student Assessment NA MINERD’s Student for the first cycle of primary Directorate staff have Assessment education analyzed received training on Directorate staff have (including by gender) and sampling, item design, received training on disseminated database sampling, item design, management, and database statistical (DLI 19) management, and statistical software. 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Application and Text PISA 2015 pilot Workshop on use and NA Workshop on use and dissemination of applied disseminati on of dissemination of international student international international assessments assessment assessment completed 30-Jun-2014 30-Jun-2020 30-Jun-2023 Page 39 of 69 The World Bank Support to the National Education Pact Project (P146831) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion System for learning Text Yes Yes (4.0) NA Yes (4.0) assessment at the primary (3.0) level (utility of learning assessment) 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Component: Improving capacity to evaluate the quality of service provided by public early childhood development centers Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Capacity strengthening of Text Discussions about the 50% of functioning NA 50% of functioning INAIPI and assessment of themes the content of Public Early Childhood Public Early Childhood QEC centers INAIPI’s strategic plan Development Centers Development Centers are assessed for are assessed for quality standards quality standards 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Page 40 of 69 The World Bank Support to the National Education Pact Project (P146831) Component: Improving capacity to enhance the process of decentralized school management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of School Number 1,021.00 1,750.00 0.00 1,021.00 Management Committees (SMCs) receiving funds 30-Jun-2014 30-Jun-2020 30-Jun-2023 directly Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Implementation of revised Text Not started Implementat ion NA Implementation SMC guidelines advanced on revolving advanced on revolving budgets and budgets and notification to SMCs. notification to SMCs. 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiary feedback Text Not started All three mechanisms NA All three mechanisms Page 41 of 69 The World Bank Support to the National Education Pact Project (P146831) included at different level in place and in place and implemented implemented 30-Jun-2014 30-Jun-2020 30-Jun-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 455,520.00 0.00 524,476.00 30-Jun-2014 30-Jun-2020 30-Jun-2023 Female beneficiaries Percentage 0.00 53.00 0.00 58.00 Comments (achievements against targets): Page 42 of 69 The World Bank Support to the National Education Pact Project (P146831) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 PDO 1. Teacher positions staffed through the revised teacher entrance exam. Outcome Indicators PDO 2. Positions to study teaching degrees at ISFODOSU filled through revised entry exam. 1. Design and implementation of ISFODOSU revised entrance examination. 2. Number of positions to study teaching degrees at ISFODOSU filled through revised entry exam. 3. Implementation of ISFODOSU secondary-level teaching degrees. 4. ISFODOSU has started the implementation of its own teaching career. Intermediate Results Indicators 5. Design and adoption of professional standards for teachers. 6. Design and implementation of competitive entrance exam for teachers. 7. Design and adoption of teaching career proposal, and implementation of teacher evaluation system. 8. Design and adjustment of INAFOCAM in-service training programs. 9. Number of teacher positions staffed through the teacher entrance exam. Page 43 of 69 The World Bank Support to the National Education Pact Project (P146831) 1. More selective system to better recruit teachers for the profession based on standards. Key Outputs by Component 2. System to better select students for teaching degrees. (linked to the achievement of the Objective/Outcome 1) 3. Improved teacher training. 4. More information available regarding teachers. Objective/Outcome 2 PDO 3. Student learning outcomes for the first cycle of primary Outcome Indicators education analyzed and disseminated. 1. Student learning outcomes for the first cycle of primary education analyzed (including by gender) and disseminated. 2. Application and dissemination of international student Intermediate Results Indicators assessments 3. System for learning assessment at the primary level (utility of learning assessment) Key Outputs by Component 1. More information on student learning outcomes is available. (linked to the achievement of the Objective/Outcome 2) 2. Dissemination of assessment results. Objective/Outcome 3 PDO 4. Percentage of QEC care centers that have been evaluated Outcome Indicators against quality standards and their results analyzed. 1. Capacity strengthening of INAIPI and assessment of QEC Intermediate Results Indicators centers. Key Outputs by Component 1. System has clear quality standards for ECE centers and (linked to the achievement of the Objective/Outcome 2) instruments for assessment are available. Page 44 of 69 The World Bank Support to the National Education Pact Project (P146831) 2. System for quality assurance is designed, functioning, and communicated. Objective/Outcome 4 PDO 5. Increasing percentage of fully functional school Outcome Indicators management committees (Juntas de Centro) implementing their school development plans. 1. Number of School Management Committees (SMCs) receiving funds directly. Intermediate Results Indicators 2. Implementation of revised SMC guidelines. 3. Beneficiary feedback included at different level. 4. Direct Project beneficiaries. 1. Communication campaigns are in place. 2. Trained SMCS. Key Outputs by Component 3. More fully functional SMCs. (linked to the achievement of the Objective/Outcome 2) 4. Clear regulation for schools in terms of process and roles of different actors. Page 45 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Juan Baron Task Team Leader(s) Zoila Catherine Abreu Rojas Procurement Specialist(s) Maritza A. Rodriguez De Pichardo Financial Management Specialist Uriel Kejsefman Team Member Jagannath S. Griffiths Palma Team Member Anna Popova Team Member Ramiro Ignacio Jauregui-Zabalaga Counsel Antonella Novali Team Member Peter Anthony Holland Team Member Carmen Veronica Del Rosario Silva Villalobos Team Member Santiago Scialabba Team Member Snjezana Plevko Program Manager Tatiana Cristina O. de Abreu Souza Team Member Victor Manuel Ordonez Conde Team Member Francis V. Fragano Social Specialist Supervision/ICR Katia Marina Herrera Sosa Task Team Leader(s) Maria Alejandra Scheker Lora Procurement Specialist(s) Lucas Carrer Financial Management Specialist Page 46 of 69 The World Bank Support to the National Education Pact Project (P146831) Paola Patricia Polanco Santos Team Member Indiana Taylor Team Member Carmen Amaro Team Member Monica Yanez Pagans Team Member Maria Pia Cravero Counsel Antonella Novali Team Member Amira Nikolas Team Member Juan Baron Team Member Asli Gurkan Environmental Specialist Juan Diego Alonso Team Member Jose C. Janeiro Team Member Francis V. Fragano Social Specialist Emanuela Di Gropello Program Manager B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY14 46.470 247,552.46 FY15 40.302 189,709.09 FY16 21.750 152,423.04 FY17 0 -1,637.43 Total 108.52 588,047.16 Supervision/ICR FY17 6.217 79,795.74 FY18 15.801 201,683.92 FY19 16.705 321,818.15 FY20 13.231 188,193.53 Page 47 of 69 The World Bank Support to the National Education Pact Project (P146831) FY21 18.499 181,397.32 FY22 20.303 217,593.16 FY23 4.786 91,740.66 FY24 .150 30,403.74 Total 95.69 1,312,626.22 Page 48 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Improving capacity to recruit and train primary 25.375 22.875 90 and secondary school teachers Improving capacity to assess student learning in 13.00 13.00 100 primary and secondary education Improving capacity to evaluate the quality of service provided by public 5.00 5.00 100 early childhood development centers Improving capacity to enhance the process of 6.50 6.50 100 decentralized school management Front-end-fee 0.125 0.125 100 Total 50.00 47.50 95 Page 49 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 4. EFFICIENCY ANALYSIS 85. The net effect of the program at the individual level is calculated as the additional benefit that a representative child obtains as a result of the program . This effect is estimated from a present discounted value (PDV) calculation. This approach estimates the stream of benefits and costs of schooling over a lifetime in the labor market with and without the program. 86. Data for this analysis are obtained mainly from the 2011 and 2020 Encuesta Nacional de Fuerza de Trabajo and the 2018 Encuesta Nacional de Hogares de Propósitos Múltiples household surveys of the labor force conducted by the National Statistical Office (Oficina Nacional de Estadística, ONE). This representative survey accounts for the population of the Dominican Republic of all ages. 87. Estimation of expected economic benefits: 88. The private benefits (returns to schooling) are measured following the standard literature on cost-benefit analysis for investments in education and by calculating the earnings over the course of the working life . Age-earnings profiles are built based on the education levels of the Dominican Republic, namely, (i) less than primary school completion (0–6th grades completed), (ii) primary school completion (7th–11th grades completed), and (iii) secondary school completion (12th grade and above completed). 89. These age-earnings profiles are constructed separately for four population groups that differ significantly in their education and labor market experiences , namely (i) men in urban areas, (ii) men in rural areas, (iii) women in urban areas and, (iv) women in rural areas. These are enriched by incorporating (i) the probability of employment in three types of employment (wage employment, self-employment, and unpaid employment), (ii) shifts in the income distribution given the likely changes in productivity and type of employment over time, (iii) changes in the probability of staying alive over the individual's working life, and (iv) the increasing urbanization of the population over time. 90. Formally, the PDV of the benefits of schooling that a representative child accumulates between the ages of 18–64 (according to the working-age international standard and the expected time to begin receiving the returns of schooling) is given by: 91. Where: - is the probability of a child aged a belonging to a population group p. These probabilities are non-stationary as the population becomes more urbanized over time. Though not explicitly incorporated in equation (1), is estimated as a function of base , which is estimated from 2018 data. The forecasted probability is based on the annualized rate of urbanization over the 2011–2018 period. Here, we assume a rate of urbanization that is constant and equal to the rate of urbanization over this period. - is the probability of a child from population group p attaining completion level l by age 18, the age at which the child is expected to end the schooling stage and start receiving expected returns in the labor market. - denotes the rate at which the expected economic benefits are discounted. It is set at 1 percent, which is a conservative value taking into account the interest rate (adjusted for inflation) of the last couple of years (see table A4.1). - denotes the probability that a child aged a, from population group p, is alive at age a. This probability comes from our own calculations and are based on data from Estimaciones y proyecciones de población 1950–2100 (Population Forecasts and Projections 1950–2100) and on a methodology given by the United Nations (2002) Page 50 of 69 The World Bank Support to the National Education Pact Project (P146831) (see table A4.1). The probability of a child aged 5 surviving to the age of 18 is assumed to be one. Given the unavailability of relevant data, it is assumed that the probability for rural and urban areas is identical. - denotes the probability of an individual aged a, from population group p, and with completion level l, has k type of employment. These profiles can change over time along with the structure of the labor market. However, this latter structural change is not accounted for in the probability calculations. - denotes the average real annual earnings received by an individual aged a, from population group p, with education level l, and employment type k. Given that this data is missing for unpaid workers, we artificially built age and education level earnings profiles by decreasing the corresponding profiles of self-employed workers by 75 percent. - denotes the annualized growth rate of real annual earnings in k type of employment with education level l. The predicted growth rate for self-employed workers is also applied to the estimated earnings of unpaid workers (outlined in the previous point). Table A4.1 Life Statistics for the Dominican Republic Probability of living Age Adult-Age specific Probability of dying at Probability of living Age from age 15 to age group death rate age x at age x x Men Women x Men Women Men Women Men Women (1) (2) (3) (4) (5) (6) (7) (8) (9) 15-19 0.0006 0.0016 15 0.003 0.008 0.997 0.992 1.000 1.000 20-24 0.0011 0.0033 20 0.005 0.017 0.995 0.983 0.997 0.992 25-29 0.0015 0.0039 25 0.007 0.019 0.993 0.981 0.992 0.976 30-34 0.0019 0.0038 30 0.010 0.019 0.990 0.981 0.984 0.956 35-39 0.0027 0.0047 35 0.014 0.023 0.986 0.977 0.975 0.937 40-44 0.0036 0.0059 40 0.018 0.030 0.982 0.970 0.961 0.914 45-49 0.0051 0.0086 45 0.025 0.043 0.975 0.957 0.943 0.884 50-54 0.0072 0.0127 50 0.036 0.063 0.964 0.937 0.918 0.841 55-59 0.0102 0.0175 55 0.051 0.087 0.949 0.913 0.882 0.778 60-64 0.0151 0.0253 60 0.076 0.126 0.924 0.874 0.831 0.691 Source: Columns 1-7 are taken from “Estimaciones y proyecciones de población 1950–2100” (2019), that are based on 2010 census data. Columns 8 and 9 are the author’s calculations following the methodology provided in United Nations (2002). Estimation of expected economic costs: 92. Estimating the expected economic costs entails predicting the direct and indirect cost of schooling at each education-completion level. These costs include (i) the household's school-related expenses, (ii) the government's spending on the education of each child, and (iii) the opportunity cost of schooling, i.e., the child's forgone earnings. 93. The PDV of the expected economic costs that a representative five-year-old child incurs from going to school between the ages of 5 and 17 is given by: Page 51 of 69 The World Bank Support to the National Education Pact Project (P146831) = ∑ [∑ ( + ) + ∑( + )− ] ( + )− = = = + ∑ [ ∑ ( + ) + ∑( + )− ] ( + )− = = = 94. Where: - The first and second terms are the expected costs of primary and secondary school completion, respectively. Based on the organization of the educational system in the Dominican Republic, the expected length of primary school completion is eight years (basic education), and four years for secondary-level completion. We assume that the costs of education for those who have not completed primary education is zero, given that the majority of those aged 18 and above without complete basic education have zero years of education. - , are the probabilities that a child aged 18, from population group p, completes primary and secondary school, respectively. - , denote the average real annual expenditure per student of the household (private) and the government (public) for a child aged a. Expenditure per student is assumed to be identical for all population given the data availability. Per student expenditure in 2020 is taken from government statistics. - denotes the probability that a child aged a, from population group p, works in k type of employment. These probabilities are calculated for children aged ten and older, given the available data. - denotes the average real annual earnings of a child aged a, from population group p, working in k type of employment. As earnings for unpaid workers are unavailable, these earnings are calculated by decreasing the age-earnings profile of self-employed child workers by 50 percent. - denotes the annual growth rate of real annual earnings in k type of employment. The rate is estimated over the 2005–2011 period and is assumed to be identical to all population groups and education levels. The predicted growth rate for self-employment is applied to the constructed earnings of unpaid employment. The expected impact of the project 95. As investment in human capital is at the heart of the 2010 –2013 National Development Strategy, the project would support the government in implementing key priority policies that would improve the management of its education system. These policies would be specifically oriented to (i) improving teacher policies, (ii) developing a comprehensive evaluation system, (iii) strengthening the quality assurance mechanisms of early childhood development services, and (iv) enhancing the institutional capacity of MINERD. 96. These policies are expected to have a direct impact on the quality of education that would improve labor market outcomes at the individual level in the medium to long term. We expect to see this impact through two channels in the life-earnings profiles: (i) school completion levels and (ii) earnings growth rates. The school completion levels would determine the lifetime earnings and the type of work in which individuals are employed. To see this, the program must change the average probability of completing each education level. By approximating the labor productivity with the labor earnings, there is a direct method to evaluate the effect of education quality on productivity. There is an expectation that each additional level of schooling produces an increase in labor productivity so that the change in earnings captures this effect. 97. Table A4.2 shows the forecasted school completion rates with and without the program. To guide our forecast to future school completion rates, we study the change in the distribution over 2011 to 2018. As table A4.4 shows, the Page 52 of 69 The World Bank Support to the National Education Pact Project (P146831) distribution of school completion levels has changed positively, as all population groups exhibit an increase in secondary school completion and a decrease in the proportion of children who only complete primary school. However, the rate of change differs among the population groups. Females in urban areas present the lowest change in school completion rates. 98. For the purpose of forecasting, we focus on the change in the probability of at least completing secondary school. To set the forecasted probability in the absence of the program, we assume that the annual growth of this probability remains the same in future years as it has been during 2005-2011 years (the trend identified in the PAD) for the following population groups: (i) men in rural areas and (ii) women in urban areas. 99. We take this forecasted annualized growth to calculate the probability that a child currently aged 5 completes secondary school when the child reaches the age of 17 . The full distribution of school completion at ages 17 to 18 is thus constructed by taking the residual probability and dividing it between the four remaining education levels in proportion to the relative participation they had in 2018 (column 7, table A4.2). 100. To adjust the predicted education level under the program, we estimate that the program will increase the annual growth rate of the probability of completing secondary school by 1 percent point . The full completion profile is then constructed as previously described (columns 8 and 9, table A4.2). 101. The second effect that we can expect from the program is related to the growth in labor productivity produced by the increase in the quality of education across each school level . We adjust the labor earnings growth rates to approximate this effect, as previously mentioned. Table A4.3 shows the average annual labor earnings in 2015 and 2020 for each population group and for two types of employment (wage and self-employment) and education levels. 102. To estimate the earnings growth without the program, we assume that the annual growth rate of earnings for these two population-employment groups will be in the future like it was in the recent past pre-project implementation (2005–2011). 103. The forecasted labor earnings growth rate with the program is constructed by adding 0.015 percent points to the forecasted annual growth rate in the absence of the program (column 7, table A4.3). 104. We calculate costs with and without the program using the predicted population group-specific completion probabilities presented in table A4.2. The increased completion of secondary school is expected as a result of both incremental spending in education as well as of the expected effectiveness and efficiency of all expenditures in public education. The activities covered by the program, as briefly mentioned above, are expected to add RD$6,585 (in 2020 RD$) to the private and public expenditure per student. This additional cost is included in the PDV of the expected economic costs of the program. Calculation of the individual economic gains 105. The individual economic gains induced by the existence of the program are obtained by estimating the discounted values of the increased costs and the benefits according to the assumptions previously outlined. These changes are presented in table A.4. The program is expected to yield a PDV of RD$105,767 per child in increased benefits (in 2020 RD$). 106. Table A4.4 also presents the results of the sensitivity analysis, where we vary some parameters of the model to verify how gains change. The scenarios considered are: (i) reducing the expected adult labor earnings growth rate gains of the program from 0.015 to 0.01 percent points, (ii) having a reduced program effect on secondary school completion of 0.5 percentage (see table A4.2 for the original), and (iii) combining scenarios (i) and (ii). As expected, Page 53 of 69 The World Bank Support to the National Education Pact Project (P146831) under these three scenarios, the expected economic gains of the project are smaller than the base scenario, but the result remains positive in all three. Table A4.4 also compares the updated results with those forecasted in the PAD (in 2020 RD$). Page 54 of 69 The World Bank Support to the National Education Pact Project (P146831) Table A4.2: Predicted school completion probability profiles for children aged 17 to 18 years, with and without the program Forecasted Forecasted Forecasted Forecasted annualized probability probability Probability Probability Annualized annualized % of of of of % Change % % Population Completion level growth completing completing completing completing 2011- change, growth group (x) rate level x level x level x in level x in 2018 2011- rate without without with 2011 2018 2018 with the the the the program program program program (1) (2) (3) (4) (5) (6) (7) (8) (9) Male, urban Less than primary 19.07 0.06 -99.68 -14.24 0.03 0.02 Primary 57.93 21.14 -63.51 -9.07 11.27 7.79 At least secondary 23.00 78.80 242.62 34.66 3.00 88.69 4.00 92.19 Male, rural Less than primary 33.27 0.13 -99.62 -14.23 0.12 0.11 Primary 56.71 32.36 -42.94 -6.13 29.49 26.68 At least secondary 10.03 67.64 574.41 82.06 1.00 70.39 2.00 73.22 Female, urban Less than primary 11.12 0.07 -99.38 -14.20 0.05 0.03 Primary 57.27 11.98 -79.08 -11.30 8.36 4.70 At least secondary 31.61 88.02 178.45 25.49 1.00 91.59 2.00 95.28 Female, rural Less than primary 10.03 0.16 -98.44 -14.06 0.02 0.00 Primary 65.69 15.83 -75.90 -10.84 1.52 0.00 At least secondary 24.29 84.17 246.52 35.22 4.00 98.47 5.00 100.00 Source: Encuesta Nacional de Fuerza de trabajo 2011, Encuesta Nacional de Hogares de Propósitos Múltiples (ENHOGAR) 2018 (ONE), and our own calculations. Column 8 is the annualized growth rate without the program identified in the PAD. Page 55 of 69 The World Bank Support to the National Education Pact Project (P146831) Table A4.3. Predicted labor earning growth rates Forecasted annualized Forecasted Annual Annual labor Growth Annual annualized Population Type of labor labor income Education level 2015- growth labor group employment earning in earning in growth 2020 2015-2020 income 2015 2020 with growth the program (1) (2) (3) (4) (5) (6) (7) (8) Less than Male, urban Wage employed primary 119941.39 141494.64 17.97 3.36 3.36 3.51 Primary 146672.17 160656.84 9.53 1.84 1.84 1.99 At least secondary 246627.51 257927.40 4.58 0.90 0.90 1.05 Less than Self employed primary 168801.50 218202.48 29.27 5.27 5.27 5.42 Primary 222066.34 248960.52 12.11 2.31 2.31 2.46 At least secondary 326229.08 366745.32 12.42 2.37 2.37 2.52 Less than Female, urban Wage employed primary 72316.70 100087.16 38.40 6.72 6.72 6.87 Primary 119941.39 117390.60 -2.13 -0.43 0.10 0.25 At least secondary 119941.39 237581.52 98.08 14.65 14.65 14.80 Less than Self employed primary 113461.33 110953.22 -2.21 -0.45 0.10 0.25 Page 56 of 69 The World Bank Support to the National Education Pact Project (P146831) Primary 136788.52 161114.88 17.78 3.33 3.33 3.48 At least secondary 199937.25 198107.52 -0.92 -0.18 0.10 0.25 Less than Male, rural Wage employed primary 100807.31 141747.60 40.61 7.05 7.05 7.20 Primary 136841.42 153675.24 12.30 2.35 2.35 2.50 At least secondary 180365.70 222411.36 23.31 4.28 4.28 4.43 Less than Self employed primary 117078.92 186682.32 59.45 9.78 9.78 9.93 Primary 168570.54 240558.36 42.70 7.37 7.37 7.52 At least secondary 193966.62 205170.24 5.78 1.13 1.13 1.28 Less than Female, rural Wage employed primary 61257.20 103500.79 68.96 11.06 11.06 11.21 Primary 68587.52 98732.32 43.95 7.56 7.56 7.71 At least secondary 138913.17 189178.32 36.18 6.37 6.37 6.52 Less than Self employed primary 99431.35 99607.01 0.18 0.04 0.04 0.19 Primary 87311.04 126132.00 44.46 7.63 7.63 7.78 At least secondary 92276.16 126735.96 37.34 6.55 6.55 6.70 Notes: Mean earnings are calculated in 2020 DOP for the population aged 18-64 years. Source: Encuesta Nacional de Fuerza de trabajo 2015 and 2020 (ONE) and own calculations. Page 57 of 69 The World Bank Support to the National Education Pact Project (P146831) Table A4.4: PDV estimates of the expected individual economic gains from the program PDV of expected PDV of expected PDV of expected Case incremental economic incremental economic incremental economic benefits (in 2020 DOP) costs (in 2020 DOP) benefits (in 2020 DOP) PAD Current PAD Current PAD Current Base 225542 274152 130747 168385 94795 105767 Reduced adult labor earnings growth rates 174993 272889 130747 168385 44247 104504 Reduced program effect on the probability of secondary school completion 184063 237351 128696 168295 55367 69056 Reduced adult labor earnings growth rates + probability of secondary school completion 134253 236094 128696 168295 5557 67799 Source: own calculations. Page 58 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 5. BORROWER’S IMPLEMENTATION AND COMPLETION REPORT 107. The Educational Pact Support project is the result of a loan agreement between the International Bank for Reconstruction and Development (IBRD, or World Bank) and the Dominican Republic, for an amount of US$50 million. 100 percent was financed with a loan from the World Bank (the Bank), while the MINERD and the Ministry of Finance assumed the commitment to make available the equivalent of US$3.63 million dollars to provide technical assistance to strengthen the implementing entities, to achieve the planned objectives. The Loan Contract is 8540-DO, signed between the Bank and the Government of the Dominican Republic, dated October 8, 2015, approved by Res. No. 671-16 of October 27, 2016, with an effectiveness date of December 9, 2016, and closing date June 30, 2023. 108. The objective of the Project is to improve the education system, particularly in the areas of teaching policies, evaluation, quality of early childhood centers, and school management. The Project supported the application of the National Education Pact by emphasizing: 1) the development of a comprehensive evaluation system; 2) strengthening quality assurance of early childhood development services; and 3) increasing the institutional capacity of the Ministry of Education. 109. The Project consists of the following components: 110. Component 1: Improving capacity to recruit and train primary and secondary school teachers. The objective of this component is to comprehensively review and adjust MINERD's recruitment and training activities for primary and secondary school teachers based on professional standards for teachers. 111. Component 2: Improving the capacity to assess student learning in primary and secondary education. This component supported the strengthening of the comprehensive evaluation system, through: (a) adjustment and implementation of the national learning strategy for the pre-university system. (b) participation in and dissemination of international evaluations. (c) the technical improvement of national tests for students. (d) the design and implementation of a system for disseminating evaluation results, with special emphasis on their use (i.e., through the optimized use of information, improving teaching practices and methods, determining institutional improvement plans, adaptation and improvement of training programs, and the content of educational materials); and the development and implementation of the national diagnostic test in the first cycle of primary school. 112. Component 3: Improving the capacity to evaluate the quality of services provided by public early childhood development centers. This component supported the institutional strengthening of INAIPI through: (a) Development of a strategic plan for INAIPI and quality standards for early childhood centers and their staff, and (b) development and implementation of an information and communication strategy, to socialize quality standards for public early childhood centers, increase public understanding of the Quisqueya Empieza Contigo programs and improve parenting practices. 113. Component 4: Improving the capacity to strengthen the decentralization process of public-school management. This component supported the decentralization of MINERD systems, through: (a) the increase in the establishment of School Management Committees in public schools. Page 59 of 69 The World Bank Support to the National Education Pact Project (P146831) (b) the review of regulations and processes to (i) increase efficiency in the establishment of SMCs and (ii) establish agile account opening processes. (c) implement a functional financial management system for the SMCs. (d) improve the training of SMCs in the design and implementation of center improvement plans and annual operating plans. (e) improve financial management and procurement practices at the school level. (f) review the follow-up, monitoring, support, and evaluation processes for improvement. Achievements by component DLI49 per component 114. The project supported the National Educational Pact, through four components with defined DLIs to monitor its achievement. The OCI accompanied and monitored MINERD entities in compliance with their respective DLI's. It is these instances that incorporated into their respective POA, the activities necessary to achieve the DLI. The Program achieved 19 of the 20 agreed indicators. 115. Component 1. Improving capacity to recruit and train primary and secondary school teachers. • DLI 1. ISFODOSU administered the standardized and revised admission exam, that is, the test that applies for admission of students to its programs. The exam was first revised in 2014, then it was implemented with the Project, and it began to be applied in 2021 to admit students to its programs. • DLI 2. MINERD carried out the competitive examination, that is, the entrance exam for all candidates requesting employment as teachers. On June 30, 2014, MINERD applied the entrance exam to hire teachers. • DLI 5. MINERD adopted the professional standards (teacher performance standards, designed by the Vice Ministry for Certification within MINERD that teachers must know and apply) for teachers and their socialization has begun. • DLI 6. MINERD aligns the administrative entrance exam for all candidates applying for employment as teachers in MINERD with the professional standards for teachers. In September 2022, MINERD applies the Opposition Contest aligned with Professional Teaching Standards. The government requested the disbursement corresponding to compliance with DLI 6: MINERD applies Competition Aligned with Teaching Professional Standards for a value of US$ 2.5 million. • DLI 8. ISFODOSU offers two new degrees for the training of teachers at the secondary level. As of 2017, both titles were already offered. • DLI 9. This indicator Design of the professional career and evaluation of teachers (based on standards) is adopted by MINERD, was cancelled, therefore, no disbursement request will be made for US$2,500,000.00. • DLI 12. INAFOCAM and ISFODOSU prepare their annual financial report using the Integrated Financial Management System (SIGEF). Reached in September 2022. • DLI 13. 20% of public-school teachers are evaluated using the new teacher and professional evaluation system. DLI 13 was reached. • DLI 17. ISFODOSU has begun implementing its own professional development plan for its educators. This was implemented since Dec 2017. The ISFODOSU Teaching Career refers to the career development and 49 Disbursement Linked Indicators. Page 60 of 69 The World Bank Support to the National Education Pact Project (P146831) evaluation system of ISFODOSU educators (teachers of teachers) focused on standards and competencies with clear incentives to stimulate the professional development of educators. • DLI 18. INAFOCAM makes modifications to continuing training programs according to the results of evaluations and/or impact evaluations. This indicator was reached in 2022. 116. Component 2. Improving the capacity to assess school learning in primary and secondary education. • DLI 3. MINERD applies PISA 2015.Workshop on the use and dissemination of an international evaluation. (Reached). • DLI 7. Implementation of the Student Assessment Strategy has begun. Indicator was met in 2016, the OIC requested the disbursement of US$ 2,375 million. • DLI 10. MINERD pilot-tested the dissemination component of the student assessment and revises the dissemination component based on the results of the pilot. Indicator was met in 2017. • DLI 14. MINERD administers a student evaluation for the first cycle of primary education. Indicator was met in 2018. • DLI 19. MINERD Evaluation Office staff have received training in sampling, element design, database management and statistical software. MINERD National Evaluation Office staff received training in sampling, item design, database management, and statistical software. 117. Component 3. Improving the capacity to evaluate the quality of services provided by Public Early Childhood Development Centers. • DLI 15. The pilot evaluation of the Early Childhood Development Centers has been carried out. In 2018, the pilot test was carried out for early childhood development centers. • DLI 20. 50% of Early Childhood Development Centers are operational and are evaluated according to quality standards. 50 percent of Early Childhood Development Centers are operational and evaluated according to quality standards (Reached in 2022). Regarding DLI 20: 50 percent of the Early Childhood Development Centers are operational and are evaluated according to quality standards, the remaining 40 percent were completed for US$1.0 Million. 118. Component 4. Improving the capacity to enhance the decentralization process of public-school management. • DLI 4. 250 additional school boards receive transfers. Completed in 2015. • DLI 11. MINERD adopts new functional spending guidelines for SMCs. Completed in 2019. • DLI 16. 1,750 SMCs are fully functional. Completed in 2023, verify information provided by Decentralization Resolution No. 02-2019. We sent the Bank the Terms of Reference for the contracting of the Project audit; the Bank team approved this document and the justification for the direct contracting of the audit firm. In August 2022, the World Bank approved the extension of the Project closing date until June 30, 2023, with the purpose of meeting the Project's pending objectives and indicators. Page 61 of 69 The World Bank Support to the National Education Pact Project (P146831) Table 1. QUANTITATIVE INDICATORS OF DECENTRALIZED COMMITTEES AS OF June 30, 2023 General Data on Committees Number of schools suitable for Decentralized Committees 7,687 Number of Formed Committees 7,277 Number of Legalized Committees 7,276 Number of Committees to Restructure 2,411 Number of Committees with RNC 5,002 Number of Committees with Bank Accounts 2,749 Number of Accounts in SIGEF 2,315 Number of decentralized Committees receiving direct transfers 1.274 119. The school committee is a decentralized management and participation body that must be created in each school. It is responsible for establishing relationships between the school and the community, whose function is to ensure that the educational process runs appropriately (Herrera, 2021). 120. In the Dominican Republic, educational centers are grouped into 18 educational regions and 122 school districts. 304 Center Boards were probabilistically selected from a database of 2,524 provided by MINERD, which corresponded to centers that had legally constituted boards and bank accounts and that had received resources. of MINERD. 121. INAFOCAM adjusts in-service training programs based on results of evaluations and/or impact evaluations. 122. We have provided technical assistance to the processes of preparing the Ministry's Induction Plan, to the 24,000 teachers selected in the last competitive examination. The Vice Ministry team reported that they have continued working based on the Bank's guide and that they have included the comments made to the induction plan presented a few months ago. The Bank offered its support in the systematization of the induction experience and in the presentation of international initiatives on classroom observation tools. Factors that affected implementation and results Positive factors ▪ In August 2022, the World Bank approved the extension of the Project closing date until June 30, 2023. So that the pending objectives and indicators could be met. ▪ The World Bank hired a consultant to support the OCI-MINERD in the validation of the DLI 16 indicator, which was achieved, and the evidence was sent. ▪ The involvement of both the authorities and the key actors of the educational system in each process was decisive in the success of the Project. ▪ Having a project tracking leader helped a lot with meeting the deliverables on schedule. Page 62 of 69 The World Bank Support to the National Education Pact Project (P146831) Negative factors ▪ Design of the professional career and evaluation of teachers / delays in the implementation by MINERD of the teaching career and the evaluation system. ▪ Finding qualified personnel in Project development management was a challenge that forced us to resort to different strategies to achieve the indicators. ▪ The COVID-19 pandemic forced the work schedule to be altered. Lessons Learned: 123. The leadership and political decision of the authorities of the Ministry of Education (MINERD) and the dedication, capacity and mystique of the entities have made it possible to achieve the indicators. 124. Intervention models, focused on raising outcome indicators in student learning in primary and secondary education, must be focused on some fundamental actions that guarantee this achievement. The idea of more actions, better results is not true. An educational model must limit its actions to those that generate improvements in learning, it must not divert physical and economic efforts. 125. The need for training and support (classroom training) is evident to achieve better learning results. The openness of teachers to these two processes is also observed. Training and support must be systematic and organized with a program that, based on diagnostic and formative evaluation results, privileges the development of teaching competencies. 126. Educational resources do not have to be the same for everyone, but they must be defined in each case to know what quality is expected. All social policies work on social groups that have unequal situations. This happens with educational policy. Not all children have the same conditions to receive education. Therefore, educational policy, in this case of attention to the first four grades, cannot work as if all children were equal, because reality is not like that. Consequently, educational policy interventions must make explicit, in each annual operational planning cycle, what their impact is in this environment of inequality. The aspiration is that the interventions act preferentially on those furthest behind, seeking to reduce inequalities; but it is possible that, in the short term, the initial interventions will not have this criterion. However, in the medium and long term, this criterion for action on the environment of inequality must be defined. Page 63 of 69 The World Bank Support to the National Education Pact Project (P146831) Table 2. PERFORMANCE EVALUATION World Bank Performance Assessment Overall Evaluation: Highly Satisfactory Assessment Justification World Bank Team Highly The World Bank accompanied MINERD before, during and after satisfactory the execution of the project with high quality standards, providing assistance that strengthened the implementing entities, thus ensuring the achievement of the declared objectives. The experience of the managers and specialists who were in charge of accompanying us was decisive in achieving compliance with the indicators and the problems that arose for the implementation of the Project. The entire World Bank team was always attentive to the execution of the Project and its efficient supervision of it allowed us to achieve the formulated indicators. MINERD performance evaluation Overall Evaluation: Highly Satisfactory Assessment Justification International Highly They managed to ensure the execution of activities towards Cooperation Office satisfactory compliance with the indicators of each instance, adding value Team with their experience, commitment, and responsibility in their functions. Ministry of Highly The involvement of the Ministry of Education in this Project Education satisfactory was without a doubt the main factor in its success. The entities were involved and promoted the achievement of the objectives of this Project. The execution of the processes was good within the limitations that were presented, which were overcome thanks to the delivery of these teams that provided adequate follow-up and proposed timely solutions to resolve obstacles in the process. Page 64 of 69 The World Bank Support to the National Education Pact Project (P146831) Eligible Expenses 127. Table 3 details the evidence of the amounts requested to comply with disbursement indicators and that have been executed under the budget items established in Loan Contract 8540, these were presented to the bank in the Eligible Expenditure Reports (EEPs), as it's shown in the following: Table 3. ELIGIBLE EXPENSES REPORT FOR THE PERIOD 11/1/14 to 9/30/20. REPUBLICA DOMINICANA: Proyecto de Apoyo al Pacto Nacional por la Educación, Préstamo IBDR No.8540-DO CERTIFICADO DE GASTOS (SOE) PARA LA CATEGORIA 1 - Informe de Gastos Elegibles (EEPs) Periodo: 01 de noviembre de 2014 al 30 de septiembre de 2020 Código Monto total de gastos Monto total de gastos Concepto Código Total Pagado por el elegibles para elegibles para Nombre de los EEPs Presupuestario Presupuestario Gobierno (DOP) financiamiento del Banco financiamiento del Banco (Auxiliar) (DOP) (USD) MINERD Transfers to SMC 491 15,674,628,285.09 1,561,908,955.98 32,419,752.27 ISFODOSU Staff Remuneration 111 1,997,897,340.21 729,798,154.42 13,335,485.94 ISFODOSU Vehicle fuel 3711 Y 3712 65,971,241.72 5,201,670.94 105,350.88 ISFODOSU Vehicle lubricants 3716 6,339,938.35 4,085.69 84.92 INAFOCAM Transfers to SMC 111 639,630,230.56 51,466,465.23 1,038,023.16 IDEICE Transfers to SMC 111 233,102,548.90 21,251,923.71 427,616.11 INABIE Transfers to SMC 111 348,720,631.89 - - INEIFI Transfers to SMC 111 191,535,956.42 2,342,381.52 48,686.72 TOTAL 19,157,826,173.14 2,371,973,637.49 47,375,000.00 128. Regarding the state of accumulated investments: Regarding DLI 9, it is cancelled, therefore, no disbursement request will be made for the amount of US$2,500,000.00. Borrower’s comments to the ICR 129. Written comments were received from the Borrower by e-mail on December 4, 2023, including: • The Borrower’s agreement with the ratings for relevance (high), efficacy (high), efficiency (substantial), and the overall outcome (satisfactory). • Additional information on the importance of the teacher standards within the framework of the proposed teaching career system. The standards provide criteria for the definition and development of profiles for teacher training, as well as the design of teacher entrance exam, performance evaluation and certification. • Regarding objective 4, the borrower sent additional information on how the audit determined that school expenses were being spent according to the POA. Samples were taken from schools from different regions, which were then visited in order to review the disbursements made in 2023. The review was carried out separately by external auditors and by staff from the OCI financial team from MINERD. The review confirmed that signatures coincided with the approved ones, that quotes were in accordance with established regulations, they reviewed the original invoices with government tax receipt numbers, checked consistency in dates and Page 65 of 69 The World Bank Support to the National Education Pact Project (P146831) amounts, and reviewed that the expenses were in accordance with the POA. The result of the report was that there was an average of 90 percent compliance. Page 66 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 6. RATINGS FROM ISRs ISR No. Date ISR Overall Project Procurement Financial Monitoring and Archived Implementation management Management Evaluation Progress 1 27-Jan-2016 MS. 2 25-Feb-2016 Project is not yet 3 07-Jun-2016 effective. 4 20-Dec-2016 MS. The Project became effective on Dec 9, 2016. Implementation activities advances in a Satisfactory manner with few delays. 5 12-Aug-2019 MS S S MS S 6 30-Dec-2017 S S S MS S The Project is advancing well, results in some areas are strong and ahead of time (e.g. student assessment, some aspects of teachers' career). Disbursements are strong, (49 percent) with only one year of effectiveness. Some other areas show some delays. 7 21-Jun-2018 S S S MS S 8 18-Sept-2018 S S S MS 9 16-Jan-2019 S S S MS S Start of the COVID-19 Pandemic. 10 20-Nov-2019 MS S S MS S The IP was downgraded to MS given de delays in the effectiveness of the additional financing. 11 16-Jun-2020 MS S S MS S 12 10-Dec-2020 MS S S MS S 13 27-May-2021 MS S S MU S (audit more than 10 months overdue) 14 02-Dec-2021 MS S S MU S 15 21-Jan-2022 S S S MS(↑) S (audit contracted and IFRS uploaded) The “Progress towards achievement of PDO” rating was upgraded from Moderately Satisfactory to Satisfactory because the Project achieved 2 of the 3 outstanding PDOs since the previous ISR. The proposed ISR FM risk rating is upgraded from Moderately Unsatisfactory to Moderately Satisfactory based on the latest actions performed by the PIU. The audit report for the period 2020/2021 was received and satisfactorily reviewed by the Bank. 16 23-Jan-2023 S S S MS S Page 67 of 69 The World Bank Support to the National Education Pact Project (P146831) ANNEX 7. REFERENCES Bendini, M. and Devercelli, A. 2022. Quality Early Learning: Nurturing Children’s Potential. Human Development Perspectives. Washington, D.C. World Bank. Bruns, B., and Luque, J. 2015. Great Teachers: How to Raise Student Learning in Latin America and the Caribbean. Washington, DC. World Bank. Bruns, B., Filmer, D., and Patrinos, H. 2011. Making Schools Work. Washington, DC. World Bank. Clarke, Marguerite. 2012. Measuring Learning: How Effective Student Assessment Systems Can Help Achieve Learning for All. Education Notes. World Bank. Washington DC. Decreto No. 639-03 que Establece el Estatuto del Docente. https://ministeriodeeducacion.gob.do/docs/marco- legal/decretos/decreto-sobre-reglamento-del-estatuto-del-docente-no-639-03-go-no-10225-del-26-de-junio-de- 2003.pdf. Educa. 2015. Formación y carrera docente: profesión que cambia vidas. XIX Congreso Internacional de Educación. Aprendo 2015. Santo Domingo, Dominican Republic. Estándares Profesionales y del Desempeño para la Certificación y Desarrollo de la Carrera Docente. 2014. https://www.ministeriodeeducacion.gob.do/docs/viceministerio-de-acreditacion-y-certificacion-docente/g051- estandares-profesionales-y-del-desempeno-para-la-certificacion-y-desarrollo-de-la-carrera-docentepdf.pdf. Government Ordenanza Number 1-2016. https://www.ministeriodeeducacion.gob.do/docs/direccion-general-de- evaluacion-y-control-de-la-calidad-educativa/uLRy-ordenanza-n01-2016pdf.pdf. IDEICE. 2013. Dominio de los contenidos matemáticos por parte del docente, como factor del éxito o fracaso escolar en el primer ciclo del nivel básico en República Dominicana. Santo Domingo: Instituto Dominicano de Evaluación e Investigación de la Calidad Educativa (IDEICE). IDEICE. Evaluación del Desempeño Docente 2017 en República Dominicana- Informe de Resultados. 2018. https://ideice.gob.do/programas/evaluacion-del-desempeno-docente Jaime Saavedra & Juan Baron. The Teaching Profession: What is the Dominican Republic Doing Right? 2018. Blog. https://blogs.worldbank.org/education/teaching-profession-what-dominican-republic-doing-right. Medir para mejorar: La calidad de las prácticas pedagógicas en la primera infancia. https://blogs.worldbank.org/es/latinamerica/medir-para-mejorar-la-calidad-de-las-practicas-pedagogicas-en-la- primera-infancia. Ministerio de Educación. 2016. Ordenanza No. 1-2016. Norma el sistema de pruebas nacionales y evaluación de los logros de aprendizaje de República Dominicana. República Dominicana. Programa de Promoción de la Reforma Educativa en América Latina y el Caribe. 2010. Informe de Progreso Educativo. Page 68 of 69 The World Bank Support to the National Education Pact Project (P146831) República Dominicana. Regulation for Teacher Training Quality in the Dominican Republic. 2015. https://www.inafocam.edu.do/transparencia/phocadownload/BaseLegal/normativa/Normativa%20para%20laformac ion%20docente%20de%20calidad%20en%20la%20Rep%20Dom%20Dic%209%202015.pdf. Teach-ECE. https://www.worldbank.org/en/topic/teachers/brief/teach-ece-helping-countries-track-and-improve- teaching-quality-in-early-childhood-education. World Bank. 2015. Project Appraisal Document for the Support to the National Education Pact Project. Report No. PAD850. Latin America and the Caribbean Region, Dominican Republic. Page 69 of 69