Document of The World Bank Report No.: PP1793 PROJECT APPRAISAL DOCUMENT FOR SMALL RECIPIENT EXECUTED TRUST FUND GRANT IN THE AMOUNT OF U.S.$2.00 MILLION TO THE ORGANIZATION OF EASTERN CARIBBEAN STATES COMMISSION FOR A SUPPORT TO IMPLEMENTATION OF THE REGIONAL EDUCATION STRATEGY July 6, 2016 Education Global Practice Latin America and the Caribbean Region This document is being made publicly available prior to approval. This does not imply a presumed outcome. This document may be updated following management consideration and the updated document will be made publicly available in accordance with the Bank’s policy on Access to Information. CURRENCY EQUIVALENTS (Exchange Rate Effective July 6, 2016) Currency Unit = East Caribbean Dollars EC$2.71 = US$1.00 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS CLASS Classroom Assessment Scoring System CQS Selection Based on the Consultants’ Qualifications CSEC Caribbean Secondary Education Certificate ECD Early Childhood Development EDMU Education Development Management Unit FM Financial Management GPE Global Partnership for Education GRS Grievance Redress Services ICB International Competitive Bidding IFR Interim Financial Report LCS Least-Cost Selection M&E Monitoring and Evaluation MOE Ministry of Education NCB National Competitive Bidding NPV Net Present Value OECS Organization of Eastern Caribbean States OESS OECS Education Sector Strategy PDO Project Development Objective QCBS Quality- and Cost-Based Selection RPS Regional Partnership Strategy SIP School Improvement Plan UNESCO United Nations Educational, Scientific, and Cultural Organization UNICEF United Nations Children’s Fund USAID U.S. Agency for International Development Vice President: Jorge Familiar Country Director: Sophie Sirtaine Senior Global Practice Director: Claudia Maria Costin Practice Manager: Reema Nayar Task Team Leader(s): Harriet Nannyonjo, Leandro Costa Organization of Eastern Caribbean States Countries Support to Implementation of the Regional Education Strategy TABLE OF CONTENTS Page I. INTRODUCTION AND CONTEXT ................................................................................... 1 A. Country Context ............................................................................................................. 1 B. Sectoral and Institutional Context ................................................................................ 1 C. Higher-Level Objectives to which the Project Contributes ....................................... 5 II. PROJECT DEVELOPMENT OBJECTIVES .................................................................... 6 A. PDO ................................................................................................................................. 6 B. Project Beneficiaries ...................................................................................................... 6 C. PDO-Level Results Indicators ....................................................................................... 6 III. PROJECT DESCRIPTION .................................................................................................. 7 A. Project Components ....................................................................................................... 7 B. Project Financing ........................................................................................................... 9 Project Cost and Financing ...................................................................................................... 9 C. Lessons Learned and Reflected in the Project Design ................................................ 9 IV. IMPLEMENTATION ........................................................................................................... 9 A. Institutional and Implementation Arrangements ....................................................... 9 B. Results Monitoring and Evaluation ............................................................................ 10 C. Sustainability ................................................................................................................ 11 V. KEY RISKS AND MITIGATION MEASURES .............................................................. 11 VI. APPRAISAL SUMMARY .................................................................................................. 12 A. Economic and Financial Analysis ............................................................................... 12 B. Technical ....................................................................................................................... 12 C. Financial Management ................................................................................................ 13 D. Procurement ................................................................................................................. 13 E. Social (including Safeguard Policies) ......................................................................... 13 F. World Bank Grievance Redress ................................................................................. 14 Annex 1: Results Framework and Monitoring ........................................................................ 15 Annex 2: Detailed Project Description ...................................................................................... 36 Annex 3: Implementation Arrangements ................................................................................. 44 Annex 4: Economic Analysis ...................................................................................................... 50 APPRAISAL DATA SHEET OECS Countries Support to Implementation of the Regional Education Strategy (P158836) PROJECT APPRAISAL DOCUMENT LATIN AMERICA AND CARIBBEAN GED04 Report No.: PP1793 Basic Information Project ID EA Category Team Leader(s) P158836 C - Not Required Harriet Nannyonjo, Leandro Oliveira Costa Lending Instrument Fragile and/or Capacity Constraints [ ] Investment Project Financing Financial Intermediaries [ ] Series of Projects [ ] Project Implementation Start Project Implementation End Date Date 01-Aug-2016 30-Aug-2019 Expected Effectiveness Date Expected Closing Date 29-Jul-2016 30-Sep-2019 Joint IFC No Practice Senior Global Country Director Regional Vice President Manager/Manager Practice Director Claudia Maria Reema Nayar Sophie Sirtaine Jorge Familiar Costin . Approval Authority Approval Authority CD Decision i . Borrower: OECS Commission Responsible Agency: OECS Commission, Education Management Unit Contact: Marcellus Albertin Title: Head, Education Development Management Unit Telephone No.: 758 4519716 Email: malbertin@oecs.org . Project Financing Data(in US$, million) Total Project Cost: 2.00 Total Bank Financing: 0.00 Financing Gap: 0.00 . Financing Source Amount Borrower 0.00 Education for All - Fast Track Initiative 2.00 Total 2.00 . Expected Disbursements (in US$) Fiscal Year 0001 0002 0003 0004 Annual 94,839 500,000 1,151,532 253,629 Cumulative 94,839 594,839 1,746,371 2,000,000 . Institutional Data Practice Area (Lead) Education Contributing Practice Areas Cross Cutting Topics [ ] Climate Change [ ] Fragile, Conflict & Violence [ ] Gender [ ] Jobs [ ] Public Private Partnership Sectors / Climate Change Sector (Maximum 5 and total % must equal 100) ii Major Sector Sector % Adaptation Mitigation Co-benefits Co-benefits % % Education Primary education 100 Total 100 I certify that there is no Adaptation and Mitigation Climate Change Co-benefits information applicable to this project. . Themes Theme (Maximum 5 and total % must equal 100) Major theme Theme % Human development Education for all 100 Total 100 . Proposed Development Objective(s) The objectives of the project are to (a) use quality learning standards to support evidence-based teaching and learning at the primary level; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector monitoring and evaluation capacity in support of evidence-based strategic management and decision making, all in the member countries. . Components Component Name Cost (US$) Component 1: Curriculum and Assessment 396,500.00 Component 2: Teacher Professional Development 1,115,000.00 Component 3: Improve School Leadership and 209,000.00 Accountability Component 4: Project implementation, monitoring, and 279,500.00 evaluation . Compliance Policy Does the project depart from the CAS in content or in other Yes [ ] No [ X ] significant respects? . Does the project require any waivers of Bank policies? Yes [ ] No [ X ] iii Have these been approved by Bank management? Yes [ ] No [ X ] Does the project meet the Regional criteria for readiness for Yes [X] No [ ] implementation? . Safeguard Policies Triggered by the Project Yes No Environmental Assessment OP/BP 4.01 X Natural Habitats OP/BP 4.04 X Forests OP/BP 4.36 X Pest Management OP 4.09 X Physical Cultural Resources OP/BP 4.11 X Indigenous Peoples OP/BP 4.10 X Involuntary Resettlement OP/BP 4.12 X Safety of Dams OP/BP 4.37 X Projects on International Waterways OP/BP 7.50 X Projects in Disputed Areas OP/BP 7.60 X . Legal Covenants Name Recurrent Due Date Frequency Institutional and Other X CONTINUOUS Arrangements Description of Covenant For purposes of project implementation oversight and strategic guidance, the Recipient shall maintain the Chief Education Officers and Planners’ Forum, with composition and responsibilities acceptable to the Bank. . Conditions Source Of Fund Name Type Description of Condition Team Composition Bank Staff Name Role Title Specialization Unit Harriet Nannyonjo Team Leader Senior GED04 (ADM Education iv Responsible) Specialist Leandro Oliveira Team Leader Economist GED04 Costa Sonia Cristina Procurement Senior GGO04 Rodrigues Da Fonseca Specialist Procurement Specialist David I Financial Sr Financial GGO22 Management Management Specialist Specialist Gabriela Grinsteins Counsel Counsel LEGLE Kerry Natelege Team Member Financial GGO22 Crawford Management Specialist M. Yaa Pokua Afriyie Safeguards Senior Social GSU04 Oppong Specialist Development Specialist Manju Ghumman Team Member Program LEGLE Assistant Mary A. Dowling Team Member Language GED04 Program Assistant Robert H. Environmental Lead GEN04 Montgomery Specialist Environment Specialist Srivathsan Sridharan Team Member Finance WFALS Analyst Tatiana Cristina O. de Team Member Finance WFALN Abreu Souza Officer Viticia Amanitu Team Member Consultant GED07 Thames Extended Team Name Title Office Phone Location . Locations Country First Administrative Location Planned Actual Comments Division v . vi I. INTRODUCTION AND CONTEXT A. Country Context 1. The small, open economies of the Organization of Eastern Caribbean States (OECS) have been trapped for years in low growth, high debt, and limited fiscal space, exacerbated by a number of external shocks. The impact of the 2008 global financial crisis was severe because tourism, remittances, foreign direct investment, and official development flows decreased sharply; growth rates plummeted; debt and fiscal imbalances increased to unsustainable levels; and labor market conditions deteriorated. In addition, natural disasters have periodically taken a large toll on the region, affecting lives, infrastructure, and economic activity. The OECS’s bleak economic performance has exacerbated social problems—unemployment has been growing at alarming rates, especially among the youth; and poverty, which was relatively high before 2008, has very likely worsened. Data for the Caribbean indicates a similar problem among youth, with youth unemployment standing at over 30 percent for women and 20 percent for men. The OECS recognizes the importance of improvement of the quality of education as part of the solution to improving social and economic development. B. Sectoral and Institutional Context 2. The OECS education system is structured as follows: (a) three years of early education (0–2 years); (b) two years of preschool or early childhood education (3–4 years); (c) 12 years of compulsory education divided into one year of kindergarten (5 years), six years of primary education (6–11 years), and five years of secondary education (12–16 years); and (d) two or more years of postsecondary/tertiary education levels. At the end of the primary school cycle (grade 6), students take the Common Entrance Examination, which is used for placement in secondary schools. At the end of the secondary cycle (Form 5), students sit for a regional examination—Caribbean Secondary Education Certificate (CSEC). 3. The OECS countries have made significant progress in expanding access to basic education in particular. The gross enrolment rate in early childhood education for students aged 3–4 years ranges from 100 in Grenada to 106 in St. Vincent, 79 in Dominica, and 105 in St. Lucia. The gross primary enrolment rate is 113 percent in St. Vincent; 126 percent in Grenada; 109 percent in Dominica; and 103 percent in St. Lucia. Enrolment at secondary level follows a similar pattern as the primary level. Gender parity has been achieved at the primary level, although small differences emerge at the secondary level (118 percent for boys and 116 for girls in Dominica; 96 percent for boys and 97 percent for girls in St. Lucia; with gender parity in Grenada and St. Vincent and the Grenadines). The progress at the secondary level was a result of a policy push to achieve universal secondary education as soon as possible. This progress has also been facilitated by the significant education expenditure as percentage of gross domestic product: 5 percent in Grenada; 7 percent in St. Vincent and the Grenadines; 5 percent in Dominica; and 5 percent in St. Lucia in 2014.1 In 2014, government expenditure on education as percentage of the total public expenditure was close to 20 percent for Dominica, 18 percent in St. 1 OECS Education Statistical Abstract 2013/14. 1 Lucia (increasing to 21 percent), 21 percent in St. Vincent and the Grenadines, and 17 percent in Grenada.2 4. Some challenges remain with regard to access, equity, and quality. Close to 20 percent of children aged 3 to 5 years were not enrolled in early education in 2013. The OECS recognizes the importance of investing in early childhood development (ECD) and is making efforts to address this challenge. Low learning achievement is the major challenge facing the education sector in the OECS. The 2010 grade 4 Literacy and Mathematics tests showed that about 50 percent of grade 4 students had not achieved mastery at their grade level. At the secondary level, the percentage of students passing the CSEC was 37 percent and below for all the states. The percentage of students who pass both English and Mathematics at CSEC is on average 23 percent across the states. Moreover, the Caribbean Examinations Council examiner’s reports indicate poor student performance on test items that require critical thinking, analysis, or communication. 5. There are several underlying causes of low education quality in the OECS, including the following: First, lack of clear learning standards to facilitate formative classroom assessment and student-centered learning. The absence of clear learning standards to guide systematic formative and summative classroom assessment leads to failure to address learning deficiencies before transitioning to the next grade or exiting the system. Students are automatically promoted to the next grade even if they have not mastered the relevant competencies. Many of these children finish school without the necessary skills and competencies to enable them to secure good jobs, increase productivity, and lift themselves out of poverty. The best-performing countries have established learning standards and have developed assessment systems to monitor their achievement (Barber and Mourshed 2007;3 Mourshed, Chijioke, and Barber 20104). Placing students’ outcomes as the main goal of educational policies requires that the learning outcomes to be achieved at every grade level are clearly specified and that the level of expected performance is clear. Second, limited continuous professional development opportunities to improve teaching, or to accompany changes to the curriculum, respond to policy changes and address identified areas of need. Professional development is even more critical in cases where teacher qualifications are low. In 2014, the proportion of untrained teachers at the primary school level ranged from 11 percent to 38 percent across the member states. As research over the past decade suggests, teacher quality is highly correlated with student learning outcomes (Rockoff 2004).5 6. Third is the low capacity of school leaders to support teaching and thereby improve student achievement. In the OECS, good teachers are promoted to become head teachers without any training, and there are limited opportunities for professional development for school leaders. Moreover, communication between schools and parents or other stakeholders is weak and instructional leadership and accountability need to be strengthened. Research has shown that 2 From OECS Education Statistical Abstract 2013/14 and OECS costing data. 3 Barber, M., and M. Mourshed. 2007. How the World's Best Performing School Systems Come Out on Top . McKinsey and Company. 4 Mourshed, M, C. Chijioke, and M. Barber. 2010. How the World’s Most Improved School Systems Keep Getting Better. McKinsey and Company. 5 Rockoff, Jonah E. 2004. “The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data.� American Economic Review 94 (2): 247–252. 2 there is no single case of a school improving its student achievement record in the absence of talented leadership (Louis et al. 2010).6 Leadership is only second to classroom instruction among school-related factors that influence student outcomes (Leithwood 2005).7 7. The OECS is taking steps to address the challenge of low learning achievement. Improving the quality of learning for every child is at the heart of the Regional Education Strategy for the OECS. The OECS has developed a Regional Education Strategy, ‘Every Learner Succeeds’, to address the common challenge of low learning achievement across member states. The strategy was endorsed, by the OECS Council of Ministers of Education at their annual meeting of May 2012, for implementation across the states. This strategy also constitutes the education chapter in the OECS Growth and Development Strategy. The OECS country-level education plans have been or are being aligned to this overarching Regional Education Sector Strategy. The results-based strategy is summarized in box 1. 8. Countries have started implementation of various aspects for the OECS Education Sector Strategy (OESS): Under the strategic imperative for improvement of quality and accountability of school leadership, the OECS is implementing various school leader development programs although these are not grounded in the recently agreed OECS professional standards for school leaders. Implementation of these standards would be a logical way forward. Regarding the strategic imperative for improvement of curriculum and assessment strategies, countries are at various stages—Dominica completed revision of its primary curriculum, Saint Lucia is in the process of piloting curriculum for various subjects, and there is limited progress in this area for Vincent and the Grenadines and Grenada. The OECS has agreed to build on the Dominica experience as a basis for clarifying learning standards at the regional level and an accompanying assessment system in line with the OECS Economic Union. Under the strategic imperative to improve teachers’ professional development, teacher professional standards have been developed but these are yet to be implemented. 9. Countries are implementing various activities focusing on improving teacher quality including teacher training activities mainly focused on pre-service training, information and communication technology, and orientation for new teachers. Although regional teacher standards are yet to be adopted by the countries, they have been used as a basis for revising and improving the existing teacher appraisal system. Countries have endorsed the teacher appraisal system and are putting systems in place for their implementation. The timing and configuration or implementation arrangements will vary across countries. Some are seeking to link the appraisal with a broader strategy to professionalize the teaching service while others are using it as a tool to establish teaching councils, which can monitor teacher quality and competency. There will be need for specific professional development aimed at building teacher capacity to implement the new initiatives focusing on improving teaching and learning for all, including the use of learning standards, monitoring student learning based on these standards, and improving classroom practices. 6 Louis, K. S., K. Leithwood, K. L. Wahlstrom, and S. E. Anderson. 2010. Investigating the Links to Improved Student Learning: Final Report of Research Findings. New York: The Wallace Foundation. 7 Leithwood, K. 2005. “Understanding Successful Principalship: Progress on a Broken Front.� Journal of Educational Administration 43 (6): 619. 3 10. The development of the regional strategy was informed by a study of students at risk in the OECS. The findings and the recommendation informed the strategic priorities and key strategies including the decision to deal with gender issues as a cross-cutting theme. The study broadly examined many factors, including those related to the socioeconomic situation, instructional practices in schools that affect access, participation, and learning. The emphasis on improving teacher competencies and changing classroom practice to make them more learner centered are strategies that are expected to reduce the at-school risks. The following ongoing studies will shed further light on equity issues and what actions need to be taken to improve equity: (i) status of student support services; (ii) out of school students; (iii) inequality of education opportunity. With support from Caribbean Development Bank, additional work has been undertaken to mainstream gender in planning and implementing education programs in the OECS. The output of this activity was a set of guidelines for school leaders and teachers in ensuring that gender sensitive issues were addressed, particularly in curriculum delivery. 4 Box 1. OESS – Program Vision, Goal, Objective, and Components Vision All citizens, at every stage of their learning journey, from early years to adulthood, are able to reach their full potential and be successful in life, at work, and in society. Goal To contribute to the socioeconomic advancement of the OECS through a quality education system that enables learners of all ages to reach their true potential Objective To improve learning outcomes, with equity Subobjectives 1. To achieve increased and more equitable access to quality ECD services 2. To improve the quality of teaching and learning and achieve greater equity in learning opportunities, with a priority focus on primary and secondary education, including (a) improved teacher quality and qualifications; (b) improved curriculum, strategies for student assessment, and instructional approaches, all effectively used for learning-centered approaches, addressing the needs of all students, including the vulnerable (boys, poor, remote, and so on); and (c) improved school leadership and management capacity, especially to support improvements in the quality of teaching and learning, evidence-based school management, and accountability. 3. To improve the quality, relevance, and affordability of tertiary and continuing education 4. To expand all students’ access to technical and vocational education and training at all levels of the education system to respond to the demands of the jobs market 5. To strengthen sectorwide leadership, management, and accountability systems and capacities to instill a culture of evidence-based strategic management and decision making Strategic Imperatives (or Components/Areas of Intervention) 1. Improve the quality and accountability of leadership and management 2. Improve teachers’ professional development 3. Improve the quality of teaching and learning 4. Improve curriculum and strategies for assessment 5. Increase (and expand) access to quality ECD services 6. Provide opportunities for all learners in technical and vocational education and training 7. Increase provisions for tertiary and continuing education Source: The OESS. C. Higher-Level Objectives to which the Project Contributes 11. The higher-level objective is to improve learning outcomes at the basic education level. The proposed project contributes to the World Bank’s twin goals of eliminating poverty and boosting shared prosperity by improving the quality of basic education. These objectives will be achieved through a mix of interventions aimed at improving teaching and learning. Improvement in quality of education and outcomes in the OECS will, in the long term, enhance employability of the citizens and support growth.8 Better education will also contribute to greater shared 8 Evidence shows that a difference of one standard deviation in test scores between countries equates to roughly two percentage points in annual long-term gross domestic product growth. See: Hanushek, Eric A., and Ludger Woessmann. 2010. Education Quality and Economic Growth. 5 prosperity because children from poorer households perform worse than their wealthier peers in school. 12. The proposed project is consistent with the Bank Group OECS Regional Partnership Strategy (RPS) FY2015–19 Framework discussed by the Board of Executive Directors on November 20, 2014. The RPS recognizes the importance of improvement in the quality of education as a way of contributing to long-term unemployment and poverty reduction. The education aspects of the project are aligned with the RPS pillar of resilience, and in particular outcome 7 focusing on establishment of quality learning standards. The proposed project focuses on improving the quality of basic education that aligns with the Education Global Practice goals of access for all and learning for all. In addition, the proposed project is aligned with the Global Partnership for Education (GPE) Goal 2 of improved and more equitable student learning outcomes through quality teaching and learning. Moreover, the project’s focus on strengthening sector monitoring and evaluation (M&E) capacity as well as school leadership directly relates to GPE Goal 3 of efficient and effective education systems. Furthermore, the proposed focus on teacher professional development and primary grade assessment relates directly to the GPE strategic objective 3 - implementing sector plans focused on improved equity, efficiency, and learning. II. PROJECT DEVELOPMENT OBJECTIVES A. PDO 13. The project development objectives (PDOs) are to (a) use quality learning standards to support evidence-based teaching and learning at the primary level; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making, all in the member countries. B. Project Beneficiaries 14. The project beneficiaries are children attending primary schools in Grenada (10,069), St. Lucia (16,268), St. Vincent and the Grenadines (12,014), and Dominica (4,866). The current teachers and school leaders will also benefit from professional development activities (285 principals and 3,335 teachers). C. PDO-Level Results Indicators (a) Percentage of primary teachers using formative assessment based on learning standards (b) Percentage of teachers rated effective on classroom practices9 9 Effective teaching practices will be determined through classroom observation that uses guidance provided by Kane, Kerr, and Pianta (2014). Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project. John Wiley & Sons. Baseline based on guidance from country statisticians on representative sampling will be established in March 2016 using Classroom Assessment Scoring System (CLASS) scores. Baseline CLASS scores will be established in August 2016. 6 (c) Percentage of primary schools with a School Improvement Plan (SIP) focused on learning outcomes (d) Percentage of school leaders reporting on progress of teachers practices and students learning to oversight bodies (e) Production of an annual report on plan implementation and performance indicators at the country level III. PROJECT DESCRIPTION A. Project Components 15. Component 1: Curriculum and Assessment (US$396,500). The best-performing countries have established learning standards and have developed assessment systems to monitor their achievement (Barber and Mourshed 2007).10 Placing students’ outcomes as the main goal of educational policies requires that the learning outcomes to be achieved at every grade level be clearly specified and that the level of expected performance be clear; grade-level learning standards need to be clarified and brought in line with international learning standards. The commitment of the OESS to ensure that every child learns will underpin the definition of the learning standards. Formative assessment will be the major focus of the reform. The project will support the following activities: (a) Reviewing the harmonized curriculum to clarify grade-level learning standards for primary education in literacy, numeracy, science, and social studies through (i) provision of consultant services and (ii) the carrying out of consultations at the regional and member country levels (b) Developing a learner-centered assessment framework to monitor the achievement of learning standards (formative assessment), through (i) provision of consultant services and (ii) the carrying out of consultations at the regional and member country levels (c) Developing and implementing guidelines for using the learning standards and formative classroom assessment through (i) the provision of consultant; (ii) the carrying out of consultations at the regional and member country levels; and (iii) the adaptation of the learning standards and formative assessment to member country level context 16. Component 2: Teacher Professional Development (US$1,115,000). Teachers’ content knowledge and classroom practices are the strongest in-school predictor of student achievement (Barber and Mourshed 2007). Teachers’ instructional effectiveness improves when they receive opportunities for continuous professional learning that focuses on the quality of teacher-student interaction around subject matter and collegial professional interactions, mentoring, and ongoing coaching and respectful monitoring systems that provide frequent feedback on the quality and 10 Barber, M. and M. Mourshed. 2007. How the World's Best Performing School Systems Come Out on Top . McKinsey and Company. 7 effectiveness of teachers’ instruction. In addition, professional development that is embedded in teachers’ everyday teaching routines is critical when new instructional policy initiatives are being introduced (Cohen and Hill 2001).11 17. This component will support the following activities: (a) reviewing the teacher training programs in member countries; (b) reviewing classroom practices in member countries to understand current practices and identify capacity gaps to inform teacher professional development; (c) developing a teacher professional development course and school-based professional learning communities to improve classroom practices; (d) carrying out competence-based professional development activities for teachers12 in the member countries through training on effective classroom practices, formative assessment, all in line with learning standards and the use of student learning data; and (e) establishing an online community of practice for teachers to access online instructional resources and share lesson plans and experiences. 18. Component 3: Improve School Leadership and Accountability (US$209,000). Research has shown that there is no single case of a school improving its student achievement record without talented leadership (Louis et al. 2010).13 Leadership is only second to classroom instruction among school-related factors that influence student outcomes (Leithwood 2005).14 19. This component will support the following activities: (a) developing a school leader training program based on professional standards, including instructional leadership to support teachers to improve instructional practices; (b) carrying out competence-based training and certification activities for school leaders based on leadership standards, including on instructional leadership and use of data in planning, management, and reporting; and (c) developing a handbook for school principals/leaders to guide school leadership and management. 20. Component 4: Project implementation, monitoring, and evaluation (US$279,500). This component will support project implementation and M&E through (a) the procurement and financial management (FM) requirements of the project; (b) process monitoring of the teacher and school leader professional development activities; (c) the establishment of the national routines for ongoing and regular monitoring of learning standards and assessment using available data and additional research; (d) the monitoring of teacher classroom practices through the use of CLASS; and (e) the carrying out of annual regional and country-level program reviews. 11 Cohen, D., and H. Hill 2000. “Instructional Policy and Classroom Performance: The Mathematics Reform in California.� The Teachers College Record 102 (2): 294–343. 12 Priority will be given to Pre-K teachers. The number of teachers in the four countries is as follows: Dominica (511); Grenada (903); St. Lucia (1,071); St. Vincent and the Grenadines (819). The number of teachers to be trained will depend on unit costs (to be obtained from the ongoing costing exercise) and available resources (St. Vincent and the Grenadines and St. Lucia might allocate IDA resources toward implementation of education-related activities. 13 Louis, K. S., K. Leithwood, K. L. Wahlstrom, and S. E. Anderson. 2010. Investigating the Links to Improved Student Learning: Final Report of Research Findings. New York: The Wallace Foundation. 14 Leithwood, K. 2005. “Understanding Successful Principalship: Progress on a Broken Front.� Journal of Educational Administration 43 (6): 619. 8 B. Project Financing 21. This will be a grant, implemented through a Recipient Executed Trust Fund arrangement between the GPE and the Bank. There is also a possibility of St. Vincent and the Grenadines and St. Lucia allocating some IDA resources to education activities under the OECS Human Development Resilience Project, and those resources will build on and scale up the proposed activities under this grant. Preliminary discussions have led to an education component structure consisting of four subcomponents. 22. At the regional level, the OECS Commission will implement the following activities in collaboration with the countries and with support from grant funding: (a) development of learning standards framework based on a review of the curriculum and assessment; (b) assessment to identify capacity gaps for teachers and school leaders; (c) preparation of professional development modules to support training of teachers and school leaders on effective classroom practices and implementation of learning standards; and (d) M&E of implementation of national activities to improve pedagogy and the promotion of knowledge sharing across countries. Project Cost and Financing 23. Table 1 provides the cost estimates and financing for the components described above. Grant Project Cost Project Components Financing % Financing (US$) (US$) 1. Curriculum and Assessment 396,500 396,500 100 2. Teacher Professional Development 1,115,000 1,115,000 100 3. Improve School Leadership and Accountability 209,000 209,000 100 4. Project implementation and M&E 279,500 279,500 100 Total Financing Required 2,000,000 2,000,000 100 C. Lessons Learned and Reflected in the Project Design 24. The importance of professional development for both teachers and school leaders when new initiatives are introduced is well established. The planned professional development aspects will be based on best practice, including imbedding training in practice and promoting collaboration, including through communities of practice. IV. IMPLEMENTATION A. Institutional and Implementation Arrangements 25. The OECS Commission will be the implementing agency for the project. Implementation at the regional level will be coordinated by the Education Development Management Unit (EDMU) of the OECS Commission. The head of the EDMU will be the project coordinator responsible for the day-to-day management of the project. The EDMU will also be the main 9 focal point for monitoring the implementation of the regional activities, including the collection and reporting of the results. It will organize regular for a, including joint reviews, for reviewing progress and sharing lessons learned from the overall program, share best practices, and promote knowledge exchange between countries. The fiduciary responsibilities will be managed by the respective units at the OECS Commission. The OECS Commission managed the Institutional Development Grant, which supported development of the Regional Education Strategy and has implemented other Bank-financed projects. The EDMU has in the past successfully coordinated a Bank grant, which supported capacity building for education statistics across six member states. This grant achieved satisfactory results and the activities supported through that grant continue to be sustained, including annual publication of education statistical digest. In addition, the OECS Commission has implemented regional education projects financed by other partners. For example, the OECS Eastern Caribbean Reform Project and a Technical and Vocational Education project financed by the Canadian and German governments, respectively. The UNICEF Eastern Caribbean Areas Office through an ongoing Program Cooperation Agreement with the OECS has been providing financial support for regional education activities. 26. The Chief Education Officers’ Forum established at the time of preparation of the OESS will act as the Project Steering Committee with chief education officers assuming chairmanship of the committee on a rotating basis as is the current practice. The committee will offer advice and guidance on the project activities as part of the overall implementation of the OESS. 27. Institutional and implementation arrangements at the national level. As part of the regional arrangements for implementing the OESS, the program implementation at the national level will be mainstreamed into the Ministry of Education (MOE). This model is in response to previous experience where projects were perceived as competing priorities outside the scope of the work program of the MOE. The national-level counterpart will be the chief education officer who may designate a national focal point. The actual implementation will be managed by the units responsible for the various program components. For example, the curriculum and assessment components will be implemented by the Curriculum Unit with the head of the Curriculum Unit having direct responsibility for related activities. Similarly, education officers with responsibility for teacher training and school leadership—usually a senior education officer—will be responsible for activities related to their respective portfolios. Implementation calls for a high level of coordination between various units and education officers. This coordination will be managed by the chief education officer or a delegated focal point. The focal point will coordinate closely with all relevant stakeholders and beneficiaries, and organize national consultation meetings of various aspects of the project. B. Results Monitoring and Evaluation 28. The project will rely on the revised results and monitoring framework developed for the implementation of the Regional Education Strategy 2012–2021. The results framework for the project detailed in annex 1 includes project-specific indicators. Quarterly and annual progress reports will be submitted to the Bank by the EDMU and will be informed by reports submitted from the countries. 29. The project-level results mapping and M&E framework are derived from the broader results mapping and M&E frameworks prepared for the OECS 10-year Regional Education 10 Strategy (2012–2021). As such, they lay out a hierarchy of impacts, outcomes, intermediate outcomes, and outputs (and related indicators), which will facilitate the tracking of progress along the project results chain. This supports the intention of the OECS, as expressed in its 10-year education strategy, to monitor and evaluate implementation progress, as well as outcomes, and their plausible links. Annex 1 thus presents (a) a mapping of the project’s results; (b) a matrix of project outcome indicators organized by project objectives; and (c) a matrix of project output indicators organized by project components. The M&E of project implementation and outcomes will rely on existing capacity and systems at the country level and at the regional level, which will be strengthened through (a) a learning-by-doing approach and (b) technical assistance, as needed. The EDMU will coordinate M&E activities at the regional level, including organizing progress reviews, and coordinating with the countries to ensure that the statisticians and planners collect the country level project data as part of monitoring sector performance. This will not present any unnecessary additional burden on those carrying out this work, because they will be collecting, analyzing, and using this information for management and decision making as a part of the overall M&E plan for the 10-year strategy. C. Sustainability 30. The sustainability of the project results is assured because it supports the existing OECS Regional Education Strategy 2012–2021, which was endorsed and is supported by all of the OECS member states. Improving quality of learning is a key priority for the heads of government (who are at the same time the ministers of finance) in the OECS. In addition, most of the GPE resources will go toward activities to strengthen system capacity and not recurrent costs. Moreover, the project will be implemented within the government system, thus increasing the chances of continuity even after the end of the project. All these increase the likelihood of continued implementation of the frameworks and actions supported by the project. V. KEY RISKS AND MITIGATION MEASURES 31. The overall risk for the project is Moderate. The political and sector strategy risks are Low. The quality of education agenda enjoys broad-based political support in the four states; it is the major focus of the OECS Regional Education Strategy and, therefore, faces a low risk of political opposition to the proposed project. The implementing agency risk is substantial both at the country and regional levels. The additional layer created by the regional aspects of the project may create implementation delays. This problem will be mitigated by having the Chief Education Officers’ Forum to oversee the implementation of project activities. In addition, this will require more effort and an experienced team from the Bank to be able to deliver on results on the proposed innovative interventions. There is a moderate risk that countries may not implement at the same pace. Addressing this risk will require putting in place a mechanism to support the countries to move at the same pace to achieve the economies of scale that the regional approach seeks to achieve. There is a moderate risk of underestimation of required supervision resources. The costs for managing this operation will be higher than a single country operation. This is a new model for delivering support to small island states using a single operation to support several countries. This has to be taken into consideration when allocating GPE resources for supervision. There is a substantial stakeholder risk mainly related to teacher and school principals’ resistance to taking on the new initiatives. This will be mitigated by ensuring strong stakeholder participation in the preparation process. Consultations will take place 11 both at regional and country levels. Every effort will be made to present the project-supported initiatives as building on what already exists rather than as new initiatives. VI. APPRAISAL SUMMARY A. Economic and Financial Analysis 32. The rationale for public sector provision and GPE support to primary education is a justifiable option. Education and human capital development are directly linked to economic growth, productivity, and the improvement of living standards.15 The low quality of education has been highlighted, by the Caribbean Growth Forum, as the major constraint to growth, employment, and poverty reduction. It is expected that as learning achievement improves, more people will enter the labor market and the economic benefits of the workforce affected by the project are expected to improve and more people will move out of poverty. The activities implemented under the project will have a direct development impact on the capacity of education systems in the OECS to provide better quality instruction, which in the long term will improve educational outcomes and labor market outcomes at the individual level. This impact will be seen through a higher probability of school completion, which improves the lifetime earnings and type of employment, and higher labor productivity growth, using labor earnings as a proxy for labor productivity. Moreover, as schools are able to offer better instruction, including to the poor students, the gap in learning outcomes between the rich and the poor will reduce. 33. The Bank has global experience in supporting government efforts to improve teacher pedagogical practices and professional development, capacity for school leadership, and monitoring student learning. It provides added value in this project through its technical expertise and financing. The Bank has and will contribute to support development of government’s capacity, under each component that supports the achievement all three PDOs. A quantitative economic analysis shows a net benefit and positive return for the GPE Project. It was considered that the project effects lead to higher returns to skills that affect the wages of beneficiaries (Hanushek et al. 2015).16 With a 10 percent discount rate, the project has a net present value (NPV) of US$11.1 million with an internal rate of return of 27.0 percent. In addition, a sensitivity analysis to consider an alternative worsening scenario, in which the effects of activities on students’ performance are assumed to be only half as large. Even in this alternative case, the project appears to be a good investment from an economic point of view, with an NPV of US$5.5 million and an internal rate of return of 20.2 percent. B. Technical 34. The OECS policy priorities supported under the project are supported by global evidence as most crucial to improving educational outcomes. First, the quality of an education system is bound by the quality of its teachers. There is emerging evidence that the right policies for professional development of and supporting teachers can transform the effect that teachers have in the classroom. Second, no education system has systematically and sustainably improved its 15 Barro, R. 2001. “Human Capital and Economic Growth.� The American Economic Review 91 (2): 12–17. 16 Hanushek, Eric A., Guido Schwerdt, Simon Wiederhold, and Ludger Woessmann . 2015. “Returns to Skills around the World: Evidence from PIAAC.� European Economic Review 73: 103–130. 12 performance without good leadership. The OECS’s move toward setting professional standards and providing competence-based professional development for school leaders will fulfill that critical element of education system management. Third, there is increasing evidence that the use of formative learning assessment can have a significant impact on learning if properly implemented. The selected areas of intervention are also reflected in the Bank’s Education Sector Strategy 2020: ‘invest early, invest smartly, and invest for all’. All these activities will, in the medium to longer term, improve the learning conditions of students and improve educational outcomes. C. Financial Management 35. FM will be handled by the OECS Commission fiduciary team, which is also handling other ongoing Bank-financed activities and has implemented several Bank-financed trust funds, including an Institutional Development Fund supporting education. Disbursement of project funds will be based on Interim Financial Reports (IFRs). The IFRs will be prepared and sent to the Bank every six months (July–December and January–June) and 45 days after the end of every reporting period. Accounting records will be kept in the accounting system ‘Sage 50’ and will be updated and monitored in accordance with the financial regulations manual of the OECS Commission. The current Operations Manual is very comprehensive and the processes and procedures once applied consistently are adequate to ensure that the financial information is accurate and funds are only used for project purposes. 36. A segregated U.S. dollar designated account and local currency project account will be opened upon effectiveness of the project at First National Bank in St. Lucia, and the bank account and bank information will be conveyed to the Bank. The operation of the bank accounts will fall under the current operations processes of the OECS Commission. D. Procurement 37. Procurement will be handled by the OECS Commission Procurement Unit, which is also handling the ongoing Bank-financed activities being implemented through the commission. The team has extensive experience working with Bank-financed projects. 38. Procurable items financed under the project will be limited to small goods and consultants and will be procured in accordance with the Bank’s ‘Guidelines: Procurement of Goods, Works, and Non-Consulting Services under IBRD Loans and IDA Credits and Grants by World Bank Borrowers’, dated January 2011 and revised July 2014; ‘Guidelines: Selection and Employment of Consultants under IBRD Loans and IDA Credits and Grants by World Bank Borrowers’ dated January 2011 and revised July 2014; the IDA Anticorruption Guidelines dated July 1, 2005, and as amended October 15, 2006; and the provisions stipulated in the Grant Agreement. E. Social (including Safeguard Policies) 39. The project is classified as Category C because it does not finance any construction or other activities that can cause potential negative impacts to the environment or natural habitats. The main thrust of the OESS is equity in learning outcomes: ‘Learning for All’. The planned activities—formative assessment, improving school leadership, teacher professional 13 development, and M&E—are all focusing on areas proven to be impactful on equity aspects. The following studies will shed further light on equity issues and what actions need to be taken to improve equity: (a) status of student support services; (b) out of school students; (c) inequality of education opportunity 40. Gender. The project will support the OECS in making teaching sensitive to the differences in learning styles of boys and girls and the impact this may have, for example, on motivation. The project also includes support in improving teacher sensitivity within teacher-student interactions as they relate to the learning needs of boys and girls. In addition, the assessment framework as well as the M&E framework for the project will provide information by gender, to better inform policy interventions. The Bank will also work with the OECS to incorporate gender aspects in the training materials to make them gender sensitive. Where applicable, indicators in the project’s M&E framework will be reported by gender. 41. Beneficiary feedback mechanisms. The project will include mechanisms to incorporate beneficiary feedback at several levels. The project will support the surveys, which will be used to incorporate feedback from teachers, parents, school leaders, and community members. This will be reflected in the results framework. The project will also promote feedback from students through the application of the new diagnostic census-based student assessment, which will include surveys on student perceptions. F. World Bank Grievance Redress 42. Communities and individuals who believe that they are adversely affected by a Bank-supported project may submit complaints to existing project-level grievance redress mechanisms or the Bank’s Grievance Redress Service (GRS). The GRS ensures that complaints received are promptly reviewed to address project-related concerns. Project-affected communities and individuals may submit their complaint to the Bank’s independent Inspection Panel, which determines whether harm occurred, or could occur, because of Bank noncompliance with its policies and procedures. Complaints may be submitted at any time after concerns have been brought directly to the Bank’s attention and Bank management has been given an opportunity to respond. For information on how to submit complaints to the Bank’s corporate GRS, please visit http://www.worldbank.org/GRS. For information on how to submit complaints to the Bank Inspection Panel, please visit www.inspectionpanel.org. 14 Annex 1: Results Framework and Monitoring OECS COUNTRIES: Support to Implementation of the Regional Education Strategy (P158836) Results Framework - Regional Level Project Development Objective Expected Impact: Improved learning outcomes (expected to happen beyond project completion but included to keep the focus on learning). Equity considerations: gender, location, disability. Project Development Objectives: The objectives of the proposed project are to (a) use learning standards to support evidence-based teaching and learning; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making, all in the member countries. Cumulative Target Values Unit of Data Resp. for Data Indicator Name Core Baseline Frequency Description Measure 2017 2018 2019 Source/Methodology Collection 2016 Impact Indicators17 Primary School Literacy Skills % of pupils literacy at or above their grade level at the key stages of grade 2 by gender, urban-rural residence Primary School Mathematics Skills % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics Project Development Objective Indicators PDO Indicator 1: Use learning standards to support evidence-based teaching and learning at the primary level. 17 These are included to keep the focus on learning but are expected to be achieved after Project closing. Project Development Objective indicators will be used to monitor Project achievements. This applies to all countries (Annex 1b-e) as well as the regional level. 15 Percentage of primary % 0 25 50 Annual MOE Number of teachers teachers using Department of using formative formative assessment Primary assessment/Total based on learning Education number of primary standards school teachers PDO Indicator 2: Improve teacher practices at the primary level. Percentage of teachers % TBD TBD TBD Annual Project Implementation MOE Number of teachers rated effective on Reports Department of assessed as classroom practices18 Primary effective/Total Education number of teachers observed using CLASS PDO Indicator 3: Improve primary school leadership and accountability. Percentage of primary % 0 25 50 Annual Project Implementation MOE Number of primary schools with an SIP Reports Department of schools with SIPs focused on learning Primary focused on outcomes Education learning/Total number of primary schools Percentage of school % 0 25 50 Annual Project Implementation MOE Number of primary leaders reporting on Reports Department of school leaders progress of teachers Primary reporting progress on practices and students Education teacher practices and learning to oversight student learning/Total bodies number of primary school leaders PDO Indicator 4: Initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making. Production of an Yes/No No Yes Yes MOE Statistics Country annual report annual report on plan and Planning on plan implementation and Units implementation and performance indicators performance indicators 18 Capacity building for respective staff of the MOE is ongoing. Baseline data will be available by August 2016. 16 at the country level . Intermediate Results Indicators Component 1 - Curriculum and Assessment Learner-centered Yes/No No Yes Once Project Implementation MOE assessment framework Reports Curriculum and and guidelines for Assessment Unit using learning standards approved Component 2 - Teacher Professional Development Teacher professional Yes/No No Yes EDMU development course developed Number of teachers Cumulative 0 270 540 Annual Project Implementation MOE trained and certified as number Reports Department of competent Primary Education Component 3 - Improve School Leadership and Accountability Handbook for school Yes/No No Yes EDMU Handbook leaders developed and printed School leadership Yes/No No Yes EDMU Course training program developed Number of school Cumulative 0 135 270 Annual Project Implementation MOE leaders certified in number Reports Department of competence-based Primary standards Education Component 4 - Project implementation, monitoring, and evaluation Number of teachers Cumulative 200 300 400 Annual Project Implementation MOE monitored through number Reports Department of CLASS Primary 17 Education Number of schools Cumulative 140 200 300 MOE analyzing and using number Department of student learning data Primary Education M&E Manual Yes/No No EDMU Manual developed Stakeholder feedback No No Yes Yes Annual Project Implementation EDMU surveys undertaken Reports Survey 18 19 ANNEX 1 B. RESULTS FRAMEWORK DOMINICA Expected Impact: Improved learning outcomes (expected to happen beyond project completion, except perhaps for Grade 2 performance). Equity considerations: gender, income, location, disability, ethnic groups Project Development Objectives: The objectivesof the proposed project are to (a) use learning standards to support evidence-based teaching and learning; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. Baseline Cumulative Target Values Resp. for Unit of Data Source Indicator Name Core (GPE Frequency Data Description Measure /Methodology 2016) 2017 2018 2019 2020 Collection Impact Indicators Primary School literacy Skills % of pupils literacy skills at or above their grade level at the key stages of grade 2 by gender, urban-rural residence Primary School Mathematics Skills % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics Project Development Objective Indicators PDO Indicator 1: Use learning standards to support evidence-based teaching and learning at the primary level Percentage of primary teachers 0 0 25 50 Annual MOE #of teachers using using formative Department formative assessment based % of Primary assessment/Total # of on learning Education primary school standards Dpt teachers Objective Indicator 2: Improve teacher practices at the primary level 20 Percentage of Number of teachers teachers rated MOE assessed as Project effective on Department effective/Total % TBD TBD TBD TBD Annual Implementatio classroom of Primary number of teachers n Reports practices19 Education observed using CLASS Objective Indicator 3: Improve primary school leadership and accountability Percentage of primary schools with a School No. of primary MOE Improvement Project schools with SIPS Department Plans (SIP) % 0% 0% 25% 50% Annual Implementatio focused on of Primary focused on n Reports learning/Total No. of Education learning primary schools outcomes Percentage of school leaders reporting on No. of primary school MOE progress of Project Department leaders reporting teachers practices % 0% 0% 25% 50% Annual Implementatio of Primary progress on teacher and students n Reports Education practices and student learning to learning/Total No. of oversight bodies primary school leaders Objection 4: initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. 19 Capacity building for respective staff of the Ministries of Education is ongoing. Baseline data will be available by August 2016 21 Production of an Country annual report annual report on on Plan plan MOE implementation and implementation Statistics performance YES/NO NO YES YES YES and performance and Planning indicators indicators at the Units country level . Intermediate Results Indicators Component 1 – Curriculum and assessment Learners-centere d assessment framework and MOE guidelines for Project YES/NO Department using learning NO YES Once Implementatio of Primary standards n Reports Education approved Component 2 – Teacher Professional Development Teacher professional YES/NO NO YES development course developed Number of MOE Project teachers trained # Department 0 0 44 87 Annual Implementatio and certified as Cumulative of Primary n Reports competent Education Component 3 – School Leadership and Accountability Handbook for school leaders YES/NO NO YES Handbook developed and printed 22 School leadership training YES/NO NO YES Course program developed Number of school leaders Project MOE certified in a # Cumulative 0 0 27 54 Annual Implementati Department competence on Reports of Primary based standards Component 4 – Project implementation, monitoring, and evaluation. Number of teachers monitored Project MOE through the # 0 32 49 65 Annual Implementati Department Classroom Cumulative on Reports of Primary Assessment Scoring System (CLASS) Number of schools # analyzing and 0 26 38 57 Cumulative using student learning data M&E Manual Manual YES/NO NO YES developed 23 ANNEX 1 C. RESULTS FRAMEWORK GRENADA Expected Impact: Improved learning outcomes (expected to happen beyond project completion, except perhaps for Grade 2 performance). Equity considerations: gender, income, location, disability, ethnic groups Project Development Objectives: The objectivesof the proposed project are to (a) use learning standards to support evidence-based teaching and learning; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. Baseline Cumulative Target Values Resp. for Unit of Data Source Indicator Name Core (GPE Frequency Data Description Measure /Methodology 2016) 2017 2018 2019 2020 Collection Impact Indicators Primary School literacy Skills % of pupils reading at or above their grade level at the key stages of grade 2 by gender, urban-rural residence Primary School Mathematics Skills % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics Project Development Objective Indicators PDO Indicator 1: Use learning standards to support evidence-based teaching and learning at the primary level Percentage of primary teachers 0 0 25 50 Annual MOE #of teachers using using formative Department formative assessment based % of Primary assessment/Total # of on learning Education primary school standards Dpt teachers Objective Indicator 2: Improve teacher practices at the primary level 24 Percentage of Number of teachers teachers rated MOE assessed as Project effective on Department effective/Total % TBD TBD TBD TBD Annual Implementatio classroom of Primary number of teachers n Reports practices20 Education observed using CLASS Objective Indicator 3: Improve primary school leadership and accountability Percentage of primary schools with a School No. of primary MOE Improvement Project schools with SIPS Department Plans (SIP) % 0% 0% 25% 50% Annual Implementatio focused on of Primary focused on n Reports learning/Total No. of Education learning primary schools outcomes Percentage of school leaders reporting on No. of primary school MOE progress of Project Department leaders reporting teachers practices % 0% 0% 25% 50% Annual Implementatio of Primary progress on teacher and students n Reports Education practices and student learning to learning/Total No. of oversight bodies primary school leaders Objection 4: initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. 20 Capacity building for respective staff of the Ministries of Education is ongoing. Baseline data will be available by August 2016 25 Production of an Country annual report annual report on on Plan plan MOE implementation and implementation Statistics performance YES/NO NO YES YES YES and performance and Planning indicators indicators at the Units country level . Intermediate Results Indicators Component 1 – Curriculum and assessment Learners-centere d assessment framework and MOE guidelines for Project YES/NO Department using learning NO YES Once Implementatio of Primary standards n Reports Education approved Component 2 – Teacher Professional Development Teacher professional YES/NO NO YES development course developed Number of MOE Project teachers trained # Department 0 0 72 144 Annual Implementatio and certified as Cumulative of Primary n Reports competent Education Component 3 – School Leadership and Accountability Handbook for school leaders YES/NO NO YES Handbook developed and printed 26 School leadership training YES/NO NO YES Course program developed Number of school leaders Project MOE certified in a # Cumulative 0 0 36 72 Annual Implementati Department competence on Reports of Primary based standards Component 4 – Project implementation, monitoring, and evaluation. Number of teachers monitored Project MOE through the # 0 53 80 107 Annual Implementati Department Classroom Cumulative on Reports of Primary Assessment Scoring System (CLASS) Number of schools # analyzing and 0 37 53 80 Cumulative using student learning data M&E Manual Manual YES/NO NO YES developed 27 ANNEX 1 D. RESULTS FRAMEWORK SAINT LUCIA Expected Impact: Improved learning outcomes (expected to happen beyond project completion, except perhaps for Grade 2 performance). Equity considerations: gender, income, location, disability, ethnic groups Project Development Objectives: The objectivesof the proposed project are to (a) use learning standards to support evidence-based teaching and learning; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. Baseline Cumulative Target Values Resp. for Unit of Data Source Indicator Name Core (GPE Frequency Data Description Measure /Methodology 2016) 2017 2018 2019 2020 Collection Impact Indicators Primary School literacy Skills % of pupils literacy at or above their grade level at the key stages of grade 2 by gender, urban-rural residence Primary School Mathematics Skills % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics Project Development Objective Indicators PDO Indicator 1: Use learning standards to support evidence-based teaching and learning at the primary level Percentage of primary teachers 0 0 25 50 Annual MOE #of teachers using using formative Department formative assessment based % of Primary assessment/Total # of on learning Education primary school standards Dpt teachers Objective Indicator 2: Improve teacher practices at the primary level 28 Percentage of Number of teachers teachers rated MOE assessed as Project effective on Department effective/Total % TBD TBD TBD TBD Annual Implementatio classroom of Primary number of teachers n Reports practices21 Education observed using CLASS Objective Indicator 3: Improve primary school leadership and accountability Percentage of primary schools with a School No. of primary MOE Improvement Project schools with SIPS Department Plans (SIP) % 0% 0% 25% 50% Annual Implementatio focused on of Primary focused on n Reports learning/Total No. of Education learning primary schools outcomes Percentage of school leaders reporting on No. of primary school MOE progress of Project Department leaders reporting teachers practices % 0% 0% 25% 50% Annual Implementatio of Primary progress on teacher and students n Reports Education practices and student learning to learning/Total No. of oversight bodies primary school leaders Objection 4: initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. 21 Capacity building for respective staff of the Ministries of Education is ongoing. Baseline data will be available by August 2016 29 Production of an Country annual report annual report on on Plan plan MOE implementation and implementation Statistics performance YES/NO NO YES YES YES and performance and Planning indicators indicators at the Units country level . Intermediate Results Indicators Component 1 – Curriculum and assessment Learners-centere d assessment framework and MOE guidelines for Project YES/NO Department using learning NO YES Once Implementatio of Primary standards n Reports Education approved Component 2 – Teacher Professional Development Teacher professional YES/NO NO YES development course developed Number of MOE Project teachers trained # Department 0 0 88 177 Annual Implementatio and certified as Cumulative of Primary n Reports competent Education Component 3 – School Leadership and Accountability Handbook for school leaders YES/NO NO YES Handbook developed and printed 30 School leadership training YES/NO NO YES Course program developed Number of school leaders Project MOE certified in a # Cumulative 0 0 39 77 Annual Implementati Department competence on Reports of Primary based standards Component 4 – Project implementation, monitoring, and evaluation. Number of teachers monitored Project MOE through the # 0 66 98 131 Annual Implementati Department Classroom Cumulative on Reports of Primary Assessment Scoring System (CLASS) Number of schools # analyzing and 0 36 52 78 Cumulative using student learning data M&E Manual Manual YES/NO NO YES developed 31 ANNEX 1 E. RESULTS FRAMEWORK FOR SAINT VINCENT AND THE GRENADINES Expected Impact: Improved learning outcomes (expected to happen beyond project completion, except perhaps for Grade 2 performance). Equity considerations: gender, income, location, disability, ethnic groups Project Development Objectives: The objectivesof the proposed project are to (a) use learning standards to support evidence-based teaching and learning; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. Baseline Cumulative Target Values Resp. for Unit of Data Source Indicator Name Core (GPE Frequency Data Description Measure /Methodology 2016) 2017 2018 2019 2020 Collection Impact Indicators Primary School literacy Skills % of pupils literacy at or above their grade level at the key stages of grade 2 by gender, urban-rural residence Primary School Mathematics Skills % of pupils by gender and urban-rural residence performing at or above the minimum standards at grade 2 in Mathematics Project Development Objective Indicators PDO Indicator 1: Use learning standards to support evidence-based teaching and learning at the primary level Percentage of primary teachers 0 0 25 50 Annual MOE #of teachers using using formative Department formative assessment based % of Primary assessment/Total # of on learning Education primary school standards Dpt teachers Objective Indicator 2: Improve teacher practices at the primary level 32 Percentage of Number of teachers teachers rated MOE assessed as Project effective on Department effective/Total % TBD TBD TBD TBD Annual Implementatio classroom of Primary number of teachers n Reports practices22 Education observed using CLASS Objective Indicator 3: Improve primary school leadership and accountability Percentage of primary schools with a School No. of primary MOE Improvement Project schools with SIPS Department Plans (SIP) % 0% 0% 25% 50% Annual Implementatio focused on of Primary focused on n Reports learning/Total No. of Education learning primary schools outcomes Percentage of school leaders reporting on No. of primary school MOE progress of Project Department leaders reporting teachers practices % 0% 0% 25% 50% Annual Implementatio of Primary progress on teacher and students n Reports Education practices and student learning to learning/Total No. of oversight bodies primary school leaders Objection 4: initiate the strengthening of sector monitoring and evaluation (M&E) capacity in support of evidence-based strategic management and decision making. 22 Capacity building for respective staff of the Ministries of Education is ongoing. Baseline data will be available by August 2016 33 Production of an Country annual report annual report on on Plan plan MOE implementation and implementation Statistics performance YES/NO NO YES YES YES and performance and Planning indicators indicators at the Units country level . Intermediate Results Indicators Component 1 – Curriculum and assessment Learners-centere d assessment framework and MOE guidelines for Project YES/NO Department using learning NO YES Once Implementatio of Primary standards n Reports Education approved Component 2 – Teacher Professional Development Teacher professional YES/NO NO YES development course developed Number of MOE Project teachers trained # Department 0 0 66 132 Annual Implementatio and certified as Cumulative of Primary n Reports competent Education Component 3 – School Leadership and Accountability Handbook for school leaders YES/NO NO YES Handbook developed and printed 34 School leadership training YES/NO NO YES Course program developed Number of school leaders Project MOE certified in a # Cumulative 0 0 34 67 Annual Implementati Department competence on Reports of Primary based standards Component 4 – Project implementation, monitoring, and evaluation. Number of teachers monitored Project MOE through the # 0 49 73 97 Annual Implementati Department Classroom Cumulative on Reports of Primary Assessment Scoring System (CLASS) Number of schools # analyzing and 0 40 57 86 Cumulative using student learning data M&E Manual Manual YES/NO NO YES developed 35 Annex 2: Detailed Project Description OECS COUNTRIES: Support to Implementation of the Regional Education Strategy 1. The objectives of the proposed project are to (a) use quality learning standards to support evidence-based teaching and learning at the primary level; (b) improve teacher practices at the primary level; (c) strengthen primary school leadership and accountability; and (d) initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making, all in the member countries. The goal is to improve learning achievement but given the short time frame for implementation (only three years) it is unlikely that changes will be seen in learning achievement during the implementation period. The expected theory of change is as follows: 2. Component 1: Curriculum and Assessment. This component will support the following activities: (a) Reviewing the harmonized curriculum to clarify grade-level learning standards for primary education in literacy, numeracy, science, and social studies through (i) 36 consultant services and (ii) the carrying out of consultations at the regional and member country levels (b) Developing a learner-centered assessment framework to monitor the achievement of learning standards (formative assessment), through (i) provision of consultant services and (ii) the carrying out of consultations at the regional and member country levels (c) Developing and implementing guidelines for using the learning standards and formative classroom assessment through (i) the provision of consultant services; (ii) the carrying out of consultations at the regional and member country levels; and (iii) the adaptation of the learning standards and formative assessment to member country level context. 3. Effective instruction relies on clear learning standards and a high-quality system of assessment so that in the course of instruction, teachers have insight into student learning and can adjust instruction accordingly. Effective assessment also ensures that stakeholders in the school community are aware of how all students (boys and girls and regardless of social economic background or disability) benefited from instruction—or how instruction might be modified. Clear learning standards will help guide classroom practices, including systematic formative and summative classroom assessment to address learning deficiencies before children transition to the next grade or exit the system. The best-performing countries have established learning standards and developed assessment systems to monitor their achievement (Barber and Mourshed 2007;23 Mourshed, Chijioke, and Barber 201024). Placing students’ outcomes as the main goal of educational policies requires that the learning outcomes to be achieved at every grade level are clearly specified and that the level of expected performance is clear. 4. The project will support the following activities: (a) consultant services to facilitate clarification of grade-level learning standards; the consultant will also ensure international best practices are incorporated while developing the learning standards; (b) consultant services to facilitate development of a learner-centered assessment framework to include formative classroom assessment based on learning standards; (c) consultations (both at the regional and country levels) to enable a wide range of stakeholders to provide inputs to the draft learning standards and assessment framework; (d) printing and distribution of the learning standards and assessment framework after incorporating inputs from stakeholder consultations; (e) further country-level forum to adapt the standards and assessment framework to the country-level context; and (f) orientation training activities for directors of studies and school leaders and education officers to the learning standards and assessment framework. 5. Learning assessment will be an integral part of a cycle of standards, curriculum, and instruction that forms an ongoing feedback loop to provide desired information to teachers, parents, and students. It will be an ongoing process of gathering evidence of what each student actually knows, understands, and can do in relation to the learning standards and goals and will 23 Barber, M., and M. Mourshed. 2007. How the World's Best Performing School Systems Come Out on Top . McKinsey and Company. 24 Mourshed, M, C. Chijioke, and M. Barber. 2010. How the World’s Most Improved School Systems Keep Getting Better. McKinsey and Company. 37 serve various purposes. The proposed assessment framework will be comprehensive and will be at three levels: (a) regional, (b) national, and (c) classroom. 6. Assessment will serve the following purposes: (a) provide information that can help ensure responsiveness to the needs of all students; (b) provide feedback to students, educators, parents, policymakers, and the public about learner’s progress and the effectiveness of educational services; (c) assessment results will be used to improve instruction to respond to student learning needs and ensure that all students attain the defined learning goals; (d) help teachers and school leaders make informed decisions about which strategies need to be strengthened, how instructional strategies can be modified depending on what students understand or misunderstand in the course of instruction, and whether or not instruction has been successful; and (e) provide valuable information on the type of professional development activities needed to improve teachers’ instructional practices. 7. The assessment framework will include three types of assessment.  Classroom or formative assessment will provide teachers and pupils with timely feedback on student knowledge and ability to guide shifts in instruction instructional plans and student learning. Research has demonstrated that formative assessments improve student learning, with the largest gains among the lowest-performing students (Black and William 1998).25 Formative assessments play a critical role in narrowing learning gaps between student groups and can be used as a tool for ensuring equitable access to learning opportunities in classrooms. Effective formative assessment processes provide immediate, actionable data about classroom learning to help teachers adjust their teaching practices. Formative tools help teachers and students gauge learning, which can lead to more targeted student questions and learning activities. Teachers can use formative assessment tools to provide corrective feedback, modify their instruction, or identify areas where additional instruction and other support is needed. Formative assessment will be the main focus of the assessment reform.  Examinations for making decisions about each individual student’s progress through the education system will be conducted at the end of the primary school cycle and will be standardized and conducted at the regional level.  System-level assessment will be undertaken at grades 2 and 4 for monitoring and providing policymakers and practitioners with relevant information of overall performance levels in the system, changes in these levels over time, and related contributing factors. These will be standardized and will be performed at national level although they can also be compared at the regional level. 8. Component 2: Teacher Professional Development. This component will support the following activities: (a) reviewing the teacher training programs in member countries; (b) reviewing classroom practices in member countries to understand current practices and identify capacity gaps; (c) developing a teacher professional development course and school-based 25 Black, Paul, and Dylan William. 1998. “Assessment and Classroom Learning.� Assessment in Education: Principles, Policy and Practice 5 (1): 7–74. DOI:10.1080/0969595980050102 38 professional learning communities to improve classroom practices (use of formative assessment, teacher-student interaction, and use of student learning data through consultant services; (d) carrying out competence-based professional development activities for teachers26 in the member countries through training on effective classroom practices, formative assessment, all in line with learning standards and the use of student learning data; and (e) establishing an online community of practice for teachers to access online instructional resources and share lesson plans and experiences. 9. Teachers’ content knowledge and classroom practices are the single strongest in-school predictor of student achievement (Barber and Mourshed 2007). Teacher professional development will commence in the second year following materials development and consensus building activities which will take place during the first year of implementation.. Teacher professional development will be in the form of school-based professional learning communities. This will include workshops and teachers’ engagement in ongoing self-reflection, peer support, experimentation, and modification of instruction and management practices based on student performance data, student work, and both learning and social behaviors embedded in teachers’ everyday teaching routine. Through an intensive process of collaborative and job-embedded learning, teachers can gain content knowledge, technical strategies, and an improved understanding of their own teaching and learning and of the various ways by which students learn (Darlington-Hammond et al. 2009).27 It is therefore an ongoing professional learning process grounded in teachers’ everyday work to support student success. 10. Teacher professional development will be competence based and will focus on improving classroom practices, including emotional support, classroom organization, instructional support28 (to ensure that teachers understand the components of their teaching that matter for their student achievement), formative assessment, and use of assessment information to tailor support to student learning needs. It will be an ongoing professional learning process grounded in teachers’ everyday instructional routines to support student success. In addition to the face-to-face professional training for teachers, the professional learning will involve teachers meeting on a regular schedule in learning teams organized by grade-level or content-area assignments and sharing responsibility for their students’ success. Learning teams will follow a cycle of continuous improvement that begins with examining student data to determine the areas of greatest student need, pinpointing areas where additional teacher learning is necessary, identifying and creating learning experiences to address these needs, developing and sharing lesson plans and assessments, applying new strategies in the classroom, reflecting on their impact on student learning, and repeating the cycle with new goals. The project will also support establishment of an online community of practice for teachers and school leaders to further enhance professional collaboration. Teacher professional development will start with Pre-K teachers. 26 The number of teachers in the four countries is as follows: Dominica (511); Grenada (903); St. Lucia (1,071); St. Vincent and the Grenadines (819). 27 Darling-Hammond, Linda, Ruth Chung Wei, Alethea Andree, Nikole Richardson, and Stelios Orphanos. 2009. Professional Learning in the Learning Profession: A Status Report on Teacher Development in the United States and Abroad. Dallas, TX: National Staff Development Council. The School Redesign Network at Stanford University. 28 Based on CLASS. Capacity building for CLASS is being undertaken in the four OECS states. 39 11. Component 3: Improve School Leadership and Accountability. This component will support the following activities: (a) developing a school leader training program based on professional standards, including instructional leadership to support teachers to improve instructional practices; (b) carrying out competence-based training and certification activities for school leaders based on leadership standards, including on instructional leadership and use of data in planning, management, and reporting; and (c) developing a handbook for school principals/leaders to guide school leadership and management. 12. Based on lessons from international experience, strategies will focus on developing and strengthening skills for dealing with the roles that contribute to improving school outcomes: (a) developing and evaluating teacher quality; (b) promoting cooperation and collaboration; (c) goal setting and performance accountability; (d) enhancing financial and human resource management, (e) outreach beyond school boarders; (f) focus on the instructional function; and (g) education of boys. Leadership learning would be based on leadership standards and grounded in practice, including analysis of classroom practice; mentoring and peer coaching; and collegial learning networks that offer communities of practice and ongoing sources of support. 13. The development of leadership capacity will be done through the following activities: (a) development of a program for training of school leaders focused on leadership practices based on agreed professional standards, including instructional leadership, and to support teachers to improve instruction; (b) competence-based training and certification of school leaders based on agreed OECS leadership standards, including on instructional leadership and use of data, planning, management, and reporting; and (c) development of a handbook for school principals/leaders to guide school leadership and management. 14. Component 4: Project implementation, monitoring, and evaluation. This component will support project implementation and M&E through (a) the procurement and FM requirements of the project; (b) process monitoring of the teacher and school leader professional development activities; (c) the establishment of the national routines for ongoing and regular monitoring of learning standards and assessment using available data and additional research; (d) the monitoring of teacher classroom practices through CLASS; and (e) the carrying out of annual regional and country-level program reviews. 15. This component will support project implementation and administration and ensure appropriate M&E of project implementation and impact, as well as of fiduciary aspects. No separate project-specific implementation unit will be established at either the OECS Commission or country level. Rather, the head of the EDMU at the OECS Commission will be the de facto project manager working closely with the chief education officers to make key policy and strategic decisions. To support the day-to-day implementation of these decisions, they will be supported by the fiduciary staff of the OECS Commission 16. Classroom practices will be monitored using CLASS—a rigorous monitoring system that can be used to provide feedback on the quality of teaching practices. Research has demonstrated that CLASS objectively identifies and assesses multiple dimensions of instructional quality that are linked to student achievement across a range of grades (pre-primary to secondary), content, and country contexts. Without improving instructional interactions between teachers and students, other resources such as qualified teachers or costly curricular materials are not likely to 40 have an impact on student learning (Cohen, Raudenbush, and Ball 2003).29 A growing body of evidence indicates that the effectiveness of teachers’ instructional practice has an impact on student learning (Tomlinson et al. 2006).30 Instructional strategies such as effective use of targeted direct instruction, modeling, and two-way feedback among students and teachers are associated with improved student achievement. The quality of the instruction teachers deliver has a striking impact on student achievement when instruction is differentiated to accommodate students’ learning styles, backgrounds, perspectives, and cultural identities while maintaining the cognitive demand of instructional tasks. Moreover, CLASS-based professional development has demonstrated that teachers can improve their teacher-student interactions leading to gains in student outcomes through providing reliable data on teacher classroom practices and a common language that identifies dimensions of effective teaching practices across subject areas and grade levels. 17. CLASS focuses on three domains of instructional interactions:  Emotional support  Classroom organization  Instructional support 18. Within each of these three domains, there are dimensions of the quality of instructional interactions rated on a 7-point scale (Low 1, 2; Mid 3, 4, 5 (effective); High 6, 7) from Pre-K to 12th grade. Emotional Support (a) Positive climate. Reflects the emotional connection between the teacher and students and among students and the warmth, respect, and enjoyment communicated by verbal and nonverbal interactions. (b) Negative climate. Reflects the overall level of expressed negativity in the classroom; the frequency, quality, and intensity of teacher and peer negativity are key to this scale. (c) Teacher sensitivity. Encompasses the teacher’s awareness of and responsiveness to students’ academic and emotional needs; high levels of sensitivity facilitate students’ ability to actively explore and learn because the teacher consistently provides comfort, reassurance, and encouragement. (d) Regard for student perspective. Captures the degree to which the teacher’s interactions with students and classroom activities place an emphasis on students’ interests, motivations, and points of view and encourage student responsibility and 29 Cohen D. K., S. W. Raudenbush, D. L. Ball. 2003. “Resources, Instruction, and Research.� Educational Evaluation and Policy Analysis. 30 Tomlinson, Carol Ann, and Jay McTighe. 2006. Integrating Differentiated Instruction and Understanding by Design: Connected Content and Kids. Alexandria, VA: Association for Supervision and Curriculum Development 41 autonomy. Classroom Organization (e) Behavior management. Encompasses the teacher’s ability to provide clear behavioral expectations and use effective methods to prevent and redirect misbehavior. (f) Productivity. Considers how well the teacher manages instructional time and routines and provides activities for students so that they have the opportunity to be involved in learning activities. (g) Instructional learning formats. Focuses on the way in which the teacher maximizes students’ interest, engagement, and ability to learn from lessons and activities. Instructional Support (h) Concept development (Grades K–3). Measures the teacher’s use of instructional discussions and activities to promote students’ higher-order thinking skills and cognition and the teacher’s focus on understanding rather than on rote instruction. (i) Content understanding (Grades 4–12). The depth of the lesson content and the approaches used to help students comprehend the framework, key ideas, and procedures in an academic discipline. At a high level, this refers to interactions among the teacher and students that lead to an integrated understanding of facts, skills, concepts, and principles. (j) Quality of feedback. Assesses the degree to which the teacher provides feedback that expands learning and understanding and encourages continued participation. (k) Analysis and inquiry (Grades 4–12). Assesses the degree to which students are engaged in higher-level thinking skills through the application of knowledge and skills to novel and/or open-ended problems, tasks, and questions. Opportunities for engaging in metacognition, that is, thinking about thinking are also included. (l) Language modeling (Grades K–3). Captures the quality and amount of the teacher’s use of language-stimulation and language-facilitation techniques. (m) Instructional dialogue (Grades–12). Captures the purposeful use of content-focused discussion among teachers and students that is cumulative, with the teacher supporting students to chain ideas together in ways that lead to deeper understanding of content. Students take an active role in these dialogues and both the teacher and students use strategies that facilitate extended dialogue. 19. CLASS is particularly relevant to the OECS for two reasons. First, there is currently no system in place to measure teacher-child interactions, and therefore, data are limited to fully evaluate the classroom environment and identify best practices. Second, CLASS will shed 42 considerable insight into the effectiveness of the teacher-training program and inform teacher professional development. 20. Capacity building for CLASS has been undertaken using the GPE grant as part of the Quality Education Framework. The capacity building covers the following areas: (a) training and certification of CLASS observers who will monitor classroom practices; (b) training of trainers; and (c) training of school leaders and instructional coaches to implement professional learning communities with coaching to improve instruction. Table 2.1 provides a summary of activities to be implemented at the regional and country levels. Regional-Level Activities to be led by the Outcome National-Level Activities OECS Commission Curriculum and (a) Review the harmonized curriculum to (a) Adapt learning standards to assessment clarify grade-level learning standards country level (b) Develop learning standards framework (b) Implement learning standards and (c) Develop a learner-centered assessment learner-centered assessment framework Teacher (a) Review teacher training programs (a) Country-level consultations and professional (b) Review classroom practices to understand promotion of improved teaching development current practices and identify gaps practices (c) Develop professional development (b) Competency-based professional program for teachers development for teachers on (d) Develop community of practice for effective classroom practices, the teachers use of formative classroom assessments, and the use of data to improve teaching Strengthening (a) Develop a competency-based program for (a) Competency-based training and leadership training and certification of school leaders certification of school leaders based on leadership standards, to support based on leadership standards to teachers to improve classroom practices support teachers to improve and use data to improve instruction classroom practices and use of (b) Develop a handbook for school leaders student learning data to improve (c) Monitor and evaluate implementation and instruction promotion of knowledge sharing across (b) Monitoring of classroom practices countries 43 Annex 3: Implementation Arrangements OECS COUNTRIES: Support to Implementation of the Regional Education Strategy Project Administration 1. The OECS Commission will be the implementing agency for the project. Implementation at the regional level will be coordinated by the EDMU of the OECS Commission. The head of the EDMU will be the project coordinator and responsible for the day-to-day management of the project. The EDMU will also be the main focal point for monitoring the implementation of the regional activities, including the collection and reporting on the results. It would organize regular fora for reviewing progress and sharing lessons learned from the overall program, share best practices, and promote knowledge exchange between countries. The fiduciary responsibilities would be managed by the respective units at the OECS Commission. The OECS Commission managed the Institutional Development Grant, which supported development of the Regional Education Strategy and has implemented other Bank-financed projects. 2. The Chief Education Officers and Planners’ Forum will act as the Project Steering Committee. This committee will offer advice and guidance on the project. Institutional and Implementation Arrangements at the National Level 3. As part of the regional arrangements for implementing the OESS, the program implementation at the national level will be mainstreamed into the MOE. This model is in response to previous experience where projects were perceived as competing priorities outside the scope of the work program of the MOE. The national-level counterpart will be the chief education officer who may designate a national focal point. The actual implementation will be managed by the units responsible for the various program components. For example, the curriculum and assessment components will be implemented by the Curriculum Unit with the head of the Curriculum Unit having direct responsibility for related activities. Similarly education officers with responsibility for teacher training and school leadership—usually a senior education officer—will be responsible for activities related to their respective portfolios. Implementation calls for a high level of coordination between various units and education officers. This coordination will be managed by the chief education officer or a delegated focal point. The focal point will coordinate closely with all relevant stakeholders and beneficiaries, and organize national consultation meetings of various aspects of the project. 4. The potential risk posed by shortage of skilled staff is lessened by the proposed approach to implementation resulting in the spread of tasks and responsibilities across several units and individuals. The proposed project activities will be included in the unit work plan that incorporates the program activities. This approach continues to work well for the regional activities that are agreed upon by the countries—since there is buy-in from the highest level of the ministry. 44 Financial Management FM Assessment and Mitigating Actions (if any) 5. Conclusion. The Bank performed an FM assessment of this proposed project in accordance with OP/BP 10.00 and the Financial Management Practice Manual (issued by the Financial Management Sector Board on March 1, 2010). In the FM specialist’s opinion, the implementing agency (the OECS Commission) has in place an FM system that will be able to provide, with reasonable assurance, accurate and timely information on the status of the funds as required by the Bank. 6. Risk assessment. The overall FM risk rating assigned to this project is Low. This is because the OECS Commission is an existing implementing agency and the grant will mainly cover consultancy-based activities, which will be monitored closely by the task team. The FM specialist will monitor the project FM risk during project implementation. The following risks with corresponding mitigation measures have been identified during the assessment. Risk Risk Risk Issues/Measures Rating Inherent risk Country level Moderate Entity and project level Low FM arrangements are in place and functioning adequately. Control risk A budget for the entire life of the project will be prepared with Budgeting Low assistance from the Bank task team. Internal controls Moderate – Accounting/financial Low – reporting Funds flow/disbursement Low – Staff has limited experience with Bank projects; however, they have a Staffing Low vast amount of experience with implementing other donor projects. External audits Low One audit will be performed covering the life of the project. 7. Implementing agency. The OECS Commission based in St. Lucia has the responsibility for the implementation and FM of this grant. The fiduciary arrangements will be intertwined in the current structure of the OECS Commission and its Finance Department, which has ample experience with implementing donor-funded projects. 8. Staffing. The Finance Department will have direct oversight of the project’s accounting. The department comprises four persons (financial controller, senior project accountant, and two accounting assistants) who are assigned to various functions and reporting relationships to ensure that there is proper segregation of duties as well as appropriate review of outputs. 9. Budgeting. A budget for all the activities of the grant for the entire implementation period will be prepared at the beginning of the grant by the technical team at the OECS Commission and the Bank. The budget will be revised each fiscal year and on an ad hoc basis, 45 based on implementation progress. The annual work plan will be derived from this master budget and included in Sage 50 (the OECS Commission’s accounting system). 10. Internal controls. The OECS Commission’s Financial Regulations and Rules of the Organization of Eastern Caribbean States (2015) has a comprehensive guidance on the daily accounting procedures. Currently the Internal Audit Unit has no staff due to recent resignations and the OECS Commission has indicated that the two posts should be filled within the next six months. However, the procedures and controls seem to be functioning effectively and efficiently. The grant will be guided by the abovementioned manual and inclusions of project-specific processes will be done as soon as the project becomes effective. 11. Funds flow and disbursement arrangements. Disbursement of project funds will be based on IFRs. Funds should only be used for implementation of the components as set out in the Financing Agreement and all disbursement must comply with the disbursement categories. A segregated U.S. dollar designated account and local currency project account will be opened upon effectiveness of the project at First National Bank in St. Lucia and the bank account and bank information will be conveyed to the Bank. The operation of the bank accounts will fall under the current operations processes of the OECS Commission. 12. Accounting and financial reporting. The Finance Department in the OECS Commission will be responsible for producing the IFRs on a semester (six-month) basis (July– December and January–June) and submitted no later than 45 days after the end of each reporting period to the Bank. These reports will provide required monitoring information and should be made available to both the internal and external auditors. Accounting records will be kept in the accounting system, Sage 50, and will be updated and monitored in accordance with the financial regulations manual of the OECS Commission. The current Operations Manual is very comprehensive and the processes and procedures once applied consistently are adequate to ensure that the financial information is accurate and funds are only utilized for project purposes. 13. External auditing. Two audits will be performed to cover the life of the project (three years) as the project’s FM risk is low and activities are not complex and will be closely monitored by the Bank’s task team. Each audit will cover 1.5 years of the 3-year life of the project. The project’s audit reports should be submitted to the Bank for review no later than six months following the end of each audit period and will be performed by the same auditor that audits the OECS Commission at that time. The OECS Commission carries out a tendering process to select the auditor every three years and that selection process is adequate. The fiscal year of the OECS Commission is July to June. Procurement General 14. Procurement for the proposed Recipient Executed Trust Fund grant will be carried out in accordance with the Bank’s ‘Guidelines: Procurement of Goods, Works, and Non-Consulting Services under IBRD Loans and IDA Credits and Grants’, dated January 2011 and revised July 2014; ‘Guidelines: Selection and Employment of Consultants under IBRD Loans and IDA Credits and Grants by World Bank Borrowers’, dated January 2011 and revised July 2014; the 46 IDA Anticorruption Guidelines dated July 1, 2005, and as amended October 15, 2006; and the provisions stipulated in the Financial Agreement. The various procurement actions under different expenditure categories are described in general below. For each contract to be financed under the Financial Agreement, the various procurement or consultant selection methods, the estimated costs, prior/post review requirements, and time frame have been agreed between the recipient and the Bank in the Procurement Plan. The Procurement Plan will be updated at least annually or as required to reflect the actual project implementation needs and improvements in institutional capacity. Procurement Arrangements 15. Procurement of goods and non-consulting services. Procurement of goods and services other than consulting services would include printing services in support of the project implementation and training materials. Procurement of non-consulting services will be carried out using International Competitive Bidding (ICB), National Competitive Bidding (NCB), Shopping and other methods indicated in the Financing Agreement. The procurement will be carried out using the Bank’s Standard Bidding Documents and other sample documents and templates, all agreed upon with the Bank. The procurement methods thresholds and prior review thresholds for works are indicated in Table 3.2, Error! Reference source not found., and Error! Reference source not found.. Domestic preferences in accordance with clause 2.55 and ppendix 2 of the guidelines will not apply. 16. Selection of consultants. Consultants’ services contracts procured under this grant will include consultancy to strengthen curriculum standards and assessment framework; consultancy to improve teacher training programs and strengthen teaching practice; consultancy to strengthen leadership and accountability of school principals; and M&E project consultancy; and so on. The following selection methods will be used: Quality- and Cost-Based Selection (QCBS), Least-Cost Selection (LCS), Selection Based on the Consultants’ Qualifications (CQS), Individual Consultants, and other selection methods indicated in the Financing Agreement. The selections will be done using the Bank’s standard Request for Proposal and other sample documents and templates, all agreed upon with the Bank. Short lists of consultants for services estimated to cost less than US$300,000 equivalent per contract may be composed entirely of national consultants in accordance with the provisions of paragraph 2.7 of the Consultant Guidelines. 17. Operating costs. This refers to incremental operating costs incurred by the OECS Commission on account of project implementation, management, and monitoring, including dissemination of project-related information and publications, office rent and utilities, office and equipment insurance, maintenance and repair, vehicle insurance, local travel, communication, translation and interpretation, bank charges, and other miscellaneous costs directly associated with the project. All operating costs are based on periodic budgets and procured using the implementing agencies’ administrative procedures acceptable to the Bank. Operating costs do not include salaries of government officials and civil servants. 18. Training costs. The grant will finance training (workshops, and so on), as needed. The training will be carried out according to training plans, which the OECS Commission will revise semiannually and as needed and submit to the Bank for no objection prior to implementation. 47 The expenses will be covered under the training category and disbursed based on the statement of expenses. 19. Procurement methods thresholds and prior review thresholds. The following procurement methods thresholds and prior review thresholds will be used: Expenditure Category Contract Value Procurement Method Contracts Subject to (Threshold) Prior Review US$, thousands 1. Works > 3,000 ICB All < 3,000 NCB First < 200 Shopping None > 100 Direct Contracting All < 100 All justifications 2. Goods > 500 ICB All < 500 NCB First < 100 Shopping None > 100 Direct Contracting All < 100 All justifications 3. Consulting Services 3.1 Firms > 300 QCBS, QBS, FBS, LCS All Contracts < 300* QCBS, QBS, FBS, LCS, First Contract and all CQS terms of reference by task team leader > 100 Single Source All < 100 All justifications 3.2 Individuals > 300 Comparison of 3 CVs All < 300 All terms of reference by task team leader Note: FBS = Selection under a Fixed Budget. * Short list may comprise only national consultants. 20. Frequency of procurement supervision. Supervision of procurement will be carried out primarily through post review done during supervision missions at least twice a year. Procurement Capacity Assessment Assessment of the Agency’s Capacity to Implement Procurement 21. An assessment of the capacity of the implementing agency (OECS Secretariat) to implement procurement actions for the project was carried out in February 2016. 48 22. The procurement will be undertaken by the EDMU within the OECS Secretariat with the support of the procurement specialist in the Functional Cooperation and Project Management Unit of the OECS Secretariat with prior experience in implementing Bank-financed projects and other donor-funded projects to work on this project procurement. The EDMU has experience in carrying out consulting procurement processes in line with Bank Guidelines. 23. The key issues/risks concerning procurement for implementation of the project have been identified during the assessment, as follows: (a) No separate project-specific implementation unit would be established either at the OECS Commission or country level. (b) It is not envisaged at this point to hire a dedicated procurement officer to the unit to directly support this project, although a procurement officer who is currently supporting a U.S. Agency for International Development (USAID) project will also be providing support. (c) The above points, (a) and (b), will be reviewed during implementation supervisions and can be changed if deemed necessary. 24. The overall project risk for procurement is Moderate. Project M&E 25. The details of the project M&E are included in annex 1. Role of Partners 26. The development partners play a key technical and financial role in the project. The GPE technical staff have worked closely with the Bank team during the preparation process and have provided comments at various stages. The development partner group, especially the United Nations Children’s Fund (UNICEF), has been working closely with the Bank team, organizing joint activities (for example, the work on school leadership). The local education group participated in reviewing and appraising the Education Sector Plan, which is the core document around which the project has been designed. Several development partners, in particular, Caribbean Development Bank, UNICEF, USAID, and United Nations Educational, Scientific, and Cultural Organization (UNESCO) are supporting specific actions within the Education Sector Plan. The project will be coordinated by the OECS Commission at the regional level and this will ensure coordination with other partners in implementation, M&E, and supervision arrangements for the project. 49 Annex 4: Economic Analysis OECS COUNTRIES: Support to Implementation of the Regional Education Strategy 1. It is expected that all four PDOs provide substantial economic benefits through their contribution to improved educational quality. The analysis estimates the economic benefits accruing from the first three PDOs: (a) use quality learning standards to support evidence-based teaching and learning at the primary level; (b) improve teacher practices at the primary level; and (c) strengthen primary school leadership and accountability. The benefits from the fourth PDO are more difficult to predict and quantify and are therefore not included in the economic analysis. The fourth PDO is (d) initiate the strengthening of sector M&E capacity in support of evidence-based strategic management and decision making in all member countries. 2. The analysis estimates the total improvement in education quality due to the project activities for participating children. These estimates were based on the results of rigorous impact evaluation of similar programs on student learning. The analysis then calculates the monetary benefit of this improved education quality in terms of gains in lifetime income. This final calculation considered the results from Hanushek et al. (2015)31 that outlined that effects of educational programs lead to higher returns to skills that affect the wages of beneficiaries. 3. The quantitative economic analysis shows a net benefit and positive return for the GPE Project. With a 10 percent discount rate, the project has an NPV of US$11.1 million with an internal rate of return of 27.0 percent. In addition, a sensitivity analysis to consider an alternative worsening scenario, in which the effects of activities on students’ performance are assumed to be only half as large, was carried out. Even in this alternative case, the project appears to be a good investment from an economic point of view, with an NPV of US$5.5 million and an internal rate of return of 20.2 percent. 4. Low quality of education has been highlighted, by the Caribbean Growth Forum, as the major constraint to growth, employment, and poverty reduction. It is expected that as learning achievement improves, more people will enter the labor market and the economic benefits of the workforce affected by the project are expected to improve and more people will move out of poverty. The PDOs are related to establish better learning standards, improve teacher effectiveness in the classroom at the primary level, and strengthen school leadership and accountability. The economic benefits of the project are expected due to the impact of better schools on learning achievement of the students (Hanushek and Woessmann 2012).32 5. Public sector provision is justified based on the high social returns of the investment and the reduction in poverty and inequality associated with it. Education in the OECS is predominantly public and the project aims to support prerequisites to delivery of quality education for all. Moreover, the project aims to support key aspects necessary to ensure the provision of a public education service of quality. In addition, other factors provide an economic rationale for public sector financing of primary education. For example, individuals acquiring 31 Hanushek, Eric A., Guido Schwerdt, Simon Wiederhold, and Ludger Woessmann. 2015. “Returns to Skills around the World: Evidence from PIAAC.� European Economic Review 73: 103–130. 32 Hanushek, E. A., and L. Woessmann. 2012. “Do Better Schools Lead to More Growth? Cognitive Skills, Economic Outcomes, and Causation.� Journal of Economic Growth 17 (4): 267–321. 50 better education contribute to the society because they are likely more engaged and responsible citizens and better parents. This also increases the overall level of productivity and growth in the economy. 6. The Bank has global experience in supporting government efforts to improve teacher pedagogical practices and professional development, capacity for school leadership, and monitoring student learning. It provides added value in this project through its technical expertise and financing. The Bank has contributed and will contribute to develop the government’s capacity under each component that supports the achievement of all three PDOs. 51