Document of The World Bank FOR OFFICIAL USE ONLY Report No: PAD1138 INTERNATIONAL DEVELOPMENT ASSOCIATION PROGRAM APPRAISAL DOCUMENT ON A PROPOSED CREDIT IN THE AMOUNT OF SDR 67.8 MILLION (US$95 MILLION EQUIVALENT) TO THE SOCIALIST REPUBLIC OF VIETNAM FOR THE ENHANCING TEACHER EDUCATION PROGRAM PROGRAM–FOR–RESULTS June 10, 2016 Education Global Practice East Asia and Pacific Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.   CURRENCY EQUIVALENTS (Exchange Rate Effective May 31, 2016) US$1 = 22,390 VND SDR1 = US$1.40288 ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank LTTU Lead Teacher Training University BOET Bureau of Education and Training M&E Monitoring and Evaluation Teacher Continuous Professional CPD Development MDGs Millennium Development Goals Ministry of Education and CPS Country Partnership Strategy MOET Training Selection Based on Consultants' CQS Qualifications MOF Ministry of Finance Ministry of Labor, Invalids and CTPA Core Teachers and Principal Advisor MOLISA Social Affairs Direct Contracting/Single Source Ministry of Natural Resources and DC/SSS Selection MONRE Environment Ministry of Planning and DLIs Disbursement-linked Indicators MPI Investment Department of Education for Ethnic DOEEM Minorities NCB National Competitive Bidding National Institute for Education DOET Department of Education and Training NIEM Management Department of Natural Resources and DONRE Environment NPV Net Present Value National Teacher Education DPI Department of Planning and Finance NTEP Program Department of Teacher Education and DTEM Management ODA Official Development Assistance ECE Early Childhood Education ODL Open Distance Learning Organization for Economic ECOP Environmental Codes of Practice OECD Cooperation and Development e-GP Electronic Government Procurement OM Operations Manual EHS Environment, Health, and Safety OMS Online Management System EIA Environmental Impact Assessment PA Performance Agreement EM Ethnic Minority PAD Program Appraisal Document Education Management Information EMIS System PAP Program Action Plan EPP Environmental Protection Plan PDO Program Development Objective Environmental and Social Systems ESSA Assessment PforR Program for Results Program for International Student ETEP Enhancing Teacher Education Program PISA Assessment FA Financing Agreement PMU Program Management Unit FBS Fixed-Budget Selection PP Procurement Plan Fundamental and Comprehensive FCER Education Reform PPC Provincial People's Committee FM Financial Management PPP Private Public Partnership FSA Fiduciary Systems Assessment QBS Quality-Based Selection FSQL Fundamental School Quality Level QCBS Quality and Cost-Based Selection Renovation of General Education GDP Gross Domestic Product RGEP Project GOV Government of Vietnam RTTI Readiness to Teach Index GPE Global Partnership of Education SA Social Assessment GRS Grievance Redress Service SBV State Bank of Vietnam International Bank for Reconstruction Social Economic Development IBRD and Development SEDP Plan Social Economic Development ICB International Competitive Bidding SEDS Strategy Information, Communication and School Education Quality ICT Technology SEQAP Assurance Program IDA International Development Association SES Socio-Economic Status Intergenerational Deaf Education IDEO Outreach Project SOE Statement of Expenditures Systematic Operations Risk Rating IIP Institutional Improvement Plans SORT Tool School Readiness Promotion INT Integrity Vice Presidency SRPP Project IO Investment Owner TA Technical Assistance International Public Sector Accounting Teacher Education Institution IPSAS Standards TEIDI Development Index Teacher Education Improvement ISAs International Standards on Auditing TEIP Project Teacher Education Management ISM Implementation Supervision Mission TEMIS Information System IT Information Technology TGE Total Government Expenditure ITE Innovative Teaching Experience TPP Trans-Pacific Partnership IVA Independent Verification Agency TTI Teacher Training Institution LCS Least-Cost Selection TTU Teacher Training University United Nations Educational Scientific and Cultural LMS Learning Management System UNESCO Organization Regional Vice President: Victoria Kwakwa Global Practice Vice President: Keith E. Hansen Acting Country Director: Achim Fock Practice Manager: Harry Anthony Patrinos Task Team Leader: Michel J. Welmond SOCIALIST REPUBLIC OF VIETNAM Enhancing Teacher Education Program Table of Contents Page I. STRATEGIC CONTEXT .............................................................................................................. 1 A. Country Context ................................................................................................................................ 1 B. Sectoral and Institutional Context ..................................................................................................... 2 C. Relationship to the Country Partnership Strategy (CPS) and Rationale for Use of Program for Results Financing ...................................................................................................................................... 6 II. PROGRAM DESCRIPTION ......................................................................................................... 9 A. Program Scope .................................................................................................................................. 9 B. Program Development Objective/s (PDO) and key results............................................................. 18 III. PROGRAM IMPLEMENTATION ............................................................................................ 20 A. Results Monitoring and Evaluation................................................................................................. 22 B. Disbursement Arrangements ........................................................................................................... 22 IV. ASSESSMENT SUMMARY ........................................................................................................ 24 A. Technical (including program economic evaluation) ..................................................................... 24 B. Fiduciary ......................................................................................................................................... 27 C. Environmental and Social Effects ................................................................................................... 30 D. Risk Assessment ............................................................................................................................. 31 E. Program Action Plan ....................................................................................................................... 32 ANNEXES Annex 1: Detailed Program Description ..................................................................................................... 34 Annex 2: Results Framework Matrix .......................................................................................................... 52 Annex 3: Disbursement-linked Indicators, Disbursement Arrangements and Verification Protocols........ 56 Annex 4: (Summary) Technical Assessment .............................................................................................. 72 Annex 5: (Summary) Fiduciary Systems Assessment ................................................................................ 88 Annex 6: Summary Environmental and Social Systems Assessment ........................................................ 94 Annex 7: Systematic Operations Risk Rating (SORT) ............................................................................... 99 Annex 8: Program Action Plan ................................................................................................................. 100 Annex 9: Implementation Support Plan .................................................................................................... 103 PAD DATA SHEET . Vietnam Enhancing Teacher Education Program . PROGRAM APPRAISAL DOCUMENT . East Asia and Pacific Region Education Global Practice . Basic Information Date: June 10, 2016 Sectors: General Education Sector (100%) Acting Country Achim Fock Themes: Education for the knowledge Director: Economy (50%) Education for all (50%) Practice Harry Anthony Patrinos/ Manager/Senior Claudia Maria Costin Global Practice Director: Program ID: P150060 Team Leader(s): Michel J. Welmond Program Implementation Period: 5 years Start Date: January 01, 2017 End Date: June 30, 2022 Expected Financing Effectiveness Date: October 1, 2016 Expected Financing Closing Date: June 30, 2022 . Program Financing Data [ ] Loan [ ] Grant [ ] Other [X] Credit For Loans/Credits/Others (US$M): Total Program 255 Total Bank 95 Cost : Financing : Total Financing Gap : 160 Cofinancing : Financing Source Amount BORROWER/RECIPIENT 160.00 IDA 95.00 IDA: New Total 255.00 . Borrower: Socialist Republic of Vietnam Responsible Agency: Ministry of Education and Training Contact: Mr. Nguyen Vinh Hien Title: Vice Minister Telephone No.: 84-4-3869 5144 Email: nvhien@moet.edu.vn Contact: Mr. Nguyen Ngoc Vu Title: Director General, Planning and Finance Department, MOET Telephone No.: 84-4-3869 2191 Email: nnvu@moet.edu.vn . Expected Disbursements (in USD Million) Fiscal Year 2017 2018 2019 2020 2021 2022 Annual 7.00 18.60 20.20 23.80 25.40 0 Cumulative 7.00 25.60 45.80 69.60 95.00 95.00 . Program Development Objective(s) To strengthen teacher education institutions to enhance teacher and principal effectiveness through improved continuous professional development. Compliance Policy Does the program depart from the CAS in content or in other significant respects? Yes [ ] No [ X ] . Does the program require any waivers of Bank policies applicable to Program-for- Yes [ ] No [ X ] Results operations? Have these been approved by Bank management? Yes [ ] No [ ] Is approval for any policy waiver sought from the Board? Yes [ ] No [ X ] Does the program meet the Regional criteria for readiness for implementation? Yes [ X ] No [ ] Overall Risk Rating: Substantial Legal Covenants Name Recurrent Due Date Frequency Institutional Arrangements (FA Reference, X Schedule 2, Section I.C) Description of Covenant: the Recipient shall maintain at all times during Program implementation adequate institutional arrangements - satisfactory to the Association as necessary to achieve the objectives of the Program - as further detailed in the Operations Manual, including the Program Management Unit, which shall be maintained at the Ministry of Education and Training, and shall be responsible for inter alia day to day management and coordination of the Program. Name Recurrent Due Date Frequency Operations Manual (FA Reference, Before effectiveness Schedule 2, Section I.D) Description of Covenant: The Recipient shall, through Ministry of Education and Training, prepare an operations manual in form and substance satisfactory to the Association, and thereafter, ensure that the Program is carried out in accordance with the arrangements and procedures set out in said manual. Name Recurrent Due Date Frequency Verification Arrangements (FA Reference, 6 months after Schedule 2, Section I.E) effectiveness Description of Covenant: The Recipient, through Ministry of Education and Training, shall: (a) not later than six months after the Effective Date, recruit an independent auditor, with experience and capacity, and under terms of reference satisfactory to the Association, and (b) thereafter cause said verification agent to: (i) carry out the verification of compliance of the DLIs/DLRs which are set forth in the table in Section IV.A.2 of this Schedule and Schedule 4 to this Agreement; and (ii) not later than thirty days after the verification of compliance of said DLIs/DLRs has been completed, prepare and furnish to the Recipient and the Association, a report on the results of said verification of compliance process of such scope and in such detail as the Association shall reasonably request. Name Recurrent Due Date Frequency Program Action Plan (FA Reference, X Schedule 2, Section I.F) Description of Covenant: The Recipient shall undertake the actions set forth in the Program Action Plan. Name Recurrent Due Date Frequency Mid-term review (FA Reference, Schedule October 31, 2019 2, Section III.C) Description of Covenant: The Recipient shall: (a) carry out jointly with the Association, by no later than October 31, 2019, a midterm review to assess the overall progress in implementation of the Program; (b) prepare and furnish to the Association, at least three (3) months prior to such review, a report, in scope and detail acceptable to the Association, needed to undertake the review, and (c) review, jointly with the Association, the report referred to in subparagraph (b) above and thereafter take all measures required to ensure the efficient completion of the Program and the achievement of the objectives thereof, based on the conclusions and recommendations of said report and the Association’s views on the matter.         VIETNAM Enhancing Teacher Education Program I. STRATEGIC CONTEXT A. Country Context 1. Vietnam has experienced rapid and inclusive economic growth since the early 1990s. Economic renovation (Doi Moi) reforms launched in 1986 have transformed it from one of the world’s poorest countries to a lower-middle-income one—with per capita gross national income (GNI) of US$1,980 in 2015. Growth has also been inclusive: incomes have risen across the income distribution, with only modest increases in inequality. The percentage of people living in extreme poverty dropped from 50 percent in the 1993 to less than three percent today. Key social indicators and access to basic infrastructure have also improved substantially. 2. Nonetheless, significant challenges remain. Although over the long term ethnic minorities have seen notable improvements in welfare, large gaps between ethnic minorities and the majority population remain. Remaining poverty is increasingly concentrated among ethnic minorities, who now constitute 14 percent of the population but 60 percent of the poor. In 2014 the poverty rate was 57.8 percent among ethnic minorities, compared to 6.3 percent among the Kinh and Hoa ethnic majority. The most recent data show worrying signs that progress on ethnic minority welfare has stalled. The infant mortality rate for ethnic minorities increased in the most recent data and is now four times the rate among the majority. Rates of child malnutrition among ethnic minorities are 32 percent and have declined only slightly in recent years. 3. Environmental degradation coupled with the impact of global climate change threaten the long-term sustainability of the country’s development. In addition, despite major infrastructure spending needs, Vietnam faces shrinking financing options on account of growing fiscal pressures and insufficient private sector participation. Furthermore, the institutions of governance that were adequate to carry the country to its lower-middle-income level are now exposing gaps that need to be addressed with urgency and boldness, especially as pressures emerge from economic integration, urbanization, an aging population, and aspirations of a rising middle class. 4. Vietnam has also achieved impressive outcomes on gender equality. Gender differences in school enrollment and attainment are minimal, and the gender wage gap is modest. But women remain outliers in private- and public sector leadership positions. There are sharp gender differences in business and particularly in government and political spheres, where the leadership is overwhelmingly male. In the last decade and a half, the share of women in the National Assembly has declined and is now at 24.4 percent. Few chairs of National Assembly committees are female. The civil service has a large share of women, but their representation in leadership positions is low, mostly at lower levels.1                                                              1 For ethnic minorities in general, and for ethnic minority women in particular, access to leadership positions (and the civil service, for that matter), is significantly lower than for Kinh and Hoa women. 1   5. These mentioned challenges are well understood by the Government, as reflected in the recent Vietnam 2035 report it jointly prepared with the World Bank Group.2 B. Sectoral and Institutional Context 6. Achievements and Challenges. Vietnam has always emphasized education as a keystone to its development policy. As one sign of this commitment, it allocates nearly 20 percent of public expenditures to education, significantly above the Organization for Economic Co-operation and Development (OECD) average of 12.9 percent.3 A national network of schools, colleges and universities has expanded dramatically improving access to education at all levels. 7. The country has essentially achieved universal primary education. It is now moving towards universal preschool education for five-year-old children and universal lower secondary education. Between 2001 and 2012, net enrollment rates at lower and upper secondary levels increased from 70 percent to 85 percent and 33 percent to 60 percent respectively.4 8. Vietnam has also made considerable progress in creating better learning conditions in schools. The Ministry of Education and Training (MOET) has developed the “Fundamental School Quality Level (FSQL)” in 2006, which establishes minimum standards for physical facilities, school organization and management, teaching materials and teacher support, and school-parent linkages for primary students. Full-day schooling for primary and lower secondary education has gradually expanded to increase learning hours. 9. Vietnam has also attained higher levels of student learning achievement. Its performance on the 2012 Program for International Student Assessment (PISA)5 surpassed the OECD country.6 10. Vietnam has made considerable progress over the last two decades with regards to the access to education by disadvantaged populations. Almost all ethnic minority children and girls enrolled in and attending primary schools,7 and minor difference in enrollment rates for lower secondary education for ethnic minorities and none with regards to gender. In terms of learning, gaps exist but have narrowed between children from majority groups and those from disadvantaged ethnic minorities. Over the last decade, ethnic minority children have registered impressive gains in mathematics, reading, and particularly Vietnamese language. However, gaps persist, in part because all populations have done better over recent years.                                                              2 Vietnam 2035: Toward Prosperity, Creativity, Equity, and Democracy. Overview Report. World Bank and the Ministry of Planning and Investment of Vietnam. February 2016 3 Education at a Glance, OECD. 2011. Page 248. 4 United Nations Education Science and Culture Organization (UNESCO). 2015. Vietnam: Education for All 2015 National Review. 5 PISA (Program for International Student Assessment) is a worldwide study by OECD in member and non-member nations of 15-year-old school pupil scholastic performance in mathematics, science and reading. Out of 65 countries, Vietnam placed 17th on math, 19th place on reading (19th place), and 8th place on science. 6 OECD. 2012. PISA 2012, Result in focus retrieved from www.oecd.org/pisa/.../pisa-2012-results-overview.pdf. 7 The remaining primary and lower secondary enrollment gaps between disadvantaged ethnic minorities and other groups are expected to close soon. The overall enrollment rate for ethnic minority children is 89 percent (steadily rising by one percent per year) but then enrollment rate for 7-9 year olds is 100 percent. With minimal primary and lower secondary school dropout rates, we can expect the rates to converge in the next 3-4 years. 2   11. While achievements are impressive, Vietnamese authorities are concerned that the education system is not well positioned for the age of globalization and international competition because it may not provide the population with the skills and competencies needed for producing greater value in tomorrow’s economy.8 Addressing these challenges is the next frontier of education development in Vietnam. Fully recognizing the importance of these issues, the 2011- 2015 Social Economic Development Plan (SEDP) states that the “quick development of high quality human resources is essential for the country’s industrialization, modernization, and the development of a knowledge-based economy.”9 The 2015-2020 SEDP confirmed this finding. 12. There is also unfinished business and persistent challenges. The access profile for ethnic minorities changes dramatically in upper secondary school. From the age of 14 (the last year of lower secondary education), ethnic minority enrollment declines, dropping from 85 percent enrollment in lower secondary school to 60 percent in upper secondary. Viewed otherwise, only a third of the poorest quintile of the student population makes this transition. This drop-off has remained steady over time and constitutes a major brake for access of ethnic minorities in higher education. 13. Vietnam has largely achieved gender equality in general and higher education, with the exception of ethnic minority girls. However, gender gaps continue, but in different forms. Whereas more women enroll in higher education than men, women are less likely to study sciences, engineering, and other technology-oriented fields. In addition, women tend to be underrepresented in management and leadership positions in the education sector. As is the case in many countries, there are substantially more female than male general education teachers, but far fewer female school directors, subject specialists, district/provincial/national administrators, or university faculty members. This means that women have far less influence on shaping education policy, teaching and learning practice and methods, and providing leadership in education reform efforts. 14. Government response. Addressing the combination of “new and old business” represents a considerable challenge for the Vietnamese education system, regardless of past successes. Teaching higher order skills in general education will mean defining new standards, developing new pedagogical approaches, setting up learning assessment and establishing and applying new incentives. These changes will particularly task schools that serve more disadvantaged populations that still struggle to meet first generation educational goals. 15. In 2013, the 11th Party Congress at its eighth meeting session adopted the Fundamental and Comprehensive Education Reform (FCER),10 which aims for education to meet “the requirements of industrialization and modernization in the socialist oriented market economy and international integration.” It constitutes a political and legislative umbrella for a broad sector-wide reform. 16. New strategies to meet new objectives. The FCER’s priority is general education (grades K-12), where learning approaches are to become less content-based and more competency- and                                                              8 Skilling Up Vietnam: Preparing the Workforce for the Modern Market Economy, World Bank. November 2013 9 Vietnamese Government. 2010. Socio-Economic Development Strategy for 2011-2015 10 Resolution No 29/NQ-TW issued on 4th November 2013 of the 8th Conference of the 11th Session of the Central Executive Committee on Fundamental and Comprehensive Education Reform. 3   quality-based. It intends for all Vietnamese students to obtain higher order cognitive and behavioral skills, such as the ability to think critically and creatively, apply knowledge from many subject areas to solve practical problems, work in teams, and convincingly communicate verbally and in writing.11 To that end, general education will introduce, inter alia, new curricula and methods of instruction, develop a comprehensive learning assessment system and reform teacher education. 17. The FCER continues government policy of emphasizing equal opportunity. Thus, all students, regardless of ethnicity, income or gender must have an equal opportunity to master these new higher order skills and competencies. 18. Past successes in increasing access to education and higher learning achievement for disadvantaged groups were the product of a wide array of interventions sustained over a period of time. The Government developed and adopted both supply12 and demand13 side strategies. The FCER aims for an 80 percent upper secondary enrollment rate--which can only be reached by a fast expansion in enrollment of ethnic minorities. Although some of the past successful strategies might remain pertinent, secondary education has become considerably more complex—with a variety of learning paths, high stakes competition for tertiary education, and new ways of teaching and learning. Thus, the changes proposed by the FCER may constitute a particular challenge for ethnic minority children because it raises the bar by introducing new skills and competencies – just when these same children have begun to become proficient at the existing learning goals. 19. However, the reform also represents an opportunity for ethnic minority children. The new pedagogy favors a tailored approach to learning and teaching that takes into consideration context, the specific characteristics of the learner and insists that all children can succeed. In consequence, the new teacher professional profile will be particularly apt, since teachers will need to tailor their teaching approaches to the specific challenges facing ethnic minority children. 20. It is difficult to predict whether the new skills and competencies will be more difficult to master for girls or boys, although the education system should develop the tools and mechanisms to detect any differences as they appear. However, as mentioned above, one of the key gender issues in the education system has to do with equal access of female personnel to leadership and management positions. Thus, efforts to reinforce teacher education will also provide an opportunity to affect the gender balance in higher-level positions in general education. 21. Teachers are at the center of the reform process. The success of such a reform will depend almost exclusively on the preparedness of the teacher to master the new pedagogical paradigm. One of the underlying principles of the FCER is that teachers should be equipped to respond to different and constantly changing contexts. The FCER will require teachers to become increasingly professional, i.e., to adapt methodologies and knowledge to real-time and concrete problems as                                                              11 Ibid. 12 For example, school construction and improvement programs targeting selected regions and/or population groups; incentives for teachers to be deployed to remote areas and to attract ethnic minorities to the teaching profession; deployment of specialized teachers or teacher aides (e.g., language assistants); development of programs for children whose mother tongue is not Vietnamese. 13 For example, school fee exemptions, discounts, school lunches, scholarships and information campaigns 4   they arise in the classroom and school. This is similar to the professional models of medical doctors, researchers, policy analysts, and other knowledge-based fields. Training and support for such teachers require greater interaction between professionals, reciprocity, hands-on mentorship, coaching, and on-time advice to solve identified problems. 22. For teachers to acquire these competencies, they will need school-based continuous professional development (CPD). Considering the nature of the expected changes to teaching methodology and professional responsibility, research has indicated that school-based CPD (provided through face-to face and online platforms) has the greatest influence on teacher competencies, classroom behavior, and effectiveness. This is primarily because this type of training and support prepare teachers to respond to quickly changing situations and a wide array of differing demands.14 By “bringing training to the teacher”, school-based CPD can help teachers more effectively master these new methods and acquire these new competencies. 23. Teacher education in Vietnam has many strengths, but it is not necessarily prepared to help teachers meet these new expectations, nor to ensure the highest quality and most responsive onsite continuous professional development. One of the biggest challenges is that teacher education actors do not operate in a coordinated manner to meet teacher and principal new needs. In response, the MOET has prepared the National Teacher Education Program (NTEP) to enable the teacher education system to adapt more effectively to the changing needs of teachers and principals.15 The NTEP focuses on one of the FCER Programs -- the Reform of Preservice and In-service Training -- to produce a comprehensive strategy for realigning teacher education to the new requirements of general education.16 24. The NTEP is a comprehensive strategy that aims to realign teacher education to the new requirements of general education. It proposes several paths of intervention: pre-service training for new teachers, retraining for some teachers to obtain additional qualifications, in-service training programs that bring teachers to training centers for specific objectives, and CPD that provides support and training to teachers inside schools and classrooms. 25. Of the different elements of the NTEP, the Government and the World Bank agreed that the Enhancing Teacher Education Program (ETEP) would focus on school-based continuous professional development. The expected changes to teacher methodology, as well as the findings of much research on teacher professionalism, point to school-based CPD (provided face-to face and online platforms) as having the greatest influence on teacher competencies and behavior. By “bringing training to the teacher”, school-based CPD can help teachers more effectively adopt new methods and master new competencies. Furthermore, since there will be little need to recruit additional teachers because of demographic shifts and a current surplus of teachers, school based                                                              14 Francesca Caena, 2011. Quality in Teachers’ Continuing Professional Development. Education and Training 2020: Thematic Working Group ‘Professional Development of Teachers’, European Commission, Directorate-General for Education and culture. 15 The National Teacher Education Program was approved by Government on April 29, 2016. 16 The MOET is currently developing a complementary program to reorganize and rationalize the network of teacher training institutions. Once MOET finalizes and approves this program, it will take the lead in ensuring that it is closely coordinated with ETEP and NTEP. LTTUs can already collaborate with other TTIs and include this action in their Performance Agreement as appropriate. 5   interventions constitute the principal lever for successfully achieving the FCER goals for general education. Figure 1: National Teacher Education Program in context C. Relationship to the Country Partnership Strategy (CPS) and Rationale for Use of Program for Results Financing Relationship to the CPS: 26. The World Bank Group Country Partnership Strategy for Vietnam (FY2012-2016) (Report 85986) consists of three pillars: (i) strengthening Vietnam’s competitiveness in the regional and global economy; (ii) increasing the sustainability of its development; and (iii) broadening access to economic and social opportunity. Vietnam’s Fundamental and Comprehensive Education Reform (FCER) shares the objectives of the first and third pillars of the CPS. The FCER aims to prepare Vietnam for greater integration into the global economy, by providing students with new and necessary skills and competencies that are increasingly driven and required by changes in the global economy. In addition, the FCER emphasizes that the opportunities to attain these higher order skills must be available to all; Government’s National Teacher Education Program, which is a key part of the FCER, places the reform of the teacher education system at the heart of its reform strategy. The Bank-Financed ETEP supports a catalytic component of the NTEP – teacher continuous professional development provided onsite. 27. In consequence, by supporting a critical aspect of FCER, ETEP contributes to achieving the World Bank Group’s Twin Goals to end extreme poverty and promote shared prosperity. Improving access to and quality of education has proven to have considerable impact on eliminating extreme poverty. In addition, high quality education that is made available in an equitable manner can increase economic growth and provide every citizen with a pathway to 6   prosperity.17 However, this can only be achieved if the education system functions effectively. Teacher effectiveness is at the heart of that challenge. ETEP helps the Government develop new mechanisms to equip teachers on an ongoing basis with new skills and pedagogical approaches that are tailored to the changing economic and social requirements of Vietnam and the specific learning conditions facing students. In this manner, ETEP directly and sustainably contributes to achieving the Twin Goals. 28. Gender equality and access to quality education among ethnic minorities will be mainstreamed in the proposed Program, thus furthering its contributions to the CPS. Concrete measures have been identified to ensure that both male and female staff and teachers have equal access to continuous professional development opportunities and that timely and adequate gender- specific support to teachers is provided. Likewise, specific interventions are designed so that continuous professional development is tailored to the needs and culture of ethnic minority teachers. Beyond that, all teachers working with ethnic minority students will be provided with relevant knowledge and skills through the Program. 29. The Bank’s education engagement in Vietnam has increasingly focused on higher order skills and competencies, while privileging the needs of disadvantaged students. The current portfolio of K-12 education operations consists of:  The School Readiness Promotion Project (SRPP - P117393) seeks to improve early childhood education by promoting school readiness for disadvantaged children through Vietnam’s existing Early Childhood Education (ECE) program. In addition to strengthening the ECE program, the SRPP expands full-day preschool enrollment, improves preschool quality assurance mechanisms, and develops preschool teachers’ and education managers’ professional expertise.  The Intergenerational Deaf Education Outreach Project (IDEO – P125581) pilots, on a small scale, three simple innovations that global experience suggests would make deaf preschoolers better prepared for the formal education system: (i) the use of sign language as the primary mode of communication; (ii) the involvement of the whole family in the learning experience of the deaf child; and (iii) "family support teams" that provide the deaf child and his/her family with early and intensive support from a trained young deaf adult and a trained sign-language interpreter.  The School Education Quality Assurance Project (SEQAP - P091747) supports the Vietnam’s full-day schooling reform program by upgrading educational infrastructure and facilities, developing human resources, and reinforcing policy frameworks. It has aimed to improve educational outcomes and completion rates for disadvantaged primary education students.  The Vietnam Global Partnership for Education - Escuela Nueva Project (VNEN - P120867) introduces new teaching and learning practices that promote active learning and de- emphasize rote memorization. The VNEN is especially notable for inaugurating and advancing                                                              17 UNICEF, 2013. Sustainable development starts with safe, healthy and well-educated children. 7   this new pedagogical approach and providing high quality and innovative school-based teacher support.  The Reform of General Education (RGE – P150058) will facilitate the curriculum revision and expanded national assessment system to measure results. It is the first project that is designed on the basis of the FCER. 30. All of these projects are designed with teacher training as an integral part and important component to ensure delivery of results, which again emphasizes the importance of the role of teachers in any reforms and/or change efforts. However, teacher training has never been the primary focus of a Bank-financed operation. The fact that teacher training is always embedded in different operations and serves as an ad-hoc solution rather than systemic support for teachers has made it difficult to track results and evaluate impact. A systemic teacher education system to support teachers more effectively is therefore needed--to garner successful lessons, ensure a systemic and comprehensive approach is put in place, and to bring benefits to all teachers (and hence students). Rationale for choice of instrument 31. This ETEP is a PforR that will support NTEP with an International Development Association (IDA) credit in the amount of US$95 million (equivalent). Specifically, ETEP will support implementation of the government’s teacher continuous professional development strategy (one dimension of the NTEP). 32. Why a PforR? Vietnam has implemented ten World Bank-funded education operations since the 1990s, most of which have supported government strategies to improve equitable access to and quality of general education. These operations have accompanied the Government’s major education reform programs and have made a clear contribution to their achievements. However, as sector objectives have become more complex, reviews have found that individual investment projects focus too narrowly on particular aspects of education systems, without keeping the overall government strategy in view from a sector-wide perspective. This has led to operations implemented in silos and thus with insufficient coordination and synergy. 33. The team considers PforR the most appropriate financial lending instrument for ETEP. First, a PforR focusses on specific results that are catalytic and can leverage an entire reform process, such as the FCER. This encourages government authorities to focus on results rather than inputs. 34. Second, Vietnam’s systems of implementing its own programs provide an acceptable level of controls and assurances. The implementation of past education operations has increased the ability of education sector authorities to deliver large programs. The overlay of additional controls to existing country systems has become less necessary over time. 35. Third, in addition to providing strategic financing for key sector results, a PforR instrument will bolster the capacity of MOET to implement and manage the entire education reform through its own systems and procedures. Assessments have concluded that country systems used by the education sector are acceptable for the implementation of ETEP but that Vietnamese authorities 8   can further improve them. The PforR approach can support these improvements to ensure that relevant government offices can internalize and sustain them well beyond the Program. 36. By privileging country systems, a PforR simplifies disbursement, financial management and procurement. Consequently, less energy, resources and time need to be mobilized to ensure proper due diligence. This has the additional advantage of allowing the education sector to benefit more fully from World Bank expertise. 37. Lastly, the Vietnam Office of Government (OOG), Ministry of Planning and Investment (MPI), and the Ministry of Finance (MOF) have indicated that the PforR approach is a preferred financing instrument for Programs supported by the Bank. From the point of view of these agencies, a PforR obliges technical agencies to justify the effectiveness of overseas development assistance based on unified and coherent programmatic frameworks. Furthermore, because PforR Operations use country systems, they tend to have lower transaction costs, more regular and timely disbursements, and fewer implementation issues overall. These characteristics are particularly salient as Vietnam graduates to IBRD. There is also prior and current experience in Vietnam with World Bank-financed PforR operations. II. PROGRAM DESCRIPTION A. Program Scope 38. The National Teacher Education Program (NTEP) establishes a new framework for the provision of training, retraining and continuous professional development to school teachers and principals in order to meet the requirements of the FCER. The Government will implemented NTEP from 2016 to 2021. The NTEP aims to achieve the following objectives by that date:  For general education and continuing education teachers and management staff: - 100 percent of teachers and principals meet qualification requirements pursuant to the Amended 2009 Law on Education and the 2012 Higher Education Law, - 100 percent of teachers and principals are trained and capable of implementing the new curriculum and using the new textbooks; - Over 70 percent of teachers and principals achieve higher professional standards; - Over 80 percent of principals are trained and awarded certificates in education management.  For teacher training institutions: - 100 percent of teacher training institutions update their training programs and curricula, build staff capacity, improve physical conditions, and restructure to provide higher quality training and CPD services; - 100 percent of teacher training institutions work closely together and share resources to meet the need for teacher education in all provinces. 9    For education management at all levels: - 100 percent of education management authorities at all levels have enhanced planning and management capacity to implement, monitor and evaluate teacher education policies; and - 100 percent of policy and legal documents in teacher training are amended and updated to respond to the reform of general education.  For schools: - 100 percent of schools have enhanced capacity to effectively implement and manage CPD at the school level. 39. NTEP is structured around six “solutions areas”.18 Each presents a set of strategies grouped by functional category or by type of teacher education.  Strengthen teacher education management and planning by i) updating and further developing the regulatory framework and related policies in support of teacher education; ii) enhancing the capacity of national and local teacher education departments; and iii) establishing an ICT-based teacher education management information system;  Renovate pre-service teacher education by (i) helping teacher training institutions prepare new curricula, training content, delivery mechanisms, and support systems for student teachers; (ii) establishing outcome standards for all training programs, (iii) developing assessment tool kits for student-teachers, and (iv) implementing relevant research activities.  Establish and deliver continuous professional development programs for teachers and school managers by: (i) developing new types of e-learning programs for the delivery of CPD, (ii) training national general education experts who can support efforts to improve CPD, (iii) establishing a cadre of Core Teachers and Principal Advisors at local levels, (iv) providing support programs for Core Teachers and Principal Advisors once deployed to schools; (v) developing materials and support tools for Core Teachers and Principal Advisors to implement CPD at the school level; and (vi) developing more systematic mechanisms to assess the quality and impact of CPD programs.  Build capacity of key lecturers and managerial staff of teacher training institutions by: (i) providing relevant training programs and courses, (ii) holding national and international conferences and/or workshops; (iii) engaging in scholar and academic exchanges, and (iv) reviewing and renovating regulations on recruitment, work position and other human resource related procedures;  Enhance the physical environment and facilities/equipment of teacher training institutions: (i) upgrading and conducting minor rehabilitation of classrooms, IT centers, teaching practice centers, laboratories, and information and technology facilities for distance and e-learning at                                                              18 Some of these actions will follow the finalization and approval of the program to reorganize and rationalize the network of TTIs. 10   the eight Lead Teacher Training Universities (LTTUs)19, (ii) supporting the development of e- libraries for the eight and (ii) enhancing the physical conditions of other teacher training institutions.  Promote socialization and international cooperation in teacher education by means of workshops, conferences, exchanges and fora. 40. Of the above solution areas, ETEP will focus primarily on efforts to improve Continuous Professional Development through the provision of school-based CPD to teachers and principals. In essence, the enhanced CPD system will provide more training and support that is of high quality, relevant, and timely -- directly to teachers and principals at the school level. Internationally, this is considered best practice for teacher training and support.20 41. The National Teacher Education Program aims to reinforce the provision of CPD through two main vehicles: (i) Core Teachers and Principal Advisors, who will be assigned to schools and school clusters to provide face-to-face training and support and (ii) an online platform that can provide needs-based, interactive training and support directly to teachers and principals. A rigorous training and support needs assessment system, based on professional standards and context-specific needs defined by teachers and principals, will inform the development of tailored training programs and applications. To ensure the dynamism and quality of the enhanced CPD system, eight LTTUs in respective catchment areas (corresponding approximately to regions) will train and provide support to Core Teachers and Principal Advisors; develop the online services and applications; and help develop the needs assessment systems. These LTTUs will provide technical leadership and guidance for this school-based CPD system. B. The Program for Results 42. This PforR consists of two parts. The first part is defined by four results areas, which are part of the NTEP, that together contribute to meeting the overall goals of ETEP over the period between 2016-2021. These four results areas constitute the principal strategies for establishing an effective and well-functioning CPD system that can provide school-based support and training to teachers and principals. These strategies or results areas consist of (i) improving capacity of Lead Teacher Training Universities and central teacher management units to enhance teacher and principal education effectiveness; (ii) developing a teacher and principal training needs assessment system to inform CPD program development; (iii) providing school-based and continuous professional development for teachers and principals; and (iv) ensuring that teachers and principals have access to CPD programs and resources through an Information and Communication Technologies-based system.                                                              19 Hanoi National University of Education, Hanoi Pedagogical University 2, Thai Nguyen University of Education, Department of Teacher Education – Vinh University, Hue University of Education, University of Education – Da Nang University, Ho Chi Minh Pedagogical University, and the National Institute of Education Management 20 Francesca Caena, 2011. Quality in Teachers’ Continuing Professional Development. Education and Training 2020: Thematic Working Group ‘Professional Development of Teachers’, European Commission, Directorate-General for Education and culture. 11   43. The second part will mobilize world-class expertise to help develop and operationalize the strategies for improving school-based CPD for teachers and principals. This technical assistance (TA) will allow MOET and LTTUs to: (i) elaborate and operationalize policies and directives, (ii) inform and improve the technical quality of the expected results and (iii) reinforce the implementation capacity, including monitoring and evaluation, of the Program. Figure 2 below presents the results chain for the overall PforR. Figure 2: Results Chain for the Enhancing Teacher Education Program Part I: Results Areas: Developing capacity of teacher training institutions to enhance quality of trainings and continuous professional development for school teachers and principals 44. Results Area 1: Improved capacity of Lead Teacher Training Universities and central teacher management units to enhance teacher and principal education effectiveness. MOET has selected eight LLTUs  to provide technical guidance for the development and the implementation of enhanced CPD for teachers and principals. The LTTUs will train and support Core Teachers, (one selected from each of 28,000 schools countrywide), and Principal Advisors 12   (one selected per seven schools) who will subsequently return to their respective schools and school clusters to provide CPD to fellow teachers and principals. Second, the LLTUs will help develop instruments that measure the CPD needs of teacher and principals. Third, the LTTUs will develop content for CPD programs that address global teacher and principal needs (e.g., preparing for the rollout of the new curriculum), teacher and principal professional needs (e.g., programs to help meet new professional standards), and programs that address specific issues or needs identified by teachers and principals. For example, for regions where there is a significant population of ethnic minorities, a LTTU could design programs and delivery methods that target the specific needs of ethnic minority teachers. 45. The LTTUs do not currently have the qualities necessary to play fully their role as technical leaders of the school-based CPD system. Each LTTU will use the Teacher Education Institutional Development Index (TEIDI) to measure their institutional capacity (see below and Box 1). Each LTTU will then prepare a Performance Agreement (PA), which is a contract signed between the LTTU and MOET. In the PA, the university commits to achieving particular results in order to receive resources necessary to enhance their capacity. Signing and implementing the PA and reporting progress on TEIDI scores for individual LTTUs are DLIs for the PforR. 46. The Bank-financed Bihar Enhancing Teacher Effectiveness Operation21 in India currently uses the TEIDI. In Bihar, TEIDI assess the readiness of teacher education institutions to deliver quality teacher education programs and to monitor their accountability. TEIDI has a quantified and weighted checklist of indicators that measure institutional performance. TEIDI can inform planning for effective decision-making at the national and provincial institutional levels. MOET will adapt the TEIDI developed in Bihar to the Vietnam context.                                                              21 The Bihar operation was approved in May 2015 and is the first PforR in the education sector. It aims to improve the effectiveness of primary school teachers. Teacher training colleges play a central role in providing support to teachers and thus their capacity is critical. So far, TEIDI has proven to be easy to use and baseline scores were easily established during the first year of the operation. The straightforward and relatively “low tech” data collection and analysis systems have contributed to its relatively easy implementation. 13   Box 1: Framework for Teacher Education Institution Development Index (TEIDI) TEIDI is a tool to measure LTTU’s efficiency and effectiveness in providing high quality teacher training and professional development. The tool includes indexes used to assess the quality and institutional effectiveness of teacher training institutions in a comprehensive manner. Each index comprises of relevant indicators and once the data is collected, the TEIDI can be calculated on their basis.  Pedagogy in preservice training programs: Ensure teachers be trained in organizing leaner- centered, activity based and participatory learning experiences;  Professional development programs: Ensure teachers be trained in organizing leaner-centered, activity based and participatory learning experiences;  Academic staff: Ensure profile of LTTU faculty developed;  Activities for and direct support to schools: Ensure LTTU practices are not isolated from the classroom practice;  ICT preparedness: Ensure LTTUs maximize the use of ICT in providing regular school-based training and support for teachers;  Governance, Partnership and Performance: Ensure the strengthening institutional organization and governance. ETEP will employ an adapted version of the TEIDI used in the Bihar operation to assess the efficiency and effectiveness of the LTTUs. TEIDI uses a comprehensive approach for the planning, management and governance of teacher education institutions and diagnosing institutional strengths and weaknesses. The index will help identify the needs for teacher education provisioning, institutional requirements, pedagogical readiness levels, efficient service delivery mechanism, including sensitivity to the needs of the disadvantaged and minority groups so as to provide CPD and improve teacher performance. 47. Results Area 2: Development of teacher and principal training needs assessment systems to inform CPD program development. Teachers and principals will record their learning and support needs and their evaluations of online CPD interventions into a Teacher Education Management Information System (TEMIS). TEMIS will also house data on the new Teacher Professional Standards that MOET is developing to be better align with the requirements of the FCER.22 Teachers and other teacher education actors can use the professional standards and their own self-assessments (and those of other actors, such as the Core Teachers) to identify and prioritize teacher CPD needs. In consequence, the TEMIS would provide an important input for teachers to develop training plans to meet the new professional standards. Furthermore, the TEMIS can easily compile results in different ways to help LTTUs and others prepare specific interventions. The TEMIS will constitute a key resource for CPD decision-making and program development. It will also help evaluate the responsiveness and impact of programs. The PforR tracks the development and deployment of the TEMIS to monitor and meet teacher and principal CPD needs as DLIs. The PforR also rewards the practice of sharing data generated from the TEMIS to different stakeholders through annual DLIs. 48. Results Area 3: School-based and continuous professional development for teachers and principals. Core Teachers are senior or well-regarded school teachers who will provide face-                                                              22 MOET established teacher professional standards in the 2000s, corresponding to the reform program introduced at that time. 14   to-face CPD in the schools where they are deployed (e.g., classroom observation, organization of peer-to-peer training; and individual coaching and mentoring). They will receive special training from the LTTUs and then return to their schools of origin to implement an annual work plan of support and training to other teachers. Similarly, Principal Advisors support principals of a cluster of six or seven schools. The Core Teachers and Principal Advisors will receive support from LTTUs throughout the year. The Core Teachers and Principal Advisors will develop annual plans detailing the training and support services they will carry out over the school year. The training and support provided to the Core Teachers and Principal Advisors by the LTTUs and the implementation of the Core Teachers’ and Principal Advisors’ annual plans constitute DLIs. 49. Core Teachers and Principal Advisors will not be new categories of civil servants. They retain their status as regular teachers and principals who are given modified work program with the same compensation, benefits, and conditions as before. Cascade trainers have played similar roles (teachers selected to be trainers for a limited period of time), but in an ad hoc manner. 50. Teachers often see their assignment as cascade trainers to be a sign of status. Becoming a Core Teacher or Principal Advisor should also be considered an appealing opportunity. In addition, Core Teachers and Principal Advisors will have fewer teaching hours,23 receive specialized training and CPD programs. It will nevertheless be necessary to establish a set of regulations for selection, deployment, training requirements, etc.24 DLI 1 for the first period will be met when MOET puts in place the regulatory framework for Core Teachers and Principal Advisors. 51. Results Area 4: Teachers and principals have access to CPD programs and resources through an Information & Communication Technology-based system. The MOET aims to put in place a comprehensive Learning Management System that can make available four basic CPD services to teachers and principals: (i) on-demand, interactive e-learning courses for an assortment of continuous professional development needs; (ii) tailored knowledge (e.g., digitized and customizable lesson plans, videos of best practice) personalized support (e.g., help desks); (iii) opportunities to collaborate with peers and experts online (e.g., A social media platform with mobile interface uniquely designed for teachers) and (iv) mechanisms to allow beneficiaries to evaluate CPD services received. 52. Teachers and principals are familiar with using ICT for training and other purposes. The MOET has an incipient Learning Management System that will provide the basis for further development. Teachers already have access to e-learning courses through this system. In addition, the MOET’s Connecting Schools Initiative has been in place for two years and provides e-learning to secondary schools. Both the VNEN and SEQAP projects are planning to use this as a platform for the transfer of their training programs. VNEN has recently launched a video competition that produced several thousand videos made by teachers on their teaching practice. Once curated, these can eventually be made available to a wider audience through the Learning Management System (LMS).                                                              23 Because of the current surplus of teachers and the relatively few hours of instruction given by each teacher 24 These regulations provide guidance and instructions to relevant agencies and institutions as to how to select, deploy, support and compensate Core Teachers and Principal Advisors. They will be introduced through MOET Circulars. DOETs and the LTTUs will use these instructions to ensure that the Core Teachers and Principal Advisors are able to fulfill their expected functions. 15   53. Implementation Support. Under this Part, ETEP will also support overall coordination, management, monitoring and evaluation, and supervision of the implementation of the Program by MOET in collaboration with concerned agencies and the independent verification agent. Part II: Technical Assistance   54. ETEP includes Technical assistance in the following areas:  Program management capacity and relevant country systems. These include financial management and environmental management. It also includes strengthening the Recipient’s institutional capacity for the implementation of the Program, in strategic and crosscutting areas such as: (i) issuing and implementing teacher education policies, standards, and regulations; ii) coordinating strategies, actions, and initiatives across teacher education institutions and agencies to ensure system-wide coherence; (iii) monitoring and evaluating the implementation, impact and cost-effectiveness of teacher education programs, interventions, and policies to inform future initiatives; and (iv) promoting knowledge sharing, innovation, and communications in the domain of teacher education.  Expertise in specific technical areas to ensure satisfactory achievement of expected results. These will include, inter alia, developing and implementing effective school-based CPD, innovative CPD programs, IT infrastructure and architecture for the online platform that will host the LMS (Learning and Management System) and TEMIS (Teacher Education Management Information System), professional profile of Core Teachers and Principal Advisors and relevant regulations and selection and appointment procedures, teacher professional standards, training needs assessment methods, and training programs in selected subject areas.  Institutional strengthening of training institutions. TEIDI will measure the effectiveness of LTTUs to provide quality and relevant services in a timely manner. Similar measures exist in certain OECD countries, but few developing countries. Consequently, the MOET will need to mobilize expertise in this area as well.  Third party validation. The MOET will need to identify an independent entity to undertake performance audits and confirm DLI achievement. Third party studies will provide external validation for key performance data and implementation activities. The MOET may wish to hire a firm or institution with strong educational expertise and country system understanding to provide this service. 55. The most critical TA activities have been included in the Program Action Plan to ensure that it is in place when ETEP becomes effective. Gender and ethnic minority informed strategies integrated into ETEP 56. Measures and strategies are included throughout the ETEP Operation to encourage the teacher education system to focus on the education needs of ethnic minorities, girls and women and female students. Programs will also aim to attract and support ethnic minority teachers and principals, as well as administrators and other education sector leaders. This is believed to help 16   ensure academic success of ethnic minority children, particularly girls, and girls in general. More women and ethnic minority Some examples of such targeted support include:  Results Area 1: As part of their institutional capacity building initiatives, LTTUs will acquire the expertise to develop CPD programs, content, delivery methods and assessment systems that help all teachers and principals teach ethnic minority children more effectively. In addition, the LTTUs will conduct research activities, invest in capacity building for educational specialists in ethnic minority issues and prioritize ethnic minority and woman lecturers in their human resource development plans.  Results Area 2: New teacher and principal professional standards will include the mastery of methods and approaches to address culturally sensitive issues, especially those related to teaching ethnic minority children and girls. CPD needs assessment tools and systems will take into consideration the capacity of teachers and principals to work with ethnic minority students and girls. Special care will be taken to evaluate the CPD needs of ethnic minority teachers and principals to inform the development of programs that effectively respond to their professional needs.  Results Area 3: The training and support programs for Core Teachers and Principal Advisors will bolster their capacity to provide CPD to teachers and principals (particularly those who are ethnic minority) that specifically responds to the needs of ethnic minority and female students. The DOETs will be strongly encouraged to select ethnic minority and female teachers and principals to become Core Teachers and Principal Advisors.  Results Area 4: A specific support mechanism will be designed throughout program implementation to ensure that ethnic minority teachers are equipped with adequate computer skills before the commencement of e-learning courses and that they will continue to receive IT support in not only access to devices but also technical problems during and after e-learning courses. 1. Program Financing 57. The estimated funding envelop for the NTEP is US$255 million over a period of five years (2017 – 2022). Table 1 shows that this includes a commitment from the Government of Vietnam (GOV) of US$160 million (63 percent of the total cost) and IDA of US$95 million (or 37 percent of the total cost). 58. There will be a separate account for ETEP funding to monitor the implementation of ETEP – funded activities. Government recurrent and capital expenditures to implement the NTEP will be within the corresponding recurrent and capital envelopes that are approved by the Government and allocated by MOF.   17   Table 1: Program Costs and Financing Source Amount (US$ m) % of Total Government 160 63 Enhancing Teacher Education Program 95 37 Total Program Financing 255 100 C. Program Development Objective/s (PDO) and key results 59. The Program Development Objective of the ETEP is: to strengthen teacher education institutions to enhance teacher and principal effectiveness through improved continuous professional development. 60. The following program key results indicators have been defined:  PDO Indicator 1: Improved performance of the eight Lead Teacher Training Universities to provide CPD, as measured by the Teacher Education Institutional Development Index.  PDO Indicator 2: Number of teachers and principals satisfied with the new school- based continuous professional development. 61. These PDO indicators will be measured annually and data will be provided from the MOET’s annual progress report on the DLIs, as well as other sources. 1. Program Beneficiaries 62. The direct beneficiaries of ETEP will include: (i) more than 600,000 teachers and principals, which account for nearly 80 percent of the total number of general education teachers and principals in the country; (ii) 28,000 core teachers and 4,000 principal advisors; (iii) 3,200 lecturers, researchers and staff from the LTTUs; and (iv) MOET staff. Indirect beneficiaries of ETEP will be general education students; pre-service teachers; and Departments of Education and Training (DOETs) and Bureaus of Education and Training (BOETs) staff. Particular focus will be paid to encourage ethnic minority and female teachers and principals to become Core Teachers and Principal Advisors. 2. Disbursement Linked Indicators 63. The ETEP DLIs will disburse against the achievement of outputs and outcomes derived from the results chain (see above) and the Results Framework (Annex 2). The disbursement model proposes five DLIs, linked to the four results areas. There will be five annual disbursement periods, thus ensuring a phased approach for the roll-out of the innovations to be introduced through the Program. The DLIs are outlined below, with details presented in Annex 3. DLIs have been 18   prioritized from the indicators described above, based upon their importance for successful implementation and the need to incentivize outputs necessary for the success of the Program. Table 2: Results, Disbursement-Linked Indicators and PDO Indicators Results Area Disbursement-Linked Indicators25 PDO indicators Results Area 1: Improved DLI 1: Institutional capacity of LTTUs to PDO 1: Improved performance capacity of Lead Teacher support the new Continuous Professional of the eight Lead Teacher Training Universities and Development system for teachers and Training Universities (LTTUs) to central teacher management principals enhanced. provide CPD as measured by the units to enhance teacher and Teacher Education Institutional principal education Development Index (TEIDI) effectiveness. DLI 2: Systems are in place to: (i) measure Results Area 2: whether teachers and principals meet new Development of teacher and professional standards; (ii) track and record principal training needs evaluations of CPD programs; and (iii) assessment systems to inform record assessments CPD needs of teachers CPD program development. and principals. Results Area 3: School- DLI 3: Number of Core Teachers and based and continuous Principals Advisors selected and trained to professional development for provide systematic face-to-face school- teachers and principals. based continuous professional development to teachers and principals. DLI 4: Number of Core Teachers and Principal Advisors providing school-based continuous professional development to teachers and principals. Results Area 4: Teachers DLI 5: Number of teachers and principals PDO 2:   Number of teachers and and principals have access to who are satisfied with ICT-based principals satisfied with the new CPD programs and resources interactive CPD services provided under school-based continuous through an Information & the Program. professional development. Communication Technology- based system. 64. Verification Protocols: Achievement of these DLIs will be verified annually and verification results will be used as a basis for disbursement. For scalable DLIs, payment will be made in proportion to the achieved results for each period after they have been verified. For non- scalable DLIs, the whole target must be achieved for each period to trigger the corresponding payment. Verification protocols are briefly described below:  Data source/agency: The Program Management Unit (PMU) will be the main data provider. All data about achievement of results for each DLI in each period should be collected by MOET before verification of results. IOs (LTTUs and MOET                                                              25 The way the DLIs are measured is described in the Verification Protocols presented in Annex 3. 19   departments) are responsible for submitting reports to MOET regarding their implementation results against DLIs. MOET will then aggregate information for results verification.  Verification Entity: An Independent Verification Agency (IVA) will be selected to verify certain results, for example, the number of core teachers and principal advisors receiving face-to-face training from LTTUs.  Verification Procedures: In order to validate disbursement requests submitted by MOET, the IVA will verify all DLIs. The results reported by MOET as achieved under the Program will be verified through a paper audit and physical inspection that tests the accuracy and quality of results claimed. In accordance with good audit practice, physical verification will take place against a sampling framework and frequency. On this basis, the IVA will prepare a results verification report which will be shared with MOET and the World Bank. A key use of the Results Verification Report will be to determine the amount of the eligible disbursement to be made on the results achieved. If the Bank finds that the disbursement request meets the terms of the credit, the Bank will disburse the corresponding funds to the MOF. Further details on the verification process for each DLI are included in Annex 3.  Scalable vs. Non-scalable DLIs: In the first year/period of ETEP, most DLIs will be non-scalable because preparation steps must be taken before further results can be achieved for the following periods. For example, rules regarding the selection and deployment of Core Teachers and Principal Advisors need to be developed before they are trained; or an ICT-based system needs to be built before e-courses can be delivered to teachers and principals. From the second year onward, most DLIs will become scalable, i.e. payments will be made in proportion to achievement of DLIs. This will allow IOs to have both incentives and flexibility in planning for and achieving results. Definitions and description of DLI achievement are specified in Table 5 Annex 3. III. PROGRAM IMPLEMENTATION 65. The MOET is responsible for the strategic management of the ETEP and, by extension, the strengthening and institutionalization of school-based teacher continuous professional development. This includes: (i) issuing and implementing relevant teacher professional development policies and regulations; (ii) establishing and monitoring new teacher professional standards; (iii) identifying teacher- and school-level demand for school-based CPD; (iv) assessing the effectiveness of CPD interventions; and (v) using assessments and evaluations to shape CPD programs and initiatives. Through its Program Management Unit (PMU), MOET will also track and report on ETEP DLIs and transmit requests for disbursements to the World Bank. MOET will operate the monitoring and evaluation (M&E) system that will enable government to report on progress toward achieving the development objective indicators and intermediate indicators of the results framework. MOET will also oversee the implementation of the LTTU Performance Agreements. 66. The Department of Education and Training (DOET) is the provincial authority for general education. It is responsible for managing teacher deployment and service within its 20   catchment areas, including in-service teacher training and CPD. DOETs will coordinate efforts to improve CPD. This will include administrative oversight of Core Teacher and Principal Advisors and ensuring all CPD stakeholders at local level implement the CPD policies. DOETs will also track training needs assessments through the TEMIS, as well as training programs taken by teachers and principals, oversee evaluations of teacher standards, and undertake quality control of the CPD apparatus and process taking a province-wide perspective. 67. The LTTUs are responsible for implementing improvements to CPD envisaged by the NTEP through training, research, online services, and support to Core Teachers and Principal Advisors, as well other interventions. Successfully accomplishing this mandate will help the MOET meet DLIs. However, the ETEP technical assessment concluded that the LTTUs will need to build their capacity to undertake many of these new responsibilities. Each LTTU will prepare a Performance Agreement to be signed with MOET. The Performance Agreement will specify the needed institutional, human and technical resources to improve its capacity. The PAs will indicate how any given input will help the LTTU meet its new mandate. LTTU-specific TEIDI targets will inform the development of the performance agreement. 68. The LTTUs will: (i) create innovative training content and delivery mechanisms, maximizing the use of ICT solutions; (ii) provide training and support for Core Teachers and Principal Advisors; (iii) coordinate with DOETs in assessing teacher training and support needs; and (iv) develop and manage demand-driven knowledge management tools. 69. The network of Teacher Training Institutions (TTIs) will soon undergo a major reorganization to respond to the dwindling need for new teachers, the transformation of many into multi-disciplinary universities and colleges, and the Government’s decision to close others. The reorganization is also mandated by the FCER and is currently under implementation. Until its finalization and approval, the MOET prefers not to consider them as key actors in the ETEP, although individual LTTUs may decide to collaborate with other TTIs and include relevant actions in their Performance Agreements. The Government may use some Technical Assistance to support analytical work (policy analysis, strategy development), technical support (planning, feasibility studies) and other technical inputs necessary for developing or implementing a TTI reorganization plan. One eventuality after the reorganization would be that certain TTIs could partner with LTTUs, either through the Program Agreements or other mechanisms. Any reorganization of the network of TTIs would need to be closely coordinated with the DOETs and the MOET. 70. The Ministry of Finance (MOF) will receive the funds disbursed by the World Bank as per the terms of the IDA credit financing agreement. It will be responsible for making the appropriate resources available to the relevant institutions (e.g., the MOET and the LTTUs through their Performance Agreements) so that key interventions can be undertaken for meeting the DLIs in a timely, efficient and effective manner. 71. Independent Verification Agency26 will provide independent confirmations of the results reported by LTTUs and the MOET. The MOET will select an IVA within six months of effectiveness as indicated in the Program Action Plan.                                                              26  An Independent Verification Agency will be contracted no later than six months after Program Effectiveness. 21   A. Results Monitoring and Evaluation 72. Monitoring and evaluation of ETEP indicators will be carried out using the Government’s own systems. The TEMIS will generate reports regarding teacher management and performance standards. Evaluation of LTTUs will be undertaken using the TEIDI system. 73. Implementation Support Missions (ISMs): Two joint implementation support missions will be carried out each year consisting of representatives of the Government and the Bank. The joint ISMs will assess ETEP implementation, the achievement of key performance indicators and progress on DLIs. These reviews will aim to provide early identification of potential problems in the achievement of DLIs and propose solutions. There will also be a midterm review of the Program. Program Reviews will be produced twice every calendar year. 74. Because of the potential risks associated with using country systems for technical assistance, it was agreed to with the Recipient that the Operations Manual would include instructions for IDA to provide systematic guidance for the execution of technical assistance. This would consist of prior agreements with IDA on key milestones in the execution of Technical Assistance such as terms of reference, selection of experts, and reception of interim and final outputs. Such measures would ensure that TA can be mobilized in a timely and effective manner with minor disruptions to the implementation of the proposed Program. The ISM is expected to monitor the implementation of these arrangements. 75. A Program Action Plan (PAP) will cover key actions Government will undertake to ensure due diligence (e.g., fiduciary and social and environmental safeguards). Annex 8 provides details. B. Disbursement Arrangements 76. Program Financing Disbursement Arrangements. Funds will flow from MOF via its treasury system to the LTTUs and the MOET. MOET will determine fund allocations on the basis of the budget proposals that are part of the performance agreements signed between the MOET and LTTUs. According to these arrangements, LTTUs will receive funds as stipulated in the performance agreements (i.e., when certain targets or criteria included in the agreements are met). Actions included in performance agreements are expected to correspond to the DLIs. 77. MOET will present annually, by the end of February, evidence of the achievement of selected DLIs (covering the period of 1 January to 31 December of the previous year).27 The IVA is tasked with verifying results by no later than 31 May each year in the form of an ETEP Results Verification Report (PRVR), which will be shared with MOET and the Bank. In order to validate the disbursement request submitted by MOET, the IVA will verify selected DLI target indicators through both a desk review and site inspections to test the accuracy and quality of results claimed. In accordance with good audit practice, physical verification will take place against a sampling                                                              27  The FSA is more detailed than the FA. The OM will be developed based on the FSA. 22   framework and frequency. A key use of the ETEP Results Verification Report will be to determine the amount of the eligible disbursement to be made based on results achieved. If the Bank finds that the disbursement request meets the terms of the Credit, the Bank will disburse the corresponding funds to a bank account designated by the Government of Vietnam and acceptable to the to the Bank. 78. Advances. The Borrower informed the Bank that advances may be needed. Advances up to 25 percent of total Operation financing (“advance”) can be made by the Bank to MOF. After consulting the LTTUs, MOET and MOF will jointly determine how much of an advance will be requested. When the DLI(s) against which an advance has been disbursed are achieved, the amount of the advance will be deducted (recovered) from the total amount due to be disbursed under such DLI(s). The advance amount recovered by the Bank is then available for additional advances (“revolving advance”). The Bank requires that the borrower refund any advances (or portion of advances) if the DLIs have not been met by the closing date, promptly upon notice by the Bank. Thus, the Recipient can withdraw up to an equivalent of SDR 16,950,000 (i.e., US$23,750,000) as an advance. 79. Program Reconciliation. If the Bank establishes, after the closing date, that the withdrawn financing balance exceeds the total amount paid for Program Expenditures, exclusive of any such amounts financed by any other financier or by the Bank under any other loan, credit or grant, the Borrower shall, promptly upon notice from the Bank, refund to the Bank such excess amount of the Withdrawn Financing Balance. The Bank shall then cancel the refunded amount of the Withdrawn Financing Balance. 80. Program Reports and Financial Statement Audit. MOET shall prepare ETEP consolidated financial statements annually, as outlined in the Results Monitoring and Evaluation framework, above. The Report will include consolidated financial statements that will cover all activities supported by ETEP, expenditures and sources of funds, in accordance with the applicable International Public Sector Accounting Standards (IPSAS). Each LTTU will prepare the financial statements for the activities implemented by the LTTU and submit it to MOET for consolidation and audit. The annual financial statement must be submitted to MOET by 15 February of each subsequent year of Program implementation, and MOET must submit consolidated Program financial statements to External Auditor by February 28. External Auditor will audit the Program financial statements. Program Audit Report and audited Program financial statements shall be submitted to the Bank within seven (7) months after the end of the financial year. 81. Baseline and Verification. While the LTTUs and MOET already have provisional baseline data in their national plan, the IVA will undertake an assessment for relevant DLIs to validate the baseline information submitted and create a starting point for the results that they will later be required to evaluate. 82. Results will be verified through financial audit, procedural verification, and physical inspection that tests the accuracy and quality of results claimed. In accordance with good audit practice, physical verification will take place against a sampling framework and frequency. The level of calibration will be detailed in the Verification Protocols. 23   IV. ASSESSMENT SUMMARY A. Technical (including program economic evaluation) 83. The technical assessment finds that the ETEP and the NTEP are well designed and technically sound with a high probability of delivering quality and sustained training and support to general teachers and principals. The MOET and the Bank team consider continuous professional development to be a key aspect of the overall reform of general education and the NTEP more specifically. The nature of the expected changes to teacher methodology, as well as the findings of much research on teacher professionalism, point to continuous professional development as having the greatest influence on teacher competencies and behavior. 84. The Program builds on successful experiences of Bank operations such as the India: Enhancing Teacher Effectiveness in Bihar which also adopts a Program for Results (PforR) instrument. ETEP Program has built on two elements of the Bihar Operation. First, it will use a web-enabled Teacher Education Management Information System (TEMIS) to ensure greater transparency in teacher education management. Designed as a tool for planning and decision- making, ETEP will use it for developing, inter alia, more demand driven CPD. Second, it will develop TEIDI (Teacher Education Institution Development Index) which is a tool to assess the readiness of teacher education institutions to deliver quality teacher education programs and to monitor their accountability. 85. The assessment examined certain possible “weak links” in program design that are summarized below, as well as whether appropriate responses are given:  The proposed changes to teacher education might represent too radical of a modification. ETEP builds on existing characteristics of the teacher education system that have proven to be successful and will introduce changes in an incremental manner. Some examples include: access to initiatives that have introduced ICT based services and e- learning to teachers; experience as cascade trainers (similar to Core Teachers); and using professional standards to determine training needs.  Teacher training universities might be reluctant to change their mandate. The Performance Agreements will leverage these institutions to focus on meeting the results defined by ETEP.  The selection and management of the Core Teachers and Principal Advisors may not be transparent and merit based. MOET will prepare detailed regulations for the selection, training, support, and management of Core Teachers and Principal Advisors. This is a DLI for the first year of the program.  The positions of Core Teachers and Principal Advisors may not be appealing. Similar positions already exist in the form of cascade trainers, which are considered as prestigious. It is expected that Core Teachers and Principal Advisors will also be considered as appealing opportunities. In addition, CTPA should have fewer teaching hours, as well as access to specialized training, as well as support from the LTTUs.  The Government will rely too much on the online platform to conduct teacher training needs assessments. The purpose of the online platform is not to determine training needs, 24   but to help track training needs and provide access to information that could help teachers better articulate their CPD needs and other actors plan and develop CPD programs.  Teachers and principals may not be sufficiently familiar with using ICT for training and other purposes. The MOET has an incipient Learning Management System that provides the basis for further development. Teachers already have access to e-learning courses through this LMS. In addition, the MOET’s Connecting Schools Initiative has been in place for two years and provides e-learning to secondary schools. Both the VNEN and SEQAP operations are will use this as a platform for the transfer of their training programs.  Teacher professional standards have proven to be of limited use because assessment processes were not rigorous. The process of measuring and recording professional standards will be reinforced.  The Lead Teacher Training Universities have had little interaction with other actors of the teacher education system. The Performance Agreements will indicate how they will better collaborate with the DOETs.  The institutional capacity of LTTUs is judged to be weak. Implementation is sequenced, with the two years mostly devoted to putting in place necessary regulatory and institutional frameworks necessary for the rest of the Program. In addition, the program will start with simple targets and focus on priority areas, such as the roll out of the new curriculum. Furthermore, ETEP will adopt the Teacher Education Institutional Development Index (TEIDI) to assess the efficiency and effectiveness of the LTTUs. TEIDI uses a comprehensive approach for the planning, management and governance of teacher education institutions and diagnosing institutional strengths and weaknesses. Furthermore, the LTTUs and the MOET will be provided with world-class Technical Assistance to ensure greater coordination and improve the technical and institutional capacity to take on these additional responsibilities, such as in areas such as planning, financial management, human resource management, CPD needs assessment, development of online CPD, and providing support to schools through Core Teachers and Principal Advisors. 86. Proposed NTEP expenditure framework. Implementing NTEP will cost US$255 million over five years. Of the US$160 million government contribution to NTEP, 79.5 percent is for recurrent expenditures. This is approximately equal to the annual non-salary allocations for teacher continuous professional development and in-service teacher training made by the DOETs and the MOET.28 ETEP essentially supports one portion of the incremental expenditures necessary for the LTTUs and MOET to meet the DLIs of the Program. For example, as part of the NTEP strategy to enhance school-based CPD, the capacity of LTTUs to train and support the Core Teachers and Principal Advisors and to develop the online platform will need to be improved; this in turn will require faculty training, technical expertise, new delivery mechanisms, software applications, and minor civil works. The overall costs of these initiatives, which will be undertaken by all eight LTTUs, is expected to be higher than the total amount of funds made available through ETEP.                                                              28 The MOF channels the budgets of National Teacher Training Universities through MOET. 25   87. Few interventions will require an increase in recurrent resources for the LTTUs, the DOETs or the MOET. In addition, there are opportunities for cost savings and resource transfers. The maintenance of the TEMIS and LMS will likely require additional resources (e.g., maintenance & repairs, recruitment of specialized technical staff); but school-based CPD will also be rendered more efficiently, generating some cost savings. Almost all activities that LTTUs and MOET will undertake for the purposes of improving school-based CPD will be one-off interventions. 88. The introduction of Core Teachers and Principal Advisors is not expected to increase recurrent costs. MOET and the DOETs will mobilize them from the current cohort of teachers. Thus, the GOV will not need to recruit any new personnel. Furthermore, the GOV will not create a new category of civil servant, with an additional set of benefits. Core Teachers and Principal Advisors will retain their original status with the same rights, duties and accountabilities. Education personnel will likely consider Core Teachers and Principal Advisors to be prestigious and sought-after positions. However, MOET and the DOETs may need to consider other incentives, such as reduced workloads and special training opportunities. These interventions are not likely to be costly. 89. In addition, certain structural changes will likely offset some cost implications of these initiatives. For example, the decline in primary education enrollments has led to a surplus of teachers. This plus the relatively low number of teaching hours compared to other countries29 should make it possible to reduce the teaching load of Core Teacher and Principal Advisors without significantly affecting student/teacher ratios. 90. Furthermore, there will be little reason to recruit new teachers for many years to come and thus there is a presently a surplus of pre-service teacher training capacity in Teacher Training Institutions. Some of these resources and excess capacity can be used for training and supporting Core Teachers and Principals Advisors, developing new CPD applications, and conducting relevant research. 91. MOET will conduct a cost and finance analysis of the future potential impact of the enhanced Continuous Professional Development System on recurrent costs for the LTTUs and MOET. Conducting this study constitutes an activity included in the Program Action Plan (PAP). 92. Monitoring and evaluation: The introduction of Performance Agreements, TEIDI, and the online platform will all contribute to greater transparency and ease of tracking results. For example, the online TEMIS will house data regarding the implementation of the annual plans of the Core Teachers and Principal Advisors, and thus enable the MOET to track progress towards the relevant DLI. 93. The Independent Verification Agency: The agency will provide independent confirmation of the results reported by LTTUs and the MOET.                                                              29 According to the Public Expenditure Review (currently being finalized) Vietnamese teachers spend many fewer hours in the classroom than their peers in other countries. Ironically, academic results are significantly higher than found in countries where the number of official hours is significantly higher. 26   94. Economic justification of the Program. A cost-benefit analysis using the present discounted value method suggests that ETEP is justifiable with a Net Present Value (NPV) of US$495 million. This return is considered to be a lower bound, given that: (i) Only (private) outcomes for the students are considered as part of the benefits of the Program; and (ii) An attempt has been made in this analysis to be conservative for all assumptions where discretion could be exercised. (See Annex 4 for details.) B. Fiduciary 95. Financial Management Systems. The Bank conducted a Fiduciary System Assessment of the financial management arrangements of ETEP and found that the arrangements provide reasonable assurance that financing proceeds will be used for the intended purposes, with due attention to the principles of economy, efficiency, effectiveness, transparency and accountability. The MOF recognizes the need to secure ETEP funds in a special deposit account, in the State Treasury at the provincial level, to manage proper budget allocation, execution and reporting of the provinces. MOF’s approach is contained in the Government’s Project Outline. The assessment identified two substantial risks:  The budget allocation process does not always ensure that sufficient funds are available for Program’s activities. In order to mitigate this risk, MOET will sign Performance Agreements with the LTTUs using performance-based budgeting. The MOET will allocate budgets to the LTTUs on the basis of LTTU performance as defined in the Performance Agreements.30  In view of the absence of an internal audit (IA) function and the inefficiency of the existing inspection function, there may be inadequate assurance of internal controls of the Program. Technical Assistance will thus be provided to build capacity of an IA function in all LTTUs involved within two years of the Financing Agreement signing.  The financial audit will be conducted in accordance with the audit terms of reference provided in the Operations Manual (OM) by an Independent Auditing Firm acceptable to the Bank. Third party independent results verification agency as well as procedures will be put in place to track DLIs and Program outputs and outcomes. 96. Procurement. The proposed Program will use Government of Vietnam systems. 97. The Fiduciary System Assessment established an overview of the implementation arrangement and existing procurement arrangement related to procurement activities under Investment Owners (IO) including the regulatory framework, procurement practices, and procurement capacity of IOs. A summary of the assessment is provided in Annex 5. In general, the assessment found that the system in place was sound and the gaps in the Government’s procurement systems were not major. However, it did identify a number of weaknesses that will                                                              30 The Performance Agreements are not equivalent to DLIs. Although LTTUs are expected to include activities (such as training and supporting a certain number of Core Teachers) that help the MOET meet DLIs, the PAs are essentially a contract between the MOET and the LTTUs. 27   be addressed through the Program Action Plan and the OM to ensure that Program fiduciary systems provide reasonable assurance that the Program expenditures will be used appropriately to achieve their intended purpose. 98. The existing legal framework for the proposed Program will follow the national procurement laws and regulations. After July 1, 2014 when the new national procurement law No. 43/2013/QH13 was adopted. Therefore, procurement is governed by such law and decree No. 63/2014/ND-CP, which was effective August 15, 2014. 99. The accountability for procurement decisions is in accordance with the procurement law and decree. The implementation of procurement activities for goods, works and consulting services for a particular university is decentralized to the institution level. MOET normally has the role of the investment decider. It is therefore, the authority of the MOET to approve the activities and the procurement plans (PPs) that are prepared by IOs. IOs have the responsibility to implement the approved PP accordingly including making any needful decisions along the procurement process such as approving bidding documents (BDs), and contract awards. 100. Procurement under the proposed Program will mainly include civil works of small and medium sized lecture halls (approx. US$200,000 – US$1.5 million), and small upgrading construction, goods (including furniture, ICT equipment etc.) packages of small and medium sized laboratories, libraries or classrooms; and consulting services including detailed designs, construction supervision, and auditing services for subprograms under the Program. There are no large contracts valued at or above Operational Procurement Review Committee (OPRC) thresholds (US$75 million for works; US$50 million for goods, US$40 million for non-consulting services, and US$20 million for consultant services) under the Program. Therefore, obviously the Program will not finance any high value contract equal or above OPRC thresholds and / or estimated monetary value is equal to or more than 25 percent of the estimated total Program expenditures. 101. Procurement issues that may negatively affect the timeliness, efficiency and economy of ETEP implementation include an insufficient number of experienced procurement staff, excessive use of direct contracting, incorrect rejection of bids, and non-application of Bank debarment/suspension lists and unavailability of budget for the activities supported by ETEP. In order to address these procurement issues, the following measures are proposed:  All proposals for detailed designs, construction supervision and bids for civil works31 carried out under LTTU Performance Agreements, whether below or above cost estimates, shall be evaluated; proposals and bids shall not be rejected only on the basis of minor, non-substantive deviations;  IOs should hire consultants who have experience in procurement of civil works and contract administration to support better implementation of the Program;  At least 60 percent of contracts for consultancies and 85 percent of contracts for works will be competitively bid annually; no contract having cost estimates above the                                                              31 As per discussions during Program preparation, civil works are expected to be minor (refurbishing and repairs). All civil works will take place on campus. 28   thresholds for using Direct Contracting DC/Single Source Selection SSS method set forth in the national public procurement law and regulation will be awarded on DC/SSS basis; and  Firms on the local, national, or Bank debarment list32 will not be allowed to participate. 102. Governance. ETEP IOs have disclosed general information such as commitment to education quality, results, fees, and other related charges through their website, in accordance with the Law on Education. Financial information of these institutions, including investments for education and financial statements, however, has not adequately been disclosed to the public.33 The IOs have already issued detailed procedures for handling grievances. Notably, MOET has, recently, established a division charged with following up on grievances to concerning agencies under the MOET’s jurisdiction, which include the all IOs. 103. ETEP IOs agree with the Guidelines on Preventing and Combating Fraud and Corruption in PforR financing, dated February 1, 2012 to the ETEP, including an obligation on all parties to cooperate with Bank investigations. 104. The ETEP IOs have also agreed with some actions, as stipulated in the Program Action Plan (PAP) that would enhance transparency and mitigate fraud and corruption risks. The actions include: (i) Disclose Operation and financial information properly; (ii) Maintain databases on public consultation, corruption, fiduciary, procurement, complaints and responses to those complaints, and beneficiary feedback to IOs and responses to such feedback. The information would be aggregated at the national level by MOET and will be reported to the Bank on a periodic basis. 105. No complaints have been recorded by the Bank complaint database in the education sector during the period of 2014 – 2015. Most of the complaints registered in the sector during the ten- year period from 2005 to 2015 concern goods contracts, hence the need for more due diligence actions with regard to contract management of the IOs. Two cases were forwarded to Integrity Vice Presidency (INT) in 2006 and 2015. No evidence of fraud and corruption has been confirmed by the INT on these cases.                                                              32 The national debarment list can be found at http://muasamcong.mpi.gov.vn/, and the World Bank debarment list at http://www.worldbank.org/debarr, and the temporary suspension list at https://clientconnection.worldbank.org. 33 This will be addressed through a covenant as described in the PAP (Annex 8). 29   C. Environmental and Social Effects Environment (including Safeguards) 106. Overall, the ESSA determined that Program is not expected to have negative environmental impacts.34 Although some upgrading activities may cause negative environmental impacts associated with civil works, such as generation of noise, dust, wastes including solid waste and wastewater, and health and safety risks, they would take place exclusively within university premises. Thus, potential negative environmental impact is assessed to be minor, temporary, localized and can be mitigated. 107. Negative environmental impacts during construction will be adequately mitigated through application of Environmental Codes of Practice (ECOP) and Worker’s Codes of Conducts (WCC) as well as communication activities in construction contracts to manage Environment, Health, and Safety (EHS) issues at construction sites. During operation of the upgraded facilities, it is envisaged that a small amount of waste would be generated by daily activities of teachers, students and university staff, and this impact will be managed by the existing waste treatment facilities in line with the Government’s regulations. An Environmental and Social Systems Assessment (ESSA) is required and prepared by the Bank Task Team to address environmental and social concerns. The final ESSA in Vietnamese language was locally disclosed through MOET website on April 12, 2016 and the English version was disclosed at the Bank’s InfoShop on June 8, 2016. Social effects 108. Principals and teachers from the participating agencies across the country, including those from ethnic minorities and other vulnerable groups, will benefit from the Program. The consultations indicated general support for the proposed objectives and design of ETEP. Given the composition of consultation participants, full support has been received from a broad range of stakeholders, from all levels of local authorities, from the provincial and district to the commune levels, and various ethnic minorities groups, such as Tay, Nung, Thai, Dao and Ede. All participants confirmed their full support for the concept and approach of the proposed ETEP, seeing them as a practical and useful way to enhance the effectiveness of teachers across Vietnam. Both male and female principals and teachers from the participating institutions would be encouraged to participate in the activities supported by the Program. Women’s Unions should encourage women and ethnic minorities to become Core Teachers, Principal Advisors and LTTU faculty. 109. Needs of people with disabilities will be considered and incorporated into the Program’s activities to the extent possible. On-site CPD program will be developed to help teachers incorporate the latest knowledge and practice with regards to teaching children with disabilities. Some LTTU Performance Agreements will include actions to develop such CPD programs.                                                              34 The ETEP will support small scale upgrading activities of the existing education facilities, likely including e- libraries, basic laboratories for teaching, and classrooms for e-learning so it would help save energy and water, reduce health risks and better manage wastes. Further, teacher’s, staff’s and student’s awareness of environmental protection are expected to be raised through the program. Such action will likely be included in the LTTU Performance Agreement and will help meet TEIDI targets, and thus DLIs. 30   110. Some challenges, related to the existing social management system were identified during consultations, including those relating to monitoring and evaluation; transparency and publicity of information; internal communication within the education sector; effective outreach communication to promote education in ethnic minority communities; and encouraging teachers, particularly from ethnic minority communities to participate in the Program (see Annex 6). 111. Communities and individuals who believe that they are adversely affected as a result of a Bank supported PforR, as defined by the applicable policy and procedures, may submit complaints to the existing Operation grievance redress mechanism or the Bank’s Grievance Redress Service (GRS). The GRS ensures that complaints received are promptly reviewed in order to address pertinent concerns. Affected communities and individuals may submit their complaint to the Bank’s Independent Inspection Panel which determines whether harm occurred, or could occur, as a result of Bank non-compliance with its policies and procedures. Complaints may be submitted at any time after concerns have been brought directly to the Bank's attention, and Bank Management has been given an opportunity to respond. For information on how to submit complaints to the World Bank’s corporate Grievance Redress Service (GRS), please visit http://www.worldbank.org/GRS. For information on how to submit complaints to the World Bank Inspection Panel, please visit www.inspectionpanel.org D. Risk Assessment 112. The Systematic Operations Risk-Rating Tool (Annex 7) presents ratings of risks. 113. The technical design of the ETEP presents some risks, but mitigation measures have been identified. The principal risks are listed first, followed by a presentation of some key responses. 114. Creating a systematic needs-based CPD for teachers requires multiple institutions that do not have experience of working collaboratively will need to work in a coordinated fashion for the success of the ETEP. For example, whereas LTTUs will be responsible for training and supporting core teachers, DOETs will designate, deploy and manage the Core Teachers and Principal Advisors. These two institutions have had little experience working together. 115. LTTUs will serve as technical leaders and guides of teacher and principal CPD. However, traditionally, the services they have essentially trained new teachers or retrained existing teachers so that they can increase in grade. Furthermore, the newly introduced on-site and needs-based continuous professional development may incur additional costs for the LTTUs. While the ETEP will create an opportunity for the LTTUs to enhance their capacity to take on their new responsibilities, more effort to institutionalize properly the newly introduced CPD apparatus will be necessary. 116. For the DLIs to be met, they will require: (i) Timely funding for implementation; (ii) Strengthened IOs, especially the LTTUs, to take on new roles and responsibilities of teacher continuous professional development, and, above all, (iii) Strong coordination among concerned stakeholders. 117. These risks will be mitigated through the following actions: 31    Strengthening of MOET’s capacity to take on the responsibility of strategic management of the teacher CPD program to ensure an effective coordination mechanism be set up among concerned CPD providers.  Performance Agreements with set criteria that promote collaboration between institutions will also be developed with MOET approval.  The Teacher Education Institution Development Index (TEIDI), which is a tool to measure LTTU’s effectiveness in providing high quality teacher training and professional development, will include indicators on cooperation with local institutions. This will help build the capacity necessary for enhanced needs-based professional development for teachers. In addition, PAs help secure sufficient funds for the LTTUs to carry out agreed to actions. 118. Since the number of new teachers required has declined, surplus resources (from the LTTU’s current preservice teacher training program) can be reallocated to support continuous professional development. A study on cost and financing of needs-based continuous professional development will be undertaken during Program implementation and will identify options and recommendations to inform the financing of the teacher CPD. 119. Several actions in the Program Action Plan focus specifically on strengthening the institutional capacity of government system/policies. This will ensure that IOs are resourced in a timely and adequate manner, and supported to achieve DLIs. 120. Finally, because of the potential risks associated with using country systems for technical assistance, it was agreed to with the Recipient that the Operations Manual would include instructions for IDA to provide systematic guidance for the execution of technical assistance. This would consist of prior agreements with IDA on key milestones in the execution of Technical Assistance such as terms of reference, selection of experts, and reception of interim and final outputs. Such measures would ensure that TA can be mobilized in a timely and effective manner with minor disruptions to the implementation of the proposed Program. E. Program Action Plan 121. The principal actions to mitigate identified risks are summarized here and presented in greater detail in Annex 8. 122. Technical. Technical capacity will be enhanced by: i) The development of a coordination mechanism between teacher education actors at national and provincial levels who are responsible for teacher continuous professional development; ii) The execution of a Plan that ensures that Program activities across LTTUs are fully coordinated and implemented in a coherent manner; and iii) A study on the cost and financing of needs-based continuous professional development. 123. Governance. Governance systems will be strengthened by ensuring that: (i) The participating agencies disclose information on the Program’s activities to the public, including financial information throughout the implementation of the Program; (ii) The participating agencies maintain a system of receiving, recording and following-up on complaints, including anonymous ones; and (iii) The MOET Inspectorate maintains a database on complaints and 32   denunciations from all the participating agencies, as a result of the implementation, and follows up on the settlement of those complaints and denunciations. 124. Financial management. The Government will build and implement an effective internal audit function for the Program and the MOET and the MOF will issue a joint decision on the performance based budgeting/allocation. 125. Furthermore, because of the potential risks associated with using country systems for technical assistance, it was agreed to with the Recipient that the Operations Manual would include instructions for IDA to provide systematic guidance for the execution of technical assistance. This would consist of prior agreements with IDA on key milestones in the execution of Technical Assistance such as terms of reference, selection of experts, and reception of interim and final outputs. Such measures would ensure that TA can be mobilized in a timely and effective manner with minor disruptions to the implementation of the proposed Program. 126. Procurement. Procurement systems will be enhanced by ensuring that: (i) All proposals for detailed designs, construction supervision and bids for civil works, whether below or above cost estimates, shall be evaluated; proposals and bids shall not be rejected only on the basis of minor, non-substantive deviations; (ii) At least 60 percent of contracts for consultancies and 85 percent of contracts for works will be competitively bid annually; no contract having cost estimate above the thresholds for using DC/SSS method set forth in the national public procurement law and regulation will be awarded on DC/SSS basis; (iii) Firms on the local, national, or Bank debarment list will not be allowed to participate; and (iv) LTTUs be required to appoint or assign qualified procurement, contract management and technical experts for PMUs, and MOET to provide annual procurement training and contract management for PMUs under LTTUs. 127. Social. Participating agencies will jointly develop a guideline to ensure the effective participation of and consultation with beneficiary principals and teachers, including those from ethnic minorities and vulnerable groups, during the implementation process. The guidelines, to be implemented by participating agencies, should fully operationalize the existing Vietnamese legislation with respect to ethnic minorities through a process of free, prior, and informed consultations     33   Annex 1: Detailed Program Description A. Program Scope   129. In 2013, the 11th Party Congress at its 8th meeting session recognized that Vietnam’s education system had performed admirably, but that overall education quality remains lower than expectations. To attend to this, Vietnam adopted the Fundamental and Comprehensive Education Reform (FCER),35 which aims for education to meet “the requirements of industrialization and modernization in the socialist orientated market economy and international integration”. It constitutes a political and legislative umbrella for a broad sector-wide reform. 130. The FCER introduces changes at every level of education – from preschool to lifelong learning. To meet the objectives of the FCER and organize the implementation of its broad strategies, the Government identified over 20 programs areas and charged the Ministry of Education and Training to prepare specific action plans to implement over the five-year period of the reform. The following Table 3 organizes the programs by education level: (i) system-wide; (ii) general education; (iii) higher education; and (iv) skills development. Table 3: FCER Action Plans SYSTEM LEVEL GENERAL HIGHER SKILLS EDUCATION36 EDUCATION DEVELOPMENT  FCER communication  Develop new curriculum  Evaluation of science  Reorganization of TVET campaign and textbooks and technology activities centers efficiency  Complete the structure  Learning assessment  Renovation of vocational of the education system  Budget allocation and professional education  Preschool education method and criteria quality  Mobilize domestic and development international resources  High school graduation  Development of higher  National Qualification education institutions Framework  Amendment of exam and university Education Law entrance exam  Information network on supply-demand of  Promote non-public  Reform teacher training education and training education network37  Incentives on public  Cultural and sports  Reform pre-service and centers institutions performance in-service training38  Recruitment based on  Teaching and actual competencies researching science and  Development of ethnic technology minority human resources  Career orientation  Infrastructure                                                              35 Resolution No 29/NQ-TW issued on 4th November 2013 of the 8th Conference of the 11th Session of the Central Executive Committee on Fundamental and Comprehensive Education Reform. 36 Many of these areas are already supported by the World Bank education program through the following operations: SEQAP, VNEN, and RGEP. 37 This program will complement the NTEP and will consist of a strategy to reorganization and rationalize the network of TTIs. It is currently under development. 38 This is the basis of the NTEP. 34    IT to support teaching and research activities  Enhance efficiency of international cooperation  Inspectorate 131. The FCER’s priority is general education (K-12 grade), where it aims to move from a content-based to a more competency- and quality-based approach to learning. It intends for all Vietnamese students to obtain higher order cognitive and behavioral skills, such as the ability think critically and creatively, apply knowledge from many subject areas to solve practical problems, work in teams, and convincingly communicate verbally and in writing.39 To that end, general education will introduce new curricula and methods of instruction, develop a comprehensive learning assessment system and reform teacher education. 132. Teacher education in Vietnam has many strengths, but it is not necessarily prepared to help teachers acquire this different professional profile and skills, nor to ensure the highest quality and most responsive onsite continuous professional development. One of the biggest challenges is that teacher education actors do not operate in a coordinated manner to meet teacher and principal new needs. The MOET has prepared the National Teacher Education Program (NTEP) to enable the teacher education system to adapt more effectively to the changing needs of teachers to meet learning objectives.40 The NTEP focuses on one of the FCER Programs -- the Reform of Preservice and In-service Training -- to produce a comprehensive strategy for realigning teacher education to the new requirements of general education. 41 The NTEP proposes several paths of intervention: pre-service training for new teachers, retraining for some teachers to obtain additional qualifications, in-service training programs that bring teachers to training centers for specific objectives and continuous professional development (CPD) that provide support and training to teachers within schools. 133. The National Teacher Education Program (NTEP) establishes a new framework for the provision of training, retraining and continuous professional development to teachers and principals so that they may build and maintain the professional profile necessary to meet the requirements of the FCER. The Program will be implemented from 2016 to 2022. The NTEP aims to achieve the following objectives by that date:  For general education and continuing education teachers and management staff: - 100 percent of teachers and principals meet qualification requirements pursuant to the Amended 2009 Law on Education and the 2012 Higher Education Law,                                                              39 Ibid. 40 The Government approved the National Teacher Education Program on April 29, 2016. 41 The MOET is currently developing a complementary program to reorganize and rationalize the network of TTIs. Once MOET finalizes and approves this program, it will take the lead in ensuring that it is closely coordinated with ETEP and NTEP, and that it effectively partners with the LTTUs. LTTUs can already collaborate with other TTIs and include this action in their Performance Agreement as appropriate. 35   - 100 percent of teachers and principals are trained and capable of implementing the new curriculum and using the new textbooks; - Over 70 percent of teachers and principals achieve higher professional standards; - Over 80 percent of principals are trained and awarded certificates in education management;  For teacher training institutions: - 100 percent of teacher training institutions update their training programs and curricula, build staff capacity, improve physical conditions, and restructure to provide higher quality training and CPD services; - 100 percent of teacher training institutions work closely together and share resources to meet the need for teacher education in all provinces.  For education management at all levels: - 100 percent of education management authorities at all levels have enhanced planning and management capacity to implement, monitor and evaluate teacher education policies; and - 100 percent of policy and legal documents in teacher training are amended and updated to response to the reform of general education.  For schools: - 100 percent of schools have enhanced capacity to effectively implement and manage CPD at the school level. 134. NTEP is structured around six “solutions areas”.42 Each presents a set of strategies grouped by functional category or by type of teacher education.  Strengthen teacher education management and planning by i) updating and further developing the regulatory framework and related policies in support of teacher education; ii) enhancing the capacity of national and local teacher education departments; and iii) establishing an ICT-based teacher education management information system;  Renovate pre-service teacher education by (i) helping teacher training institutions prepare new curricula, training content, delivery mechanisms, and support systems for student teachers; (ii) establishing outcome standards for all training programs, (iii) developing assessment tool kits for student-teachers, and (iv) implementing relevant research activities.  Establish and deliver continuous professional development programs for teachers and school managers by: (i) developing new types of e-learning programs for the delivery of CPD, (ii) training national general education experts who can support efforts to improve CPD, (iii) establishing a cadre of Core Teachers and Principal Advisors at local levels, (iv) providing support programs for Core Teachers and Principal Advisors once deployed to schools; (v) developing materials and support tools for Core Teachers and Principal Advisors to implement                                                              42 Some of these actions will follow the finalization and approval of the program to reorganize and rationalize the network of TTIs. 36   CPD at the school level; and (vi) developing more systematic mechanisms to assess the quality and impact of CPD programs.  Build capacity of key lecturers and managerial staff of teacher training institutions by: (i) providing relevant training programs and courses, (ii) holding national and international conferences and/or workshops; (iii) engaging in scholar and academic exchanges, and (iv) reviewing and renovating regulations on recruitment, work position and other human resource related procedures;  Enhance the physical environment and facilities/equipment of teacher training institutions: (i) upgrading and conducting minor rehabilitation of classrooms, IT centers, teaching practice centers, laboratories, and information and technology facilities for distance and e-learning at the eight Lead Teacher Training Universities (LTTUs)43, (ii) supporting the development of e- libraries for the eight and (ii) enhancing the physical conditions of other teacher training institutions.  Promote socialization and international cooperation in teacher education by means of workshops, conferences, exchanges and fora. 135. Of the above solution areas ETEP would focus on school-based continuous professional development. The nature of the expected changes to teacher methodology, as well as the findings of much research on teacher professionalism, point to school-based CPD (provided through face- to face and online platforms) as having the greatest influence on teacher competencies and behavior. By “bringing training to the teacher”, school-based CPD can help teachers more effectively adopt new methods and master new competencies. Furthermore, since there will be little need to recruit additional teachers because of demographic shifts and a current surplus of teachers, school based interventions constitute the principal lever for successfully achieving the FCER goals for general education. 136. Vietnam possesses a keen advantage as it enters a new generation of education sector development. It is building from the base of a well-functioning education system that has provided exceptional results. Despite “unfinished business”, the system has produced learned children distributed quite equitably across Vietnam. Many better-endowed countries would be envious of this result. However, the challenges of such an ambitious shift in educational expectations should not be underestimated and the National Teacher Education Program constitutes a promising strategy in that regard. 137. Figure  1Figure 3 provides a representation of the nested relationship of the FCER, the General Education Reform, the National Teacher Education Program and ETEP.                                                              43 Hanoi National University of Education, Hanoi Pedagogical University 2, Thai Nguyen University of Education, Department of Teacher Education – Vinh University, Hue University of Education, University of Education – Da Nang University, Ho Chi Minh Pedagogical University, and the National Institute of Education Management 37   Figure 3: National Teacher Education Program in context   B. Enhancing Teacher Education Program (ETEP) 138. The Government’s teacher continuous development strategy emphasizes four key interrelated areas. First, the Program intends for a select group of teacher training universities to provide the necessary technical support for improved teacher education. They will train the Core Teachers and provide them with continuous professional support after they return to their schools; provide content, innovative approaches and interactive support to teachers deployed in schools through the online platform. They will also conduct research and evaluations to improve the effectiveness and quality of training. They will be the principal user of the improved assessment systems. 139. Second, the program reinforces assessment systems for teacher performance so that training and support can target teacher needs and ensure greater accountability for the quality of training. MOET intends to modify teacher professional standards to align them with the new curriculum and teaching methodology. The standards, feedback from Core Teachers, and self- assessment will be the basis of teacher training plans. A Learning Management System (LMS), which will be located on the online platform, will record these will serve to help teachers and others track the implementation of annual CPD plans, provide training institutions and authorities with a basis for developing training programs, and enable training programs to be assessed.44                                                              44 The education authorities will not use the online platform to assess teacher performance or training needs directly. Rather, it provides a database for recording training needs and the quality of training. In addition, teachers, trainers, and other key stakeholders will be able to use the database to conduct analyses for the purposes of accountability and to improve the quality of services provided. 38   140. Third, it introduces a professional cadre of core teachers who are deployed in every school to provide school-based training, coaching, mentoring, and just-in-time support to individual teachers and teams of teachers at the school level. It also develops Principal Advisors to support pedagogic managers from a cluster of schools. 141. Lastly, it establishes a multi-purpose and integrated online platform for CPD that provide access to e-learning, individualized e-support, knowledge, and interactions with peers and experts. Thus, training and support are brought to the teacher through on-site expertise and through expertise provided at a distance, but customized to the needs of the teachers. 142. These different strategies do not constitute hermetically sealed packages. For example, the training and support provided by core teachers and through the online platform will be complementary. Core teachers will help other staff use the online platform more effectively. They will also use the online platform themselves to access knowledge, interact with other Core Teachers and stay linked to the Lead Teacher Training Universities. In addition, Core Teachers will serve as mentors for teacher interns. The Core Teacher will use training needs assessment system to help shape CPD programs. The online platform will serve multiple purposes for all aspects of teacher education. C. Program for Results 143. This PforR consists of two parts. The first part is defined by four results areas, which are part of the NTEP, that together contribute to meeting the overall goals of ETEP over the period 2016-2021. These four results areas constitute the principal strategies for establishing an effective and well-functioning CPD system that can provide school-based support and training to teachers and principals. These strategies or results areas consist of (i) improving capacity of Lead Teacher Training Universities and central teacher management units to enhance teacher and principal education effectiveness; (ii) developing a teacher and principal training needs assessment system to inform CPD program development; (iii) providing school-based and continuous professional development for teachers and principals; and (iv) ensuring that teachers and principals have access to CPD programs and resources through an Information and Communication Technologies-based system. 144. The second part will mobilize world-class expertise to help develop and operationalize the strategies for improving school-based CPD for teachers and principals. This technical assistance (TA) will allow MOET and LTTUs to: (i) elaborate and operationalize policies and directives, (ii) inform and improve the technical quality of the expected results and (iii) reinforce the implementation capacity, including monitoring and evaluation, of the Program. Figure 4 below presents the results chain for the overall PforR. 145. The ETEP will support the Government’s proposed NTEP with an IDA credit in the amount of US$95 million to support implementation of the Government’s teacher continuous professional development strategy (one dimension of the NTEP) The PforR will disburse against results that support the enhancements of teacher continuous professional development. 39   Part I: Results Areas: Developing capacity of teacher training institutions to enhance quality of trainings and continuous professional development for school teachers and principals 146. Results Area 1: Improved capacity of Lead Teacher Training Universities and central teacher management units to enhance teacher and principal education effectiveness. MOET has selected eight LLTUs  to provide technical guidance for the development and the implementation of enhanced CPD for teachers and principals. The LTTUs will train and support Core Teachers, (one selected from each of 28,000 schools countrywide), and Principal Advisors (one selected per seven schools) who will subsequently return to their respective schools and school clusters to provide CPD to fellow teachers and principals. Second, the LLTUs will help develop instruments that measure the CPD needs of teacher and principals. Third, the LTTUs will develop content for CPD programs that address global teacher and principal needs (e.g., preparing for the rollout of the new curriculum), teacher and principal professional needs (e.g., programs to help meet new professional standards), and programs that address specific issues or needs identified by teachers and principals. For example, for regions where there is a significant population of ethnic minorities, a LTTU could design programs and delivery methods that target the specific needs of ethnic minority teachers. 147. The LTTUs do not currently have the qualities necessary to play fully their role as technical leaders of the school-based CPD system. Each LTTU will use the Teacher Education Institutional Development Index (TEIDI) to measure their institutional capacity (see below and Box 1). Each LTTU will then prepare a Performance Agreement (PA), which is a contract signed between the LTTU and MOET. In the PA, the university commits to achieving particular results in order to receive resources necessary to enhance their capacity. Signing and implementing the PA and reporting progress on TEIDI scores for individual LTTUs are DLIs for the PforR. 148. The Bank-financed Bihar Enhancing Teacher Effectiveness Operation45 in India currently uses the TEIDI. In Bihar, TEIDI assess the readiness of teacher education institutions to deliver quality teacher education programs and to monitor their accountability. TEIDI has a quantified and weighted checklist of indicators that measure institutional performance. TEIDI can inform planning for effective decision-making at the national and provincial institutional levels. MOET will adapt the TEIDI developed in Bihar to the Vietnam context.                                                              45 The Bihar operation was approved in May 2015 and is the first PforR in the education sector. It aims to improve the effectiveness of primary school teachers. Teacher training colleges play a central role in providing support to teachers and thus their capacity is critical. So far, TEIDI has proven to be easy to use and baseline scores were easily established during the first year of the operation. The straightforward and relatively “low tech” data collection and analysis systems have contributed to its relatively easy implementation. 40   Box 2: Framework for Teacher Education Institution Development Index (TEIDI) TEIDI is a tool to measure LTTU’s efficiency and effectiveness in providing high quality teacher training and professional development. The tool includes indexes used to assess the quality and institutional effectiveness of teacher training institutions in a comprehensive manner. Each index comprises of relevant indicators and once the data is collected, the TEIDI can be calculated on their basis.  Pedagogy in preservice training programs: Ensure teachers be trained in organizing leaner- centered, activity based and participatory learning experiences;  Professional development programs: Ensure teachers be trained in organizing leaner-centered, activity based and participatory learning experiences;  Academic staff: Ensure profile of LTTU faculty developed;  Activities for and direct support to schools: Ensure LTTU practices are not isolated from the classroom practice;  ICT preparedness: Ensure LTTUs maximize the use of ICT in providing regular school-based training and support for teachers;  Governance, Partnership and Performance: Ensure the strengthening institutional organization and governance. ETEP will employ an adapted version TEIDI used in the Bihar operation to assess the efficiency and effectiveness of the LTTUs. TEIDI uses a comprehensive approach for the planning, management and governance of teacher education institutions and diagnosing institutional strengths and weaknesses. The index will help identify the needs for teacher education provisioning, institutional requirements, pedagogical readiness levels, efficient service delivery mechanism, including sensitivity to the needs of the disadvantaged and minority groups so as to provide CPD and improve teacher performance. 149. Results Area 2: Development of teacher and principal training needs assessment systems to inform CPD program development. Teachers and principals will record their learning and support needs and their evaluations of online CPD interventions into a Teacher Education Management Information System (TEMIS). TEMIS will also house data on the new Teacher Professional Standards that MOET is developing to be better align with the requirements of the FCER.46 Teachers and other teacher education actors can use the professional standards and their own self-assessments (and those of other actors, such as the Core Teachers) to identify and prioritize teacher CPD needs. In consequence, the TEMIS would provide an important input for teachers to develop training plans to meet the new professional standards. Furthermore, the TEMIS can easily compile results in different ways to help LTTUs and others prepare specific interventions. The TEMIS will constitute a key resource for CPD decision-making and program development. It will also help evaluate the responsiveness and impact of programs. The PforR tracks the development and deployment of the TEMIS to monitor and meet teacher and principal CPD needs as DLIs. The PforR also rewards the practice of sharing data generated from the TEMIS to different stakeholders through annual DLIs.                                                              46 MOET established teacher professional standards in the 2000s, corresponding to the reform program introduced at that time. 41   150. Results Area 3: School-based and continuous professional development for teachers and principals. Core Teachers are senior or well-regarded school teachers who will provide face- to-face CPD in the schools where they are deployed (e.g., classroom observation, organization of peer-to-peer training; and individual coaching and mentoring). They will receive special training from the LTTUs and then return to their schools of origin to implement an annual work plan of support and training to other teachers. Similarly, Principal Advisors support principals of a cluster of six or seven schools. The Core Teachers and Principal Advisors will receive support from LTTUs throughout the year. The Core Teachers and Principal Advisors will develop annual plans detailing the training and support services they will carry out over the school year. The training and support provided to the Core Teachers and Principal Advisors by the LTTUs and the implementation of the Core Teachers’ and Principal Advisors’ annual plans constitute DLIs. 151. Core Teachers and Principal Advisors will not be new categories of civil servants. They retain their status as regular teachers and principals who are given modified work program with the same compensation, benefits, and conditions as before. Cascade trainers have played similar roles (teachers selected to be trainers for a limited period of time), but in an ad hoc manner. 152. Teachers often see their assignment as cascade trainers to be a sign of status. Becoming a Core Teacher or Principal Advisor should also be considered an appealing opportunity. In addition, Core Teachers and Principal Advisors will have fewer teaching hours,47 receive specialized training and CPD programs. It will be nevertheless be necessary to establish a set of regulations for selection, deployment, training requirements, etc. DLI 1 for the first period will be met when MOET puts in place the regulatory framework for Core Teachers and Principal Advisors.    153. Results Area 4: Teachers and principals have access to CPD programs and resources through an Information & Communication Technology-based system. The MOET aims to put in place a comprehensive Learning Management System that can make available four basic CPD services to teachers and principals: (i) on-demand, interactive e-learning courses for an assortment of continuous professional development needs; (ii) tailored knowledge (e.g., digitized and customizable lesson plans, videos of best practice) personalized support (e.g., help desks); (iii) opportunities to collaborate with peers and experts online (e.g., A social media platform with mobile interface uniquely designed for teachers) and (iv) mechanisms to allow beneficiaries to evaluate CPD services received. 154. Teachers and principals are familiar with using ICT for training and other purposes. The MOET has an incipient Learning Management System that will provide the basis for further development. Teachers already have access to e-learning courses through this system. In addition, the MOET’s Connecting Schools Initiative has been in place for two years and provides e-learning to secondary schools. Both the VNEN and SEQAP projects are planning to use this as a platform for the transfer of their training programs. VNEN has recently launched a video competition that produced several thousand videos made by teachers on their teaching practice. Once curated, these can eventually be made available to a wider audience through the LMS.                                                              47 Because of the current surplus of teachers and the relatively few hours of instruction given by each teacher 42   155. Implementation Support. Under this Part, ETEP will also support overall coordination, management, monitoring and evaluation, and supervision of the implementation of the Program by MOET in collaboration with concerned agencies and the independent verification agent. Box 3: School-based teacher education shows results in Palestine The Teacher Education Improvement Project An increasing amount of research shows that teacher quality is one of the most significant determinants of student learning.1 This has been exemplified by the West Bank and Gaza Teacher Education Strategy, which has been supported by the Bank-financed Teacher Education Improvement Project (TEIP). TEIP comprises two components: (i) expanding Practicums for preservice teacher trainees; and (ii) providing in-service training to under-qualified teachers to help them transition from ‘subject teachers’ into ‘class teachers’. The World Bank and others have studied this experience intently, using both qualitative and quantitative approaches. Some of this research focused specifically on the TEIP, whereas other work examined similar initiatives addressing the education needs of Palestinian children. Here are selected findings:  Higher performing schools tend to have more teachers who have benefited from a greater amount of professional development.  Successful schools tend to have teachers who are confident, have high levels of job satisfaction, and benefit from on-going professional development and training.  Beneficiaries of the inservice component adopted a more open and searching (i.e., professional) attitude, as evidenced by the establishment of ‘communities of practice’ that used ICT chat room facilities to allow for discussions with peers and faculty/trainers.  Student teachers who benefited from more Practicums tended to: (i) maintain a flexible, creative and motivating classroom environment; (ii) use ICT more regularly and (iii) employ different teaching strategies.  Teachers receiving school-based inservice support tended to encourage students to engage in experimentation and collaborative work projects; and (ii) employed methods that met the needs of different types of students. The experience of the Kufor Quod Girls’ Secondary School is particularly illustrative. The director employed school-based teacher supervisors who assessed teacher performance and implemented customized training programs. They successfully served to transfer effective, adaptive and innovative teaching approaches.3 TIEP also supported the development of the Readiness to Teach Index (RTTI) comprising 21 competencies based on 21 professional standards. Participating institutions used the RTTI to evaluate student teachers and as a framework for developing teacher education programs. The design of ETEP has been informed by programs such as TEIP that emphasize hands-on school-based training and continuous professional development. Similar to TEIP, it intends to use ICT to connect teachers to each other and to the world of education research and practice. And it adopts a rigorous measure of teacher professionalism to guide the development of teacher education that is more adapted and adaptable to the needs of teachers in the classroom. These are all proven methods to further professionalize teachers and improvement the learning experience. 1. Noah Yarrow et al. (2014). M. Cochran-Smith & K.M. Zeichner (eds), (2005). Studying teacher education: The report of the AERA panel on research and teacher education. London: Lawrence Erbaum. 2. World Bank. 2014. Learning in the face of adversity: the UNRWA education program for Palestine refugees. Washington, DC: World Bank Group. 3. Hana Brixi, Ellen Lust, and Michael Woolcock, (2015) “Trust, Voice, and Incentives: Learning from Local Success Stories in Service Delivery in the Middle East and North Africa,” Washington, DC: World Bank. doi: 10.1596/978-1-4648-0456-4. License: Creative Commons Attribution CC BY 3.0 IGO   43   Part II: Technical Assistance   156. ETEP includes technical assistance in the following areas:   Program management capacity and relevant country systems. These include financial management and environmental management. It also includes strengthening the Recipient’s institutional capacity for the implementation of the Program, in strategic and crosscutting areas such as: (i) issuing and implementing teacher education policies, standards, and regulations; ii) coordinating strategies, actions, and initiatives across teacher education institutions and agencies to ensure system-wide coherence; (iii) monitoring and evaluating the implementation, impact and cost-effectiveness of teacher education programs, interventions, and policies to inform future initiatives; and (iv) promoting knowledge sharing, innovation, and communications in the domain of teacher education.  Expertise in specific technical areas to ensure satisfactory achievement of expected results. These will include, inter alia, developing and implementing effective school-based CPD, innovative CPD programs, IT infrastructure and architecture for the online platform that will host the LMS (Learning and Management System) and TEMIS (Teacher Education Management Information System), professional profile of Core Teachers and Principal Advisors and relevant regulations and selection and appointment procedures, teacher professional standards, training needs assessment methods, and training programs in selected subject areas.  Institutional strengthening of training institutions. TEIDI will measure the effectiveness of LTTUs to provide quality and relevant services in a timely manner. Similar measures exist in certain OECD countries, but few developing countries. Consequently, the MOET will need to mobilize expertise in this area as well.  Third party validation. The MOET will need to identify an independent entity to undertake performance audits and confirm DLI achievement. Third party studies will provide external validation for key performance data and implementation activities. The MOET may wish to hire a firm or institution with strong educational expertise and country system understanding to provide this service. 157. Selected technical assistance activities have been included in the Program Action Plan to ensure that the most critical TA is in place when ETEP becomes effective. Gender and ethnic minority informed strategies integrated into ETEP 158. Measures and strategies are included throughout the ETEP Operation to encourage the teacher education system to focus on the education needs of ethnic minorities, girls and women and female students. Programs will also aim to attract and support ethnic minority teachers and principals, as well as administrators and other education sector leaders. This is believed to help ensure academic success of ethnic minority children, particularly girls. Some examples of such targeted support include: 44    Results Area 1: As part of their institutional capacity building initiatives, LTTUs will acquire the expertise to develop CPD programs, content, delivery methods and assessment systems that help all teachers and principals teach ethnic minority children more effectively. In addition, the LTTUs will conduct research activities, invest in capacity building for educational specialists in ethnic minority issues and prioritize ethnic minority and woman lecturers in their human resource development plans.  Results Area 2: New teacher and principal professional standards will include the mastery of methods and approaches to address culturally sensitive issues, especially those related to teaching ethnic minority children and girls. CPD needs assessment tools and systems will take into consideration the capacity of teachers and principals to work with ethnic minority students and girls. Special care will be taken to evaluate the CPD needs of ethnic minority teachers and principals to inform the development of programs that effectively respond to their professional needs.  Results Area 3: The training and support programs for Core Teachers and Principal Advisors will bolster their capacity to provide CPD to teachers and principals (particularly those who are ethnic minority) that specifically responds to the needs of ethnic minority and female students. The DOETs will be strongly encouraged to select ethnic minority and female teachers and principals to become Core Teachers and Principal Advisors.  Results Area 4: A specific support mechanism will be designed throughout program implementation to ensure that ethnic minority teachers are equipped with adequate computer skills before the commencement of e-learning courses and that they will continued to receive IT supports in not only access to devices but also technical problems during and after e-learning courses. D. Program Financing 159. The estimated funding envelop for the NTEP is US$255 million over a period of five years (2017 – 2021). Table 1 shows that this includes a commitment from the Government of Vietnam (GOV) of US$160 million (63 percent of the total cost) and the ODA source of US$95 million (or 37 percent of the total cost). 160. There will be a separate account for ETEP funding to monitor the implementation of ETEP – funded activities. Government recurrent and capital expenditures to implement the NTEP will be within the corresponding recurrent and capital envelopes which are approved by the Government and allocated by MOF. E. Program Beneficiaries 161. The direct beneficiaries of ETEP will include: (i) more than 600,000 teachers and principals, which account for nearly 80 percent of the total number of general education teachers and principals in the country; (ii) 28,000 core teachers and 4,000 principal advisors; (iii) 3,200 lecturers, researchers and staff from the LTTUs; and (iv) MOET staff. Indirect beneficiaries of ETEP will be general education students; pre-service teachers; and Departments of Education and 45   Training (DOETs) and Bureaus of Education and Training (BOETs) staff. Particular focus will be paid to encouraging ethnic minority and female teachers and principals to become Core Teachers and Principal Advisors. F. Results Chain 162. One of the objectives of the NTEP is to enhance the continuous professional development provided to general education teachers and principals in schools. In essence, the enhanced CPD system will provide more training and support that is of high quality, relevant, and timely -- directly to teachers. Internationally, this is considered best practice for teacher training and support. The CPD will be provided through two vehicles: (i) Core Teachers and Principal Advisors, who will be assigned to schools and school clusters to provide face-to-face training and support and (ii) an online platform that can provide needs-based, interactive training and support directly to teachers and principals. A rigorous training and support needs assessment system, based on professional standards and context-specific needs, will inform the development of training programs and applications tailored to individual teachers and principals, as well as to school staff. To ensure the dynamism and quality of the enhanced CPD system, eight Lead Teacher Training Universities (LTTUs) in respective catchment areas (corresponding approximately to regions) will train and provide support to Core Teachers and Principal Advisors; develop the online services and applications; and help develop the needs assessment systems. These LTTUs will constitute the technical leaders of this school-based CPD system.   46   Figure 4: Results Chain for the Enhancing Teacher Education Program for Results   G. Program Key Results and Disbursement-Linked Indicators 163. The Program Development Objective of the ETEP is: to strengthen teacher education institutions to enhance teacher and principal effectiveness through improved continuous professional development.  The following program key results indicators have been defined:  PDO Indicator 1: Improved performance of the eight Lead Teacher Training Universities to provide CPD, as measured by the Teacher Education Institutional Development Index.  PDO Indicator 2: Number of teachers and principals satisfied with the new school- based continuous professional development. 164. These PDO indicators will be measured annually and data will be provided from the MOET’s annual progress report on the DLIs, as well as other sources. 47   165. Intermediate Level Indicators are:  Intermediate Indicator 1: Number of LTTUs that sign Performance Agreements with MOET (cumulative).  Intermediate Indicator 2: Number of TEMIS-generated annual reports on CPD needs by DOET  Intermediate Indicator 3: Number of deployed Core Teachers and/or Principal Advisors having completed six or more priority training modules.  Intermediate Indicator 4: Number of teachers completing all online modules of the CPD program for the new curriculum through the Learning Management System. H. Disbursement-linked Indicators 166. The ETEP DLIs will disburse against the achievement of outputs and outcomes derived from the results chain (see above) and the Results Framework (Annex 2). The disbursement model proposes five DLIs, linked to the four results areas. There will be five annual disbursement periods, thus ensuring a phased approach for the roll-out of the innovations to be introduced through the Program. The DLIs are outlined below, with details presented in Annex 3. DLIs have been prioritized from the indicators described above, based upon their importance for successful implementation and the need to incentivize outputs necessary for the program’s success. 167. Verification Protocols: Achievement of these DLIs will be verified annually and verification results will be used as basis for disbursement. For scalable DLIs, payment will be made in proportion to the achieved results for each period after they have been verified. For non- scalable DLIs, the whole target must be achieved for each period to trigger the corresponding payment. Verification protocols are briefly described below:  Data source/agency: The Program Management Unit (PMU) will be the main data provider. All data about achievement of results for each DLI in each period should be collected by MOET before verification of results. IOs are responsible for submitting reports to MOET regarding their implementation results against DLIs. MOET will then aggregate information for results verification.  Verification Entity: An Independent Verification Agency (IVA) will be selected to verify certain results, for example, the number of core teachers and principal advisors receiving face-to-face training from LTTUs.  Verification Procedures: In order to validate disbursement requests submitted by MOET, the IVA will verify all DLIs. The results reported by MOET as achieved under the Program will be verified through a paper audit and physical inspection that tests the accuracy and quality of results claimed. In accordance with good audit practice, physical verification will take place against a sampling framework and frequency. On this basis, the IVA will prepare a results verification report which will be shared with MOET and the World Bank. A key use of the Results Verification Report will be to determine the amount of the eligible disbursement to be made on the results achieved. If the Bank finds that the disbursement request meets the terms of the credit, the Bank 48   will disburse the corresponding funds to the MOF. Further details on the verification process for each DLI are included in Annex 3.  Scalable vs. Non-scalable DLIs: In the first year/period of ETEP, most DLIs will be non-scalable because preparation steps must be taken before further results can be achieved for the following periods. For example, rules regarding the selection and deployment of Core Teachers and Principal Advisors need to be developed before they are trained; or an ICT-based system needs to be built before e-courses can be delivered to teachers and principals. From the second year onward, most DLIs will become scalable, i.e. payments will be made in proportion to achievement of DLIs. This will allow IOs to have both incentives and flexibility in planning for and achieving results. Definitions and description of DLI achievement are specified in Table 5 of Annex 3. V. Institutional Arrangements for Implementation A. Institutional Responsibilities 168. The Ministry of Education and Training (MOET) is responsible for the strategic management of the ETEP and, by extension, the strengthening and institutionalization of school- based teacher continuous professional development. This includes: (i) issuing and implementing relevant teacher professional development policies and regulations; (ii) establishing and monitoring new teacher professional standards; (iii) identifying teacher- and school-level demand for school-based CPD; (iv) assessing the effectiveness of CPD interventions; and (v) using assessments and evaluations to shape CPD programs and initiatives. Through its Program Management Unit, MOET will also track and report on ETEP DLIs and transmit requests for disbursements to the World Bank. MOET will operate the monitoring and evaluation (M&E) system that will enable government to report on progress toward achieving the development objective indicators and intermediate indicators of the results framework. MOET will also manage the implementation of the LTTU Performance Agreements. 169. The Department of Education and Training (DOET) is the provincial authorities for general education. It is responsible for managing teacher deployment and service within its catchment areas, including in-service teacher training and CPD. DOETs will coordinate efforts to improve CPD. This will include administrative oversight of Core Teacher and Principal Advisors and ensuring all CPD stakeholders at local level implement the CPD policies. DOETs will also track training needs assessments through the TEMIS, as well as training programs taken by teachers and principals, oversee evaluations of teacher standards, and undertake quality control of the CPD apparatus and process taking a province-wide perspective. 170. The LTTUs are responsible for implementing improvements to CPD envisaged by the NTEP through training, research, online services, and support to Core Teachers and Principal Advisors, as well other interventions. Successfully accomplishing this mandate will help the MOET meet DLIs. However, the ETEP technical assessment concluded that the LTTUs will need to build their capacity to undertake many of these new responsibilities. Each LTTU will prepare a Performance Agreement to be sign with MOET and MOF. The Performance Agreement will specify the needed institutional, human and technical resources to improve its capacity. The performance agreements will indicate how any given input will help the LTTU meet its new 49   mandate. LTTU-specific TEIDI targets will inform the development of the performance agreement. 171. The LTTUs will: (i) create innovative training content and delivery mechanisms, maximizing the use of ICT solutions; (ii) provide training and support for Core Teachers and Principal Advisors; (iii) coordinate with DOETs in the in assessing teacher training and support needs; and (iv) develop and manage demand-driven knowledge management tools. 172. Because of the potential risks associated with using country systems for technical assistance to support the LTTUs, it was agreed to with the Recipient that the Operations Manual would include instructions for IDA to provide systematic guidance for the execution of technical assistance. This would consist of prior agreements with IDA on key milestones in the execution of Technical Assistance such as terms of reference, selection of experts, and reception of interim and final outputs. Such measures would ensure that TA can be mobilized in a timely and effective manner with minor disruptions to the implementation of the proposed Program. 173. The network of Teacher Training Institutions (TTIs) will soon undergo a major reorganization. This Program constitutes a response to the dwindling need for new teachers, the transformation of many into multi-disciplinary universities and colleges, and the Government’s decision to close others. The reorganization is also mandated by the FCER and is currently under implementation. It will take the ETEP and the NTEP as points of departure. Until its finalization and approval, the MOET prefers not to consider them as key actors in the ETEP, although individual LTTUs may decide to collaborate with other TTIs and include relevant actions in their Performance Agreements. The Government may use some of the Technical Assistance provided to support analytical work (policy analysis, strategy development), technical support (planning, feasibility studies) and other technical inputs necessary for developing or implementing a TTI reorganization plan. One eventuality after the reorganization would be that certain TTIs could partner with LTTUs, either through the Program Agreements or other mechanisms. Any reorganization of the network of TTIs would need to be closely coordinated with the DOETs and the MOET. 174. The MOF, through the State Bank of Vietnam (SBV), will receive the funds disbursed by the World Bank as per the terms of the IDA credit financing agreement. It will be responsible for making the appropriate resources available to the relevant institutions (e.g., the MOET and the LTTUs through their Performance Agreements) so that LTTUs can undertake key interventions necessary for meeting the DLIs in a timely, efficient and effective manner. 175. Independent Verification Agency48 will provide independent confirmations of the results reported by LTTUs and the MOET. The MOET will select an IVA within six months of effectiveness as indicated in the Program Action Plan.                                                              48  An Independent Verification Agency will be contracted no later than six months after Program Effectiveness 50   Figure 5: ETEP Institutional Implementation Arrangement49                                                                49 The Government Program to reorganize and rationalize the network of Teacher Training Institutions is currently under development and will complement the NTEP and ETEP. Once this program is finalized and approved, other TTIs may be added to the ETEP institutional arrangements. LTTUs are nevertheless free to collaborate with other TTIs and even include such actions in their Performance Agreements. 51   Annex 2: Results Framework Matrix Cumulative Target Values50 Core Data DLI Results Indicators Unit Baseline Period Data source Year 1 Year 2 Year 3 Year 4 Year 5 collection Dec 2017 Dec 2018 Dec 2019 Dec 2020 Jun 2022 The Program Development Objective (PDO) is: to strengthen teacher education institutions to enhance teacher and principal effectiveness through improved continuous professional development Program Development Objective Indicators PDO Indicator 1: Improved performance of the eight Lead Teacher 10% 15% increase Training increase Annual TEIDI Baseline over the Universities Percentage over the assessment established baseline in LTTUs, (LTTUs) to provide 1 of TEIDI N/A baseline in Annual reports for each TEIDI score MOET CPD as measured score TEIDI Annual M&E LTTU for each by the Teacher score for reports LTTU Education each LTTU Institutional Development Index (TEIDI). PDO Indicator 2: Number of teachers and principals TEMIS reports satisfied with 2 Number 0 0 0 160,000 320,000 480,000 Annual Annual M&E MOET school-based reports continuous professional development.                                                              50 Targets may be different from DLIs. The Results Framework represents probable achievements of the Program, whereas the DLIs are binding agreements. 52     Cumulative Target Values Core Data DLI Results Indicators Unit Baseline Period Data source Year 1 Year 2 Year 3 Year 4 Year 5 collection Dec 2017 Dec 2018 Dec 2019 Dec 2020 Jun 2022 IR 1: Number of LTTUs that sign Annual M&E Performance 1 Number 0 4 8 8 8 8 Annual MOET reports Agreements with MOET. IR 2: Number of TEMIS-generated TEMIS annual reports on 2 Number 0 0 0 15 30 45 Annual Annual M&E MOET CPD needs by report DOET. IR 3: Number of deployed Core Teachers and/or LMS/ MOET Principal Advisors 3 Number 0 0 6,900 13,800 20,700 28,600 Annual Annual M&E having completed reports six or more priority training modules. IR 4: Number of teachers completing all online modules LMS/ of the CPD program Number 0 0 0 320,000 480,000 640,000 Annual Annual M&E MOET for the new reports curriculum through the LMS.   53   Cumulative Target Values Core Data DLI Results Indicators Unit Baseline Period Data source Year 1 Year 2 Year 3 Year 4 Year 5 collection Dec 2017 Dec 2018 Dec 2019 Dec 2020 Jun 2022 Intermediate Results Indicators IR 1: Number of LTTUs that sign Annual M&E Performance 1 Number 0 4 8 8 8 8 Annual MOET reports Agreements with MOET. IR 2: Number of TEMIS-generated TEMIS annual reports on 2 Number 0 0 0 30 40 50 Annual Annual M&E MOET CPD needs by report DOET. IR 3: Number of deployed Core Teachers and LMS/ MOET Principal Advisors 3 Number 0 0 0 6,900 13,800 28,600 Annual Annual M&E having completed reports six or more priority training modules. IR 4: Number of teachers completing all online modules LMS/ of the CPD program Number 0 0 0 320,000 480,000 640,000 Annual Annual M&E MOET for the new reports curriculum through the LMS. 54   Table 4: Description of Indicators Indicator Name Indicator Description/Definition PDO Indicators PDO Indicator 1: Improved performance of the eight The TEIDI survey instrument will be developed by MOET (in collaboration with key stakeholders, such as Lead Teacher Training Universities (LTTUs) to provide LTTU authorities). Each LTTU will use the surveys to determine its annual TEIDI score (1-100). Each LTTU CPD as measured by the Teacher Education Institutional establishes its baseline score during Year 1 or Year 2 (TEIDI baselines and targets are established as part of Development Index (TEIDI) each LTTU’s Performance Agreement, some of which will be signed during Year 1 and others during Year 2). Satisfaction with school-based CPD is measured by a composite index based on questionnaires filled out by teachers and principals through the Teacher Education Management Information System (TEMIS). The PDO Indicator 2:   Number of teachers and principals questionnaire will ask teachers and parents to rate school-based CPD in terms of: (i) availability; (ii) relevance satisfied with the new school-based continuous to meeting new teacher and principal standards; (iii) usefulness in the classroom; and (iv) contribution to professional development.  improved student outcomes. The Operations Manual describes assessment methods and criteria. The TEMIS will provide the first survey results during Period 3. Responses can be disaggregated by gender and ethnicity of teachers and principals. Intermediate Results Indicators Each LTTU signs a Performance Agreement (PA) that describes the actions it will undertake to achieve TEIDI targets and will contribute to DLIs. The accepted format of Performance Agreements will be developed with IR 1: Number of LTTUs that sign Performance WB’s technical assistance to be included in the Operations Manual. There are a total of eight Lead Teacher Agreements with MOET and MOF (cumulative). Training Universities supported by ETEP, each of which is expected to have a PA in place by the end of the second year of the program. TEMIS-based reports on CPD needs come from many sources: external assessments of professional standards; IR 2: Number of TEMIS-generated annual reports on teacher and principal self-assessments; suggestions made by teachers, principals, Core Teachers, Principal CPD needs by DOET Advisors, DOET authorities and other authorized teacher education actors. The format of the DOET reports will be determined in the Operations Manual. When possible, data well be disaggregated by sex and ethnicity. IR 3: Number of deployed Core Teachers and/or MOET will identify 6 priority training modules that each Core Teacher and Principal Advisor should complete. Principal Advisors having completed six or more Three priority training modules will be provided during the initial training provided at the LTTUs. The others priority training modules. will be made available online or through other in-service training mechanisms. MOET will begin rolling out the reform of general during the 2018-2019 school year, corresponding approximately with the second year of ETEP implementation. MOET plans to develop several types of support IR 4: Number of teachers and school principals and training to prepare teachers, particularly through CPD. As part of this array of support, the Ministry intends completing all online modules of the CPD program to elaborate a set of online modules that will be made available through the LMS. As it is expected that the LMS through the Learning Management System. will be fully operational by Period 3, the first measurement will take place during that period. The target by the end of Year or Period 5 is 640,000 teachers and school principals completing mandatory modules of the CPD program through the LMS system, of which 600,000 are teachers and 40,000 are school principals. 55   Annex 3: Disbursement-linked Indicators, Disbursement Arrangements and Verification Protocols Table 5: Disbursement-Linked Indicator Matrix Total DLI As % of Allocation Total Timeline for DLI achievement DLI description DLI Baseline (Million Financing US$) Amount Period 1 Period 2 Period 3 Period 4 Period 5 DLI 1: Institutional capacity of LTTUs to support the new Continuous Professional Development system for teachers and principals enhanced LTTUs have A total of eight (8) LTTUs have conducted Each LTTU conducts Each LTTU conducts Each LTTU conducts Performance Agreements inadequate TEIDI baseline assessment and signed an annual TEIDI an annual TEIDI an annual TEIDI (PAs) are signed with the capacity to Performance Agreements with MOET. assessment and MOET assessment and MOET assessment and MOET MOET by all eight support the publishes the TEIDI publishes the TEIDI publishes the TEIDI LTTUs. The PAs include continuous scores on its website. scores on its website. scores on its website. actions that will enhance professional LTTU institutional development. capacity to meet TEIDI targets and that will contribute to meeting DLIs. Each of a total of eight (8) PAs is priced at Each of the total of Each of the total of Each of the total of $1,125,000 to be signed in Periods 1-2. eight (8) LTTU annual eight (8) LTTU annual eight (8) LTTU annual However, a minimum number of 4 signed PAs is TEIDI assessments TEIDI assessments TEIDI assessments required before disbursements commence. published on the published on the published on the MOET website is MOET website is MOET website is Any undisbursed allocation from Periods 1 and priced at $750,000. priced at $750,000. priced at $750,000 2 will be withheld until corresponding Performance Agreements are signed. Any undisbursed Any undisbursed Any undisbursed allocation will be allocation will be allocation will be reallocated or reallocated or reallocated or cancelled. cancelled. cancelled. Allocated amount 27 28 % US$ 9,000,000 US$ 6,000,000 US$ 6,000,000 US$ 6,000,000 DLI 2: Systems are in place to: (i) measure whether teachers and principals meet new professional standards; (ii) track and record evaluations of CPD programs; and (iii) record assessments of CPD needs of teachers and principals. Systems are in place to: Existing MOET issues a MOET establishes a Up to Fifteen (15) Up to Thirty (30) Up to Forty-Five (45) (i) measure whether systems to circular describing Teacher Education DOETs publish a DOETs publish a DOETs publish a teachers and principals assess teacher guidelines and Management TEMIS-generated TEMIS-generated TEMIS-generated 56   Total DLI As % of Allocation Total Timeline for DLI achievement DLI description DLI Baseline (Million Financing US$) Amount Period 1 Period 2 Period 3 Period 4 Period 5 meet new professional and principal procedures to assess Information System CPD Annual Report CPD Annual Report CPD Annual Report standards; (ii) track and CPD needs teachers and principals (TEMIS) that: (i) for its respective for its respective for its respective record evaluations of are not against new tracks teacher and province. The Annual province. The Annual province. The Annual CPD programs; and (iii) aligned with professional standards. principal professional Report present trends Report present trends Report present trends record assessments CPD requirements standards rating; (ii) on professional on professional on professional needs of teachers and of FCER nor records beneficiary standards, beneficiary standards, beneficiary standards, beneficiary principals. adequately evaluations of CPD assessments of CPD assessments of CPD assessments of CPD inform the programs and programs and programs and programs and development interventions; and (iii) assessments of CPD assessments of CPD assessments of CPD of CPD records teacher and needs of teachers and needs of teachers and needs of teachers and programs principal CPD needs. principals. principals. principals. Undisbursed Each TEMIS- Each TEMIS- Each TEMIS- Each TEMIS- allocation will be generated Annual generated Annual generated Annual generated Annual reallocated or Report for any given Report for any given Report for any given Report for any given cancelled. DOET is priced at DOET is priced at DOES is priced at DOES is priced at US$ 133,333 for an US$ 133,333 for an US$ 133,333 for an US$ 133,333 for a annual total of up to annual total of up to annual total of up to total of up to forty-five fifteen (15) Annual fifteen (15) Annual thirty (30) Annual (45) Annual Reports. Reports. Reports. Reports. Remaining Remaining allocation allocation from Year from Years or Periods Any remaining Any remaining or Period 3 is 3 and 4 is available allocation from this allocation from this available for for disbursements at Year or Period may be Year or Period may be disbursements at $133,333 per Annual carried over to carried over to $133,333 per Annual Report in excess of subsequent Periods subsequent Periods Report in excess of target number of target number of Reports for Year or Reports for Period 4. Period 5. Remaining allocation Remaining allocation from Periods 3 and 4 from Periods 3, 4 and may be carried over to 5 will be reallocated the subsequent Period. or cancelled after Period 5. 57   Total DLI As % of Allocation Total Timeline for DLI achievement DLI description DLI Baseline (Million Financing US$) Amount Period 1 Period 2 Period 3 Period 4 Period 5 Allocated amount: 18 19% US$ 2,000,000 US$ 4,000,000 US$ 2,000,000 US$ 4,000,000 US$ 6,000,000 DLI 3: Number of Core Teachers and Principals Advisors selected and trained to provide systematic face-to-face school-based continuous professional development to teachers and principals. LTTUs train 25,000 Core A corps of MOET issues a LTTUs provide basic training to 25,000 Core Teachers and 3,600 Principal Advisors. Teachers and 3,600 trained Circular that defines Principal Advisors to school-based the rules and provide face-to-face specialists that regulations for the school-based CPD to provide selection, roles, teachers and principals. systematic responsibilities and face-to-face working conditions of school-based Core Teachers and CPD to Principal Advisors. teachers and principals does not exist. For every two of the six modules of the basic training completed by a Core Teacher (up to 25,000 Core Teachers) or a Principal Advisor (up to 3,600 Principal Advisors), US$ 233 will be disbursed. Every Core Teacher and Principal Advisor is expected to complete six modules to be fully certified Consequently, a total of three disbursements can be made per Core Teacher or Principal Advisor over the life of the Program. Any remaining undisbursed allocation will be reallocated or cancelled after Period 5. Allocated amount: 22 23 % US$ 2,000,000 US$ 20,000,000 DLI 4: Number of Core Teachers and Principal Advisors providing school-based continuous professional development to teachers and principals. 25,000 Core Teachers A corps of Up to 6,000 Core Up to 12,000 Core Up to 18,000 Core Up to 25,000 Core and 3,600 Principal trained Teachers and 900 Teachers and 1,800 Teachers and 2,700 Teachers and 3,600 Advisors provide face- specialists that Principal Advisors Principal Advisors Principal Advisors Principal Advisors N/A to-face school-based provide provide school-based provide school-based provide school-based provide school-based continuous professional systematic continuous continuous continuous continuous development to teachers face-to-face professional professional professional professional school-based development to development to development to development to 58   Total DLI As % of Allocation Total Timeline for DLI achievement DLI description DLI Baseline (Million Financing US$) Amount Period 1 Period 2 Period 3 Period 4 Period 5 and principals every CPD to teachers and principals teachers and principals teachers and principals teachers and principals year. teachers and during Period 2, as per during Period 3, as per during Period 4, as per during Period 5, as per principals their individual annual their individual annual their individual annual their individual annual does not exist. plan. plan. plan. plan. Each Core Teacher Each Core Teacher Each Core Teacher Each Core Teacher and Principal Advisor and Principal Advisor and Principal Advisor and Principal Advisor having provided having provided having provided having provided school based CPD is school based CPD is school based CPD is school based CPD is price at US$200 for an priced at US$ 200 for priced at US$ 200 for priced at $200 for an annual total target of an annual total target an annual total target annual total target of up to 6,000 Core of up to 12,000 Core of up to 18,000 Core up to 25,000 Core Teachers and 900 Teachers and 1,800 Teachers and 2,700 Teachers and 3,600 Principal Advisors. Principal Advisors. Principal Advisors. Principal Advisors. Any remaining Remaining allocation Remaining allocations Remaining allocations allocation from this from Period 2 is from Periods 2 and 3 from Periods 2, 3 and Period may be carried available for are available for 4 are available for over to subsequent disbursements at $200 disbursements at $200 disbursements at $200 Periods. for each Core Teacher for each Core Teacher for each Core Teacher and Principal and Principal and Principal Advisors in excess of Advisors in excess of Advisors in excess of the targets for Period the targets for Period the targets for the 3. Remaining 4. Remaining Period 5. Remaining allocation from allocation from undisbursed Periods 2 and 3 may Periods 2, 3 and 4 allocations from 2, 3, be carried over to may be carried over to 4 and 5 will be subsequent Periods. subsequent Period. reallocated or cancelled after Period 5. Allocated amount: 14 15% 0 N/A US$ 1,380,000 US$ 2,760,000 US$ 4,140,000 US$ 5,720,000 DLI 5: Number of teachers and principals who are satisfied with ICT-based interactive CPD services provided under the Program MOET establishes a Up to 480,000 teachers Learning Management Up to 160,000 Up to 320,000 Up to 480,000 and principals are System (LMS) that teachers and principals teachers and principals teachers and principals satisfied with their access 0 provides ICT-based are satisfied with ICT- (additionally Years or are satisfied with ICT- to ICT-based interactive interactive CPD based interactive CPD Periods Two and based interactive CPD CPD services every year. services. services. Three) are satisfied services.  59   Total DLI As % of Allocation Total Timeline for DLI achievement DLI description DLI Baseline (Million Financing US$) Amount Period 1 Period 2 Period 3 Period 4 Period 5 with ICT-based interactive CPD services.  Each teacher or Each teacher or Each teacher or principal satisfied with principal satisfied with Principal satisfied interactive ICT-based interactive ICT-based with interactive ICT- CPD services is priced CPD services is priced based CPD services is at $12.50 for an at $12.50 for an priced at $12.50 for an annual total target of annual total target of annual total target of up to 160,000 for up to 320,000 for up to 480,000 for Period 3. Remaining Period 4. Remaining Period 5. Remaining allocation from this allocation from Period allocation from Years Period is available for 3 is available for or Periods 3 ad 4 is disbursement in disbursements at available for subsequent Periods. $12.50 for each disbursements at teacher or principal in $12.50 for each excess of the target for teacher or principal in Period 4. Remaining excess of the target for allocation from Period 5. Remaining Periods 3 and 4 is undisbursed allocation available for from Periods 3, 4 and disbursements in 5 will be reallocated subsequent Period. or cancelled after Period 5. Allocated amount: 14 15% N/A US$ 2,000,000 US$ 2,000,000 US$ 4,000,000 US$ 6,000,000 Total Financing 95 100% US$ 8,500,000 US$ 16,880,000 US$ 17,760,000 US$ 23,140,000 US$ 28,720,000 Allocated:       60   Table 6: DLI Verification Protocols Protocol to evaluate achievement of the DLI and data/result verification Scalability of Review Disbursements Year or Definition51/ Description of achievement DLI Period Data source/ Verification (Yes/No) Procedure52 agency Entity DLI 1: Institutional capacity of LTTUs to support the new Continuous Professional Development system for teachers and principals enhanced. MOET will meet this DLI when at least four A total of eight (8) LTTUs sign Performance Agreements with LTTUs have MOET. The Performance Agreements will conducted TEIDI include activities that contribute to meeting baseline assessment DLIs and activities that contribute to meeting and signed The MOET PMU Director will collect copies of the signed TEIDI targets. The LTTUs will use the PA Independent Performance Performance Agreements and submit them to the MOET and templates that are described in the Operations Verification 1-2 Agreements with Yes Performance the Bank. The IVA will determine whether the PAs are of a Manual. Each LTTU will conduct a TEIDI Agency MOET. Agreements quality satisfactory to the Bank in terms of process, content baseline assessment prior to drafting its PA. (IVA) signed and format, as described in the Operations Manual. The TEIDI assessment instruments and between each procedures will be described in the Operations LTTU and the Manual. MOET Each LTTU conducts This DLI will be met when a LTTU publishes Annual The MOET PMU Director will confirm whether an annual an annual TEIDI an annual TEIDI report on the MOET website, TEIDI TEIDI report for a given LTTU has been published on the assessment and MOET which presents the findings of an annual TEIDI Reports for MOET website and report to the MOET and the Bank. The publishes the TEIDI assessment. 3-5 No each LTTU IVA IVA will determine whether each published annual TEIDI scores on its website. report is of a quality satisfactory to the Bank in terms of its MOET development process and procedures, as described in the website Operations Manual.                                                              51 Additional definitions are given below. 52 Actual sampling methodology and size may be subject to modification 61   Protocol to evaluate achievement of the DLI and data/result verification Scalability of Review Disbursements Year or Definition51/ Description of achievement DLI Period Data source/ Verification (Yes/No) Procedure52 agency Entity DLI 2: Systems are in place to: (i) measure whether teachers and principals meet new professional standards; (ii) track and record evaluations of CPD programs; and (iii) record assessments CPD needs of teachers and principals. MOET issues a This DLI will be met when MOET issues a The MOET PMU Director will send copies to the Bank and Circular describing Circular that describes the guidelines and MOET of the officially issued Circular that describes the guidelines and procedures to assess teachers and principals guidelines and procedures to assess teachers and principals procedures to assess against new professional standards. The MOET against the new professional standards. The IVA will teachers and principals will officially establish the new standards for MOET determine whether the guidelines and procedures described in 1 No IVA against new teachers and principals before it issues the website the Circular are of a quality satisfactory to the Bank in terms professional standards. above-mentioned Circular. The Circular will be of the development process and procedures, as defined in the of a quality satisfactory to the Bank in terms of Operations Manual. its development process and procedures, as defined in the Operations Manual. MOET establishes a The MOET PMU Director will submit a report on the Teacher Education This DLI will be met when the MOET establishment of the TEMIS to the MOET and the Bank. The Management establishes a TEMIS that can report whether IVA will determine whether the TEMIS meets the criteria Information System teachers and principals meet professional defined in the Operations Manual. (TEMIS) that (i) tracks standards; can record evaluations by teachers whether teachers and and principals; and can record CPD needs as MOET 2 principals meet expressed by teachers and principals No IVA website professional standard; (ii) records beneficiary The TEMIS will be of a quality satisfactory to evaluations of online the Bank, as described in The TEMIS will be CPD program; and (iii) compatible with the LMS as per the relevant teacher and principal terms described in the Operations Manual. CPD needs. DOET publishes The MOET PMU Director will collect the evidence that the TEMIS-generated This DLI will be met when a DOET publishes a TEMIS generate DOET Annual Reports on CPD have been 3-5 Yes MOET report IVA CPD Annual Reports TEMIS-generated CPD Annual Report for its produced and validated by the DOET authorities. This for their respective respective province. The Annual Report evidence will be submitted to the MOET and the Bank. The provinces that present presents trends on professional standards, IVA will determine whether the TEMIS reports are of a 62   Protocol to evaluate achievement of the DLI and data/result verification Scalability of Review Disbursements Year or Definition51/ Description of achievement DLI Period Data source/ Verification (Yes/No) Procedure52 agency Entity trends on professional beneficiary assessments of CPD services and quality satisfactory to the Bank in terms of structure, content standards, beneficiary assessments of CPD needs of teachers and and data disaggregation, as described in the Operations assessments of online principals. Manual. CPD services and assessments of CPD All information presented in the Annual needs of teachers and Reports can be disaggregated by gender and principals by DOET. ethnicity. The Operations Manual will describes the disaggregation methodology. DLI 3: Number of Core Teachers and Principals Advisors selected and trained to provide face-to-face school-based continuous professional development to teachers and principals. MOET issues a This DLI will be met when the MOET issues a The MOET PMU Director will collect evidence of the Circular that defines Circular that defines the rules and regulations issuance of the Circular defining the rules and regulations the rules and regarding the selection, roles, responsibilities, regarding the selection, roles, responsibilities, and working regulations for the and working conditions of Core Teachers and conditions of Core Teachers and Principals Advisors and selection, roles, Principal Advisors, of a quality satisfactory to 1 No MOET IVA submit to the Bank and MOET. The IVA will determine responsibilities and the Bank. whether the Circular is of a quality satisfactory to the Bank in working conditions of These rules and regulations will ensure that EM terms of its development process and procedures, as defined Core Teachers and and female candidates for Core Teacher and in the Operations Manual. Principal Advisors Principal Advisor positions to guard against any discrimination. LTTUs provide basic This DLI will be met every time a Core TEMIS 2-5 training to 25,000 Teacher or Principal Advisor completes two of Core Teachers and the six training modules necessary for their LTTUs The MOET PMU Director will collect evidence from LTTUs 3,600 Principal certification. Typically, Core Teachers and and the TEMIS on the number of basic training modules Advisors. Principal Advisors are expected to complete the completed by each Core Teachers and Principal Advisors and six modules over a three Year or Period submit it to the MOET and the Bank. The IVA will then timeframe. Consequently, three disbursements Yes IVA conduct an independent performance audit of the basic can be made for each Core Teacher or Principal training provided to Core Teachers and Principal Advisors, to Advisor over the life of the Program. The determine whether the training provided was of a quality and number of Core Teachers and Principals against quantity satisfactory to the Bank, as described in the which disbursements can be made will depend Operations Manual. on how many receive the initial training course of two modules at a LTTU. The modules are 63   Protocol to evaluate achievement of the DLI and data/result verification Scalability of Review Disbursements Year or Definition51/ Description of achievement DLI Period Data source/ Verification (Yes/No) Procedure52 agency Entity provided face-to-face or online by a LTTU. Core Teachers and Principal Advisors will nevertheless be able to practice in schools after having completed two modules at a LTTU. All data regarding Core Teachers and Principal Advisors can be disaggregated by gender and ethnicity to guard against any discrimination. DLI 4: Number of Core Teachers and Principal Advisors providing school-based continuous professional development to teachers and principals. The DLI will be met when a Core Teacher or The MOET PMU Director will submit a consolidated report 25,000 Core Teachers Principal Advisor implements his or her annual of the number of Core Teachers and Principal Advisors who and 3,600 Principal plan to provide CPD at their school or cluster. have implemented their Annual Plan to the MOET and the Advisors provide face- Al the beginning of the school year, Core Bank based on the TEMIS reports. The IVA will conduct an to-face school-based Teachers and Principal Advisors prepare an independent performance audit of the implementation of the continuous annual plan, which is endorsed by the principal annual Core Teachers and Principal Advisors CPD plan. professional or the DOET and recorded in the TEMIS. At development to the end of the school year, principals and teachers and principals DOET authorities will record whether the MOET report every year. 2-5 annual plan was implemented as expected. The Yes based on the IVA Operations Manual will provide further details TEMIS data on the process. Responses can be disaggregated by gender and ethnicity of Core Teachers and Principal Advisors, as well as by the ethnic gender composition of teachers and students in schools. The Operations Manual describes assessment methods and criteria, as well as disaggregation methodology. DLI 5: Number of teachers and principals who are satisfied with ICT-based interactive CPD services provided under the program 64   Protocol to evaluate achievement of the DLI and data/result verification Scalability of Review Disbursements Year or Definition51/ Description of achievement DLI Period Data source/ Verification (Yes/No) Procedure52 agency Entity MOET establishes a Learning Management This DLI will be met when the MOET has The MOET PMU Director will submit evidence that ICT- System (LMS) that developed a Learning Management System, based interactive CPD platform has been established and provides ICT-based which is an ICT-based interactive platform that submit it to the MOET and the Bank. The IVA will confirm 2 No MOET report IVA interactive CPD provides access to all online CPD programs and that the platform is of a quality satisfactory to the Bank in services. services. The LMS will meet the relevant terms terms of its development process and main characteristics, as of the Operations Manual. defined in the Operations Manual. MOET report The MOET PMU Director will submit evidence of the Up to 480,000 teachers This DLI will be met when the indicated based on the number of teachers and principals satisfied with the ICT- and principals are number of teachers and principals express their TEMIS data based interactive CPD services they received. The IVA will satisfied with ICT- satisfaction with the ICT-based interactive CPD then conduct an independent performance audit of the TEMIS based interactive CPD services. Satisfaction with CPD services will assessment of the ICT-based interactive CPD services to services every year. be measured by a composite index that consists confirm that the survey is of a quality and quantity acceptable of two dimensions. The first groups the kind of to the Bank, as described in the Operations Manual. interactive support provided, such as a social media platform, video-taped lesson plans, help- desks, chat rooms, and classroom observation via camcorder. It is expected that not all these services will come online at the same time. The second dimension measures the perceived 3-5 quality of ICT-based interactive CPD services Yes IVA with regards to, inter alia: (i) availability; (ii) relevance to meeting new teacher and principal standards; (iii) usefulness in the classroom; and (iv) contribution to improved student outcomes. Responses can be disaggregated by gender and ethnicity of teachers and principals. Consolidated reports will only be generated for individual DOETs and at the national level. The Operations Manual describes assessment methods and criteria, as well as disaggregation methodology. 65   Table 7: Bank Disbursement Table # DLI Bank Of which Deadline for Minimum DLI Maximum DLI Determination53 of Financing Amount to be financing Financing DLI value to be value(s) expected disbursed against achieved and verified DLI allocated available Achievement1 achieved to to be achieved for value(s) 4 to the DLI for Prior trigger Bank results disbursements of disbursements Bank Financing 2 purposes 3 1 Institutional capacity of LTTUs 28 % n/a 30 June 2022 US$ 4,500,000 US$ 27,000,000 A minimum of four Performance Agreements to support the new Continuous must be signed so that a scalable disbursement Professional Development can be made of US$ 1,125,000 for each system for teachers and Performance Agreements signed and verified. principals enhanced Annual TEIDI assessments published so that a scalable disbursement can be of US$ 750,000 for each Annual Report. 2 Systems are in place to: (i) 19% n/a 30 June 2022 US$ 2,000,000 US$ 18,000,000 US$ 2,000,000 will be disbursed when the measure whether teachers and MOET issues a Circular describing guidelines principals meet new and procedures to assess teachers and principals professional standards; (ii) against new professional standards. US$ track and record evaluations of 4,000,000 will be disbursed when MOET CPD programs; and (iii) record establishes a Teacher Education Management assessments of CPD needs of Information System (TEMIS). A scalable teachers and principals. disbursement can be made of US$ 133,333 for each Annual Report published by a DOET (up to 15, 30, & 45 Annual Reports are expected for Periods 3, 4, & 5 respectively). Remaining allocations from one Period may be carried forward to subsequent Periods. As a consequence, the maximum number of Annual Reports against which disbursements can be made will also increase for the subsequent Period question. 3 Number of Core Teachers and 23 % n/a 30 June 2022 US$ 2,000,000 US$ 22,000,000 US$ 2,000,000 will be disbursed when MOET Principals Advisors selected issues a circular that defines the rules and and trained to provide regulations for the selection, roles,                                                              53 Refer to Disbursement Mechanism below for further details. 66   # DLI Bank Of which Deadline for Minimum DLI Maximum DLI Determination53 of Financing Amount to be financing Financing DLI value to be value(s) expected disbursed against achieved and verified DLI allocated available Achievement1 achieved to to be achieved for value(s) 4 to the DLI for Prior trigger Bank results disbursements of disbursements Bank Financing 2 purposes 3 systematic face-to-face school- responsibilities and working conditions of Core based continuous professional Teachers and Principals Advisors. A scalable development to teachers and disbursement can be made of US$ 233 for every principals. two of the six modules of basic training completed by a Core Teacher or Principal Advisor. 4 Number of Core Teachers and 15 % n/a 30 June 2022 US$ 1,380,000 US$ 14,000,000 A scalable disbursement can be made of US$ Principal Advisors who provide 200 for each Core Teacher or Principal Advisor school-based continuous having implemented his or her annual school- professional development to based CPD plan. (6,000, 12,000, 18,000 & teachers and principals 25,000 Core Teachers and 900, 1,800, 2,700 and 3,600 Principal Advisors for Years or Period s 2, 3, 4, & 5 respectively). Remaining allocations from one Period may be carried over to subsequent Periods. As a consequence, the maximum number of Core Teachers and Principal Advisors against which disbursements can be made will also increase for the subsequent Period in question. 5 Number of teachers and 15 % n/a 30 June 2022 US$ 2,000,000 US$ 14,000,000 US$ 2,000,000 will be disbursed in Year 2 when principals who are satisfied the LMS is established. A scalable disbursement with ICT-based interactive can be made of US$ 12.5 for each teacher or CPD services provided under principal satisfied with the interactive ICT based the program CPD services (160,000, 320,000, & 480,000 for Years or Periods 3, 4, & 5 respectively). Remaining allocations from one Period may be carried over to subsequent Periods. As a consequence, the maximum number of teachers and principals against which disbursements can be made will also increase for the subsequent Period in question. 67   Disbursement Mechanism: The following further describes determination of annual disbursement amount against each achieved and verified DLI values. 1. DLI #1 a) For Periods 1 and 2: i) A total of eight (8) PAs are expected to be signed in the first two (2) Periods, and each signed PA is priced at $1,125,000, or a total maximum allocation of $9 million. However, there is a minimum of four (4) PAs to be signed in the first Period, i.e. to trigger Bank disbursements in the first Period, a minimum of four (4) PAs needs to be signed. Otherwise, Bank disbursements will be withheld until a minimum of four (4) PAs are signed. ii) Each additional signed PA after the first four (4) signed PAs is priced at $1,125,000 and is scalable over these two (2) Periods. iii) Any remaining allocation for the first two (2) Periods will be withheld until corresponding PAs are signed in subsequent Periods. b) For Years 3, 4 and 5: i) An amount of US $6 million is allocated for each of Years or Periods 3-5, for a total maximum allocation of US $18 million. ii) Publication of an annual TEIDI assessment by each of the eight (8) LTTUs is priced at US$750,000, and Bank disbursements in any given Period depends on the number of annual TEIDI assessments that are published. There is no minimum requirement. Any remaining allocation in a given year cannot be carried over to subsequent Periods, but can be available for reallocation or cancellation. 2. DLI #2 a) For Period 1: i) An amount of US $2 million is allocated and is a pass/fail, i.e., if the target is not achieved in this designated Period, allocated funds will be reallocated or cancelled. b) For Period 2: i) An amount of US $4 million is allocated and is a pass/fail, i.e. if target is not achieved in this designated Period, allocated funds will be reallocated or cancelled. c) For Periods 3-5: 68   i) Allocations of US $2 million, US $4 million, and US $6 million are made for Periods 3-5 respectively, for a total maximum allocation of US $12 million. ii) Each TEMIS-generated Annual Report for any given DOET is priced at US $133,333 and the annual disbursement amount depends on the number of Annual Reports published on the MOET website (up to an agreed number for each Period) as follows: a. For Period 3: Up to fifteen (15) DOETs publish TEMIS-generated Annual Reports on the MOET website for a total potential disbursement of US $2 million. Remaining allocation is available for disbursements in subsequent Periods. b. For Period 4: Up to thirty (30) DOETs publish TEMIS-generated Annual Reports on the MOET website for a total potential disbursement of US $4 million. If there are any remaining allocations from the previous Period, however, they can be disbursed if more than 30 Annual Reports are published (at US $133,333 per report until remaining allocations are exhausted). c. For Period 5: Up to forty-five (45) DOETs publish TEMIS-generated Annual Reports on the MOET website. If there are any remaining allocations from the previous Periods, however, it can be disbursed if more than forty-five (45) Annual Reports are published (at US $133,333 per report until remaining allocations are exhausted). iii) For Periods 3 and 4: If annual target are not met, remaining allocations are available for disbursements in the subsequent Periods. iv) Remaining allocations after Period 5 will be cancelled or reallocated. 3. DLI #3 a) For Period 1: i) An amount of $2 million is allocated and is a pass/fail, i.e., if target is not achieved in this designated Period, allocated fund will be reallocated or cancelled. b) For Periods 2-5: i) A total maximum cumulative allocation of US $20 million can be disbursed. ii) Each two of six priority modules are completed by a Core Teacher or Principal Advisor are priced at US $233 and the annual disbursement amount depends on the number of priority modules completed by Core Teachers and Principal Advisor in a given Period. 4. DLI #4 a) For Period 1: No target or allocation is set. 69   b) For Periods 2-5: i) Allocations of US $1.38 million, US $2.76 million, US $4.14 million and US $5.72 million are made for Periods 2-5 respectively, for a total maximum allocation of US $14 million. ii) Each Core Teacher or Principal Advisor implementing his or her annual plan is priced at US $200 and annual disbursement amounts depend on the number of Core Teachers or Principal Advisors who have implemented his or her plan in a given year or period, subject to annual parameters as follows: a. For Period 2: Up to 6,000 Core Teachers and 900 Principal Advisors implement their annual plans. Remaining allocations from this Year or Period are available for disbursements in subsequent Periods. b. For Period 3: Up to 12,000 Core Teachers and 1,800 Principal Advisors implement their annual plans. If there are any remaining allocations from the previous Period, however, they can be disbursed if more than 12,000 Core Teachers and 1,800 Principal Advisors implement their annual plan (at US $200 per Core Teacher or Principal Advisor until remaining allocations are exhausted). Remaining allocations from this Period are available for disbursement in subsequent Periods. c. For Period 4: Up to 18,000 Core Teachers and 2,700 Principal Advisors implement their annual plans. If there are any remaining allocations from the previous Years or Periods, however, they can be disbursed if more than 18,000 Core Teachers and 2,700 Principal Advisors implement their annual plan (at US $200 per Core Teacher or Principal Advisor until remaining allocations are exhausted). Remaining allocations from this Period are available for disbursement in subsequent Period. d. For Period 5: Up to 25,000 Core Teachers and 3,600 Principal Advisors implement their annual plans. Any remaining allocations after Year or Period 5 will be reallocated or cancelled. 5. DLI #5 i) For Period 1: No target or allocation is set. ii) For Period 2: An amount of US $2 million is allocated and is a pass/fail, i.e., if target is not achieved in this designated Period, allocated fund will be reallocated or cancelled. iii) For Periods 3, 4 and 5, each teacher or principal satisfied with the CPD services is priced at $12.50 and annual disbursement amounts depend on the number of satisfied teachers and principals, subject to the annual parameters: iv) For Period 3: Up to 160,000 teachers and principals satisfied with the CPD services. Remaining allocations from this Period are available for disbursements in subsequent Years or Periods. v) For Period 4: Up to 320,000 teachers and principals satisfied with the CPD services. If there are any remaining allocations from the previous Period, however, they can be disbursed if more than 320,000 teachers and principals satisfied with the CPD services (at US $12.50 per teacher or principal until remaining allocations are exhausted). Remaining allocations from this Period are available for disbursements in subsequent Period. 70   vi) For Period 5: Up to 480,000 teachers and principals satisfied with the CPD services that year. If there are any remaining allocations from the previous Periods, however, they can be disbursed if more than 480,000 teachers and principals satisfied with the CPD services (at US $12.50 per teacher or principal until remaining allocations are exhausted). Any remaining allocations after Period 5 will be reallocated or cancelled. 71   Annex 4: (Summary) Technical Assessment 176. In preparing the technical assessment, the team reviewed data on the current teacher education system on the supply side, including the network of teacher training institutions (TTIs), provincial Departments of Education and Training (DOETs) and district-level Bureaus of Education and Training (BOETs), and demand side, consisting of general school across sixty-three provinces of the country, as well as other sources of data from associated projects and programs. The team has also met with all related departments of the Ministry of Education and Training (MOET) and other Ministries and Government bodies; completed field visits to eight LTTUs - the lead providers of teacher education services; and organized a workshop with these eight LTTUs and relevant MOET departments. The Technical Assessment, together with other assessments, were used to provide inputs for developing the provisional Systemic Operations Risk Ratings (SORT) and Program Action Plan. 177. The Government and the World Bank team consider CPD to be perhaps one of the most important aspects of the overall reform of general education. The nature of the expected changes to teacher methodology, as well as the findings of much research on teacher professionalism, point to continuous professional development as having the greatest influence on teacher competencies and behavior. By “bringing training to the teacher”, enhanced CPD (provided face-to face and online) can help teachers more effectively adopt new methods and master new competencies. Furthermore, CPD shaped to meet the training and support needs of individual teachers or teachers within one school are particularly effective because they respond to specific conditions of instruction. Thus, this assessment considers that the ETEP’s focus on CPD a well-supported strategic choice that is likely to have the most catalytic effect for the success of the reform. I. Strategic Relevance 178. Vietnam has impressive teachers and managers. This in great part speaks to a robust teacher education system established over the last two decades. Both pre-service and in-service training has been effective, particularly considering the pace the corps grew to meet demand. 179. To keep up with changing demands and challenges, MOET enacted several reforms over the last fifteen years to improve the effectiveness of general education teachers, such as new professional standards used to both evaluate teachers and determine their professional development needs. To help teacher meet these standards, MOET developed new teacher training approaches, as well as an extensive menu of continuous professional development opportunities. Furthermore, the MOET introduced many important innovations, such as the collective development of lessons plans at the school level, peer support programs, and use of mentors for new teachers. 180. During the same period, MOET put in place a new regulatory framework for Teacher Training Institutions (TTI) to improve management of teacher education, as well as introduce a new career structure. Education managers, TTI lecturers and staff, and Departments of Education 72   and Training (DOET) and Bureaus of Education and Training (BOETs)54 tend to be well qualified reflecting the success of the existing incentive structure. Furthermore, a relatively equitable deployment of teachers speaks to good planning. 181. However, changes in learning objectives necessarily mean changes in what teachers need to know and do. Thus, it is time for the teacher education system to adapt once again to new circumstances, expectations and aspirations. 182. Teachers tend to rely on rote and “chalk and talk” methodologies.55 To date, this has apparently served most students well. However, much research and experience indicate that these approaches are often less effective when the objective is to meet complex learning goals. This plus the diverse character of different communities (often changing rapidly because of internal migration) and the range of learning challenges within a single classroom, requires teachers to take on a more sophisticated set of teaching methodologies. 183. Essentially, the FCER will require teachers to become increasingly professional, i.e., to adapt methodologies and knowledge to real-time problems as they arise in his or her classroom and school. This is similar to the professional models of medical doctors, researchers, policy analysts and other knowledge-based fields. Training and supporting such teachers requires greater interaction between professionals, reciprocity, hands-on mentorship and coaching and on-time advice to solve identified problems. In sum, a key factor for helping teachers during this new phase is the quality of continuous professional development (CPD). Box 4: Today’s teachers will need to  Master new integrated subjects;  Use facts as a starting point, not an end point for teaching;  Encourage learning by doing;  Make changes to lessons, activities or methodology on the spot when needed;  Have high expectations of all their students;  Continuously evaluate their own practice;  Seek new evidence-based international and national knowledge to inform their teaching;  Work collaboratively with other teachers - sharing knowledge, providing feedback and serving as mentors and coaches; and  Form strong relationships with students, as well as their families and communities. 184. Continuous professional development in the best education systems often has three basic qualities.56                                                              54 DOET are education authorities answerable to provincial governments and BOETs authorities to district governments. 55 Such as recitation, lecturing, and seat work, with a considerable focus on discipline. 56 Bill and Melinda Gates Foundation, 2014. Teachers Know Best: Teachers’ views on professional development. http://collegeready.gatesfoundation.org/wp-content/uploads/2015/04/Gates-PDMarketResearch-Dec5.pdf. Department of Education and Training, Victoria, Australia, 2005. Professional Learning in Effective Schools: The seven Principles of Highly Effective Professional Learning. http://www.education.vic.gov.au/Documents/school/teachers/profdev/proflearningeffectivesch.pdf. 73    Training and support services must be directed towards the school level, without diluting the quality of that support;  Knowledge about innovations and best practice needs to be more freely available to all teachers and in forms that can help teachers meet real-time challenges; and  A more comprehensive assessment and feedback mechanism for training needs should be in place to drive the development of support and training for teacher in the school and classroom. 185. Teacher CPD in Vietnam has elements of the above features, but is not organized along these lines. CPD is not systematic, although the potential is there. Teacher CPD has been of good quality and value but it should now be conceptualized as a coherent system that ensures that CPD is evidence-based, demand-driven, can be modified to respond to feedback. The NTEP and ETEP come at the right time to address this issue. II. Program Technical Soundness 186. The technical assessment finds that the NTEP in general, and the continuous professional development program supported by ETEP in particular, to be well-designed and technically sound with a high probability of delivering quality and sustained training and support to general education teachers. Technical experts and key Vietnam stakeholders consulted during the assessment agreed that achieving the objectives of the FCER would require modifications to the existing teacher education system. The technical assessment also concludes that the NTEP’s strategies for improving continuous professional development for general education teachers and principals are appropriate for achieving this objective. 187. However, the assessment team warned that the proposed changes to teacher education might represent too radical a modification. Implementing teacher education reform is rarely an easy exercise because it can challenge well-ingrained cultures of teacher practice. However, ETEP builds on existing characteristics of the teacher education system that have proven to be successful and will introduce changes in an incremental manner. For example, schools and education authorities regularly assign mentoring, training and oversight roles to seasoned teachers recognized for their expertise. The development of a cadre of Core Teachers builds on this experience. Vietnam already provides e-learning to teachers. MOET has used standards (which they have regularly updated) for several decades to measure teacher performance and to design training programs. Finally, teacher-training institutions already develop materials for teacher self- study, in-service teacher training programs and e-learning programs. 188. Furthermore, several programs exist throughout Vietnam that provide similar types of teacher support, often financed by development partners. For example, the Global Partnership for Education funds the Vietnam Escuela Nueva Project, where counsellors provide school staff engage in school-based support and mentoring activities. Several programs have piloted interactive online teacher training. The School Education Quality Assurance Program has conducted systematic evaluations of training and training needs. 189. However, the teacher education system conducts most of these initiatives in an ad hoc manner, without a strategic and integrated vision to ensure long-term effectiveness or 74   sustainability. With the efforts support by ETEP, the teacher education system will be able to delivers continuous professional development in a systematic and sustainable fashion and tailored to the specific needs of teachers. 190. Finally, the evident quality of Vietnam general education and the existing levels of teacher professionalism should facilitate the implementation of FCER overall and the NTEP more specifically. Building on a successful education system has some advantages. A. Evaluation of Technical Risks 191. Each results area supported by ETEP constitutes one facet of the overall strategy to enhance teacher continuous professional development. Core Teachers and online interactive training are complementary; a rigorous assessment system allows training providers to take into account local context and priorities; and using LTTUs as the leaders of continuous professional development can link teachers in classrooms to the world of education innovation. 1. Lead Teacher Training Universities as drivers for change 192. The MOET has selected eight national Teacher Training Universities57 to facilitate an enhanced teacher continuous professional development provided face-to-face and online. As such, these Lead Teacher Training Universities (LTTUs) will provide the necessary technical and institutional guidance to ensure that teachers receive continuous professional development aligned with both their specific needs and FCER requirements. They will train CTPA to ensure that “cascade training” does not dilute once it arrives at the school level. Supporting Core Teachers will allow for greater interaction between teachers within the same school; the teacher education online platform will promote interaction nationwide. The online platform should also facilitate knowledge transfer, access to innovative training programs, and evidence-based program development. The LTTUs will be the primary clients of the improved assessment systems. 193. The Government has chosen these eight institutions for three reasons. First, the MOET only chose amongst National Teacher Training Universities (of which there are fourteen) because these have considerably more technical, logistical, academic and institutional capacity than the other 110 teacher training institutions in Vietnam.58 Furthermore, MOET decided that working with all fourteen National LTTUs from the start of the Program would be unwieldy. Finally, MOET selected these TTUs because of their geographic location. These eight institutions are located such as to serve the seven regions of Vietnam in an equitable manner.59 194. One concern raised by the technical assessment team was that the Teacher Training Universities might be reluctant to change their mandate. Most have focused on pre-service training of secondary school teachers and teacher retraining. MOET and several DOETs have contracted                                                              57 Approximately 110 institutions nation-wide provide pre-service and in-service teacher training: National Teacher Training Universities, education departments within other universities, regional teacher training colleges, education departments within colleges and specialized training institutes for education managers. 58 The MOET is currently developing a program to reorganize and rationalize the network of TTIs. 59 The eighth institution, the National Institute of Education Managers trains, supports and undertakes research on school principals and other education system administrators. It will be responsible for training and supporting Principal Advisors. 75   individual faculty members to design or provide training, but rarely at an institutional level. The MOET responds to this concern in two ways. First, the Performance Agreements signed with the LTTUs will leverage these institutions to focus on the meeting the results defined by ETEP. The Performance Agreements stipulate that the institutional improvement plans prepared and implemented by the LTTUs must meet certain criteria if the institutions are to receive funding. 195. Second, the MOET intends to establish a Teacher Education Institutional Development Index that measures the capacity of these institutions to perform certain key functions. Progress along this index will represent one of the Disbursement-linked Indicators of ETEP, and a determining aspect of the Performance Agreement. These elements constitute satisfactory responses to this risk. 2. Core teachers as change agents at school level 196. The notion of core teachers is not new to Vietnam. Typically, DOETs identify senior teachers with proven records of accomplishment to receive training that MOET or they organize. These teachers then become trainers in their own right. The selection tends to be ad hoc (although projects financed by development partners may require them to meet particularly standards) and there is no guarantee that they would be chosen to perform the function again. Under the model that will be supported by ETEP, the Core Teachers will take on a more formal and permanent role and will be responsible for coordinating, organizing and delivering onsite professional development activities at schools. They will also help teachers do their training needs assessments. Being a Core Teacher or Principal Advisor will be a long-term assignment. 197. Essentially, the Core Teachers and Principal Advisors will constitute the “glue” of the enhanced CPD system, providing a bridge between teacher training universities, regional authorities, and teachers and principals. The LTTUs will provide them with training and – more importantly - ongoing support after they return to their schools. School and DOET authorities will be responsible for selecting and managing the Core Teachers (DOETs more generally and school authorities on a daily basis). In addition, Core Teachers can help innovations, training, and support services developed by LTTUs, the MOET/DTEM, DOETs, and other partners to have the greatest impact. 198. The technical assessment team expressed concern as to whether the selection and management of the Core Teacher and Principal Advisor would be transparent and merit based. During discussions, most stakeholders indicated that teachers selected to be trainers or specialists were usually respected, apt and accountable. Education authorities will select Core Teachers on a set of criteria developed by MOET in coordination with provincial DOETs and LTTUs. Education authorities will carried out the selection process in such a manner to ensure that EM teachers will not face any discrimination concerning their nomination as Core Teachers or Principal Advisors. 199. The technical assessment team questioned whether there were sufficient incentives for a teacher to want to become a Core Teacher or Principal Advisor. Core Teachers and Principal Advisors will not constitute new categories of civil servants. Considering that they retain their status as regular teachers and principals with the same compensation, benefits, work conditions and responsibilities as before, why would they want to become Core Teachers or Principal Advisors? It is important to note, however, that cascade trainers provided these services on top of 76   their other teaching responsibilities. Teachers and principals tended to consider this role as a sign of status. Similarly, teachers and principals should also consider the role of Core Teachers and Principals Advisors as appealing opportunities. They will likely have few teaching hours, have access to specialized training and CPD programs, and benefit from priority access to online services. It will be nevertheless be necessary to put in place a special set of regulations for selection, deployment, training requirements, etc. Putting in place the regulatory framework for Core Teachers and Principal Advisors is included in DLI 1 for the first period. 3. ICT-based support for teaching and learning 200. For MOET, the development of an online platform to support teacher education is a high priority. Authorities point to other countries that leverage technology to address teachers’ need for more collaborative and customized learning opportunities. In Vietnam, there have also been efforts to apply ICT solutions to improve professional development at scale. Some LTTUs have started to provide e-learning training programs. However, design and quality of e-learning programs/modules need substantial improvements. 201. ETEP will support the development of an online platform dedicated to teacher education that can provide the basis for several tools and applications to the benefit of teachers and principals. A Teacher Education Management Information System (TEMIS) will track teachers’ profiles as defined by new teacher performance. The online platform will house just-in-time help desks; applications that help prepare lesson plans, innovative and easily accessible classroom materials, data analysis tools and other resources. A Learning Management System (LMS) will enable teachers to develop training plans and provide access to self-paced and guided e-learning courses. Video capabilities will allow for interactive learning and classroom observation. Social media applications will facilitate teacher exchange, sharing, and peer networking.60 202. The technical assessment team warned that ICT is not a panacea. MOET recognizes this, which is why education authorities conceptualize the online platform to enhance the teacher education system, not replace any particular component. The Core Teachers, the LTTUs, and school management will be able to augment their capabilities by harnessing ICT applications. However, more importantly, local CPD actors can help teachers get the most out of online applications. Providing two complementary types of school-based support can be attractive: teachers will be get help through the online platform and through Core Teachers (both supported by the LTTUs). Core Teachers will be selected based on a set of criteria developed by MOET in coordination with provincial DOETs and LTTUs. The selection will be carried out in such a manner to ensure that EM teachers will not face any discrimination with regards to the nomination as Core Teacher or Principal Advisor. Furthermore, the online system will also serve as a resource for Core Teachers who will frankly have a challenging job to do. 203. Teachers and principals are familiar with using ICT for training and other purposes. The MOET has an incipient Learning Management System that will provide the basis for further                                                              60 One barrier to effective professional learning is teachers’ lack of time to participate in and/or finish a course because they are already too busy with teaching. Summer vacation is the only time when they can put teaching aside and spend time on professional learning. A mobile application that allows teachers to follow a course at anytime and anywhere would significantly address this barrier. 77   development. Teachers already have access to e-learning courses through the LMS. In addition, the MOET’s Connecting Schools Initiative has been in place for two years and provides e-learning to secondary schools. Both the VNEN and SEQAP operations are planning to use this as a platform for the transfer of their training programs 4. Assessment of teachers’ professional development needs 204. MOET hopes to improve considerably the training needs assessment system for CPD. The strategy is to develop a system that both helps teachers mobilize the support necessary to meet new professional standards, as well as to respond to more individual needs, priorities and the specific school context. 205. The assessment team expressed concerns that the Government’s program will rely too much on the online platform to conduct teacher needs assessment. However, the purpose of the online platform is not to determine training needs, but is rather an instrument to help track training needs and provide access to information that could help teachers better articulate their training needs. Teachers will develop their annual CPD plans with the help and support of the Core Teacher, the school principals and their peers (those in the school and those they “meet” online). Subsequently, the school principal will approve the plans and record them in the LMS. 206. The team was impressed by the quality of the initial set of new professional standards that have developed by the MOET, but they remarked that their application could pose three challenges. First, the MOET will need to develop objective criteria for measuring whether (and to what degree) a teacher meets any particular standard. Second, the use of teacher professional standards should not be the only tool used to capture teachers’ CPD needs – the use of the TEMIS should allow for multiple sources of information to establish training and support needs (including, self- assessment). Finally, previous performance evaluations based on standards have proven to be of limited use because assessment processes were not rigorous. The technical assessment team advised that the most important factor will the means by which standards are measured – and how this information will be used -- not the standards per se. B. Evaluation of institutional and managerial risks 207. The teacher education system consists of four principal actors: teacher-training institutions, MOET, DOET and the schools. Whereas the roles and responsibilities of the general education schools will not change significantly, those of these other actors will. 208. MOET will be responsible for the strategic management of enhanced continuous professional development. This includes issuing and implementing relevant teacher professional development policies and regulations; establishing and monitoring new teacher professional standards; identifying teacher-level demand for CPD; tracking the effectiveness of CPD interventions; using assessments and evaluations to shape CPD programs and initiatives; and piloting the National Teacher Education Program, including the enhancements to CPD system. The DOETs will continue to manage and provide CPD at the district and provincial level, but with increasing emphasis on oversight, coordination and quality assurance. 78   209. The technical team observed that the national Teacher Training Universities have typically not had much interaction with the other actors of the teacher education system. This has contributed to the “disconnect” many teachers feel when participating in training and CPD. In response, MOET will use the Performance Agreements as a basic motivating instrument (criteria will include activities that require collaboration with the DOETs). 210. NTEP is an ambitious program with many pieces that should function in a coordinated manner. The technical assessment team expressed concerns regarding the capability of the MOET, the LTTUs and other actors to implement in this regard. 211. Recognizing institutional and managerial shortcomings, the MOET has emphasized the importance of sequencing program implementation. The first year will be devoted to putting in place the regulatory and institutional frameworks necessary for key parts of the program to function (e.g., Core Teachers, role of LTTUs, new professional standards, formats for LTTU Performance Agreements, and the TEIDI). MOET will elaborate many of the details for rolling out the innovations described above during the first year. MOET intends to do with international technical support. 212. In addition, the program will start with simple targets and focus on priority areas. For instance, supporting the roll out of the new curriculum will likely constitute the defining objective of the NTEP during the first years. Core Teachers will focus on helping teachers adopt the new curriculum. In addition, MOET intends to develop the new standards in phases, with the first defined by the new curriculum. At first, the online platform will focus more on helping teachers master the curriculum and new teaching methodologies through help desks and interactive learning. 213. Finally, the key to the success of the NTEP will be the technical and institutional capacity of all actors to take on their new responsibilities, particularly the LTTUs who must guarantee the quality and relevance of CPD materials and approaches. Although the LTTUs represent the best of teacher education in Vietnam, their organizational, technical, and managerial capacity will need significant reinforcement. MOET will also require considerable support as it assumes its role as the strategic managers of teacher education reform. Here is a summary of the principal institutional and managerial constraints:  Weak management capacity of the Direction of Teachers and Education Managers: DTEM’s mandate has been to manage the teaching force of Vietnam and ensure that they are provided with sufficient professional support. As part of its mandate, DTEM has been coordinating a number of teacher training programs and activities to keep them up-to-date. However, these programs are mostly ad-hoc in nature and thus not effective enough to generate a visible change in quality of the teaching staff although certain efforts did produce encouraging results.  Structural limitations of the Lead LTTUs: Presently, most LTTUs are organized based on subject areas (Math, Literature, Biology, Chemistry, Geography, etc.). The FCER now requires that subjects be more integrated at lower levels of general education. LTTUs will need to undergo a reorganization to response to this new requirement. In addition, their 79   lecturers will also have to be equipped with new capabilities to deliver new teacher education programs.  Low ICT preparedness and inadequate ICT infrastructure: Although some LTTUs have had experience in delivering online training programs, their design has not been attractive and facilitative to teachers, especially those who have low computer skills. This would be a big challenge in this Program in that online professional development courses/modules will have to be attractive and user-friendly enough to motivate teachers’ professional learning. At the same time, the capabilities of current ICT infrastructure at LTTUs and MOET level is too modest to meet the demand of the Program design. 214. To address these constraints, ETEP combines considerable design support during preparation, results-based disbursements to incentivize performance, and high quality technical assistance during implementation. The results of the assessments necessary to prepare a PforR have identified the most important institutional constraints. The proposed actions will enhance the capacity of the LTTUs and national and provincial departments responsible for teacher education to take on their new responsibilities. 215. The following activities have been jointly identified with the MOET for capacity building and system strengthening:  Modernized teacher education management through the development and use of: (i) a TEMIS for effective teacher education planning and management; (ii) a TEIDI to measure teacher education institutional effectiveness; and (iii) teacher professional standards to assess teacher CPD needs.  Capacity building for LTTUs in the areas of program development and pedagogy including ICT preparedness, institutional governance and school engagement to ensure its high quality and relevant CPD support for teachers. An action is built in the Program Action Plan and its results will be measured by TEIDI.  An online platform for teacher education: The platform will serve several purposes, including the provision of interactive e-learning and support services; a LMS for tracking teacher training and support needs, completion of training programs; and evaluation of services, etc.  Teacher performance assessment: The MOET has revised and updated secondary education teacher professional standards with support from ADB. However, a set of professional standards for general education teachers including those at primary education level and strengthened assessment techniques and external review should be developed to ensure greater integrity and accuracy.  E-learning and online teacher support services for teachers and Core Teachers (e.g., program design, multi-media content development, contact program structuring, and assessment techniques for effective curriculum supplementation).  Implementing agency capabilities for result-based monitoring, fiduciary, environmental, and social management (particularly with regards to the needs of ethnic groups).  Coordination between LTTUs, TTCS and teacher education management departments at national and provincial levels and schools. 80   C. Program expenditure framework 216. The NTEP is financed by the Government budget and ETEP. The total Program value is US$255 million to be disbursed in five years (2017 – 2022). This ETEP will support the government’s proposed NTEP with an IDA credit in the amount of US$95 million (equivalent). ETEP will use the PforR financing instrument to support implementation of the government’s teacher continuous professional development strategy (one dimension of the NTEP) and provide technical assistance to reinforce country systems for ETEP implementation and in technical areas related to CPD (e.g., online programs of teacher support). The PforR will disburse against results that support the enhancements of teacher continuous professional development (see Table 8). Table 8: Program Costs and Financing by Program Area and Source of Funding (US$ million)* % of Program Current Incremental program budget budget budget budget Preservice teacher training, retraining 112 110 2 43.8 and related curriculum development In-service teacher training and related 48 8 40 18.9 curriculum development Infrastructure development for teacher 83 - 83 32.6 training institutions Other 12 3 9 4.7 Total 255 120 135 100 % total 100 47 53 Of which: Government of Vietnam 160 120 40 63 Enhancing Teacher Education Program 95 - 95 37 Total program budget 255 100 * The table regroups expenditures of the six NTEP Solution Areas into four categories to simplify analysis. 217. Over two thirds of the Government contribution to NTEP is for recurrent expenditures. This is approximately equal to the annual non-salary allocations for teacher continuous professional development and in-service teacher training made by the DOETs and the MOET.61 The sixty-three provincial governments allocate resource to their respective DOETs. MOET considers that all teacher training and support is part of the NTEP, and thus includes these recurrent expenditures as part of the program. 218. ETEP essentially supports part of incremental expenditures of the LTTUs and MOET necessary to implement the NTEP and to meet the DLIs. For example, ETEP supports increasing the capacity of LTTUs to: (i) train and support the Core Teachers and Principal Advisors and (ii) develop and use the online teacher CPD platform. These in turn will require faculty training, the                                                              61 The MOF channels the budgets of the National Teacher Training Universities through MOET. 81   development of CPD programs, new CPD delivery mechanism, software applications, and minor civil works. 5. Financial sustainability 219. The technical assessment considers that the changes to the teacher education system introduced by NTEP to be sustainable, particularly if the Government continues to allocate at least 20 percent of public resources to the education sector62 220. The technical assessment concludes that improvements to CPD that result from the implementation of the ETEP will have a minimal impact on overall government expenditures. The TEMIS will modernize and render more efficient the collection and analysis of CPD data; the LMS will do the same for online CDP services. This increased efficiency is expected to offset any need for additional resource allocations to finance, for example, maintenance and repairs or specialized technical staff). Table 9: Public expenditure on education has rapidly increased over the past decade (VND trillion) 2000 2002 2009 2010 2011 2012 GDP Current 476.6 579.8 1,809.1 2,157.8 2,779.9 3,245.4 2000 prices 476.6 538.1 849.1 903.6 960.0 1,010.4 TGE Current 103.2 133.4 508.0 586.3 706.4 913.6 2000 prices 103.2 123.8 238.4 245.5 244.0 284.4 Education Current 15.6 22.6 93.0 118.5 143.5 192.1 2000 prices 15.6 21.2 62.7 72.4 71.2 80.8 as % of GDP 3.3 3.9 5.1 5.5 5.2 5.9 as % of TGE 15.1 16.9 18.3 20.2 20.3 21.0 Sources: MOF for 2009-12; Vietnam & the World Bank (2005) for 2000 and 2002. Note: GDP figures were updated from the 2004 PER for 2000 and 2002, resulting in a slight change in the education expenditure as a percentage of GDP for 2000 and 2002  221. The MOET and DOETs will select Core Teachers and Principal Advisors from the current cohort of teachers and principals. Thus, the government will not need to recruit any new personnel for this initiative. Furthermore, the government does not intend to create a new category of civil                                                              62 As required by the Circular No 127/2009/TT-BTC dated June 19, 2009. To achieve the overall education budget target at 20 percent of the total state budget, central government requires provinces to allocate at least 20 percent of their respective budgets for education. It also requires provinces to allocate at a minimum the amount determined by the formula and other budget lines, as discussed below, to education. In this sense, the centrally defined education budget is earmarked, at least in theory. 82   servant, with an additional set of benefits, for teachers or principals who serve this function. The Core Teachers and the Principal Advisors will retain their status with the same rights, duties and accountabilities. Education personnel will likely consider these positions to be prestigious and sought-after positions. However, MOET and the DOETs may need to consider other incentives, such as reduced workloads and special training opportunities. The decline in primary education enrollments has led to a surplus of teachers have a relatively low number of teaching hours compared to other countries.63 222. Furthermore, because there will be little demand to recruit new teachers for many years to come, Teacher Training Institutions that have essentially provided preservice training may need to consider reallocating their resources to other ends.64 Some TTIs now provide programs for teachers to attain higher credentials (and thus opportunities for promotions and higher salaries (for a fee). Others have converted some of their facilities for teaching in other disciplines altogether. Savings could also be used to support new CPD programs or conducting relevant research. Financial predictability 223. The technical assessment concluded that the Government has put in place budget and expenditure management systems and practices that will allow ETEP to reach its expected results. However, there are two sources of difficulty. The first the budget cycle is often inconsistent resulting in very late budget decisions and fund transfers, which are disruptive to investment programs. Second, the Government develops medium term plans and annual plans against notional resource availability, an important part of which is outside of central government control. Thus, the key issue is to address delays in budget approval, and better linking annual budgeting and planning process to multi-year plans. B. Economic and Financial Analysis 224. This economic analysis follows ETEP’s value chain wherein more effective teacher continuous professional development leads to higher quality teaching and learning. Those who benefit from this better quality instruction will in turn be able to earn a higher level of income. 225. The MOET intends to enhance the quality and quantity of both face-to-face and online teacher CPD, reflecting similar trends worldwide to bring training closer to teachers. In this analysis, we label these improvements “Enhanced CPD”. The economic analysis will calculate the net present values of the Program investments for two scenarios: with and without the enhanced CPD. This will help assess the cost – benefit ratios of these two options and the feasibility of a continuation of the program using the Government’s own budget in the future.                                                              63 According to the Public Expenditure Review (currently being finalized), Vietnamese teachers spend many fewer hours in the classroom than their peers in other countries. Ironically, academic results are significantly higher than found in countries where the number of official hours is significantly higher. 64 This is one reason why the government plans to reorganize and rationalize the network of TTIs. 83   1. Overall economic benefit estimation 226. The main benefit stream arising from the program is attributable to the improvement in the wages that students can expect to earn after completing their formal schooling. The great majority of children entering primary school today (at age six) will join the work force twelve years from now (age 18). Most will work until the age of 65 and therefore, reap any benefits from enhanced CPD for approximately 47 years. The modeling adopted here use conservative and empirically based estimates to the extent possible. 227. The Program should contribute to higher student completion rates at primary, lower secondary and upper secondary levels. Better-trained teachers should also help improve the quality of learning, and thus students will be more likely to complete the level in which they are enrolled. With higher levels of education, these students will have more opportunities to find jobs with better pay. This is the key reasoning used for this economic analysis. 228. A Mincer regression based on Vietnam Household Living Standard Survey (VHLSS) 2012 data helped determine income levels for various levels of formal schooling. Accordingly the income of primary school completers is expected to increase by US$0.8 per annum (over those who have not completed primary schooling), the income of lower secondary school completers is expected to increase by US$0.9 per annum and the income of upper secondary school completers is expected to increase by US$1.4 per annum. 229. In order to identify the number of beneficiaries, differences in completion rates were calculated for before and after the Program. These were calculated as the difference of the projected completion rate without the Program and the decrease in dropout rate thanks to the Program intervention. As the teachers will receive CPD over three consecutive years, the change in completion rate is assumed to start in the third year of the Program, i.e. 2019. 230. The change in dropout rate is calculated as the multiplication of change in score due to teacher training by the correlation between learning outcome and dropout rate. In this case, the PISA scores are used for calculation, because it a comprehensive and reliable source of information. 231. The improvement in scores due to teacher training is calculated as below: A = B*C*D*E, whereby A: Improvement in scores due to teacher training B: Impact of teacher training on quality of education. Since there is no available data for Vietnam, the team uses the ratio of the India’s Bihar Teacher operation based on the assumption that the impact is similar in two Asian developing countries. C: Number of years of teacher training. It is assumed as 1 as teachers will receive both face- to-face and online training. D: Standard deviation of PISA score, estimated at 16.4 percent. 84   2. Overall economic cost estimation 232. The economic cost estimates of enhancing CPD uses the following assumptions: a) Costs include total program expenditures as well as estimated future costs estimated by year. b) After the completion of ETEP, the government budgets will continue to finance teacher education. The future costs after ETEP closes include maintenance of assets created during program implementation, as well as any recurrent expenditures. 3. Combining cost and benefit calculation 233. The model indicates a conservative estimate of program value of benefits minus costs being US$495 million using the lower scenario. The combined use of face-to-face and online CPD may lower some operational costs and raise maintenance costs. The benefit-to-cost ratio of this scenario is 3.5. 234. To calculate the NPV of ETEP, the team estimated the stream of benefits and costs over the period of 2017 – 2036. Below are the parameters used: a) The impact of the program on the increase in completion rate only start from year three. b) All financial stream forecasts have been made after duly accounting for and adjusting for the expected rate of inflation; long-term inflation is calculated to be roughly 5.35 percent based on historical data. c) Using data from the 2015 Labor Force and Employment Survey, the assessment team estimates the future unemployment rate to be 2.42 percent for school completers. The calculations undertaken factors out the unemployment rate form the economic benefits before computing net cash flow. d) The analysis assumes that work force participation rate for school completers to be 90.5 percent in accordance with the 2011 Labor Force and Employment Survey. The team has factored in the anticipated participation rate has to adjust the economic benefits before computing net cash flow. e) The discount rate used for this analysis is 12 percent. 235. Three key parameters are modeled with a Monte Carlo simulation with 100,000 runs to allow a range of possible values: the earnings premium at different levels of education; the responsiveness of earnings to improvements in school quality and the discount rate. Running the estimates with these parameters generates mean value of US $494.8 million. 85   236. Below are the forecasts for the NPV: Figure 6: NPV Forecast Statistics: Forecast values (US $)65 Trials 0.1 million Base Case 359.5 million Mean 494.8 million Standard Deviation 4,2 million The ICT scenario forecasts a mean NPV of US$ 494.8 million; negative NPV’s are in the 0 percent percentile. 237. The sensitivity of the NPV calculations in the Monte Carlo simulation shows that the uncertainty regarding program benefits comes from the wage premium for upper secondary education, as shown in the figure below. The program intervention affects mainly the quality correlation and the sensitivity analysis shows that the variability in the quality correlation would not influence the benefit calculation. This is an important finding because it indicates the economic value of the program even under lower than expected outcomes regarding the magnitude of program impact.                                                                65 Rounded to the million 86   Figure 7: NPV Sensitivity Calculations 238. In summary, enhanced CPD that introduced by the NTEP will have a positive benefit/cost ratio and thus makes the investment worthy. 87   Annex 5: (Summary) Fiduciary Systems Assessment Purpose of Fiduciary Systems Assessment (FSA) 239. This Assessment follows the principles governing fiduciary assessment for Program-for- Results as set out in OP/BP 9.00 (PforR Financing). This notes that the Program fiduciary systems should provide “reasonable assurances that the financing proceeds will be used for intended purposes, with due attention to the principles of economy, efficiency, effectiveness, transparency, and accountability.” The FSA of the Program integrates findings in three areas:  The Procurement systems were assessed to determine the degree to which the planning, bidding, evaluation, contract award and contract administration arrangements and practices provide a reasonable assurance that the Program will achieve the intended results through its procurement processes and procedures;  The Financial management systems were assessed to determine the degree to which the relevant planning, budgeting, accounting, internal controls, funds flow, financial reporting and auditing arrangements provide a reasonable assurance on the appropriate use of Program funds and safeguarding of its assets; and  The Governance systems were assessed to determine the extent to which they address risks of fraud and corruption, including the practice of disclosing information by the MOET and LTTUs, the use of complaint mechanisms, and how such risks are managed and mitigated in light of the government’s commitments under the Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results Financing (Anti-Corruption Guidelines or ACGs). Procurement 240. The procurement under the ETEP to be supported is carried out in accordance with the national procurement laws and regulations. The procurement activities under ETEP will be governed by Public Procurement Law No. 43/2013/QH13 approved by Vietnam National Assembly on November 26, 2013 and effective from July 1, 2014; Decree No. 63/2014/ND-CP dated June 26, 2014 issued by the Vietnam Government for detailing the implementation of selective articles of the Law on Public Procurement regarding Contractor Selection; and relevant Circulars. 241. A review of the Public Procurement Law No. 43/2013/QH13 and the Decree No. 63/2014/ND-CP found that they: (i) require procurement planning to be prepared and approved in a timely manner with proper linkage to budget availability; (b) mandate open competitive bidding as the default procurement method and provide detailed procedures for open competitive bidding; (c) provide a reasonably clear allocation of responsibilities and accountabilities to the procuring entities, employers and other owners; and (d) provide sanctions against unacceptable practices including fraud and corruption. In addition, the Government has issued various Model Bidding Documents66 for goods, works and consulting services (including forms and templates of contract) and bid evaluation templates for use by procurement practitioners, in which the NCB Harmonized                                                              66 Please refer to Circulars No. 05/2015/TT-BKHDT dated June 16, 2015; No. 03/2015/TT-BKHDT dated May 6, 2015 and No. 01/2015/TT-BKHDT dated February 14, 2015. 88   Bidding Documents have been issued altogether with Asian Development Bank and The World Bank in July 2015. 242. The open competitive bidding procedures described in the Laws and Regulations are found to generally adequately accommodate transparency, economy and efficiency of procurement process. 243. MOET is responsible for organizing training courses as part of their capacity building responsibilities. In addition, the Government has established an e-Government Procurement system (available at muasamcong.mpi.gov.vn). This e-GP system allows procuring entities to electronically handle bidding process (e-tendering) in a more transparent and fair manner with lower administrative costs. Participating provinces should be encouraged to gradually adopt this electronic procurement system for the Program. 244. There are risks that existing procurement procedures and processes will result in a low level of competition in competitive bidding that will impact on cost-effectiveness and competitiveness. First, existing procedures may lead to an unjustified rejection of bids due to minor deviations and omissions and as a result of bid prices exceeding cost estimates. Secondly, direct contracting procedures may be abused in ways that impact on cost-effectiveness and competitiveness. To mitigate these risks it is recommended that procurement actions are included in the Program Action Plan, and effectively implemented during Program execution. Additionally, IOs should be encouraged to use the e-GP system for managing procurement. MOET should provide detailed guidance on procurement matters in the Operations Manual. 245. The primary risk to contracting is that a shortage of cash flows causes delays to contract payments which result in slow construction and late delivery of works and services. To mitigate this risk, the IOs should appropriately utilize Program advances, all DLI disbursements and mobilize other temporary funding sources for contract payments. Effective, timely budgeting, procurement planning and execution, and reporting procedures are important elements that should be in place to mitigate this risk. 246. There is a risk that a fragmented approach to procurement planning will affect the timeliness of Program implementation and service delivery, that unrealistic or late procurement planning for critical large works and consulting services contracts will result in completion delays, and that willful packaging of works and services into small contracts for the use of direct contracting will occur. These practices, should they occur would affect cost-effectiveness and competitiveness. To mitigate these risks it is recommended that the IOs plan procurement for entire activities and phases with priority being given to critical large works and consulting services. IOs should also be encouraged to bundle small contracts into larger ones for the use of competitive bidding to the extent possible. MOET should provide detailed guidance on these matters in the Operations Manual. 247. The IOs have hired consultants to provide supervision and construction management services. It is reported that the contract execution have normally been completed on schedule. There were also a number of cases of delay mainly due to shortage of fund, weather severe conditions and weak capacity of the contractors. Cost overruns also did not appear to be an issue 89   under the Program as the agencies generally managed to complete works within the agreed budget. It is noted that no serious disputes requiring resolution by a court were reported under the Program. 248. In Vietnam the construction of civil works, especially larger works, is often delayed because of shortages of funds for contracting (or delays in funds allocation process). It is envisaged that implementation of civil works contracts under the Program may face similar challenges. In particular, civil works under the Program would require regular cash flow (including 20-30 percent of contract prices as advance payments, monthly payments etc.). Due to the nature of the PforR instrument, a major portion (75 percent or more) of disbursements can only be made once the works have been completed and certified. A cash flow shortage is assessed to be one of the biggest challenges to good contract management and timely delivery of works and services under the Program. To effectively address this challenge, IOs would have to ensure proper management of the advance disbursement and other partial disbursements as well as to find temporary funding sources. 249. The Bank’s experiences in Vietnam shows that procurement complaints were frequently received under Bank-financed operations and most of them were anonymous. Bank policy requires anonymous complaints to be seriously considered and reviews of such complaints have helped to identify and address many procurement problems and unacceptable practices. The lack of a clear, consistent and affirmative policy and procedure for these complaints is a clear weakness in the Program’s complaints handling system. Under the Program, IOs should therefore be encouraged to accept and review anonymous complaints. 250. The Program is subject to Vietnam’s national debarment procedures. Prohibition from participation in future tenders is governed by the Law on Procurement which provides a defined set of “prohibited acts” and commission of which can lead to debarment. For procurement, initial sanctioning decisions will be made by the investment owners. Lists of debarment within a given province are prepared by the provincial Department of Planning and Investment (DPI) and are submitted to the PPC and MPI on an annual basis. Provincial officials gave the team inconsistent information regarding whether tenders consistently refer to the MPI sources and the DPI lists to determine bidder eligibility. The list of debarred firms/companies are published in the MPI website: muasamcong.mpi.gov.vn. Under this Program, the Bank’s debarment list and temporarily suspension list have not been applicable. Therefore, it is recommended that these Bank’s lists will be applicable when the Bank supports the Program. The IOs under the Program reported that no firm/company has been debarred so far under the Program. Financial Management 251. The Bank has assessed the financial management arrangements of the Program and found that they provide a reasonable assurance that the financing proceeds will be used for the intended purposes, with due attention to the principles of economy, efficiency, effectiveness, transparency and accountability. The overall Financial Management risk after mitigation measures is assessed as Substantial. 252. The principal law governing budget management is the revised State Budget Law approved by the National Assembly (NA) in 2002 which came into force for the 2004 budget year. This 90   revised Law has strengthened the legal platform for budget management. It clarifies the powers and responsibilities of stakeholders participating in the budget process. The budget process takes place between July and around December when budget allocation is entered into the Treasury and Budgetary Management Information System (TABMIS)67 by the MOET’s Department of Planning and Finance. 253. The identified risk relating to budgeting and planning process is that the current budget allocation process does not ensure that sufficient funds are allocated for the Program activities and for the performing LTTUs. In order to mitigate this risk, for the Program, a performance based budgeting process will be introduced in which MOET will sign a Performance Agreement (PA) with each participating LTTUs. The budget allocation to LTTUs will be based on the actual performance of the LTTUs that is measured by the indicators, equivalent to the DLIs, set out in the PA. A Program deposit account will be created in the State Treasury at the LTTU level to manage proper budget allocation, execution and reporting of the LTTUs. 254. As regard to accounting and financial reporting, the capacity of the ST and each IO is assessed as adequate given the TABMIS system (at the ST) and the accounting systems (at the LTTU) are fully operational and the accounting staff are capable for handling accounting and reporting tasks. For the Program, it is proposed that each IO will prepare Financial Reports that will be sent to the MOET for consolidation. The frequency, timing and template of the Financial Reports will follow the guidance from the MOET and be indicated in the PAs signed between MOET and the LTTUs. An independent verification agency will also be selected to provide independent confirmation of the results reported by the participating LTTUs. 255. Internal controls exist in IOs with the formally documented Internal Expenditure Regulation (IER). While the framework of internal controls for expenditure and specific internal controls at IOs are adequate, the comprehensiveness and quality of internal controls procedures documented in the IER varies from University to University. Under the Program, the IER of the IOs will be reviewed and a standard IER with suggested contents will be developed for the use of the Universities, and for the MOET’s reference when reviewing for endorsement of those IERs. 256. The absence of an Internal Audit function and inefficiency of the existing Inspection function which may not provide adequate assurance for internal controls of the Programs is a Substantial FM risk. Under the Program, technical assistance will be provided to build capacity of an Internal Audit (IA) function in all IOs involved within 2 years of the Financing Agreement signing. The financial audit will be conducted in accordance with the audit terms of reference provided in the Program Operations Manual (OM) by an Independent Auditing Firm acceptable to the Bank. Third party independent results verification agency as well as procedures will be put in place to track DLIs and operation outputs and outcomes.                                                              67 TABMIS is a computerized system used by State Treasury to record public expenditures. 91   Governance 257. The 2005 Anti-corruption Law and its amendments in 2007 and 2012, and the 2012 Law on Higher Education stipulate the obligation of the education institutions to disclose information to the public on the management and use of public budget, state property and assets, staff and financial resources for education, the support provision and other investment for education and other revenues. In particular, Circular 09/2009 of MOET details that public education institution must disclose and make publicly available the commitment of education quality; education accreditation results; condition to ensure quality of education; the collection, management and use of tuition fees, recruitment charges, other revenues from consultancy, transfer of technology, investment for education and other financial revenue and expenditure items. The Program IOs have done a good job on disclosing information on their general operations such as commitment of education quality, education results, and fees and related charges through their website. Financial information of the institutions including investments for education, however, was not yet adequately disclosed to the public. 258. The 2005 Anti-Corruption Law and its amendments in 2007 and 2012, and the 2011 Laws on Complaints and on Denunciation entitle citizens to submit complaints on fraud and corruption (amongst others) to the government People’s Committee at the appropriate level. A complaints handling system will receive, record, and provide necessary investigative actions where it is deemed legitimate. A parallel complaints “system” is also instituted in the Government Inspectorate (GI) offices under line ministries and departments at the central and provincial level. The GI’s main functions are to inspect both compliance and allegations of fraud and corruption in their respective sectors. 259. The Program IOs have already issued detailed procedures for handling grievances. There were a very few number of complaints registered at the IOs. This is largely thanks to the transparency effort that the agencies have been implementing over the years, including the annual public employee conferences where staff in each agency can openly voice their concern to its leadership. The MOET’s Inspectorate is charged with compiling all the complaints and denunciations at all institutions under the MOET’s jurisdiction in a systemic manner, according to Circular 15/2013 of the MOET. A new division under the MOET Inspectorate has been established since early 2015 to handle grievances in relation to the operations of the MOET and its sub- ordinate agencies. MOET would, from 2016 onwards, have proper registries of grievances and how they have been handled within the MOET’s mandate. 260. Under ETEP, complaints of all kinds, which could include allegations of fraud or corruption, should mainly be submitted in writing to the key IOs. Where crimes are alleged, allegations will be passed to the police and prosecutors for handling. In addition to the written complaints channel, any citizen may also attend Meeting Sessions with MOET leaders, held at least monthly, to raise issues of fraud and corruption. Staff and students of the LTTUs can also attend periodic meetings with the agencies’ leaders and raise concerns of fraud and corruption. Although government agencies are not required to handle anonymous complaints, all the key IOs maintain the practice of accepting, recording, and giving these complaints some consideration when they contain credible evidence of fraud and corruption. 92   261. The Government, through MOET, the MOET Inspectorate and the IOs will improve reporting systems and establish a database in order to strengthen the framework for complaints handling and combat fraud and corruption under the Program. The Government has confirmed its commitment to the implementation of the Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results financing and will require the application of World Bank suspension and debarment lists that will be made readily available to participating cities. These actions will be reviewed during Program supervision. In addition, the Government will oblige all Program participants to cooperate with Bank investigations. To this end, the World Bank has also received evidence of a written communication from a competent central government authority (i.e. MOET) to the relevant IOs (such as LTTUs, education management units etc.), which sets out the obligations of those authorized to cooperate with any investigations under the Program, including any investigations that may be necessary by the World Bank. 262. The Program IOs have committed to actions that are stipulated in the PAP to help enhance transparency and minimize risks of fraud and corruption. These actions include that the Program IOs disclose financial information properly and maintain databases on (i) public consultation, corruption, fiduciary, procurement, and Program implementation complaints and responses to those complaints; and (ii) beneficiary feedback to IOs and responses to such feedback. The information would be aggregated at the national level by MOET and the MOET Inspectorate and will be reported to the Bank on a periodic basis. These actions will be further detailed in the Operations Manual. 263. There have been no complaints recorded by the Bank complaint database in the education sector during the period of 2014 – 2015. Of the number of complaints registered concerning the education sector, which is about ten percent of the total complaints of the portfolio during the ten year period from 2005 to 2015, two complaints were substantiated and forwarded to INT for further actions in 2006 and 2011. No evidence of fraud and corruption has been disclosed by the INT on the two cases. Most of the complaints registered in the sector so far concerns with goods contracts, hence the need for more due diligence actions with regard to contract management of the Program IOs. 93   Annex 6: Summary Environmental and Social Systems Assessment Environment system assessment 264. Upgrading activities supported under the Program could cause adverse environmental impacts associated with civil works, such as generation of noise, dust, vibration, wastes, and health and safety risks. However, upgrading activities would take exclusively place within university premises, so potential negative impacts are insignificant, transitory and manageable. The potential impacts will be mitigated by proper construction practices and/or environmental codes of practices under supervision of the responsible agencies and local communities. During operation of upgraded facilities, a small amount of wastes would be generated from daily activities of teachers, students and staff of LTTUs, including solid waste and wastewater. The wastes will be managed through the existing waste treatment and collection facilities under inspection of the responsible agencies in line with the government regulations. 265. Vietnam has environmental assessment requirements that differentiate investment operations by potential impacts. An Environmental Impact Assessment (EIA) is mandatory for activities with potential large scale adverse impacts and an Environmental Protection Plan (EPP) is required for activities with more limited impacts. Activities requiring an EIA are further divided into those that require central environmental agency review and approval (MONRE), and those that are reviewed and approved at the provincial environmental agency (DONRE). All proposed upgrading activities will require an EPP with delegated review and approval required at district level. 266. There are potential risks related to inadequate environmental supervision stemming from insufficient human and financial resources in the IOs, as well as poor environmental management by contractors. 267. The potential adverse effects are generally well known and understood by the implementing authorities. It is expected that effects will be manageable with known and demonstrated mitigation measures. 268. An Operational Manual (OM) will be developed which will include guidance to LTTUs on screening activities to avoid significant impacts, inclusion of Environmental Codes of Practice (ECOP) and Worker’s Codes of Conducts (WCC) as well as communication activities in construction contracts to manage Environment, Health, and Safety (EHS) issues at construction sites. Contractors will also be required to submit copies of their quarry license as a means of controlling EHS issues. Construction supervisors will be contracted on behalf of Program owners to supervise the contractors on both engineering aspects and EHS issues related to construction activities. Program owners and their PMUs are the entities accountable to the government to ensure that all aspects of the investments, including environmental management, follow Vietnamese legislation. While PMUs are subjected to fines for violations, they can also penalize construction supervisors/contractors for non-compliance of contractual requirements/conditions. Capacity building activities will also help to improve contractor performance relating to compliance with environmental regulations. 94   269. Institutionally, the environmental management function is fully established from central government to the city/district levels and is capable of managing the environmental aspects associated with the Program. However, a few shortcomings in the performance of the environmental management system have been identified, including: (i) weaknesses in environmental screening and assessment; (ii) weaknesses in post-EPP monitoring and supervision practices, and (iii) challenges in ensuring environmental compliance during the construction phase. Given small scale and simplicity of upgrading activities, the overall environmental risk is considered Low. 270. In order to promote institutional sustainability, the environmental management capacity of the participating LTTUs should be developed. A capacity building program including formal and on-the-job trainings should be developed and implemented. A qualified environmental specialist should be mobilized at the central level to provide technical advisory services to the PMUs in the early stage of the Program implementation and prepare an environmental training plan for the Program. The environmental training plan should cover all key stakeholders such as designated PMU staff, construction supervisors, and contractor site engineers. The city/district Environmental Divisions should also be engaged in the Program’s environmental capacity building. 271. As part of the preparation of ESSA, public consultation meetings with key stakeholders including eight LTTUs were carried out from 9 to 18 November 2015 to: (i) brief participants on the Environmental and Social Systems Assessment approach under the proposed Operation; (ii) solicit feedback on the key findings and recommendations of the ESSA; and (iii) identify possible recommendations for the proposed action plan. In general, stakeholders expressed their strong support for the implementation of the proposed Operation which they considered would help to address the need for infrastructure upgrading of the LTTUs. This feedback was carefully considered by the Bank Task Team and will be addressed through the Operation where appropriate. 272. The final ESSA in Vietnamese language was locally disclosed through MOET website on April 12, 2016 and the English version was disclosed at the Bank’s InfoShop on June 8, 2016. Social system assessment 273. Principals and teachers from the participating agencies, including those from ethnic minorities and other vulnerable groups, will benefit from the Program. The consultations indicated general support for the proposed objectives and design of the Program. Given the composition of consultation participants, full support has been received from a broad range of stakeholders, from all levels of local authorities, from the provincial and district to the commune levels, and various ethnic minorities groups, such as Tay, Nung, Thai, Dao and Ede. All these participants confirmed their full support for the concept and approach of the proposed Program, seeing them as a practical and useful way to enhance the effectiveness of teachers from general schools across Vietnam. Both male and female principals and teachers from the participating institutions would be encouraged to participate in the Program’s activities. The role of school-based Women’s Unions would be enhanced to maximize benefits and minimize adverse impacts on beneficiaries, especially those from ethnic minorities and other vulnerable groups, thus making the Program more successful and socially sustainable. On-site CPD program will be developed to help teachers incorporate the latest knowledge and practice with regards to teaching children with disabilities, to the extent possible. Some LTTU Performance Agreements will include actions to develop such CPD programs. 95   274. Monitoring and evaluation: The Department for Planning and Finance takes responsibility for preparing and managing education and training indicators, providing guidance on collection, recording and implementation procedures for education and training statistics in centrally administered cities and provinces, education establishments, regulations on supply and use of data, reporting formats in the whole education sector. However, Department of Education for Ethnic Minorities is preparing a proposal for establishing a data management system on education and training relating to ethnic minorities groups. A pilot will be conducted in ethnic minority boarding schools. The system aims to code and digitalize data of every ethnic minority student from enrolment to graduation in an effort to improve the M&E system in the education sector and also to provide timely policy interventions in case of necessity. 275. Transparency and publicity of information: For many years, information on education management relating to EM groups has been published in MOET’s website (www.moet.gov.vn). However, the published information remains insufficient and inadequate. Therefore, MOET now plans to develop and improve the information system. 276. Internal communication within the education sector. The conventional channels of communication between teachers and management are hierarchical in the order from teachers to head teachers, to principals, BOETs, and then to DOETs. There is a lack of interaction between the related parties under the communication mechanism, as presented earlier in this assessment. Some important issues and feedback may be missed or under-reported through such a communication line. Sensitive comments or requests may not be expressed explicitly via immediate managers. OMS, online management system, deployed by DOET, has been in operation, has improved the situation, enabling schools to communicate with BOET and DOET. According to the consultation, principals find this system useful. The consultations showed that Yen Bai DOET has deployed its own email system to enable e-communication between DOET, BOET and schools. 277. More effective outreach communication to promote education in ethnic minorities’ communities: Special attention should be paid to designing teaching methods to promote the education of ethnic minorities groups. Specific characteristics, such as boarding and semi-boarding conditions, regional specifications, and cultural and ethnic identities, should be taken into account. A bilingual communication handbook should be prepared for students, teachers and principals to use within schools and in ethnic minorities communities. An online dictionary may be developed for some stabilized languages, such as Hmông, Êđê, Jrai, Bahnar, Chăm, and Khmer, as a useful tool for daily conversations amongst administrators, teachers and students. 278. Despite the legal framework provided for ethnic minority development investments and activities, operational procedures for adequate implementation are lacking. Overcoming prejudices constitutes a barrier for participation of ethnic minorities and brings a significant challenge for effective decentralization. It will be important to require appropriate measures under the Program Action Plan to involve all local people, including ethnic minority people in the Program activities to ensure their meaningful participation and consultation in every step of the Program implementation, including planning. 96   279. Gaps in the legal social management framework have been identified in regard to social screening and assessment, consultation and participation, including for ethnic minorities, as well as weakness of the existing capacity for effective management at the institutional level, leading to a lack of institutional sustainability. From a social management perspective, the following measures are recommended: (a) Improve the transparency and accountability of implementing organizations through the maintenance of a database on corruption/ fiduciary/ procurement and Program implementation feedback and complaints. This database should integrate the complaints received by all implementing bodies. The database should include anonymous complaints received where possible. In addition, the database should collect and summarize all actions taken in response to complaints and allegations, including the results of any investigations conducted by investigative bodies in the IOs. The database information should be reviewed and approved by the respective MOET Inspectorate and reported to the Bank on an annual basis. Further details of the arrangements for this action should be included in the OM. (b) Social screening and assessments should be conducted to maximize benefits and minimize adverse impact to beneficiary principals and teachers. Measures should be taken to address the major constraints as identified in this assessment, including less effective acquisition and utilization of some new approaches and contents under the Program by teachers from various ethnic minorities areas; inadequate consultation and participation of beneficiaries from ethnic minorities and other vulnerable groups in Program activities; poor IT accessibility in some regions and amongst some target groups; and low motivation to join the Program among some teachers and principals. Particular attention should be paid to the screening of activities in low income and ethnic minorities’ areas. Specific information on social screening processes to be used under the Program should be included in the OM. (c) The Program will enhance the consultation with and participation of beneficiaries, including those from ethnic minorities groups, in the Program activities to ensure their meaningful participation and consultation in every step of the Program implementation, including planning, design and implementation. Further details on how participation and consultation will be ensured under the Program will be included in the OM. (d) ETEP should encourage the following social development measures: (i) Women’s Unions are tapped as resources to encourage women and ethnic minorities to become Core Teachers, Principal Advisors and LTTU faculty and (ii) development of CPD interventions that help teachers work more effectively with disabled children. (e) ETEP will have an Operational Manual which should develop guidelines to: (a) strengthen implementation of national regulations about information disclosure, consultation and participation and redress mechanisms within the ETEP framework, including strengthening approaches to consultation and participation with special attention to ethnic minorities people and other vulnerable groups; (b) boost gender equality and women participation activities; and (c) enhance the quality of education received by disabled students. (f) The above mentioned guidelines will be supplemented with capacity building. A suitable M&E system should be developed to reflect these changes and track the implementation. 97   (g) The IOs will strengthen grievances and redress mechanisms and their documentation. They will need to develop operational guidance and undertake capacity building to improve the conflict resolution and properly track and document all grievances. (h) The ETEP should carefully consider the potential social related political and reputational risks when activities supported by the ETEP are appraised. 98   Annex 7: Systematic Operations Risk Rating (SORT) VIETNAM: Enhancing Teacher Education Program Stage: Appraisal Systematic Operations Risk-Rating Tool (SORT) Risk Category Rating (H, S, M, L) Political and Governance Low Macroeconomic Low Sector Strategies and Policies Low Technical Design of Project or Program Substantial Institutional Capacity for Implementation and Sustainability Substantial Fiduciary Substantial (Financial management and procurement), Moderate (Governance) Environment and Social Low (Environment), Moderate (Social) Stakeholders Moderate Other Moderate OVERALL Substantial 99   Annex 8: Program Action Plan Cove Resp. Completion Action Description DLI Due Date nant Party Measurement Technical. MOET (i) The Recipient shall design and adopt a Within 12 Issuance of a mechanism – satisfactory to the months of the circular on Association - to coordinate teacher effectiveness teacher continuous professional development of the credit continuous initiatives and activities among teacher professional education providers and concerned development teacher management departments at which includes national and provincial levels. the regulated coordination mechanism Plan to be A Capacity (ii) The Recipient shall develop and developed Building Plan execute a Capacity Building Plan for within 12 developed, the LTTUs to ensure sustainable months of the costed and provision of teacher school based effective date executed [in continuous professional development of the FA and cooperation support, all in form and substance implemented with LTTUs] satisfactory to the Association. throughout the period of the Operation (iii) The Recipient shall carry out a study – Within 36 Study validated under the terms of reference acceptable months of the by MOET to the Association - to identify options effectiveness and recommendations for financing of the credit teacher- continuous professional development. Governance: (i) The Recipient shall disclose First reports MOET Evidence of information on the Program’s activities on disclosure information to the public, including financial submitted to disclosure information throughout the the Bank in implementation of the Program; March 2018 (ii) The Recipient shall maintain a system Following LTTUs Log of of receiving, recording and following- evidence and complaints and up on complaints, including anonymous reports will be reports ones, and; submitted annually     Cove Resp. Completion Action Description DLI Due Date nant Party Measurement (iii) The Recipient shall maintain a database Database in Database on complaints and denunciations from place MOET verified all the participating agencies and follows up on the resolution of those complaints and denunciations. Financial Management. Internal audit (i) The Recipient shall (a) review LTTUs capacity Within two MOET internal audit capacity; and (b) prepare assessed and years of /MOF and execute – or cause to be executed - LTTUs effectiveness plans to build the capacity of internal capacity of the credit audit function in LTTUs building plans developed, Within six costed, (ii) The Recipient shall issue guidance on months of approved and the performance based effectiveness executed [in budgeting/allocation of the credit cooperation with LTTUs] Procurement. Competitive Procurement Implemented MOET applied as per the new Recipient regulations throughout the and in addition: period of the Program (i) All proposals for detailed designs, construction supervision and bids for civil works, whether below or above cost estimates, shall be evaluated; proposals and bids shall not be rejected only on the basis of minor, non-substantive deviations; (ii) At least 60 percent of contracts for consultancies and 85 percent of contracts for works shall be competitively bid annually; no contract having cost estimate above the thresholds for using DC/SSS method set forth in the national public procurement law and regulation will be awarded on DC/SSS basis; and (iii) Firms on the local, national, or Bank debarment list shall not be allowed to participate. Social. The Recipient shall develop a guideline, Within twelve MOET Formal acceptable to the Association, to facilitate months of issuance of the meaningful consultation and inform effectiveness guideline participation of beneficiary principals and of the credit teachers, including those from ethnic 101   Cove Resp. Completion Action Description DLI Due Date nant Party Measurement minorities and vulnerable groups, during the ETEP implementation process. The guideline, which will be implemented by LTTUs should fully operationalize the existing Vietnamese legislation with respect to ethnic minorities through a process of free, prior, and informed consultations. Operational Manual. (i) The Recipient will prepare an Before MOET OM agreed by Operational Manual. effectiveness the Bank of the credit (ii) The Recipient will execute the Implementation Operation according to the Manual. Support Implemented Missions will throughout the review period of the implementation Operation quality. 102   Annex 9: Implementation Support Plan 280. Implementation support for the Program will be in the form of close support by the Bank team as well as Technical Assistance provided through the Program. Bank team assistance 281. The Program will require considerable support from the Bank team particularly during its early stages, especially in acquiring technical assistance required to achieve intended results. One challenge will be to coordinate the actions agreed in the Program Action Plan with operational activity on the ground, ensuring that concerned stakeholders and IOs act on a coordinated manner. Support will also be required for the LTTUs with their planning to ensure that available funding can be absorbed and result delivered in time and within expected budget envelops. The team recognizes that a PforR transfers performance risk to the IOs, and lays down a challenge to change numerous operational practices and norms, particularly at the local level. 282. The MOET will also require implementation support to introduce new ways of providing continuous on-site professional development support for teachers and to enhance capacity of the LTTUs to sustain the innovative approach the Program will introduce in teacher CPD. The focus of the Bank implementation support will be on implementation quality and on making the result based incentive system work to its full potential. This will include: (i) reviewing implementation progress (including that of the Program Action Plan) and achievement of Program results and DLIs; ii) providing support on resolving emerging Program implementation issues and on building institutional capacity; (iii) monitoring adequacy of system performance, and monitoring compliance with legal agreements; (iv) supporting the Government in monitoring changes in risks; and (v) ongoing technical support. 283. Key to effective implementation support will be the coordination of its timing with critical points in the timelines of planning and verification of results or payment requests to the World Bank. The first implementation support mission will take place as soon as possible after effectiveness to provide direct feedback on the quality of plans developed by the LTTUs. This will include review of Performance Agreements developed the LTTUs. 284. An outline of the implementation support required is given in the tables below:   103   Table 10: Main focus of Implementation Support Time Focus Skills Needed Resource Partner Role Estimate First Development of roles, Legal, 2 implementation TBC twelve work plan and governance, support missions months incentives for Core financial plus 2 technical Teachers and management, missions Principal Advisors technical 2*12 people*2 (teacher Development of weeks = 48 education), Performance weeks procurement, Agreements between social and Plus LTTUs and MOET; environmental 2*5 people*1 Establishment of safeguard week = 10 weeks arrangements for specialists independent Total: 58 person verification weeks over 12 months Development of plan for MOET to ensure that LTTUs activities are coordinated and cost-effective. Strengthening of the M&E system 12-48 Reviewing Legal, 2 implementation months implementation governance, support mission progress, cross financial plus mid-term checking linkages management, review between technical 2*4 years*8 MOET/DOET/LTTUs (teacher people*2 weeks planning and results, education), = 128 weeks providing support in procurement, case of disputes social and 2*12 people*2 relating to verification environmental weeks = 24 (LTTUs or safeguard weeks MOET/IVA) specialists, Total: 152 person M&E weeks Other Independent audit/ Independent 2 people for 5 assessment of technical weeks verification of results expertise Total: 10 person weeks 104   Table 11: Task Team Skills Mix Requirements for Implementation Support Skills Needed Number of Staff Weeks Number of Trips Comments Legal 6 n/a Procurement 15 n/a Financial Management 15 n/a Governance 10 n/a Environment 15 n/a Social safeguards 15 10 specialist M&E 20 n/a Education specialist 100 12 105