Emergency Operation for Development Project (EODP) – Education component: Teacher Training Implementation & Lessons Learned for national scale up Acknowledgement................................................................................................................................... 2 Introduction ............................................................................................................................................ 3 Objective of this report .......................................................................................................................... 4 Design of the teacher and counselor training ....................................................................................... 5 Step one: Training needs assessment ...................................................................................................................5 Step two: Design of the training materials ...........................................................................................................8 Implementation of the teacher and counselor training activities ..................................................... 11 Evaluation of the Teacher and Counselor Training .......................................................................... 13 Student Performance in Arabic .......................................................................................................................... 20 Student Performance in Mathematics ................................................................................................................ 20 Lessons Learned ................................................................................................................................... 23 Content and Training ......................................................................................................................................... 23 Implementation................................................................................................................................................... 23 References ............................................................................................................................................. 27 1 Acknowledgement The World Bank team would like to thank all stakeholders from Government of Iraq for their collaboration which contributed to the development of this report, including but not limited to: Duraid Dhari Daas (Director General of the Educational Planning Directorate), Safaa Lafta Yonus (Previous Director General of the Educational Planning Directorate), Abdul Shaheed Jwad Muein (Previous Director General of the Educational Planning Directorate), Aseel Yasin Jasim, Previous (Director General of Directorate of preparation and training), Shurooq Bashar Talii (Assistant DG of Directorate of preparation and training), Bushra Aftan (Legal Counselor, Project Management Team – PMT), Muna Al Ani (Engineer -PMT), Anas Ghanawi (Engineer, Previous PMT), Faisal Shafia (Engineer, Previous PMT), Nabeel Rfiq Mohammed (Project Team, Directorate of preparation and training), and Eng. Mohanad Hussien Ali (Project Team, Directorate of preparation and training) . The World Bank also would like to extend sincere appreciation to Shurooq Bashar Talii, Assistant Director General of the Directorate of Preparation and Training, for her outstanding leadership in overseeing our teacher training programs. The Iraq’s Emergency Operation for Development Project (EODP) – Teacher Training Report was prepared by Aarya Rajendra Shinde (Consultant) and Elisabeth Sedmik (Economist), with reviews and input provided by Elaine Ding (Research Analyst), Ghada Hameed (Senior Education Specialist), and Zainab Alsammak (Partnerships Analyst). The implementation of the teacher training from the World Bank’s side was led by Anna Boni, under the overall guidance of Nathalie Lahire (Senior Economist, TTL) and Elisabeth Sedmik (Economist, TTL). 2 Introduction Years of conflict in Iraq had a severe impact on child welfare and service delivery. Despite the significant milestones achieved by Iraq's education system during the 1970s and 1980s, including near-universal enrollment, gender parity, and a remarkable reduction in illiteracy among the 15–45 age group (World Bank, 2023), years of conflict and structural inefficiencies have had lasting negative impact on the education sector and child welfare. These inefficiencies encompass aspects such as equity, access, quality, governance, assessments, and financing at all educational levels, among others (World Bank, 2023), creating considerable barriers for Iraqi children in accessing quality education and learning opportunities (UNICEF, 2018). Iraq's education system faces an acute shortage of qualified teachers, a key factor in improving learning outcomes and conveying skills. Almost three-quarters of primary education teachers lack a university degree (World Bank, 2023). Teacher training opportunities are limited and unequal, with a lack of consistent teacher training programs and quality assurance mechanisms. However, research consistently demonstrates that effective teaching practices and quality teacher training positively impact student learning outcomes, including academic achievement and social-emotional development (McCallum, 2021; OECD, 2023). Investing in teacher training and development is therefore crucial for improving education quality and ensuring that students reach their full potential. Moreover, resources and attention to mental health and psychosocial issues among children are critical needs. The conflict has exposed Iraqi children to significant emotional distress, leading to long-term mental health and psychosocial issues. There is also a shortage of counselors in Iraq. Access to trained counselors is essential for addressing students' mental health needs, fostering resilience, and establishing a supportive learning environment that promotes their overall well-being and academic performance (UNICEF, 2022; Johnson et al., 2023). Without adequate support, students may struggle to navigate the psychological and emotional challenges resulting from conflict and displacement. According to the World Bank's Human Capital Index (HCI), a child born in Iraq today is projected to achieve approximately 41% of their potential productivity as they mature (World Bank, 2022). This index is comparatively lower than its neighbors in the MENA region, reflecting the need for further development (World Bank, 2021). Investing in teacher training and support, strengthening the skills of counsellors, improving learning materials and infrastructure, and addressing the underlying socio-economic factors contributing to the learning crisis can be effective means to improve the country’s human capital outcomes. To support the Government of Iraq's (GoI) stabilization and reconstruction efforts, the World Bank’s Emergency Operation for Development Project (EODP) was developed. The EOPD aims to reconstruct and restore public services and infrastructure in recently liberated areas 3 in Iraq, and the 2017 Additional Financing (AF) expanded the scope of the project to additional sectors such as agriculture, irrigation, urban services, and education (World Bank, 2017). The education component of the EODP-AF project (component 9) aims to restore education services in areas of Iraq that have been impacted by conflict, while also providing a basis for future development in the education sector. Through its three sub-components, there is a focus on strengthening the foundation and socioemotional skills of children in the early grades and promoting economic empowerment for the women. The sub-components are as follows: • Sub-component 1: Rehabilitation, Reconstruction, Upgrading and Equipping of Education Infrastructure (26 schools) in selected governorates of Iraq • Sub-component 2: Support to Teachers and Counselors • Sub-component 3: Institutional Strengthening, Sector Development Support and Project Management The present report focuses on activities within sub-component 2: Support to Teachers and Counselors, which had the following main objectives: • Improving early grade teachers' (grades 1-3) ability to support basic numeracy and literacy skills through adaptive teaching strategies and positive disciplinary strategies. • Enhancing school counselors' and teachers' ability to provide psychosocial support services to students in grades 1-3 in the targeted governorates. • Piloting a cascade model of training that could be reapplied in future operations or governmental programs. To achieve these objectives, the following activities were designed under the EODP project: 1. Train 5000 teachers and counselors (“educators”) in the most crisis affected areas. 2. Provide capacity building to the Ministry of Education (MOE) to design and implement teacher training, and 3. Provide policy recommendations and lessons learned for future teacher training activities undertaken by the MOE. Objective of this report This report documents the content and design, implementation, and evaluation of the teacher and counselor training implemented from 2021-2022 by the MOE in Iraq, under the World Bank's EODP-AF. Additionally, this report proposes areas for expansion and improvement of the teacher training, based on an evaluation conducted during implementation of the training. The report includes policy recommendations that aim to enhance the effectiveness and impact of any future training activities undertaken by the MOE to improve student learning. 4 The MOE’s Project Management Team (PMT) oversaw the implementation of the education component, while International Development for Education and Leadership (IDEAL LLC) served as the external consultant responsible for designing and implementing the training. The Project Coordination Unit (PCU) in the Reconstruction Funds for Areas affected by Terroristic Operations (REFAATO) handled procurement and contract management. The General Directorate for Educational Supervision supervised the training and conducted assessments, while the Training Directorates in the governorates were responsible for implementation, coordination, and monitoring at the governate level. Design of the teacher and counselor training Step one: Training needs assessment In a first step to design an effective teacher training, a needs assessment was conducted in December 2019 to understand the existing status of teaching resources and teacher training and support in the country. The assessment was conducted through a review of essential documents presented by the MOE and collecting questionnaires from a sample of teachers and counselors from the five targeted areas. The needs assessment highlighted the following issues: • Overcrowded schools lacking essential equipment, materials, and necessary facilities. • High classroom populations, with an average of 40 to 50 students per class, posing significant challenges for teachers, counselors, and school administrators. • Many counselors lacking basic resources such as office space for record-keeping, materials, and meeting areas for sessions with students or parents. • Inconsistency in the frequency of school inspections, with some reporting monthly visits by inspectors while others mentioned only one or two visits per year. • Existence and organization of parents/teachers/counselors’ meetings were unclear, with a consensus on general meetings once or twice a year but no details on need-based meetings. • Lack of evidence of the Training Directorate's involvement in verifying whether trained teachers applied their acquired knowledge. • Uncertainty regarding whether inspectors were aware of the skills and knowledge acquired by teachers and counselors through training, raising questions about their effectiveness in evaluating schools' performance. • Lesson plans were primarily checked by school directors, with only one teacher reporting inspections by inspectors. • Lesson plans exhibited a wide variety of styles and organization, suggesting a need for standardization through the use of templates for all subjects or subject groups. • The potential use of teachers' guides for each student textbook, as discussed in an Instructional System Design workshop sponsored by the EU and British Council, was not evident in the collected lesson plans. 5 Using the teacher and counselor questionnaires distributed to the six governorates, the assessment also evaluated needs in three main areas: teachers, counselors, and other needs. The teachers’ questionnaires explored three main areas: 1. Difficulties faced by teachers within the classroom • Teachers identified difficulties faced within the classroom, including the presence of students with severe behavioral problems, students with learning disabilities, and students with other mental challenges. • Teachers indicated a need for a classroom management module as part of their training, • Teachers also indicated the need for having a module on accommodation strategies, differentiated instruction and the use of teaching aid material. 2. Support available to teachers • Support from inspectors and MoE trainers was not seen as significant, suggesting the need for improvements in their roles to enhance support to teachers. 3. Policy and Teacher Guide • Teacher-parent interaction guidelines and teacher performance evaluation criteria were areas that needed further development • A clearer teacher-performance evaluation criteria was suggested so that teachers could better understand what is needed of them. • There is a need to establish clear classroom and playground rules as well as define emergency procedures, including fire drills, evacuation drills (earthquake), and COVID-19-related protocols. These guidelines should be documented, incorporated into the curriculum, and scheduled for implementation. The questionnaire given to counselors involved three areas of concern in addition to details of their work, workload, and population size. 1. Difficulties faced with the student population • Counselors identified difficulties faced by students, including those affected by traumatic experiences, exhibiting bullying and aggressive behavior, and dealing with phobias and fears. These issues were attributed to factors such as witnessing traumatic events, displacement, and challenging family circumstances. Counselors emphasized the need for training objectives related to psychosocial support and strategies for addressing these student challenges. 2. Support available to counsellors • Support from outside the school community, such as inspectors and MoE trainers, was deemed insufficient. 3. Policy Guide for Counselors 6 • It was suggested that the policies and guidelines related to counselor-teacher-parent interactions and counselor performance evaluation criteria needed further development. Other Identified Needs: • Teachers needed tools to unify lesson planning, such as templates, to ensure compliance with required components and standards. • The analysis of previous trainings indicated that teachers had acquired substantial knowledge and skills. However, further monitoring and evaluation were needed to assess the application of these skills across schools. • Overall, the needs assessment highlighted various areas where improvements and training modules were necessary, including classroom management, psychosocial support, policy awareness, and lesson planning standardization. It also emphasized the importance of structured support systems for teachers and counselors. The needs assessment highlighted the following recommendations for the training: 1. Include a classroom management module and a differentiated instruction module for teachers in the training. 2. Encourage teachers and counselors to participate in a well-coordinated program that involves all stakeholders, including school personnel, parents, supervisors, trainers, regional offices, and MOE Directorates. This program should provide support and follow-up to ensure effective implementation. 3. Emphasize the importance of coordination among teachers, counselors, and parents to enhance children's communication skills and foster positive classroom behavior. 4. Recommend addressing the need for the Directorate of Supervision to conduct school visits with increased frequency as part of the training plan. Trainers should focus on verifying the implementation of skills and strategies presented in the training modules, while supervisors should evaluate teachers' performance and collaborate with school directors to suggest further training or module reviews for teachers facing difficulties in applying the training material. 5. Based on counselor questionnaires, consider the following objectives for inclusion in the training program: • Establish unified terminology and clear limits for intervention across all schools to improve communication and promote communities of practice. • Provide trainees with an overview of internationally recognized standards. • Ensure a common understanding among trainees regarding the impact of crises and disasters on children. • Equip trainees with tools for observing and documenting observation sessions. • Develop counseling skills to support students, parents, and colleagues. • Offer an overview of the effects of the pandemic on children and suggest measures to help them cope. 7 • Provide trainees with psychosocial support skills and procedures, including activities, self-expression, drama, music, art, and psychomotor activities. • Teach trainees how to design intervention programs or activities within the school. 6. Factor in all elements, such as parent meetings for collaborative problem-solving, student referrals for medical and psychosocial support, and support from other psychosocial personnel, when developing training program objectives. 7. Integrate these recommendations into the training program to enhance its effectiveness. Step two: Design of the training materials Based on the needs assessment, training materials were collaboratively designed by the MOE and IDEAL, with technical support from the World Bank in the period spanning from January 5, 2020, to August 15, 2020. The training materials were divided into three main packages: 1. Adaptive teaching strategies and positive disciplinary strategies for math and Arabic language teachers in grades 1-3 (IDEAL, 2021). 2. Psychosocial Support package for school counselors working with students in grades 1-3 (IDEAL, 2021). 3. Psychosocial Support package for teachers working with students in grades 1-3 (IDEAL, 2021). Package 1: Adaptive Teaching Strategies and Positive Disciplinary Strategies This package aimed to improve learning outcomes of students by improving teachers’ ability to apply adaptive and positive disciplinary teaching strategies for Math and Arabic in grades 1-3. This package is rooted in four key theoretical frameworks: constructivism, Piaget's cognitive development stages, multiple intelligences, and the principles of inclusive and learner-friendly classrooms (IDEAL, 2020). Constructivism promotes active learning and emphasizes learners' role in constructing knowledge, while the theory of Multiple Intelligences underscores the importance of addressing diverse forms of intelligence in education. The objective of this package is to establish an inclusive, safe, and stimulating learning environment, especially in conflict-affected areas. It aims to equip teachers with effective strategies and considerations for diverse learners, ultimately promoting effective teaching and learning through adaptive and positive disciplinary approaches. This package comprises six modules designed to enhance teaching skills. These modules include: 8 1. Lesson Management Skills: This module concentrates on elements of effective teaching, emphasizing student-centered learning and authentic assessment, with a focus on practical life applications. 2. Differentiated Instruction: The module aims to empower teachers to meet individual learner needs through differentiation, allowing each student to engage in the learning process with enthusiasm, confidence, and competence. 3. Active Learning: Teachers are guided to design lessons using active strategies such as play-based learning, art, and stories. These strategies are derived from mathematics and the Arabic language subjects and are integrated into an effective model that considers evaluation and assessment while accommodating learner differences. 4. Classroom Management: This module assists teachers in behavior modification by collaboratively designing a code of conduct involving key stakeholders in education, including students, parents, teachers, and administration. It underscores the importance of creating a safe and stimulating environment as a fundamental prerequisite for effective teaching and learning. 5. Mathematics and Arabic Language Modules: These subject-specific modules provide targeted guidance for teaching mathematics and the Arabic language, aligning with the principles of the overall package. (Summary of Training Materials, 2020) Package 2: Psychosocial support for teachers and Package 3: Psychosocial support for counselors The shortage of counselors in schools has made it necessary for teachers to take on educational counseling, support counselors in their duties, and even replace them where counseling departments are unavailable. In light of this reality, training teachers in counseling skills is crucial for providing support and assistance to students and schools experiencing stressful conditions. (IDEAL, 2020). In response to this need, the Psychosocial Support Training Program developed with the objective to equip educational leaders with the knowledge and skills necessary to provide psychosocial support and promote the psychological well-being and resilience of students in grades 1 to 3 in areas affected by terroristic operations. Effective programs addressing mental health and support require an integrated approach with unified global standards and a common language. The Psychosocial Support Training Program was designed through a comprehensive review of guides from international organizations such as UNICEF, UNESCO, UNRWA and the International Red Cross, reflecting best practices and considering principles and guidelines for mental health and psychosocial support. It includes interactive methods, guidance for trainers, real case studies, and a large number of psychosocial support activities based on scientific foundations (IDEAL, 2020). The Psychosocial Support Training Program consists of two distinct packages: Psychosocial support for teachers and Psychosocial support for counselors. Each of these packages 9 comprises a total of five modules. The teacher and counselor packages shared modules and activities, building a common ground for both teams to bridge the gap in psychosocial support services in schools. Given the difficult conditions Iraqi students face, this approach is essential to ensure basic knowledge and skills of psychosocial support. The table below provides a summary of the modules: Table 1: Overview of Psychosocial Support Training Program Modules Module Focus Module 1: Psychosocial Support Module 1 provides essential insights into general psychosocial in Communities Affected by support topics, with a specific focus on their relevance during Terroristic Operations (For both the COVID-19 crisis. It serves as a foundational knowledge teachers and counselors) base essential for progressing to subsequent modules. Module 2 (For Counselors): This module concentrates on the intricacies of diagnosing the Counseling Skills and its Role in need for psychosocial support, the tools employed in the Diagnosing the Need for diagnostic process, and the development of crucial counseling Psychosocial Support skills for effectively providing support during crises, including the COVID-19 pandemic. Module 2 (For Teachers): The Module 2 delves into the qualities that define supportive Role of Teachers in Psychosocial teachers and their pivotal role in cultivating a nurturing Support educational environment. It addresses effective intervention strategies for managing students' behavioral challenges and facilitating their safe return to schools, particularly in the context of the COVID-19 pandemic. Module 3: Psychosocial Support Module 3 explores a diverse array of strategies employed to Strategies (For both teachers and evaluate psychosocial support services. This comprehensive counselors) examination encompasses techniques such as psychological first aid, psychological debriefing, as well as artistic mediums like drama, drawing, and music. Module 4: Building a Support This module sheds light on the crucial roles played by various System for the Psychosocial supporting entities within the domain of psychosocial support. Support Team (For both teachers It addresses the involvement and contributions of the school, and counselors) parents' council, family, and the broader community in the pursuit of effective psychosocial support. 10 Module 5: Planning Psychosocial Module 5 provides a detailed roadmap for planning Support Programs in Schools psychosocial support programs within the school setting. (For both teachers and Additionally, it offers practical applications of these planning counselors) procedures, ensuring a systematic and comprehensive approach to program development. Implementation of the teacher and counselor training activities The project activities were carried out from December 2019 to December 2022, and successfully completed two years before project closing. The following describes first the training design and secondly, the implementation of the training. Firstly, a 10-day training of 95 Master Trainers was conducted between March 5 and March 15, 2021. The training took place in Erbil, Iraq. Since the training was administered during the COVID-19 pandemic, safety measures were implemented during the training to ensure the well- being of the participants, while being conducted in person. These Master Trainers were selected from the central MOE and Education Offices at the governorate level. The training of 95 Master Trainers utilized a variety of strategies, including play-based learning, case studies, mind maps, role-playing, collaborative learning, working groups, and problem-solving skills (IDEAL, 2021). After the initial training of the 95 trainers, a comprehensive plan for training the 5,000 trainees, consisting of teachers and counselors, was meticulously devised and submitted, alongside the impact evaluation strategy. This process encompassed identifying the evaluation strategy, crafting the associated evaluation tools, outlining the training implementation plan, submitting the initial draft for approval by the MOE, engaging in necessary revisions, and ultimately finalizing the draft. These processes took place between April 1, 2021, and May 15, 2021. Subsequently, between July 25, 2021, and September 15, 2021, the 95 trainers participated in a refresher session covering the various training packages they had previously received. During this timeframe, these trainers also underwent a dedicated workshop to equip them with the skills required to execute the evaluation plan effectively. By September 15, 2021, all training sessions had been concluded, and the trainers were fully prepared to deliver the training and put the evaluation plan into action. 11 From September 2021 to January 2022, the 95 Master Trainers took on the responsibility of training the 5,000 teachers and counselors, referred to as "educators" (IDEAL, 2022). The cascade model saw first the training of 95 Master Trainers who would then in turn train 5000 teachers and counselors. Table 2 provides an overview of project beneficiaries in various governorates, encompassing 95 trainers and a total of 5,000 trainees, consisting of 4,346 teachers and 654 counselors (IDEAL, 2022). Table 2: Training recipients by governorate Governate No. of Trainers No. of Trainees No. of teachers No. of Counsellors Baghdad K1 7 250 215 35 Baghdad K3 7 250 215 35 Anbar 16 800 550 250 Salaheddin 14 700 570 130 Kirkuk 12 500 342 158 Ninawa 21 2000 1988 12 Diyala 12 500 466 34 General 6 Directorate Total 95 5000 4346 654 Throughout the project's duration, various challenges emerged. These challenges encompassed administrative hurdles as well as difficulties arising from the unexpected circumstances of the pandemic. Some of them are as follows: 1. The COVID-19 Pandemic: The COVID-19 pandemic and resulting lockdowns presented significant challenges throughout the project's implementation. The need for continuous communication, in-person trainings, and evaluations was hindered by school closures, lockdown measures, and social distancing protocols, disrupting the planned implementation stages. For example, the training of the 95 trainers, initially scheduled for March 2020, was rescheduled to March 2021, aligning with the subsequent school year of 2021-2022. As a consequence, the TOT training report's timeline was extended from April 2020 to April 2021, and the design of the implementation plan was similarly pushed from April-May 2020 to 2021. 2. Multiple stakeholders: The project involved multiple parties responsible for supervision and decision-making, including the MOED, REFAATO, IDEAL and World Bank supervision support. This required continuous follow up and communication. 12 3. Geo-demographic considerations: The diverse geographic characteristics within the governorates and demographic features, such as the expansive areas of certain governorates like Anbar and Ninawa among the targeted ones, coupled with security concerns and transportation challenges, necessitated meticulous planning. Evaluation of the Teacher and Counselor Training The evaluation of training effectiveness was developed between April and May 2021 and the data collection and impact assessment phase took place between September and November 2022. During this period, several key activities took place, including the formation of supervision, data collection, and data analyzing committees. Trainers were deployed according to the established schedule, and the training program proceeded as planned. The evaluation process followed the predetermined strategy, involving the collection of evaluations and questionnaires, followed by extensive efforts in the comprehensive analysis and interpretation of the collected data. The training effectiveness evaluation plan was based on the Kirkpatrick model. The Kirkpatrick model is a widely used framework to evaluate the effectiveness of training programs (Smith 2021). The model provides a structured approach to training evaluation and helps organizations identify areas for improvement in their training programs. It consists of four levels of evaluation: • Reaction: evaluates participants' immediate reactions and satisfaction with the training program. • Learning: assesses how much knowledge and skills participants have acquired during the training. • Behavior: examines whether the participants have applied what they learned in their job or daily life. • Results: measures the impact of the training on business outcomes or organizational goals. A variety of tools were developed to gather information, with each tool aligning with a different level of the Kirkpatrick model. Table 3 outlines the evaluation tools associated with each level of the Kirkpatrick Model, facilitating comprehensive assessment of training program effectiveness. 1. Interviews with trainees on Adaptive Learning strategies and Psychosocial learning. 2. Daily feedback from the trainers on issues that emerged during the training, such as training activities, trainee performance, and learning difficulties. 13 3. Daily and final surveys from trainees on the training course, activities, and trainers. This feedback from trainees on the training course, activities, and trainers was collected to identify areas of improvement in the training program. 4. Classroom performance observation grid for the 95 training leaders 5. Assessment scale to measure the effectiveness of the 95 training leaders 6. Pretests and posttests conducted before and after the training to measure the knowledge of the trainees (teachers and counselors). 7. Assessment scale to measure the effectiveness of the teachers and counselors. 8. Student performance in Mathematics and Arabic language tests. 9. Children’s psychological well-being scale. Table 3: Evaluation tools mapped to the different levels of the Kirkpatrick model Level of Evaluation Tool (Kirkpatrick Model) Reaction - Interviews with trainees on Adaptive Learning strategies and Psychosocial learning Gathering feedback through - Daily feedback from the trainers on issues that questionnaires, polls, or interviews. emerged during the training, such as training activities, trainee performance, and learning difficulties - Daily and final surveys from trainees on the training course, activities, and trainers. This feedback from trainees on the training course, activities, and trainers was collected to identify areas of improvement in the training program Learning - Pre and posttests conducted before and after the training to measure the knowledge of the Evaluating the knowledge and skills trainees acquired by trainees through pre- and post-training metrics, which are vital tools for this assessment Behavior - Classroom performance observation grid for the 95 training leaders - Assessment scale to measure the effectiveness At this level, performance improvement of the 95 training leaders is tracked through statistical analysis - Assessment scale to measure the effectiveness comparing pre- and post-training of the teachers and counselors performance. It also involves evaluating performance three months 14 to a year after training, allowing trainees to apply what they've learned. Assessments include performance observation grids, skill scales, daily preparation plans, and quarterly/annual distribution records prepared by trainees Results - Student performance in Mathematics and Arabic language tests This level provides clear indicators of - Children’s psychological well-being scale the return on investment from the training program, measuring tangible outcomes that align with the program's objectives. It assesses improvements in work quality, increased productivity, reduced complaints, enhanced quality, improved morale, and innovation in work methods To facilitate the evaluation process, the MOE's technical committee assigned 45 supervisors from the directorates of education to assess a specific number of teachers – from both the cohort that received training (“treatment group”) as well as a cohort that did not receive training (“control group”). Additionally, the 95 training leaders were tasked with evaluating students in grades 1, 2, and 3. The MOE employed specific criteria for selecting teachers and counselors from both the control and treatment groups. They chose teachers from schools situated in districts far from the central governorate, with these districts being significantly impacted by terroristic operations. In addition, students from both treatment and control group teachers were assessed. Overall, data from 1603 teachers and counselors, as well as 2750 students in grades 1-3 across the six governorates was collected. The IDEAL team collaborated with department directors, acting as field coordinators, to communicate evaluation visit dates to supervisors and school principals. To improve data entry and analysis, the evaluation forms were digitized using an online survey tool. The chosen online survey tool was selected for its ability to collect data offline, given the limited internet access in the targeted governorates. This allowed for the forms to be used without an internet connection and data to be synced with servers when the evaluator connected to a network. Data was collected for each of the following tools: 1. Interviews with trainees on Adaptive Learning strategies and Psychosocial learning 15 The objective of the interviews with trainees on adaptive learning and psychosocial support was to assess trainers' competence in psychosocial support and adaptive learning strategies. This process encouraged teachers and counselors to provide feedback, express their views, facilitate ongoing enhancements, and gather input for future training endeavors. The interviews were distributed among a representative sample of the study population. A total of 1,346 trainees for Psychosocial learning, and 1411 trainees for adaptive learning, including both teachers and counselors, participated in the interviews, and they were selected from the overall sample of 5,000. The findings from the interviews on Psychosocial Learning reveal positive perceptions of the training leaders' proficiency in psychosocial support. An overwhelming 97% answered Yes to all questions, demonstrating the training's substantial impact on leaders' skills. Notably, the lowest ratings were related to infrastructure and timing issues, possibly requiring MOE intervention. The positive responses from the trainees indicate that the TOT training had a direct and clear impact on the development of the training leaders' professional skills, allowing them to effectively apply the enriched knowledge they gained in the field of psychosocial support. The findings from the interviews on Adaptive learning had similar results, with 96% of teachers and counselors responding affirmatively to all questions regarding the proficiency of training leaders in conveying knowledge and skills related to adaptive learning and positive disciplinary approaches. The data indicated a minimal 4% negative response rate, signifying the successful achievement of training objectives in enhancing the professional skills of the leaders. This positive outcome was attributed to the effective delivery of training materials and the utilization of active training methods by the leaders. Issues pertaining to logistics and administration, as evident in the lowest percentage responses, were unrelated to the effectiveness of training leaders. For detailed results, please refer to the annex. 2. Daily feedback from the trainers on issues that emerged during the training, such as training activities, trainee performance, and learning difficulties. Two daily surveys are conducted with trainees to gather feedback on the training course, activities, and trainers. The purpose of these daily surveys is to continually assess and identify areas for improvement in the ongoing training program and report on issues that arose during the training, including training activities, trainee performance, learning difficulties, and other relevant matters. This feedback was an essential part of the training process, ensuring that any issues could be addressed promptly and effectively. 3. Final survey from trainees on the training course, activities, and trainers. 16 Feedback from trainees on the training course, activities, and trainers was collected to identify areas of improvement in the training program. The purpose of the final survey given to trainees was to gather their feedback on the training course, activities, and trainers. This feedback was collected to identify areas for improvement in the training program. 4. Classroom Performance Observation results for the 95 Training Leaders Training Leaders were evaluated by MOE supervisors using a Classroom Performance Observation Grid. The objective of the Classroom Performance Observation Grid was to assess and evaluate key aspects of the trainer's performance during the training sessions. This included evaluating the trainer's organizational skills, their ability to effectively present the training materials, the level of interaction with trainees, and their knowledge of the training topics. The grid aimed to provide valuable insights into the trainer's overall performance and identify areas for improvement in delivering the training program. Results indicated that a substantial 73% of the essential skills required for educational leaders were fully achieved, with 26% needing further development, and only a minimal 1% going unnoticed. This assessment shed light on their training and performance, highlighting areas for potential improvement. 5. Assessment Scale to measure the effectiveness of 95 Training Leaders. Similar to the Classroom performance observation grid, Training Leaders were evaluated by trainees using an Assessment Scale. The Assessment Scale aimed to assess the trainer's effectiveness on a scale ranging from one to five in three key areas: planning for the training session, the execution of training activities, and the use of evaluation methods during the training sessions. In the assessment, trainees were requested to provide ratings on a scale ranging from 1 (Not Recognized) to 5 (Very Effective). The outcomes of the assessment predominantly fell within the categories of effective and very effective. A detailed breakdown reveals that 32% of supervisors perceived the performance of training leaders as very effective, while an additional 52% considered it effective. The proportion of those who deemed it partially effective stood at 13%. Conversely, the lower levels of Not Effective and Not Recognized received minimal representation, with figures not exceeding 4% and 0%, respectively. These findings underline the positive evaluation of training leaders' performance by the trainees, with a substantial majority rating it as effective or very effective. 6. Pre-tests and post-tests conducted before and after the training to measure the knowledge of the trainees (teachers and counselors). The MOE shared the assessment tool with 5,000 teachers, and 4,752 teachers and counselors responded. This group underwent evaluation across four areas: General Knowledge, Objectives, Key Terms, and Learning Styles. 17 The objective of conducting pre-tests and post-tests before and after the training was to comprehensively evaluate the knowledge levels of trainees, including teachers and counselors. The pre-test aimed to assess the trainees' baseline knowledge before each module, while the post-test measured the knowledge and skills they had acquired upon completing the module. These assessments were integral to gauging the effectiveness of the training program, specifically in equipping teachers and counselors with adaptive learning strategies and psychosocial support expertise. By measuring their knowledge, performance, and participation, the objective was to drive improvements in their professional behavior and facilitate a positive contribution to the overall school environment. Figure 1 displays the average improvement in each area, with the highest enhancement observed in Objectives, followed by Learning Styles. Overall, there was a significant 36.91% increase in scores after the training, surpassing the scores obtained in the psychosocial support assessment. These assessment results, similar to the Psychosocial Support Assessment, indicate the positive impact of the training program on the participants' professional knowledge and their understanding of adaptive learning strategies and positive disciplinary approaches Figure 1: Impact of the training on knowledge related to Adaptive Learning Strategies Improvement Rates -Adaptive Learning Strategies 60% 49.45% 44.80% Percentages 36.91% 40% 28.93% 24.46% 20% 0% General Knowledge Objectives Key Terms Learning Styles Overall Areas of Measurement 7. Assessment scale to measure the effectiveness of the teachers and counselors The objective of the Assessment Scale to measure the effectiveness of teachers and counselors was to capture changes in their professional behavior within the school environment. The Assessment Scale was specifically designed to assess the effectiveness of the training workshops across four domains for teachers: lesson planning, lesson execution, lesson evaluation, and communication and interaction in the classroom. For counselors, it focused on areas related to academic support, psychosocial support, support for particular groups, and support for the public- school system. 18 Among the trained teachers, 60.3% achieved mastery, 23.6% needed further development, and 16.1% did not recognize the topic. In contrast, only 21.2% of the untrained teachers achieved mastery, with 47.4% needing development and 31.4% not meeting the identified criteria. These results clearly indicate that trained teachers performed significantly better overall compared to those who did not participate in the training. The findings underscore that participating teachers acquired fundamental knowledge through the training, leading to improvements in their professional performance and effectiveness. These positive outcomes emphasize the importance of developing and enhancing skills that align with the training objectives. Moreover, the findings consistently revealed that trained teachers across all Iraqi cities outperformed their untrained counterparts. For counselors, the analysis focused on identifying the need for psychosocial support, evaluating support strategies and systems, and assessing their planning for psychosocial support programs. Furthermore, their capacity to provide academic support, support for special needs students, and assistance in supporting school systems were examined. The results demonstrated that participating counselors who received training achieved a notably higher percentage (61.2%) in terms of high and entirely satisfactory effectiveness compared to those who did not participate (19.1%). This positive impact of training was evident in their overall performance scores, which fell within the range of 51 to 60, showcasing minimal variation. 8. Student performance in Mathematics and Arabic language tests The objective of the Student Assessment was to evaluate and assess the development of students' skills in mathematics and Arabic language across different grade levels. This assessment, known as the Achievement Tests, was based on the content of the Iraqi curriculum and aligned with the general goals and competencies set by the Ministry for each subject in grades 1 to 3. Teachers designed these tests using recognized methods, aligning them with specific lessons. A flexible methodological template was provided to accommodate variations, ensuring tailored assessments. Data collection involved 2750 students across 6 governorates (7 directorates) and was carried out by the 95 training leaders. The assessment specifically focused on students' progress in mathematics topics such as numbers, addition, subtraction, multiplication, division, measurement, shapes, and geometry, as well as their skills in Arabic language, encompassing listening, speaking, reading, and writing. A total of 2,676 students from grades 1 to 3, comprising both participants and non-participants, underwent assessments of their performance in Arabic and Math. The distribution of students across grades is seen in Table 5. 19 Table 5: Distribution of Students Across Grades for Arabic and Math Assessment Participants Non-participants Total Grade 1 410 482 892 Grade 2 417 476 893 Grade 3 416 475 891 Total 1243 1433 2676 Student Performance in Arabic This evaluation encompassed various aspects of the Arabic language, including Listening, Speaking, Reading, Phonetics, and Writing. The assessment of students' performance involved categorizing them into three groups: Poor, Fair, or Good. Students were assigned ratings based on their performance in these areas, reflecting their overall proficiency in the Arabic language. Grade 1 students showed the highest overall ratings, with 68.8% of participants rated good compared to 65% in Grade 2 and 60.8% in Grade 3. Participants in Grade 1 exhibited a notable difference in performance compared to non-participants across various sub-fields, with a range of 15 to 24 percentage points higher for participants. For Grade 2, the difference between participants and non-participants in ratings varied from 9 percentage points to 16 percentage points. Writing had the highest difference (16 percentage points), while reading and phonetics had the lowest (9 percentage points). Writing was the only sub-field where more than 70% of the participating students were rated good. Grade 3 had the lowest average number of students with a "good" rating across all fields. The difference between participants and non-participants ranged from -9% to 19%, with listening showing the highest difference and phonetics the lowest. Reading and speaking had minimal differences as well. Overall, the results indicate the positive impact of training on teachers and subsequent improvements in students' performance, with Grade 1 students demonstrating the highest achievements. Student Performance in Mathematics 20 The students' ratings and tests were based on the Math curriculum, with topics varying across grades and increasing in difficulty. Grade 1 students were assessed on seven topics: Styles, Numbers, Addition, Subtraction, Shapes, Classification, and Measurement. Grade 2 students had eight topics: Numbers, Addition, Subtraction, Multiplication, Fractions, Shapes, Measurement, and Data Representation and Interpretation. Grade 3 students were evaluated on eight topics as well: Numbers, Addition and Subtraction, Multiplication, Division, Fractions, Measurements, Shapes, and Data Representation and Interpretation. Unlike their performance on the Arabic assessment, the average scores for Grade 1 participants were the lowest (56%) compared to Grade 2 (60.8%) and Grade 3 (59.3%) students. Participants performed similarly to non-participants in the topic of Styles, with only a 2-percentage point difference. The performance difference ranged between 6 and 8 percentage points for other areas like Shapes and Measurement. However, there was a significant disparity in the topic of Classification, where 63% of participants were rated as good compared to only 24% of non-participants. The performance in Grade 2 showed variability across different topics. The difference in overall scores between participants and non-participants ranged from zero to 44 percentage points. In terms of Measurements, more non-participants (53%) were rated as "good" compared to participants (51%). There was no difference in the percentage of students rated "good" for the Addition topic between participants and non-participants. However, participating students had significantly higher percentages rated as "good" in topics like Fractions (69%), Shapes (70%), and Data Representation and Interpretation (65%) compared to non-participants. In Grade 3, the performance gap between participants and non-participants ranged from 3 to 32 percentage points. Unlike Grade 2, where non-participants performed better in Measurements, the majority of Grade 3 participants (69%) received a good rating in Measurements, with a significant 32 percentage point difference. However, there was only a 3- percentage point difference between participants (48%) and non-participants (45%) in terms of "good" ratings for Multiplication. The results indicate consistent outperformance by participants compared to non- participants across all grades. Grade 1 participants had lower average scores than Grades 2 and 3, particularly in Shapes and Measurement. Notably, there was a substantial disparity observed in the Classification topic, where a higher percentage of Grade 1 participants excelled compared to non-participants. In Grade 2, participants excelled in Fractions, Shapes, and Data Representation, while Grade 3 participants demonstrated significant improvement in Measurements, with only a minor difference in Multiplication ratings. Figures 2 and 3 display the overall totals for participants and non-participants in each grade for both assessments. 21 Figure 2: Impact of Training on Arabic Figure 3: Impact of Training on Math Impact of Training on Arabic Impact of Training on Mathematics 80 77.8 77.2 Average score 80 63.7 67.8 65.3 75.1 Average scpre 55.6 59.2 57.6 75 60 69.3 69.3 40 70 67.1 20 65 0 60 No-participants Participants No-participants Participants Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3 9. Children’s psychological well-being scale A total of 2,676 students, including 1,243 participants and 1,433 non-participants, participated in the Children's Psychological Well-Being Scale assessment. Data collection was carried out by the 95 training leaders. The Children’s Psychological Well-Being Scale assessed children's psychological well-being through three main dimensions: 1. Empathy and Positive Relationships: This dimension focused on assessing a child's ability to empathize and maintain positive relationships with others. It included statements related to cooperative work with friends, enjoyment of various activities, feeling at peace with family, familiarity with school friends, preference for group activities, and emotional responses to sharing and companions' feelings. Some of the statements included phrases like "they felt at peace with their family in their home" and "they liked learning new things through group activities." 2. Self-Satisfaction: This dimension evaluated a child's level of self-satisfaction and contentment. It included statements related to self-perception, self-esteem, and overall satisfaction with oneself. Some of the statements mentioned that "they loved themselves," "they felt loved by their friends," and "they felt optimistic about the future." 3. Independence and Decision Making: This dimension assessed a child's level of independence and ability to make decisions. It included statements related to a child's autonomy, decision-making skills, and independence in various aspects of life. Some of the statements indicated that "they felt confident when doing things their way," "they loved discovering new places," and "they could take care of those younger than them. 22 Participating students showed significantly higher ratings for "yes" responses (58%-61.8%), while non-participants had lower scores (13.8-21.9). Conversely, non-participants had higher scores for "no" responses (30.5%-34.2%) compared to participants (14.8%-18%). The training positively impacted participating students' well-being, with a high achievement rate of 59.7% for "yes" responses and overall performance scores ranging from 45% to 54%. Lessons Learned Content and Training The training program resulted in noteworthy outcomes. Firstly, teachers and counselors experienced a significant improvement in their knowledge and understanding of psychosocial support, with an average score increase of 38.37%. This demonstrates the effectiveness of the training in enhancing their expertise in this area. Secondly, participants showed a boost in their scores on the adaptive learning strategies assessment, indicating a positive impact on their knowledge and comprehension. This suggests that the training successfully equipped them with effective and innovative approaches to enhance the educational process. Trained teachers demonstrated higher levels of effectiveness and performance in implementing the training they had received in classrooms, emphasizing the importance of comprehensive training to enhance their teaching abilities. Similarly, participating counselors who received training displayed higher effectiveness scores compared to non-participants, underscoring the value of the training in improving their performance in providing psychosocial support. Furthermore, the training program positively affected students' psychological well-being, as evidenced by their higher ratings in areas such as empathy, self-satisfaction, and independence. This indicates the training's success in promoting positive mental well-being among students. Notably, Grade 1 students showcased the highest overall ratings and notable improvements in Arabic language performance, highlighting the positive impact of the training on their language skills. Additionally, participants generally outperformed non-participants in mathematics, particularly in specific topics like Classification, demonstrating the training's effectiveness in enhancing students' mathematical abilities in targeted areas. Implementation An essential aspect that would have facilitated a more comprehensive impact evaluation would have been the establishment of a baseline for teachers' and counselors' knowledge and 23 practices, as well as children's learning experiences and achievements, prior to project initiation. By conducting a comprehensive assessment and data collection before project start, that would have provided valuable insights into the existing strengths, weaknesses, and areas for improvement. This baseline data would serve as a reference point for measuring the impact and effectiveness of the project over time, allowing for targeted interventions and informed decision- making to enhance the overall outcomes and ensure meaningful progress. These difficulties highlight the importance of addressing data gaps and ensuring adequate resources for conducting thorough assessments. The originally scheduled evaluation, set to take place between May 2020 and May 2021, was postponed and rescheduled for the period from September 2021 to November 2022. Conducting the evaluation right after the training is crucial for several reasons. First, it allows for immediate feedback on the effectiveness of the training program, enabling timely adjustments and improvements. Second, participants' retention of knowledge and skills is at its highest immediately after training, making it an optimal time to assess their performance and application. Lastly, conducting evaluations promptly helps to maintain the relevance and accuracy of the data collected, as recall bias and other factors that may affect responses are minimized. Although we acknowledge that the evaluation was delayed due to unforeseen restrictions and measures imposed as a result of COVID-19, this serves as a valuable lesson for future training programs. It highlights the importance of adapting to unforeseen circumstances and implementing strategies to ensure timely evaluation even in challenging situations. Aligning with the earlier point on thorough planning, it is crucial to emphasize the significance of fostering ownership and streamlining communication. This approach not only helps minimize unanticipated delays and resource needs during training initiation but also addresses time-consuming processes and communication challenges. By promoting faster approvals, cultivating willingness among trainers and educators, and fostering a learning culture, stakeholders at all levels can contribute to a more efficient and effective implementation process, benefiting the overall project's success Considering the COVID-related delays experienced, for future endeavors, it remains crucial to underscore the importance of meticulous planning and the anticipation of potential challenges during the training initiation phase. This proactive approach will serve to mitigate unforeseen delays and reduce the necessity for extra resources. By thoughtfully addressing potential obstacles during the planning stage, teams can enhance their preparedness, ultimately leading to improved efficiency and effectiveness during training implementation. 24 Measuring the impact of the training is critical. Within the framework of the Kirkpatrick model, one of the key levels is results, which assesses the training's influence on organizational goals and business outcomes. Correspondingly, it is crucial to determine whether the training was effectively implemented in the classroom, embraced by teachers, and ultimately had an impact on students. The ultimate measure of success lies in integrating the acquired knowledge into teachers' instructional practices and counselors' daily routines. Consequently, meticulous planning and provision for comprehensive follow-up evaluations are imperative to gauge these outcomes effectively. Based on the successes and challenges, following are some recommendations: 1. Continuous and Comprehensive Training: Given the difference in the responses of trained and untrained teachers and counselors, and the positive impact of training, it is recommended to establish regular and comprehensive training programs for teachers and counselors. Based on the implementation experience and teacher training conducted for this project, it will be helpful to develop a long-term comprehensive capacity development plan for future teacher training initiatives. This will ensure a systematic and effective approach to enhancing teachers' skills and knowledge, leading to improved educational outcomes. This will foster continuous improvement in their knowledge of psychosocial support, adaptive learning strategies, and relevant areas. Ongoing training opportunities will enable educators and counselors to enhance their expertise and stay up to date with innovative approaches in education and support. 2. Conduct a comprehensive baseline assessment: Data collected before project initiation provide valuable insights into areas for improvement and serve as a reference point for measuring the project impact and effectiveness. 3. Target Interventions for Grade 1 Students: Given the lower performance of Grade 1 students in mathematics compared to other grades, it is essential to implement targeted interventions and provide additional support specifically designed for this grade. These interventions should address the unique challenges faced by this grade and provide tailored instruction, resources, and support to enhance their mathematical abilities. By targeting early interventions, students can develop a strong foundation in mathematics, setting them up for success in higher grades. 4. Enhance Instruction in Specific Assessment Topics: To address low performance in specific assessment topics at each grade level, it is crucial to identify these areas and implement targeted policies. These policies should prioritize improving instruction and allocating necessary resources for the identified topics. Moreover, providing additional 25 training and support for teachers will enable them to effectively teach these challenging areas, leading to improved student performance and academic outcomes. 5. Address Implementation Challenges: To overcome the challenges faced during the project's implementation phase and incorporate lessons learned, it is essential to implement comprehensive policies that address these issues effectively. The following recommendations aim to address the implementation challenges mentioned above: a. Impact of the COVID-19 Pandemic: Establish flexibility and contingency plans to ensure project continuity in the face of future uncertainties and disruptions similar to the COVID-19 pandemic b. Lack of coordination and delays in decision-making: Empower the Directorate General (DG) at the ministry level. By granting the DG the authority to delegate and decentralize decision making power, responsibilities can be distributed more effectively. This approach can streamline the decision-making processes, leading to expedited project implementation and improved outcomes. By decentralizing authority, relevant stakeholders at different levels can be engaged, enhancing coordination and promoting local ownership of the project. c. Time-consuming administrative processes: In line with the earlier recommendation, streamline and simplify administrative procedures by reducing bureaucratic steps and implementing efficient digital systems. d. Geo-demographic considerations: Customize project timelines and contractual agreements to accommodate the unique administrative and logistical requirements of each governate, considering their diverse geographic and demographic characteristics. To ensure comprehensive coverage and effective implementation of the project, it is recommended to involve relevant offices and stakeholders at the governorate level, particularly during the delegation process. The trainings conducted at the governorate level present a valuable opportunity to engage and include local offices and stakeholders, enhancing the efficiency and effectiveness of the training implementation. e. Follow-up: It may have been beneficial to conduct a follow-up test or analysis a few months after the final reporting to assess the level of retention and usage of the acquired knowledge and skills. This will provide valuable insights into the long-term effectiveness and application of the training program. 26 References International Development of Education and Leadership - IDEAL. (2022). Consulting Services for Training of Trainees and Material Design EODP-RF/C16-1 Mission 6 – Final Report. Johnson, K. F., Kim, H., Molina, C. E., Thompson, K. A., Henry, S., & Zyromski, B. (2023). School counseling prevention programming to address social determinants of mental health. Journal of Counseling & Development, 00, 1– 14. https://doi.org/10.1002/jcad.12471 Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publishers. McCallum, F. (2021). Teacher and Staff Wellbeing: Understanding the Experiences of School Staff. In: Kern, M.L., Wehmeyer, M.L. (eds) The Palgrave Handbook of Positive Education. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-64537-3_28 Organization for Economic Cooperation and Development. (2023). “Schools as hubs for social and emotional learning: Are schools and teachers ready?". OECD Education Spotlights, No. 4, OECD Publishing, Paris. https://doi.org/10.1787/f6d12db7-en. RTI International. (2012). Education Data for Decision Making (EdData II): Iraq Education Surveys– MAHARAT. Analysis of Student Performance in Reading and Mathematics, Pedagogic Practice, and School Management. https://pdf.usaid.gov/pdf_docs/PA00JBRG.pdf UNICEF Iraq. (2018). Education. https://www.unicef.org/iraq/what-we-do/education UNICEF. (2022). Learning Brief on Mental Health and Psychosocial Support (MHPSS) in Education. https://www.unicef.org/media/134766/file/Global%20Multisectoral%20Operational%20F ramework.pdf World Bank. (2015). Project Appraisal Document on a Proposed Loan in the Amount of US$ 350 Million to the Republic of Iraq for an Emergency Operation for Development. World Bank. (2017). Combined Project Information Documents / Integrated Safeguards Data Sheet World Bank. (2017). Project paper on a proposed additional loan in the amount of US$400 million to the Republic of Iraq for the Emergency Operation for Development Project. 27 World Bank. (2018). World Development Report 2018: Learning to realize education's promise from https://openknowledge.worldbank.org/handle/10986/28340 World Bank. (2021). What is Learning Poverty? Retrieved from https://www.worldbank.org/en/topic/education/brief/what-is-learning-poverty World Bank. (2022). Project Appraisal Document On A Proposed Grant In The Amount Of Us$10 Million Equivalent To The Republic Of Iraq For The Innovations Towards Learning In Lagging Iraqi Governorates Project. World Bank. (2023). Concept Note- Support to Education and Skills Development in Iraq. 28