SCHOOLS, STUDENTS, AND TEACHERS IN LEBANON RESEARCH FOR RESULTS PROGRAM 2021 SCHOOLS, STUDENTS, AND TEACHERS IN LEBANON RESEARCH FOR RESULTS PROGRAM 2021 © 2021 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. This report was prepared by the World Bank Education Global Practice Middle East & North Africa Unit and included, in alphabetical order, Hana Addam El-Ghali, Mary Bend, Sheena Fazili, Nadine Franji, Nathalie Lahire, Anahita Matin, and Elisabeth Sedmik. This report is based on background reports prepared by each of the RAND Corporation, Management Systems International, the World Bank and the American University of Beirut. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Cover design: Sarah Alameddine TABLE OF CONTENTS 4 EXECUTIVE SUMMARY 7 ‫الملخص التنفيذي‬ 10 Abbreviations and Acronyms 11 Introduction 11 Sector Context 13 Purpose and Objectives of Research for Results 15 1. Overview of R4R Studies and Methodology 15 Overview of Studies 25 2. Key Findings of the R4R Studies 25 Findings from the School Study 32 Findings from the Student Assessment Study 40 Findings from the Teacher Study 49 Findings from the Vulnerability Study 53 3. Policy Recommendations 53 System Reform Recommendations 54 Recommendations for Teacher Professional Development Reform 55 Recommendations for Reform of the School Environment 56 Recommendations for Student-Centered Support 57 References List of Figures 19 Figure 1.1. Teacher Content Evaluation (sections 1 and 2 examples) 20 Figure 1.2. CLASS Domains 22 Figure 1.3: Post Observation Instrument Construct 32 Figure 2.1. Grade 4 Score Difference by School Type 33 Figure 2.2. Grade 7 Score Difference by School Type 34 Figure 2.3. Upward Learning Mobility as a Challenge 35 Figure 2.4: Grade 7 Rounds Average Score Distribution 36 Figure 2.5: Grade 4 Rounds Average Score Distribution 37 Figure 2.6. Grade 4 Total Score by School Type 37 Figure 2.7: Grade 7 Total Score by School Type 38 Figure 2.8. Grade 4 Total Score by Gender and School Type 38 Figure 2.9. Grade 7 Total Score by Gender and School Type 39 Figure 2.10. Grade 4 Scores by Governorate 39 Figure 2.11. Grade 7 Scores by Governorate 40 Figure 2.12. Average Score Statistics for Teacher Content Knowledge Evaluation—Section 1 41 Figure 2.13. Average Score Statistics for Teacher Content Knowledge Evaluation—Section 2 42 Figure 2.14. Teacher Evaluation Score in Grade 7 French Language—By Shift 43 Figure 2.15. CLASS Domain Analysis Score, Lebanon 44 Figure 2.16. CLASS Score—Subject Domain Analysis, Lebanon 45 Figure 2.17. CLASS Score Distribution in Lebanon by Shift 45 Figure 2.18. Instructional Support Distribution in Lebanon by Shift 46 Figure 2.19. CLASS Score Distribution in Lebanon by School Type 4 EXECUTIVE SUMMARY Lebanon’s education sector consists of a public school system, and a larger, diverse private school system. Substantial effort and progress have been made towards universal basic education; however, Lebanon’s performance remains far below most countries in the Middle East and North Africa (MENA) region and the rest of the World on international student assessments (across both public and private schools). Student migration to public schools started in 2019 and is expected to continue in 2021-2022 and beyond as the financial situation in Lebanon persists and the economic status of families continues to deteriorate. In 2021, it is estimated that more than 82 percent of the population live in poverty, 1 an increase from 55 percent in 2020.2 This report presents the findings from the second phase of the Research for Results (R4R) program. The R4R program is a partnership between the Ministry of Education and Higher Education (MEHE), the World Bank, the UK Foreign, Commonwealth and Development Office, and the United States Agency for International Aid (USAID). The R4R’s primary objective is to generate quantitative evidence on student and teacher performance, school environment and management, and qualitative evidence related to vulnerable youth at risk of dropping out. This evidence-based analysis is supported by policy recommendations and are featured in the new Government five-year general education strategy (2021-2025). The R4R program was conducted in two phases, first analyzing Lebanon’s education sector from a political economy perspective, and second, analyzing key school-level factors that affect learning, teaching and retentions. 1. The first phase of the R4R (R4R Volume 1) focuses on the political economy of education in Lebanon and includes an assessment of the performance of Lebanon’s bifurcated system in terms of learning outcomes, equity, and efficiency, while explaining its foundations and underlying dynamics ( Abdul-Hamid and Yassine 2020). 2. The second phase of the R4R (Volume 2, this report) builds on R4R Volume I by deep diving into the various school-level factors that affect learning, teaching and retention. The analysis included in this Volume 2 also addresses the education of vulnerable communities and refugee children who constitute a large proportion of school-age youth in Lebanon. 1 https://news.un.org/en/story/2021/09/1099102 2 ESCWA 2020 5 The second phase R4R program generates quantitative evidence on student and teacher performance, school environment and management, and qualitative evidence related to vulnerable youth at risk of dropping out. This R4R program Volume 2 encompasses four studies that were conducted between 2016 and 2018 and comprise a nationally representative sample of 145 schools including public schools operating the first and second shifts of the day respectively, free private schools, and fee-based private schools. This sample is featured in all the studies described below except for the vulnerability study, for which the target group was different as it focused on children and youth who were at risk of dropping out or had dropped out of school. 1. The School Study aimed to provide an understanding of the enabling context of schools in Lebanon including school finances, management, teachers, school environment, and facilities. This study took place during the 2016–2017 and 2017–2018 academic years and focused on examining the contexts and processes of public and private schools in four key dimensions: (1) school-based management; (2) teachers; (3) school finance; and (4) school environment and facilities. 2. The Student Assessment Study aimed to provide a better understanding of student learning outcomes and perceptions in Lebanese classrooms. The assessment was administered at the beginning and end of the academic year 2016–2017, focusing on Grades 4 and 7 in mathematics (taught in Arabic, English, or French, depending on a given school’s language of instruction), Arabic (reading and writing) and foreign languages (French or English reading and writing). The study included 5,806 students from Grade 4 and 4,335 students from Grade 7 participating in the assessments. The assessments were aligned with grade-appropriate skill levels and learning outcomes as outlined in the national curriculum. 3. The Teacher Performance Study collected data on teacher knowledge and classroom practices to provide an understanding of teachers’ classroom practices and content knowledge performance in Lebanese schools. The study focused mainly on Grades 4 and 7 classrooms in subjects similar to the student assessment study (i.e., mathematics taught in English or French, Arabic, and foreign languages—either French or English). In total, there were 599 teachers in the first-shift schools, and 108 teachers in the second-shift schools. Classroom teacher observations (CLASS) and an evaluation of teachers’ general content knowledge were used as the primary tools for this study. 4. The Vulnerability Study aimed to identify the risks and opportunities for improving educational experiences among vulnerable Lebanese and refugee children. The study examined the educational experiences of 1,800 vulnerable children in Lebanon within different community types and across different ages and grade levels. It explored the risks, protective factors, and trends contributing to dropping out of school among vulnerable Lebanese and Syrian students. 6 Summary of Key Recommendations Evidence from the four studies provide a wealth of information on the current status of Lebanon’s education system and across key education indicators. At the system level, public school administration needs to shift toward a more decentralized approach while strengthening school-based management, empowering principals and building a community of learning in which parents and the larger community carry a joint responsibility along with shared accountability. Additionally, the curriculum has not been reformed since 1997 and should focus on skills and learning that are required in a fast paced and changing world. Teaching practices vary by subject in Lebanon, yet there is an underlying trend that they must shift from a reliance on rote learning and memorization towards facilitating higher order thinking. Only then can the quality of instruction and student learning outcomes improve. The studies brought out further that there is a difference between the quality of teaching at first-shift public schools and the quality at second-shift, especially with regard to teaching Arabic language. A recent World Bank report, Advancing Arabic Language Teaching and Learning: A Path to Reducing Learning Poverty in MENA, provides solutions to this matter and highlights the inherent challenges in learning Arabic at schools as the Modern Standard Arabic (MSA) taught is different than the way children speak at home (Gregory et al. 2021). The same report also addresses the challenges that Arabic language teachers themselves face as a product of ineffective Arabic language education, highlighting how these teachers are often not comfortable using Arabic as a medium of instruction. Improving the quality of teaching is therefore closely linked to the need to allocate teachers who specialize in the subject they teach while equipping them with the pedagogic skills to facilitate growth in student learning. The studies shed light that the current system is not sufficiently supporting students with academic weaknesses and does not help them to improve. Lowest performing students for the most part remained low performers as illustrated by the student assessment study. Early intervention programs, and remedial programs must be institutionalized to focus on the needs of struggling students and retain them in school . The vulnerability study highlighted that bullying is a key push-out factor driving vulnerable students towards school drop-out . A safe, conducive, and nurturing learning environment, free of bullying but with high levels of stimulation (including ICT) is critical for improving academic achievement and supporting the development of social and emotional skills. ‫‪7‬‬ ‫الملخص التنفيذي‬ ‫خاص أكبر‬‫يتكون قطاع التعليم في لبنان من نظام مدرسي رسمي أو حكومي ونظام مدرسي ّ‬ ‫ّ‬ ‫تقدم كبير نحو تعميم التعليم األساسي؛ ومع ذلك‪ ،‬ال‬ ‫ّ‬ ‫حقيقية وإحراز‬ ‫ّ‬ ‫تم بذل جهود‬‫ومتنوع‪ .‬وقد ّ‬ ‫ّ‬ ‫وبقية‬ ‫ّ‬ ‫يزال أداء لبنان أدنى بكثير من معظم البلدان في منطقة الشرق األوسط وشمال أفريقيا‬ ‫والخاصة)‪ .‬بدأت حركة نزوح‬ ‫ّ‬ ‫الرسمية‬ ‫ّ‬ ‫كل من المدارس‬‫للطالب (في ّ‬ ‫العالم في التقييمات الدولية ّ‬ ‫تستمر في‬ ‫ّ‬ ‫أن‬ ‫ع‬ ‫ّ‬ ‫المتوق‬ ‫ومن‬ ‫‪2019‬‬ ‫عام‬ ‫في‬ ‫ة‬ ‫الرسمي‬ ‫ّ‬ ‫المدارس‬ ‫إلى‬ ‫الخاصة‬ ‫ّ‬ ‫الطالب من المدارس‬ ‫العام الدراسي ‪ 2022-2021‬وما بعده مع استمرار الوضع المالي في لبنان واستمرار تدهور الوضع‬ ‫سكان لبنان يعيشون في الفقر في عام‬ ‫ّ‬ ‫أن أكثر من ‪ ٪82‬من‬‫االقتصادي لألسر‪ .‬تشير التقديرات إلى ّ‬ ‫‪4‬‬ ‫‪ 3،2021‬وذلك زيادة من ‪ ٪55‬من عام ‪.2020‬‬ ‫يعرض هذا التقرير نتائج المرحلة الثانية من برنامج البحث من أجل النتائج (‪ .)R4R‬برنامج ‪R4R‬‬ ‫هو شراكة بين وزارة التربية والتعليم العالي‪ ،‬البنك الدولي‪ ،‬وزارة الخارجية والتنمية البريطانية‬ ‫يتمثل الهدف األساسي لـ ‪ R4R‬في‬ ‫ّ‬ ‫(‪ )FCDO‬والوكالة األمريكية للمساعدة الدولية (‪.)USAID‬‬ ‫ّ‬ ‫ّ‬ ‫كمية حول أداء الطالب والمعلمين‪ ،‬البيئة المدرسية وإدارتها‪ ،‬واألدلة النوعية المتعلقة‬ ‫إنتاج ّ‬ ‫أدلة ّ‬ ‫التسرب من المدرسة‪ّ .‬‬ ‫يتم دعم هذا التحليل القائم‬ ‫ّ‬ ‫المعرضين لخطر‬ ‫ّ‬ ‫باألطفال االكثر عرضة للمخاطر‬ ‫االستراتيجية الخماسية (‪ )2025-2021‬التعليم العام‬ ‫ّ‬ ‫ويتم عرضها في‬‫ّ‬ ‫ّ‬ ‫األدلة من خالل توصيات‬ ‫على‬ ‫التي تم تطويرها من قبل وزارة التربية والتعليم العالي‪.‬‬ ‫تم تنفيذ برنامج ‪ R4R‬على مرحلتين‪ ،‬األولى عبر تحليل قطاع التعليم في لبنان من منظور‬ ‫ّ‬ ‫ّ‬ ‫الرئيسية على مستوى المدرسة التي تؤثر‬ ‫ّ‬ ‫االقتصاد السياسي‪ ،‬والثانية عبر تحليل العوامل‬ ‫ّ‬ ‫التعلم والتعليم واإلستبقاء‪.‬‬ ‫على‬ ‫ّ‬ ‫المجلد ‪ )1‬على االقتصاد السياسي للتعليم في لبنان‬ ‫ّ‬ ‫تركز المرحلة األولى من تقرير ‪R4R( R4R‬‬ ‫‪.1‬‬ ‫ ‬ ‫والفعالية‪ ،‬مع‬ ‫ّ‬ ‫ّ‬ ‫المتشعب من ناحية نتائج التعلم‪ ،‬اإلنصاف‬ ‫ّ‬ ‫وتشمل تقييم أداء النظام اللبناني‬ ‫شرح أسسها والديناميات األساسية (‪.)Abdul-Hamid and Yassine 2020‬‬ ‫المجلد ‪ 1‬من ‪ R4R‬من خالل‬ ‫ّ‬ ‫ّ‬ ‫(المجلد ‪ ،2‬هذا التقرير) على‬ ‫‪ .2‬تستند المرحلة الثانية من ‪R4R‬‬ ‫ّ‬ ‫ّ‬ ‫الغوص العميق في العوامل المختلفة على مستوى المدرسة التي تؤثر على التعلم والتعليم‬ ‫واالستبقاء‪ .‬يعالج ً‬ ‫أيضا التحليل الوارد في هذا المجلد ‪ 2‬تعليم المجتمعات الضعيفة واألطفال‬ ‫يشكلون نسبة كبيرة من الشباب في ّ‬ ‫سن الدراسة في لبنان‪.‬‬ ‫ّ‬ ‫الالجئين الذين‬ ‫ّ‬ ‫كمية على أداء الطالب والمعلمين‪ ،‬البيئة‬ ‫أدلة ّ‬ ‫تنتج المرحلة الثانية من برنامج ‪ّ R4R‬‬ ‫المعرضين‬ ‫ّ‬ ‫ّ‬ ‫المتعلقة باالطفال االكثر عرضة للمخاطر‬ ‫النوعية‬ ‫ّ‬ ‫ّ‬ ‫واألدلة‬ ‫المدرسية وإدارتها‪،‬‬ ‫وتضم‬ ‫ّ‬ ‫ّ‬ ‫التسرب‪ .‬يشمل المجلد ‪ 2‬من برنامج ‪ R4R‬أربع دراسات أجريت بين عامي ‪ 2016‬و ‪2018‬‬ ‫ّ‬ ‫لخطر‬ ‫الرسمية التي‬ ‫ّ‬ ‫عينة تمثيلية على المستوى الوطني من ‪ 145‬مدرسة بما في ذلك المدارس‬ ‫ّ‬ ‫‪3 https://news.un.org/en/story/2021/09/1099102‬‬ ‫‪4 ESCWA 2020‬‬ ‫‪8‬‬ ‫مجانية‪ ،‬ومدارس ّ‬ ‫خاصة‬ ‫خاصة ّ‬‫تعمل في المناوبتين الصباحية وبعد الظهر على التوالي‪ ،‬مدارس ّ‬ ‫العينة في جميع الدراسات الموضحة أدناه باستثناء دراسة االطفال‬ ‫ّ‬ ‫مدفوعة األجر‪ .‬تظهر هذه‬ ‫أنها ّ‬ ‫ركزت على األطفال‬ ‫االكثر عرضة للمخاطر‪ ،‬حيث كانت المجموعة المستهدفة مختلفة بما ّ‬ ‫التسرب من المدرسة أوكانوا قد ّ‬ ‫تسربوا من المدرسة‪.‬‬ ‫ّ‬ ‫معرضين لخطر‬ ‫ّ‬ ‫والشباب الذين كانوا‬ ‫ الدراسة المدرسية‪ :‬هدفت الدراسة المدرسية إلى توفير فهم للسياق التمكيني‬ ‫‪.1‬‬ ‫للمدارس في لبنان بما في ذلك الوضع المالي واالداري‪ ،‬والمعلمين‪ ،‬والبيئة المدرسية‪،‬‬ ‫والمرافق‪ .‬أجريت هذه الدراسة خالل العامين األكاديميين ‪ 2017-2016‬و‪ 2018-2017‬وركزت على‬ ‫دراسة سياقات المدارس العامة والخاصة من خالل أربعة أبعاد رئيسية‪ )1( :‬اإلدارة الذاتية‬ ‫للمدرسة ؛ (‪ )2‬المدرسين ؛ (‪ )3‬تمويل المدرسة و (‪ )4‬البيئة المدرسية والمرافق‪.‬‬ ‫ دراسة تقييم الطالب‪ :‬هدفت دراسة تقييم الطالب إلى توفير فهم أفضل لنتائج تعلم‬ ‫‪.2‬‬ ‫الطالب وتصوراتهم في الفصول الدراسية اللبنانية‪ .‬تم إجراء التقييم مرتين‪ ،‬في بداية‬ ‫ونهاية العام الدراسي ‪ ،2017-2016‬مع التركيز على الصفين الرابع والسابع في المواد التالية‪:‬‬ ‫اعتمادا على لغة التدريس في‬ ‫ً‬ ‫الرياضيات (تدرس باللغة العربية أو اإلنجليزية أو الفرنسية‪،‬‬ ‫مدرسة معينة)‪،‬العربية (قراءة وكتابة) واللغات األجنبية (الفرنسية أو اإلنجليزية قراءة وكتابة)‪.‬‬ ‫طالبا من الصف السابع الذين شاركوا في‬ ‫ً‬ ‫طالبا من الصف الرابع و‪4،335‬‬‫ً‬ ‫شملت الدراسة ‪5،806‬‬ ‫التقييمات‪ .‬تم مواءمة التقييمات مع مستويات المهارات المناسبة للصف ونتائج التعلم كما‬ ‫هو موضح في المناهج الدراسية الوطنية‪.‬‬ ‫ دراسة أداء المعلم‪ :‬جمعت دراسة أداء المعلم بيانات حول المعرفة لدى المعلم‬ ‫‪.3‬‬ ‫وممارسات الفصل الدراسي لتوفير فهم لممارسات الفصل الدراسي للمعلمين واألداء‬ ‫المعرفي للمحتوى في المدارس اللبنانية‪ .‬ركزت الدراسة بشكل أساسي على الصفين‬ ‫الرابع والسابع في الفصول الدراسية في مواد مشابهة لدراسة تقييم الطالب (أي الرياضيات‬ ‫التي تدرس باللغة اإلنجليزية أو الفرنسية‪ ،‬العربية واللغات األجنبية (الفرنسية أو اإلنجليزية))‪.‬‬ ‫معلما في مدارس‬ ‫ً‬ ‫معلما في مدارس الدوام الصباحي‪ ،‬و ‪108‬‬ ‫ً‬ ‫في المجموع‪ ،‬كان هناك ‪599‬‬ ‫دوام بعد الظهر‪ .‬تم استخدام مراقبة معلم الفصل (‪ )CLASS‬وتقييم المعرفة العامة بالمحتوى‬ ‫للمعلمين كأدوات أساسية لهذه الدراسة‪.‬‬ ‫ دراسة االطفال االكثر عرضة للمخاطر‪ :‬هدفت دراسة االطفال االكثر عرضة للمخاطر إلى‬ ‫‪.4‬‬ ‫تحديد المخاطر والفرص لتحسين فرص التعلم لألطفال اللبنانيين االكثر عرضة للمخاطر‬ ‫والالجئين‪ .‬سلطت الدراسة الضوء على التجارب التربوية لـ ‪ 1800‬طفل من الفئة االكثر عرضة‬ ‫للمخاطر في لبنان ضمن مجتمعات مختلفة ومن مختلف األعمار والمستويات الصفية‪ .‬كشفت‬ ‫الدراسة المخاطر وعوامل الحماية التي تواجه هذه االطفال واالتجاهات التي تسهم في‬ ‫التسرب من المدرسة بين الطالب اللبنانيين االكثر عرضة للمخاطر والالجئين السوريين‪.‬‬ ‫ملخص التوصيات الرئيسية‬ ‫توفر األدلة من الدراسات األربع هذه ثروة من المعلومات حول الوضع الحالي لنظام التعليم في‬ ‫لبنان وعبر مؤشرات التعليم الرئيسية‪.‬‬ ‫على مستوى النظام العام‪ ،‬من الضروري أن تتحول إدارة المدارس الرسمية إلى نحو نهج أكثر‬ ‫المركزية مع تعزيز اإلدارة الذاتية للمدرسة‪ ،‬باالضافة الى ضرورة تمكين المديرين وبناء مجتمع‬ ‫تعليمي يتحمل فيه األهل وأولياء شؤون الطالب والمجتمع األكبر مسؤولية مشتركة ً‬ ‫جنبا إلى‬ ‫جنب مع المساءلة المشتركة‪ .‬باإلضافة إلى ذلك‪ ،‬لم يتم إصالح المنهج منذ عام ‪ 1997‬ولذلك‬ ‫يجب أن يركز المنهج الجديد على المهارات ونظام التعلم المطلوبين في عالم سريع الخطى‬ ‫ومتغير‪ .‬تختلف ممارسات التدريس باختالف المادة في لبنان‪ ،‬ولكن هناك اتجاه أساسي مفاده‬ ‫‪9‬‬ ‫أنه يجب أن يتحول نظام التعليم من االعتماد على الحفظ والتسميع إلى نظام يرتكز على تحفيز الفكر بأعلى‬ ‫الدرجات‪ .‬عندها فقط يمكن أن تتحسن جودة التدريس ونتائج تعلم الطالب‪ .‬كما أوضحت الدراسات أن هناك ً‬ ‫فرقا بين‬ ‫جودة التعليم في المدارس الحكومية في الدوام الصباحي وجودة التعليم في دوام بعد الظهر‪ ،‬ال سيما فيما يتعلق‬ ‫حديثا عن البنك الدولي بعنوان النهوض بتعليم اللغة العربية وتعلمها‪ :‬طريق‬ ‫ً‬ ‫بتدريس اللغة العربية‪ .‬وفي تقرير صدر‬ ‫حلوال لهذه المسألة‪ ،‬كما ويسلط هذا التقرير‬ ‫ً‬ ‫دم‬ ‫قّ‬‫للحد من فقر التعلم في منطقة الشرق األوسط وشمال إفريقيا‪ ،‬تُ َ‬ ‫الضوء على التحديات الكامنة في تعلم اللغة العربية في المدارس في منطقة الشرق األوسط وشمال افريقيا‬ ‫حيث أن اللغة العربية الفصحى الحديثة التي يتم تدريسها مختلفة عن الطريقة التي يتحدث بها األطفال في المنزل‬ ‫أيضا التحديات التي يواجهها مدرسو اللغة العربية أنفسهم كنتاج لتعلم‬ ‫(‪ .)Gregory et al. 2021‬يتناول التقرير نفسه ً‬ ‫فعال‪ ،‬ويبرز كيف أن هؤالء المعلمين ً‬ ‫غالبا ما ال يشعرون بالراحة عند استخدام اللغة العربية‬ ‫اللغة العربية بشكل غير ّ‬ ‫ً‬ ‫ارتباطا وثيقا بالحاجة إلى تعيين مدرسين متخصصين في‬ ‫ً‬ ‫كوسيلة للتعليم‪ .‬لذلك فإن تحسين جودة التدريس يرتبط‬ ‫سونها مع تزويدهم بالمهارات التربوية لتسهيل النمو في تعلم الطالب‪.‬‬ ‫المادة التي ُي َ‬ ‫د ّر ُ‬ ‫كاف‬ ‫ٍ‬ ‫أيضا على أن النظام الحالي ال يدعم الطالب الذين يعانون من ضعف أكاديمي بشكل‬‫ً‬ ‫سلطت الدراسات الضوء‬‫ّ‬ ‫أداء في الغالب من ذوي األداء المنخفض (أي أن فرص تقدمهم‬ ‫ّ‬ ‫وال يساعدهم على التحسن‪ .‬يبقى ّ‬ ‫الطالب األقل ً‬ ‫جدا) كما هو موضح في دراسة تقييم الطالب‪ .‬يجب إضفاء الطابع المؤسسي على برامج التدخل‬ ‫االكاديمي ضئيلة ً‬ ‫المبكر والبرامج العالجية للتركيز على احتياجات الطالب المتعثرين والمساعدة على ابقائهم في المدرسة‪.‬‬ ‫المعرضين للخطر نحو‬ ‫ّ‬ ‫مر هو عامل رئيسي يدفع الطالب‬ ‫نّ‬‫الت َ‬ ‫أبرزت دراسة االطفال االكثر عرضة للمخاطر أن َ‬ ‫التنمر‪ ،‬ولكن بمستويات عالية‬ ‫ّ‬ ‫تعليمية آمنة ومواتية وراعية وخالية من‬ ‫ّ‬ ‫التسرب من المدرسة‪ّ .‬‬ ‫إن وجود بيئة‬ ‫ّ‬ ‫األهمية لتحسين التحصيل األكاديمي ودعم‬ ‫ّ‬ ‫من التحفيز (بما في ذلك تكنولوجيا المعلومات واالتصاالت) أمر بالغ‬ ‫تنمية المهارات االجتماعية والعاطفية‪.‬‬ 10 ABBREVIATIONS AND ACRONYMS CERD Center for Educational Research and Development CLASS Classroom Assessment Scoring System DOPS Direction d’Orientation Pédagogique et Scolaire FCDO Foreign, Commonwealth and Development Office GDP Gross Domestic Product MEHE Ministry of Education and Higher Education MENA Middle East and North Africa MSI Management Systems International NGO Non-governmental Organization OOSC Out-of-School Children PISA Programme for International Student Assessment RACE Reaching All Children with Education R4R Research for Results SEL Social and Emotional Learning TIMSS Trends in International Mathematics and Sciences Study USAID United States Agency for International Aid 11 INTRODUCTION Sector Context Lebanon’s education sector is a tale of two systems: a public school system run by the government, and a larger, diverse private school system comprising well-established religion-based schools alongside others. Once known as “the classroom of the Middle East” for the reputation of its schools and universities, Lebanon’s education system has faced significant challenges in recent decades. Schools in Lebanon frequently face resource constraints and a security environment, both of which interrupt the academic calendar, disrupt student attendance, and make it difficult to develop and maintain a cadre of high-quality teachers. As a result, many of the country’s most talented and promising citizens emigrate. In Lebanon, roughly two-thirds of school students are enrolled in private schools.5 Very few countries in the world have a greater private sector share of primary or secondary school students.6 Although progress has been made toward universal basic education, substantial room remains for improvement. Net primary school enrollment— the share of primary-level students of a given age who are enrolled in the grade (year class) appropriate for that age—was 82 percent in 2016, while net secondary enrollment was 65 percent (UNESCO Institute for Statistics 2018). Thus, a significant proportion of students are not progressing through the education system in line with international benchmarks. Lebanon performs far below average on international assessments and below other countries in the Middle East and North Africa (MENA) region. Lebanon participated in the Trends in International Mathematics and Sciences Study (TIMSS) in 2015 and 2019 and in the Programme for International Student Assessment (PISA) in 2015 and 2018. In TIMSS 2019, Lebanon’s eighth grade students scored an average of 429 in mathematics, or 71 points below the international average score of 500. Further, this represented a decline of 13 points from the country’s performance in 2015 (CERD 2018). In science, Lebanon’s eighth graders ranked in the bottom quartile of participating countries, and their achievement had declined 21 points from 2015 to 2019 5 Lebanon’s school education system includes first-shift public schools, second-shift public schools, free private schools, and fee-based private schools. 6 According to World Bank 2015 data, only Macao SAR, China, Belize, and the United Arab Emirates had a higher share of primary level students enrolled in private schools than did Lebanon; similarly, Macao SAR, China, Bangladesh, and Guatemala had a higher share of secondary level students enrolled in private schools than did Lebanon (World Bank 2018). 12 (CERD 2018). PISA assesses the academic progress of students ages 15 to 16 in mathematics, science, and reading. In 2018, Lebanese PISA participants scored very poorly in all three subjects. Student underperformance amounted to either three or four years of schooling below the Organisation for Economic Cooperation and Development (OECD) average, depending on the subject. Performance in reading was especially low at 58 points below the MENA average, which in terms of productive classroom time equates to almost two years below the MENA average. In all three subjects, roughly two-thirds of participating students did not meet the basic level of proficiency, leaving them at risk of exclusion (World Bank 2019). While both private and public schools performed below international standards, the overall results of these internationally benchmarked assessments mask important differences between the public and private school systems in Lebanon. According to a World Bank analysis of PISA data, students enrolled in Lebanon’s private schools are on average two academic years ahead of students enrolled in the country’s public schools (Gajderowicz and Jakubowski, forthcoming). In Lebanon’s National Examination, taken in Grades 9 and 12, students enrolled in private schools also significantly outperform their public school counterparts. Public spending on education is low compared to that in other MENA countries with economies of a similar size. In 2015, for example, Lebanon’s public spending on education was 2.1 percent of gross domestic product (GDP) and 6.3 percent of total public expenditure. Government expenditure on education has further decreased from 2.0 percent of GDP in 2019 to 1.8 percent in 2020. However, after factoring in public subsidies for private schools (0.40 percent of GDP) and parents’ contributions (1.45 percent of GDP), total spending on schools as a share of GDP was almost 4.00 percent, which is closer to the MENA average (4.47 percent of GDP) and indeed the average for middle-income countries (4.40 percent of GDP) (UNESCO 2018; World Bank 2017; World Bank 2018). Lebanon’s beleaguered education system has been further challenged by the influx of Syrian refugees (which includes around 400,000 school-age children registered with the United Nations High Commissioner for Refugees [UNHCR]). Syrian refugee children currently represent 37 percent of the total student population in public schools in Lebanon (CERD 2020). This influx led to the introduction of a second shift of instruction in the afternoon to accommodate demand. Teachers also have to manage different curricula and language needs (discussed below) to accommodate the various needs of Lebanese and Syrian children, adjustments which impact the quality of learning for all students. Despite government efforts and the support of the international community to provide education to the Syrian population, almost one-half of Syrian refugees between the ages of 5 and 17 remain out of formal schooling (UNHCR 2019). This has both short- and long-term consequences for children, families, and society. For families coping with the daily struggles of displacement, schooling presents an added burden. However, an absence of schooling today is likely to contribute to a life of poverty tomorrow, eroding human capital development and exacerbating inequality. Both outcomes could contribute to the risk of future conflict and destabilization in the region. Lebanon and its public services, including the school system, have struggled to accommodate the ongoing influx of refugees while simultaneously addressing deeper systemic issues that beset the funding and delivery of a service of adequate quality. The protracted nature of the Syrian crisis, and the related increase in demand for schooling, have resulted in further strains on severely stretched public services, including public education for both refugee and host 13 community children. Moreover, the recent economic and financial crises during the 2020– 2021 academic year have led to the arrival at the front gates of public schools of an additional 50,000 students whose parents can no longer afford to pay private school fees elsewhere.7 It is further expected that more children will transition from private to public schools in the forthcoming academic year 2021–2022 due to the deteriorating economic situation in the country. The many crises, contraction of the economy, and an increase in poverty rates will likely lead to more parents shifting their children to public schools in the coming years, as well as a higher number of student dropouts. Families who used to prioritize education for their children in Lebanon, as seen in high private investments in education, will now be forced to prioritize other needs such as food and health. Education will increasingly become a commodity that only a few can afford. Just like health workers, teachers have started to leave the country, with many more expected to seek greener pastures elsewhere, further putting a strain on an already fragile system. According to the head of the Teachers’ Union, almost 15% of the private school teachers had left the country by September 2021 (El Deeb 2021). The future looks grim if quick action is not taken to reform the sector with a shift from a quality of education that benefits a few to one that ensures every child in Lebanon has access to quality education. Purpose and Objectives of Research for Results The Research for Results (R4R) program is a partnership between MEHE, the World Bank, the UK Foreign, Commonwealth and Development Office, and the United States Agency for International Aid (USAID). The program leverages education research and system-wide analyses, along with stakeholder engagement and communication, to improve decision- making on the provision of education in Lebanon. The R4R’s primary objective is to generate new evidence on student and teacher performance, insights into schools, and factors related to equity and vulnerability. This evidence will be used to provide data-driven policy recommendations to strengthen the efficiency and quality of education services in Lebanon and provide recommendations to policy makers for a post-Reaching All Children for Education (RACE) Phase 2. The R4R program comprises system-level political economy analyses (R4R Volume 1), combined with research on education service delivery (R4R Volume 2). 1. The first phase of the R4R (R4R Volume 1) focuses on the political economy of education in Lebanon and includes an assessment of the performance of Lebanon’s bifurcated system in terms of learning outcomes, equity, and efficiency, while explaining its foundations and underlying dynamics (Abdul-Hamid and Yassine 2020). 2. The second phase of the R4R (Volume 2, this report) builds on the political economy study of R4R Volume I by deep diving into the various school-level factors that affect learning and teaching. The analysis also focuses on service delivery within a nationally representative sample of schools (145 schools including public first and second shifts, free private, and fee-based private schools). The analysis included in this Volume 2 also 7 Preliminary numbers from the School Information Management System (SIMS) at the Ministry of Education and Higher Education (MEHE) in May 2021. 14 addresses the education for vulnerable communities and refugee children who constitute a large proportion of school-age minors and young adults in Lebanon. This R4R Volume 2 encompasses the following four studies: 1. School study: to understand the enabling context of schools in Lebanon (school finances, management, teachers, school environment, and facilities); 2. Student assessment study: to understand student learning outcomes and perceptions in Lebanese classrooms focusing on Grades 4 and 78 in mathematics (taught in Arabic, English, or French, depending on a given school’s language of instruction), Arabic (reading and writing) and foreign languages (French or English reading and writing); 3. Teacher performance study: understanding teachers’ classroom practices and content knowledge performance in Lebanese schools, mainly in Grades 4 and 7 classrooms in subjects similar to the student assessment study (i.e., mathematics taught in English or French, Arabic, and foreign languages—either French or English); and 4. Vulnerability study: to identify the risks and opportunities for improving educational experiences among vulnerable Lebanese and refugee children. 8 Grades 4 and 7 were selected for the R4R studies because they are the entry grades for Cycles 2 and 3, both of which are transitional grades within the Lebanese education system. Additionally, the corresponding age groups for these grades are often selected in international or other large-scale assessments. 15 1. OVERVIEW OF R4R STUDIES AND METHODOLOGY 9 The study leveraged at a nationally representative sample of 145 schools representing public schools operating the first and second shifts of the day respectively, free private schools, and fee-based private schools. The same sample was applied in all the studies described below except for the vulnerability study, for which the target group was different. Overview of Studies SCHOOL STUDY The school study, conducted with support from the RAND Corporation,10 and executed by MRO (Market Research Organization) that was commissioned on behalf of RAND, took place during the 2016–2017 and 2017–2018 academic years and focused on examining the contexts and processes of public and private schools in four key dimensions: (1) school-based management; (2) teachers; (3) school finance; and (4) school environment and facilities. The school study aimed to address two broad research questions: ƒ What variation is seen among public schools, fee-based private schools, and free private schools in the areas of finance, school management, teacher quality, views on teaching and learning, and the provision of a positive school learning and physical environment? ƒ At public schools that have adopted two shifts to educate Syrian refugees, how similar or different are the practices they implement (during the first shift, the second shift, or both) in terms of those four key dimensions? 9 The studies employed standard statistical tests (such as t-tests) for statistical inference. In addition, reliability tests were applied for the various studies. For example, mean differences in student assessment rounds were cross-validated by comparing their Information Resources and Technology (IRT)-based Rounds 1 and 2 mean ability estimates on a logit scale. For the teacher study, using the Classroom Assessment Scoring System (CLASS tool), a correlation analysis was undertaken that shows significant and high internal consistency of results. For details refer to the background reports. 10 RAND Corporation is an American nonprofit global policy think tank. 16 The sample included data from: (1) school principals at each sample school; (2) five teachers per school who taught either mathematics, languages, science, or social science; (3) three parents per school in a subsample of 55 schools; (4) the financial managers or individuals in charge of school finances at 30 schools (often the principal) who completed the finance questionnaire; and (5) administrators in charge of staffing and enrollment data who provided basic information about the school. Data were collected using face-to-face interviews with principals, parents, the individuals responsible for school finances, and administrators. In addition, there was a self-administered, but proctored, teacher questionnaire. Two trained enumerators spent one full day at each school collecting data on the school environment and facilities. All questionnaires were administered in a paper-and-pencil format. STUDENT ASSESSMENT STUDY The student assessment study, supported by Management Systems International (MSI) 11 and administered during the academic year 2016–2017, aimed to provide a better understanding of effective classroom practices (teaching and learning) and how they affect the quality of education. The study was administered to Grades 4 and 7 students as these grades were considered transitional grades in each of the primary and middle schools. In both Rounds 1 and 2, there were 5806 students from Grade 4 and 4335 students from Grade 7 participating in the assessments. The student assessment study aimed to address the following research questions: ƒ How much do students learn over the academic year (Rounds one and two of the R4R assessment? ƒ How do students’ end-of-year exam scores12 compare with their assessment scores? Each Grade 4 and 7 cohort underwent two rounds of assessment, at the beginning and end of the school year, respectively. These focused on the following classes: ƒ Arabic: reading and writing; ƒ Foreign language: reading and writing (English or French); and ƒ Mathematics: (taught in English, French, or Arabic). Each assessment comprised 30 multiple choice questions with four options. All assessments were aligned with grade-appropriate skill levels and learning outcomes as outlined in the national curriculum. In Round One, data collection took place in 145 schools13 during the 2016– 2017 academic year. In Round Two, data collection took place in 142 of the 145 schools14 that participated in Round One. The remaining three schools could not be covered in Round Two due to strong resistance on the part of the school directors. Round Two assessments were aligned with Round One assessments with respect to content and psychometric properties. If any differences in difficulty were observed in Round Two assessments as compared to Round One assessments, then a test score equating procedure was implemented to account for the 11 MSI is an international development firm. 12 Exams are not harmonized. In other words, exams for Grades 4 and 7 are prepared by the schools and cannot be compared between schools. 13 The Round One assessments were administered in 120 schools (public, private, and subsidized private schools; plus 25 second-shift schools) randomly selected by MEHE and the World Bank. 14 The Round Two assessments were administered in 117 schools of the 120 Round One schools. 17 differences. Within each school, researchers endeavored to administer Round Two tests to the same students who had been tested in Round One. However, some students tested in Round One could not be tested in Round Two due to their absence on the day of testing or because they had moved to another school. To make up for the loss of students in Round Two, researchers also tested new students who had not participated in Round One testing. The sample students’ end of grade exam scores were collected from the respective schools. Researchers developed a template incorporating the following data: school code, school name, shift, grade, section, student ID (created in Round One), and student name. School directors were asked to fill in the template with the students’ end-of-grade marks in Arabic, foreign language (English or French), and mathematics, and their respective maximum possible scores. After the data were collected, researchers transformed the marks into percentages so that the marks collected from different schools could be reported on the same percentage scale. Rounds One and Two Test Equivalence—to Measure Student Improvement during the Academic Year In order to estimate student improvement (trend over time from the beginning to the end of an academic year), it was necessary that student scores in Round Two be brought into the Round One score scale. The psychometric procedure for bringing Rounds One and Two scores onto the same scale is called “test equating.”15 A subset of items from Round One assessments (about 30 percent of the total) was included in the Round Two assessments, such that the item difficulty parameters established in Round One remained unchanged in Round Two. Rounds One and Two assessments were equivalent with respect to content and level of difficulty, allowing for a time trend analysis of scores. There were some discrete variations in difficulty level observed along the ability scale for some assessments. However, these variations in difficulty level were accounted for when transforming students’ Round Two scores into equivalent Round One scores through the test equating procedure. The student assessment study is closely linked to the teacher performance study and was conducted with the students who were observed in class. However, linkages in analysis were not made as it was not possible to link the individual teacher with their student assessment results (see data limitations). TEACHER PERFORMANCE STUDY The teacher performance study collected data on teacher knowledge and classroom practices to provide useful insights for teacher professional development policies. Studying and measuring the quality of teaching (subject knowledge and pedagogical practices) in Lebanese classrooms contribute to 15 A subset of items (about 30 percent of total number of items) from Round one instruments was included in the Round two instruments, and their item difficulty parameters estimated in Round one were kept fixed in Round two to estimate difficulty parameters of the remaining Round two items. Test characteristics curves (TCC) were created for both Rounds one and two to evaluate the equivalence of the instruments. 18 understanding the gaps in teaching, particularly in light of the continued challenges within the sector from past and recent crises. The teacher study took place in the 2016–2017 academic year, with schools being visited twice during the school year. Just like the student assessment study, the teacher study primarily focused on Grade 4 and Grade 7 classrooms in the following subjects : ƒ Arabic language ƒ Foreign language (either English or French depending on the school’s language of instruction) ƒ Mathematics: (either in English or French depending on the school’s language of instruction) Firstly, each teacher was evaluated on their content knowledge aligned to the curriculum level they taught. Secondly their classroom was observed live, using the Classroom Assessment Scoring System (CLASS), and lastly, each teacher was interviewed post classroom observation about their challenges as a teacher, their professional development needs, lesson planning, methods of student evaluation, and so forth. The teacher performance study aimed to address the following research questions: 1. How knowledgeable are the teachers in the content they teach? What is the performance of teachers (including preparation and activities both in and out of the classroom) across different types of schools? 2. What is the extent and quality of classroom practices available by teachers across different types of schools? 3. What is the support available to and used by teachers in the classrooms across different types of schools? Teacher study methodology: (I) Teacher content knowledge evaluation In addition to the live classroom observations, teachers who participated in the R4R study were evaluated on their general content knowledge aligned with the minimum knowledge of the curriculum level, subject, and grade they taught. The teacher content knowledge evaluation was designed under the assumption that understanding teachers’ instructional practices in the classroom will help MEHE design programs to establish goals that focus on effective learning, implement and promote reasoning and problem-solving skills among students, build procedural fluency from conceptual understanding, and elicit and support evidence of students’ critical thinking abilities. Therefore, all the teacher assessments were designed such that the teachers marked a student’s mock exam. The benefit of conducting the evaluation in the form of correcting a student test was to mainly understand whether or not the teachers were able to apply their knowledge in their teaching. Each evaluation test consisted of two sections: the first section of the assessment included 10 multiple choice questions (4 choices per each question). The teachers were asked to identify whether the student answers were (1) correct/incorrect, and (2) if the answer was incorrect, provide a correct answer. 19 FIGURE 1.1. TEACHER CONTENT EVALUATION (SECTIONS 1 AND 2 EXAMPLES) Grade 7 Math in English Example—Section 1 Read the text and then answer the questions Grade 4 English Example—Section 2 The second section of the teacher content knowledge assessment comprised three questions from a student’s mock test which had an incorrect answer. For each question, the teachers were asked to (1) provide the correct response, (2) provide a reason behind the student’s mistake or source of student’s confusion, and (3) provide a guiding suggestion (teaching remediation) that would help the student understand his/her mistake and learn the target leaning objective of that concept/question. The second and third follow-up questions about the source of student confusion and a guiding suggestion were presented as multiple-choice options to teachers. An example for each section of the assessment is provide in figure 1.1. 20 Teacher study methodology: (II) CLASS tool The purpose of live classroom observation was to analyze key elements of the learning environment that contribute to learning. These elements can best be assessed through direct observation (Goe, Biggers, and Croft 2012). CLASS,16 developed by Robert Pianta at the University of Virginia, assesses classroom settings on 12 components (dimensions), measuring three broad domains: emotional support, classroom organization, and instructional support. The main aim of CLASS observation is the potential to identify strengths and address specific weaknesses in teachers’ practices. At the upper elementary level, CLASS produces qualitative ratings of teacher performance on a scale from 1 to 7 on 12 dimensions across three broad domains: (1) emotional support; (2) classroom organization; and (3) instructional support (Bruns and Luque 2015) as shown in figure 1.2. FIGURE 1.2. CLASS DOMAINS Measures specific teacher’s behavior that promotes students’ academic achievement and engagement by fostering positive relationships and motivation. This domain also measures whether EMOTIONAL or not the teachers are being aware and responsive to students’ SUPPORT needs and allowing students to willingly take learning risks (Hu et al. 2016). The emotional support domain consists of positive climate, teacher sensitivity, and regards for student perspective dimensions. Describes teachers’ behaviors that establish and affect structure for learning through organization and management of students’ behavior, time, and attention in the classroom. Classrooms function CLASSROOM best and provide the most opportunities to learn when students are ORGANIZATION well-behaved, consistently have things to do, and are interested and engaged in learning tasks. The classroom organization domain consists of productivity, behavior management, and a positive or negative climate. Refers to specific teaching behaviors that push the students thinking to deeper understanding and more advanced performance skills. These behaviors help students build knowledge, promote INSTRUCTIONAL higher order thinking skills, expand students' learning through SUPPORT specific feedback, and use discussion to deepen students’ academic understanding. The instructional support domain consists of institutional learning formats, content understanding, analysis and inquiry, quality of feedback, and instructional dialogue. 16 The CLASS observation tool has been extended to classrooms worldwide. Initial evidence from research in international contexts suggests that the predictive utility of classroom interactions as captured by the CLASS extends across cultures (Cadima et al. 2010; Malmberg and Hagger 2009; Pakarinen et al. 2010). The focus of the CLASS instrument observers is on the process of learning rather than on “what” is being taught. 21 CLASS is a scientifically developed and largely applied classroom observation tool, both for its theoretical framework and for its accumulation of robust empirical evidence demonstrating connections between teachers’ quality of classroom behaviors and children’s developmental competencies (for example, social skills and learning behaviors), school readiness skills, and even long-term academic outcomes (Hamre et al. 2014; Pianta and Hamre 2009). The CLASS tool for upper elementary and secondary classrooms requires the observers to make standardized, structured judgments and assign a numeric score on a 1–7 Likert scale grouped in three ranges: a score of 1 or 2 (the low range); a score of 3, 4, or 5 (the mid-range); and a score of 6 or 7 (the high range). Scores of 1 or 2 indicate the quality of teacher-student interaction is low or there is a lack of interaction between teachers and students, the management of classroom is done poorly, and teaching is purely rote. Scores of 3, 4, or 5, are given when classrooms show a mixture of effective teacher-student interactions with periods when interaction are ineffective or absent. Scores of 6 or 7 mean that effective teacher-student interactions are consistently observed throughout the classroom observation period (Coflan et al. 2018). Low range Mid-range High range 1 2 3 4 5 6 7 The classroom observation (using the CLASS tool) focused on assessing the quality of instruction given by 707 teachers in a total of 145 schools from a nationally representative sample across Lebanon in Grades 4 and 7, including in public (first and second shifts), free private, and fee-based private schools.17 The counselors at the Department of Guidance and Counseling or Direction d’Orientation Pédagogique et Scolaire (DOPS) at the Ministry of Education and Higher Education implemented the study in 53 first-shift public schools and 25 second-shift public schools. Meanwhile, the contractor, Info Pro, conducted the data collection in 64 private schools. In total, there were 599 teachers (85 percent) in the first-shift schools, and 108 teachers (15 percent) in the second-shift schools. There were 375 teachers in public schools (53 percent), 71 teachers in free private schools (10 percent), and 261 teachers (37 percent) in fee-based private schools. In order to gain an in-depth understanding of a teacher’s instructional effectiveness, each classroom was visited twice, and during each visit two enumerators observed the lesson for two cycles, each consisting of 15-minute lessons. The inter-reliability rate for observers who observed Grade 4 teachers was 96%, and for Grade 7 teachers it was 98%. Enumerators then performed a post observation interview with all the teachers in the sample, asking a range of questions about their lesson plans, opportunities for professional development, students’ behavioral issues, the teachers’ own morale, and so forth. Teacher study methodology: (III) Post observation interview The R4R study used a post observation instrument and interviewed the teachers (whose classrooms were observed live) to better understand the teachers’ philosophy of teaching and the planning behind the delivered class. Results from post observation interviews— coupled with live classroom observation data (using CLASS) and teacher content knowledge evaluations—will help policy makers better identify options to improve Lebanon’s education system. The post observation instrument included basic information about the students, notes about the lesson plan, observations about teacher success or challenges, educational 17 Fee-based private schools are financed by students’ tuition fees. Free private schools are private schools that are privately owned but subsidized by the government, religious associations, charities, and other organizations. 22 resources, assessments, educational needs, teacher’s perspective on parent participation, teacher’s professional development need, and teachers’ methods of student evaluation and feedback using samples of student evaluation methods used by teachers during the school year (such as annual or quarterly exams, homework, etc.) (figure 1.3). FIGURE 1.3: POST OBSERVATION INSTRUMENT CONSTRUCT Teacher professional development information Educational Teachers' resources and challenges support P OS T OBS ERVATION Student evaluation Lesson plan techniques TEACHERS’ CHARACTERISTICS Out of the teachers whose classrooms were observed and interviewed, around 20 percent of them were male and 80 percent were female teachers, with one-third of the teachers in the sample (31 percent) having five years or less years of teaching experience. Table 1.1 shows the summary statistics for teachers’ characteristics in the sample. Around one-half of the teachers (49 percent) have a bachelor’s degree and 20 percent have a bachelor’s degree in education. About one-third (36 percent) of the teaching force in the sample specialize in subjects other than the subjects that their classrooms were observed for (i.e., Arabic, English, French, and math). This indicates that Grade 4 and Grade 7 teachers are specialized in a subject that is often not as relevant to the subject they teach. Only around 19 percent of the teachers have a teacher training diploma. 23 TABLE 1.1. SUMMARY DESCRIPTION OF TEACHERS’ CHARACTERISTICS Percent Gender Male 20 Female 80 Years of teaching experience <= 5 years 31.2 > 5 years and ≤10 years 21.2 > 10 years and ≤15 years 13.9 > 15 years and ≤20 years 13.3 > 20 years and ≤30 years 12.3 > 30 years 9.4 Education level Secondary school diploma 9.9 Secondary school diploma (with teacher training certificate) 5.9 Bachelor’s degree (university degree) 48.8 Bachelor’s degree (in education) 20.2 Post graduate diploma (DEA-DES) 4.7 Master’s degree 9.2 Master’s degree in education 1.0 Area of specialization Arabic 26.3 French 10.6 English 15.1 Math (in English and French) 15.3 Others 36.0 VULNERABILITY STUDY The vulnerability study, supported by the American University of Beirut, was conducted during the academic years of 2017–2018 and 2018–2019 and aimed to provide a better understanding of the reasons why vulnerable Lebanese and Syrian refugee children either remain in school or drop out of school. The study examined the educational experiences of 1,800 vulnerable children in Lebanon within different community types and across different ages and grade levels. It explored the risks, protective factors, and trends contributing to a persistence in enrollment among vulnerable Lebanese and Syrian students. It also examined the experiences of Lebanese and Syrian out-of-school children (OOSC), addressing their family and personal backgrounds, and the reasons that led them to drop out. Finally, the study identified best practices in classroom instruction, school environment, and home support in order to ensure that vulnerable children remain at school and are successful. 24 To learn about these issues, the study focused on five cohorts of vulnerable children: (1) Syrian (refugee) children enrolled in schools, for either the first-shift or second-shift schools; (2) Syrian (refugee) children enrolled in nonformal education programs; (3) Syrian (refugee) children who are out of school; (4) vulnerable Lebanese children enrolled in schools; and (5) vulnerable Lebanese children who do not attend school. The study adopted a mixed method approach using quantitative and qualitative tools. In addition to quantitative assessment tools, the study also included a background questionnaire on children’s characteristics (including academic achievement—as reported by schools) and family information. Qualitative tools included semi-structured interviews with children attending school, interviews with children not attending school, and focus group discussions with parents, teachers, and school counselors. Two rounds of data collection were conducted between May and November 2018 in eight governorates: Akkar, Baalbeck-Hermel, Beirut, Beqaa, Mount Lebanon, North Governorate, Nabatiyeh, and South Governorate. The design for sample selection was based on five stratifying variables: governorate, school shift (first or second), nationality (Syrian or Lebanese), grade (4 or 7), and gender. The first round of data collection identified the most marginalized student populations and causes exacerbating their vulnerability. The second round captured progress or decline as reported by students (mainly during Round One) and examined in greater depth the profile of the most marginalized group identified during Round One. Data synthesis challenges in triangulating results across studies The challenges discussed in this section highlight the challenges that arose in making cross- study comparisons, given inconsistency in data collection. Since different agencies were responsible for producing different data sets, the school IDs were coded differently, making it difficult to match schools when other school specific information was not available. In some cases, school IDs used in the various studies were linked to the Center for Educational Research and Development (CERD) school IDs; however, this was not the case for the vulnerability study. Therefore, it was not possible to evaluate or triangulate the results across all studies at the school level. While almost all studies could be linked at the school level, except for the vulnerability study, results at the individual level could not be linked, making a more granular analysis impossible. After the studies were completed, efforts were made to synthesize information—specifically drawing linkages—which required returning for clarification to various stakeholders, including researchers, field staff, and data collection agencies. While the final version of the data set is reflective of all studies, there remained two primary linking gaps that could not be addressed. First, the teacher observation study did not record any variables to uniquely identify teachers. Although data could be aggregate to the school level, it was not possible to analyze the data at the specific teacher level. Therefore, it was not possible to link individual students’ assessment scores to their individual teachers nor to the classroom observations. All relevant variables from the teacher observation study as well as the final class scores were averaged at the school level. Secondly, in the vulnerability study the school identifiers for the quantitative phase of the study could not be linked to the master data set (which included data from the other R4R studies). As a result, it was not possible to link the findings of the vulnerability study and the findings from the other studies at the school level. 25 2. KEY FINDINGS OF 18 THE R4R STUDIES Findings from the School Study CHARACTERISTICS OF PUBLIC AND PRIVATE SCHOOLS On average, private schools provided 13 to 17 percent more instructional days than public schools, which is equivalent to a total of six to seven hours of formal learning per day for 170 to 189 days per year. Private schools served a much larger population of Lebanese students than did first-shift and second- shift public schools.19 First-shift public schools had a higher proportion of teachers with a bachelor’s degree than other types of schools, and public schools relied on a higher proportion of teachers who were not permanent (either on contracts or seconded20 for temporary assignments) than fee-based private schools. On average, the proportion of permanent teachers in free private schools was at least twice that of teachers in public schools for mathematics, Arabic, and foreign languages. Second-shift public schools had the largest proportion of teachers without formal education. In public schools, permanent teachers taught fewer hours, on average, than permanent teachers at private schools. Furthermore, permanent teachers in public schools taught, on average, similar or fewer hours than teachers on contracts. A higher proportion of public schools reported teacher shortages in mathematics, Arabic, foreign languages, and sciences than did private schools. Teachers comprised a higher proportion of staff in public schools than private schools. On average about 80 percent of staff in first-shift and second-shift public schools were teachers, compared to 75 percent of staff in private schools. Public schools and free private schools lacked a nurse or special needs instructor. Roughly 75 percent of free private and fee-based schools owned their facilities, compared to only 50 percent of first-shift public schools and 64 percent of second-shift public schools. 18 For further information on the analysis of the R4R studies, please refer to the original reports. 19 First-shift public schools refer to the morning shifts in public schools, and second- shift public schools refer to the afternoon shifts in public schools. 20 Seconded teachers are teachers who receive reimbursement for their services from donors, school funds, and/or parents’ councils. They were hired in response to the Syrian crisis to teach Syrian students in Lebanese public schools. Seconded teachers work without annual contracts, and they do not have an ID number at the Ministry of Education and Higher Education (MEHE) that guarantees them any legal rights, such as an increase in hourly wages or health insurance. 26 SCHOOL MANAGEMENT Private school principals reported higher educational qualifications and more managerial experience than principals at the other types of schools. On average, principals at private schools had 15 years of experience as a principal, including an average tenure of 11 years in their current school. In comparison, principals in public schools had an average of 7 to 8 years of experience as a school principal. In addition to having more years of experience working as principals, private school principals had more formal education, on average, than public school principals. In first- shift and second-shift public schools, 14 percent and 12 percent of principals, respectively, had only a high school diploma, compared to 6 percent in free private schools. In fee-based private schools, 96 percent of principals had a bachelor’s degree and 43 percent had advanced degrees. Across all types of public and private schools, the majority of principals had prior experience as teachers. In public schools, all principals surveyed had teaching experience. The percentage of private school principals with such experience was comparatively lower, but still quite high: 69 percent of principals in free private schools and 79 percent in fee-based private schools had teaching experience. Private schools also reported having a larger administrative cadre than public schools (79 percent of private schools have more than 11–15 people on staff), including the school principal, assistant principal, a bursar, a psychologist, and a secretary. The only administrative position that was available in 96 percent of public schools was an assistant director (naazir). The administrative cadres in private schools also met more frequently than at the public schools and spent more time on administrative tasks and less on instructional tasks. Diverse stakeholders are involved in schools’ financial management. Over one-half of fee- based private schools had just one person managing the finances. The principal was more likely to be involved in school finances than any other individual or committee across all school types. However, the percentage of principals who reported being involved in making financial decisions was much lower in public first-shift schools (68 percent) and fee-based private schools (50 percent) than second-shift public schools (92 percent) and free private schools (88 percent). Therefore, second-shift public and free private school principals are quite engaged in managing the finances of the schools. Free private schools were least likely to have parent councils. Only 56 percent of free private schools were found to have parents councils, although all schools, irrespective of type, are required by MEHE to have parent councils. On the other hand, all public schools reported having parent councils, as did 92 percent of fee-based public schools. Parents councils in first-shift public schools met more frequently than those in second-shift public schools. In general, parent councils in public and fee-based private schools engaged in similar activities, such as facilitating parent communication and providing input on limited school matters. Parent councils in free private schools engaged in fewer activities. The percentage of time principals spent on various daily activities varied slightly across the different school types. Overall, principals tended to spend at least one-third of their time on administrative duties. They reported spending about one-sixth of their total time on each of the following activities: observing teachers, interacting with parents, addressing student discipline problems, and providing feedback to teachers. Almost all teachers agreed that their school principals were effective managers who respected them, welcomed feedback, and worked collaboratively with them. There were slight differences across school types on some measures of leadership by the principal: for 27 example, teachers in fee-based private schools were more likely to report that their principals ran efficient meetings, and teachers in both types of private schools were more likely than their public school counterparts to agree that the principal clearly articulated her or his vision for improving the school. Teachers rated their principals to be effective; however, principals reported a constraint imposed by insufficient parent interest in participating in school affairs. A substantial proportion of public school principals (28 percent of first-shift public and 36 percent of second-shift public principals) reported that insufficient parent involvement limited their effectiveness as leaders. SCHOOL INSPECTION AND TEACHER EVALUATION School inspection: Private schools were inspected far less frequently than public schools.21 Private schools received inspection visits from the Department of Private Education (60 percent of fee-based private schools reported not receiving any such visits), while public schools were monitored by both the Central Inspection Board and the Educational Inspectorate. The average number of inspection visits by the Educational Inspectorate was 13 for first-shift public schools and 19 for second-shift public schools, compared to two and three visits, respectively, from the Central Inspection Board. For private schools, the Department of Private Education made two visits on average to free private schools, and roughly one visit to fee-based schools. Almost all (96 percent) of public school principals reported that evaluating the content and quality of exams was one purpose of inspection visits. Most principals found feedback from these visits to be helpful (that is, when feedback was provided). Teacher evaluation: In most schools, principals reported that teachers were evaluated at least once a year. Different types of schools emphasized different factors in evaluating their teachers. Fee-based private schools were more likely to take feedback from principals, subject coordinators, and parents into account compared to other school types. Free private schools were the least likely to take the feedback of other teachers into account. All principals with underperforming teachers reported having taken action to address such issues. Public school principals were more likely to have read comments from the Department of Guidance and Counseling or Direction d’Orientation Pédagogique et Scolaire (DOPS) coaches at MEHE or to have reported a teacher to MEHE (this only applies to public schools). Principals from free private schools were more likely to have given a teacher a verbal notification or to have fired a teacher. Principals from fee-based private schools were more likely to have given a teacher a written warning notification or deducted pay from a teacher’s salary. In all school types other than free private schools, about 40–50 percent of principals reported having underperforming teachers in the 2016–2017 school year. In free private schools, only 13 percent of principals reported having underperforming teachers. Classroom observation: Method and frequency of classroom observation of teachers varied across the different types of schools. The majority of teachers across all school types reported they were observed in their classroom at least twice during the school year 2016–2017. Most teachers reported receiving feedback after being observed. In most types of schools, the principal was the most likely individual to have observed a teacher’s classroom at least once and to have also visited teachers’ classrooms more frequently than any other 21 It is important to note that it is within the responsibility of the Department of Private Education at MEHE to visit private schools. This department visits all free private schools annually and fee-based private schools on a needs basis. 28 school personnel. Across all types of schools, the assistant principal was the least likely to provide feedback. Teachers overwhelmingly reported that the feedback they received was helpful. Public schools had a higher proportion of teachers receiving feedback on content and quality of tests. Fewer teachers in first-shift public schools, however, reported receiving feedback on teaching practices than did teachers in fee-based private schools and second- shift public schools. Public school teachers reported receiving the least amount of feedback on proficiency of language of instruction compared to private school teachers. PROFESSIONAL DEVELOPMENT (PRINCIPAL AND TEACHER) Professional development for principals and teachers: More than 50 percent of the principals (ranging from 52 percent of principals in second-shift public schools to 63 percent of principals in free private schools) reported that they did not receive professional development in 2016–2017. Those who did receive training reported typically one to five days of training. Most training days were provided by the Center for Educational Research and Development (CERD) for both public and private school principals.22 Similarly, a substantial proportion of teachers did not receive training in 2016–2017. About one-half of the fee-based private school teachers, and over one-third of free private school and first-shift public school teachers reported not receiving training. However, teachers were more likely than principals to have received at least one day of professional development training throughout the school year. A much higher proportion of public school teachers reported receiving training from CERD than private school teachers. Private school teachers also received training internally at their schools. Topics addressed in training varied somewhat by school type, with a larger proportion of private school teachers reporting receiving training on educational technology, ways to integrate low performing students into their classes, and classroom management, compared to public school teachers where training focused mainly on active teaching methods and evaluations. Teacher training was perceived as generally helpful, as it enabled teachers to adopt new practices (a range of 78–85 percent of the teachers reported having changed their practices on the basis of training received). Private school teachers were far more likely than their public school counterparts to report receiving training on educational technology (more than 60 percent in private schools compared to 36 percent of public school teachers) and on integrating low performing students into their classes and otherwise addressing their needs (56 percent and 42 percent in free and fee-based private schools, respectively, compared to 27 percent of teachers in second-shift public schools and only 9 percent in first-shift public schools). Similarly, about one-half of fee-based private school teachers reported that training addressed classroom management, compared to (at most) one-third of public school teachers. On the other hand, a larger proportion of free private school teachers reported learning about student assessment during the training, compared to about one-fourth of the teachers (or less) in the other school types. About 50 percent of private school teachers reported collaborating with other teachers between one to three times a week, and in some cases as often as every day, while most public school teachers reported collaborating a couple of times a year, or at best twice a month, with about 10 percent of teachers in each type of public school reporting never collaborating with other teachers at all. 22 It is within the mandate of CERD to train teachers and principals from public and private schools, although those from private schools do not usually attend these trainings but rather often conduct their own trainings. 29 TEACHER WORKLOAD AND STUDENT SUPPORT Teachers, on average, reported spending about 33 percent of their time teaching. On average, teachers in public schools spent 31 to 32 percent of their day teaching, compared to a little over 35 percent in private schools (36 to 37 percent of their day). The percentage of time teachers spent on other activities, such as planning lessons and marking exams, was similar across all school types. Public school teachers were more likely to have underperforming students and a lower percentage of gifted students in their classes compared to private school teachers. In public schools, about 39 percent of teachers in first-shift public schools and 42 percent in second- shift public schools reported that at least one-third of their classes comprised low academic achievers, compared to 4 percent of fee-based private school teachers and 10 percent of teachers in free private schools. Therefore, public schools seem to have more underperforming students than fee-based private schools. The strategies teachers used to deal with underperforming students were similar across all school types, although public school teachers were less likely to provide after-class support to low performers, and instead referred them to be tested for learning disabilities. Despite employing these various strategies, at most 25 percent of teachers at each type of school felt that they were “quite successful” in improving the performance of low academic achievers. About 25 percent of teachers in first-shift public schools reported their efforts as successful in dealing with special needs students, compared to 14 percent and 16 percent in second-shift public schools and fee-based private schools, respectively. PARENTS AND COMMUNITY ENGAGEMENT About two-thirds of parents whose children were enrolled in public schools had not completed high school compared to less than one-third of private school parents. About 40 percent of private school parents reported holding a bachelor’s degree, while the remaining parents reported having completed high school or a vocational or technical school. Almost all parents reported receiving information about their children’s academic performance, behavior, activities, and the performance of their school.23 Ninety percent of parents with children enrolled in first-shift public schools indicated that they received information on how schools used parent funds, compared to 77 percent of parents in second- shift public schools. About 80 percent of parents with children in first-shift public schools reported receiving information on school policies, compared to at least 90 percent of parents with children in other school types. Overall, parents agreed that their children’s schools have been responsive to their opinion or complaints, and that the schools initiate discussion of their children’s academic progress and behavior. Furthermore, almost all parents reported receiving a variety of information from the schools. A slightly smaller proportion of parents of children in public schools reported receiving information, particularly regarding the use of parents’ funds and school policies. According to principals’ reports, about 75 percent or more of the schools collaborate with community organizations to provide presentations or workshops to the school, classes, or parents. Over 80 percent of private school principals and about 75 percent of second- 23 School performance in the context of the study refers to school management, school engagement, financial decision-making, and improvement of school environment and facilities. 30 shift public school principals indicated working together with community organizations to organize academic or artistic competitions for students. A much lower proportion reported working with community organizations to host career fairs or raise money to support the schools and students. SCHOOL ENVIRONMENT AND EXTRACURRICULAR ACTIVITIES A school’s environment and facilities are important in how they make teachers and students feel about the school. In general, a higher share of private schools had the assistance of adults to manage traffic and pedestrians, designated drop-off zones, and adequate parking. A higher share of public schools had both exterior and interior facilities in need of repair compared to private schools. While all schools had functioning electricity during the enumerators’ visits, a significantly higher share of private schools had an electric generator or service contract as a backup, than did the other school types. Both students and staff lavatories in private schools were rated better maintained and cleaner than those in public schools. Around 70 percent of staff lavatories in private schools were very clean, compared with 50 percent of those in free private schools and fewer than 50 percent in public schools. Vandalism was also observed at some participating schools. About 33 percent of second- shift public schools had signs of vandalism to their interior, compared to about 20 percent of first-shift public schools and 15 percent or less of private schools. Most schools had railings on stairs, and one-half or more of schools had a functioning public announcement system in case of emergencies. Regarding availability of specialty rooms, a significantly greater share of private schools had an indoor gym and a multipurpose room. A greater share of public schools had health resources such as a clinic and a nurse on campus, while private schools were more likely to contract out health services. All parents reported that their children’s school provided extracurricular activities. The most common activities across all school types were school plays or musicals, and field trips. Parents with children in second-shift public schools reported fewer opportunities for their children to volunteer or to participate in scholarly competitions, compared to parents with children in other types of schools. Parents viewed physical fighting, student conflict, students not respecting staff and other students, and teachers harming students to be greater problems in public schools and free private schools than in fee-based private schools. SCHOOL FINANCE While the MEHE pays teachers’ salaries and large capital expenditures from its budget, public schools have a discretion over two types of funds, the school fund and the parent’s fund. They receive these directly, enabling them to procure certain services. For every public school, the MEHE contributes to a school fund for each student enrolled. Parents are also expected to contribute with a small fee for each student; however, this requirement has been relaxed more recently due to economic hardship. Public schools that enroll Syrian students and other refugees receive additional funds from donors, particularly for running second shifts in the afternoon. Donor funding has also supported the financing of the parents’ fund for all children enrolled in public schools (Grades 1 to 9) regardless of their nationality as part of the RACE 2 program. Private schools draw their resources from student tuition fees, although some schools (free private schools) receive a government subsidy for enrolling low-income students. Fee-based private schools also receive indirect financing from the 31 government through education subsidies given to civil servants, who then enroll their children at these schools. Private schools are typically run by a religious association or independently, and governance and budgeting vary depending on that arrangement. Among the public schools in the study sample, all the second-shift public schools reported revenues in 2015–2016 in line with their expectations, as did a majority of first-shift public schools. Similarly, all free private schools reported that revenues were in line with their expectations, compared to only two out of four fee-based private schools. In general, both public and private schools reported that they were able to track and spend their funds. This was less the case among free private schools where only one school of the four sampled indicated not facing any difficulty spending their funds. Among all the schools in the sample, second-shift public schools reported that the funds they received were sufficient to meet their needs. Only five of the eleven first-shift public schools in the study’s sample indicated that the school fund was sufficient to meet their needs, and seven of eleven indicated that the parent fund was sufficient to meet their needs. Among private schools, two of five free private schools and two of four fee- paying private schools reported that the funds were sufficient to meet their needs. Therefore, school financing seemed to be insufficient among many schools within the sample. CHALLENGES SCHOOLS FACE Principals across all school types cited the top two challenges as lack of support staff (such as psychologist or social worker) and inadequate information and communication infrastructure (including computers and connectivity). The third challenge most frequently identified by principals was inadequate school facilities (for first-shift public schools), inadequate student basic skill preparation (for second-shift public schools), large numbers of students in classes (at free private schools), and difficulty in managing student discipline (at fee-based private schools). The top challenges identified by teachers, which focused more on student preparedness and parents, differed the challenges identified by principals. There was consensus among teachers across various schools on the three top challenges: insufficient parent participation or support, student misbehavior, and inadequate student preparation in basic skills. 32 Findings from the Student Assessment Study Students in the participating classes had to take a test (Round One assessment) at the beginning of the school year and another (Round Two assessment) at the end of the school year to determine overall student progress in three subjects: Arabic, a foreign language (French or English), and mathematics, and a Student Perception Survey was completed by each student during the Round 2 student assessment. In Round One, a baseline assessment was administered at the beginning of the school year which assessed students on knowledge from the previous grade as well as content covered thus far in the grade they were in, and in Round Two, an assessment was administered at the end of the school year to determine overall student progress at that grade level. The assessment consisted of 30 questions of equal weight. One of the intended ways in which this study sought to measure student learning growth by grade level and subject area was by analyzing the differences in scores between Round One and Round Two assessments. According to the Management Systems International (MSI), Round Two assessments were made equivalent to Round One assessments with respect to content and psychometric properties in order to ensure comparability of scores. Consistent across both Grades 4 and 7, as well as across school types, students’ growth was found to be positive overall. These differences in scores remain of statistical significance. One of the intended ways to measure student learning was by making a comparison of the Research for Results (R4R) assessment scores with end-of-year scores, however, the scope of reliance on end-of-year scores is limited given that these were not nationally administered but rather designed and administered at the individual school level, thus making validity inconsistent. For Grade 4 students, the average score percentage in Round One was 35.68, whereas in Round Two it rose to 41.18. This is a substantially low average regardless of the type of school. Students from fee-based private schools showed the greatest improvement (seven points) despite having also started from the strongest position. Free private schools show the least progress, although students from these schools started off better than those in public schools, with students from public second-shift schools performing the lowest in Round One. While score differences are positive across most categories, there seems to be a fall in mathematics scores (tested in Arabic) among students from first-shift public schools and free private schools: down 1.6 and 2.6 points, respectively (figure 2.1). FIGURE 2.1. GRADE 4 SCORE DIFFERENCE BY SCHOOL TYPE 14 12 11.5 10 8 7.9 7.2 7.5 6.8 6.8 6.8 6 6.3 6.6 6.0 6.7 5.7 5.7 5.4 5.6 4.8 4.8 4.8 5.1 4.9 4 4.4 1.9 2 1.4 0.5 0.7 0.3 0 -2 -1.6 -2.6 -4 Math in English Math in French English French Arabic Math in Arabic Total First-Shift Public Schools Second-Shift Public Schools Free Private Schools Fee-based Private Schools Source: Author calculations. 33 For Grade 7 students, the average score in Round One was 42.64, whereas in Round Two it rose to 45.06. Students from second-shift public schools showed the greatest improvement, 4.8 points on average. Students from these schools also started off lowest compared to other school types. There seemed to be a fall in French language assessment scores for students from first-shift public schools and free private schools: down by 1.4 and 4.3 points, respectively. Students’ scores from fee-based private schools dropped by 2.0 points in mathematics when tested in French (figure 2.2). FIGURE 2.2. GRADE 7 SCORE DIFFERENCE BY SCHOOL TYPE 10 9.0 8 6 6.5 5.7 5.5 4.9 4 3.0 3.0 2 2.3 2.4 2.2 2.1 2.1 0.5 1.5 0.6 1.5 0.6 1.7 1.1 0 -0.7 -2.0 -2 -1.4 -4 -4.3 -6 Math in English Math in French English French Arabic Total First-Shift Public Schools Second-Shift Public Schools Free Private Schools Fee-based Private Schools Source: Author calculations. UNDERPERFORMING STUDENTS In order to understand students’ performance better, including any likelihood that underperforming students will improve their performance over time, inter-quintile mobility was tracked. For this purpose, students were clustered into five quintiles, where the first quintile consisted of the lowest performing students in Round One, and the fifth quintile consisted of the highest performing students in Round One. When assessing inter-quintile mobility for aggregate scores of students, results showed that low performing students were 50 percent more likely to continue to perform the worst in their cohort across different time periods (in this case rounds). Moreover, only 2.13 percent of those in the lowest quintile in Round One managed to rise to the highest quintile in Round Two. Underperforming students mostly do not catch up. Grade 4 students who fell in the first quintile, the lowest performing group of students in Round One, remained clustered around low scores in Round Two. This confirms that poorly performing students will continue to underperform, with the slimmest of chances that they might climb to membership in the highest performing quintile group. 34 Consistent with Grade 4 findings, a small share of students are seen to rise up to the highest performing group in Grade 7, with a majority continuing to remain in the first quintile (figure 2.3). This clearly implies that upward learning mobility remains a challenge among the underperforming students, and programs designed to focus on their needs are key for equitable learning growth. FIGURE 2.3. UPWARD LEARNING MOBILITY AS A CHALLENGE Tracking Low Performing Students in Grade 4 From Round 1 to Round 2 30 1 2 25 3 4 20 5 Round 1 scores 15 10 5 0 0 20 40 60 80 100 Round 2 scores Source: Author calculations. Almost one-half of the students in the lowest performing quintile in Round One (roughly 43 percent of the 565 students) were from the first-shift public schools, 24 percent were from the second-shift public schools, 25 percent were from fee-based private schools, and only 8 percent were from the free private schools. Among the lowest performing students in Grade 7 (421 students), roughly 60 percent were from the first-shift public schools, 15 percent were from second-shift public schools, 23 percent were from fee-based private schools, and less than 1 percent of the students were from free private schools. WELL PERFORMING STUDENTS The highest performing students in Grade 7 attended fee-based private schools. In the fifth quintile, which consists of students who perform best in Round One, roughly 71.00 percent of the students were from fee-based private schools, roughly 16.00 percent were from first- shift public schools, 12.00 percent from free private schools, and only 0.79 percent were from second-shift public schools. A similar pattern is seen in Round Two, where roughly 74 percent of the students were from private fee-based private schools and only 1.28 percent were from second-shift public schools. Among Grade 4 students, nearly 62 percent of the highest performing students attended fee- 35 based private schools, 17 percent attended first-shift public schools, nearly 16 percent attended free private schools, and fractionally more than 5 percent were from second-shift public schools. The same trend is seen in Round Two scores, where nearly 67.00 percent of students were from fee-based private schools whereas only 4.66 percent were from second-shift public schools. Therefore, growth was observed between Rounds 1 and 2 across all school types. However, the equity gap in learning among school types is clearly evident in that private schools begin much better off than public schools and then show much greater progress over time. It is important to note that positive growth of school averages in absolute terms must not be construed to show progress relative to other school providers. Furthermore, results among students in public schools continue to remain low compared to their counterparts in private schools. The overall distribution of scores across school types is presented in figures 2.4 and 2.5. As is evident, the distribution for public school scores across both grades is significantly lower than that of private schools. The distribution of round average scores in private schools is also more dispersed than public schools where the highest density of students fall below 50 percent. FIGURE 2.4: GRADE 7 ROUNDS AVERAGE SCORE DISTRIBUTION First-shift public schools Second-shift public schools .08 .06 .04 .02 0 Density Free private schools Free-based private schools .08 .06 .04 .02 0 20 40 60 80 20 40 60 80 Source: Author calculations. 36 FIGURE 2.5: GRADE 4 ROUNDS AVERAGE SCORE DISTRIBUTION First-shift public schools Second-shift public schools .06 .04 .02 0 Density Free private schools Free-based private schools .06 .04 .02 0 0 50 100 0 50 100 Source: Author calculations. DISAGGREGATING RESULTS BY CATEGORIES 1. SCHOOL PROVIDER Overall, the results from this study show that private schools perform better than public schools. Scores from both rounds of assessments were averaged out to analyze the overall performance of students across time. An average of Round One and Round Two scores for Grade 4 indicate that second-shift public schools perform the worst, while fee-based private schools perform the best. This trend continues for schools across grade 7 as well. Grade 4 When comparing the two types of public schools based on shifts, schools with the first shift perform slightly better on Round One and Round Two averages of the student assessment than schools with the second shift that primarily host non-Lebanese students (first-shift: 34 percent; second-shift: 32.8 percent). This trend is reversed when comparing end-of-year student exam scores,24 with a second-shift public school average of 53.2 percent compared to 51.5 percent in first-shift public schools (figure 2.6). 24 Please note the limitation in using the end-of-year exam score as a comparison, as it is not a harmonized measure across school types. 37 FIGURE 2.6. GRADE 4 TOTAL SCORE BY SCHOOL TYPE 70 68.3 60 63.5 50 53.2 51.5 40 44.4 41.2 30 34.8 32.8 20 10 0 First-Shift Second-Shift Free Private Fee-based Public Schools Public Schools Schools Private Schools R4R Assessment School End of Year Exam Source: Author calculations. Grade 7 When comparing schools’ average scores across providers in Grade 7, fee-based private schools and free private schools perform almost identically relative to public schools. The end-of-year average for Grade 7 in free private schools was as high as 64 percent, which is about 3 percent higher than at fee-based private schools. This difference is significant at the 99 percent confidence interval along with differences among most other provider types. The difference in the assessment is significant at the 90 percent confidence interval across the two private providers (figure 2.7). FIGURE 2.7: GRADE 7 TOTAL SCORE BY SCHOOL TYPE 70 60 64.0 60.9 50 49.3 49.4 40 45.9 47.1 39.7 30 36.0 20 10 0 First-Shift Second-Shift Free Private Fee-based Public Schools Public Schools Schools Private Schools R4R Assessment School End of Year Exam Source: Author calculations. Similar to Grade 4 results, first-shift public schools were found to outperform students in second-shift public schools on the assessments administered. The reverse is true for end- of-year exam scores, where second-shift school students have a higher school average of 47.1 percent (figure 2.7). 38 2. GENDER In both Grades 4 and 7, girls outperform boys on average at both the assessments as well as the end-of-year exam across all types of schools. These score differences are statistically significant at the 99 percent confidence interval. These results are promising in view of the efforts by the Government of Lebanon, as well as the World Bank, to bridge the gender divide in learning (figures 2.8 and 2.9). These results are in line with achievement trends from the official exam results for Grades 9 and 12 for the years 2016–2017, 2017–2018, and 2018–2019, where the official exams showed that results (grades) were significantly better for females. FIGURE 2.8. GRADE 4 TOTAL SCORE BY GENDER AND SCHOOL TYPE 70 60 70.7 65.9 65.9 50 61.2 55.5 54.3 40 50.8 49.1 45.3 43.5 42.9 30 39.5 36.0 33.9 33.8 31.7 20 10 0 Male Female Male Female Male Female Male Female First-Shift Public Schools Second-Shift Public Schools Free Private Schools Fee-based Private Schools R4R Assessment School End of Year Exam Source: Author calculations. FIGURE 2.9. GRADE 7 TOTAL SCORE BY GENDER AND SCHOOL TYPE 70 60 65.5 64.9 62.1 57.4 50 51.9 51.2 49.2 48.5 47.0 47.2 40 44.8 43.1 40.1 39.3 36.9 37.0 30 20 10 0 Male Female Male Female Male Female Male Female First-Shift Public Schools Second-Shift Public Schools Free Private Schools Fee-based Private Schools R4R Assessment School End of Year Exam Source: Author calculations. 39 3. GOVERNORATE For Grade 4 students, the South Governorate is the highest performing (figure 2.10). The differences across governorates are statistically significant throughout. FIGURE 2.10. GRADE 4 SCORES BY GOVERNORATE South 64.1 44.0 North 54.4 32.2 Nabatieh 61.1 39.7 Mount Lebanon 60.4 41.5 Beqaa 57.1 36.4 Beirut 55.6 37.9 0 10 20 30 40 50 60 70 School End of Year Exam R4R Assessment Source: Author calculations. For Grade 7 students, Mount Lebanon is the best performing governorate overall, with differences across governorates remaining statistically significant (figure 2.11). FIGURE 2.11. GRADE 7 SCORES BY GOVERNORATE South 58.4 45.5 North 48.1 39.9 Nabatieh 51.7 41.8 Mount Lebanon 58.2 47.9 Beqaa 50.0 42.3 Beirut 51.0 44.2 0 10 20 30 40 50 60 70 School End of Year Exam R4R Assessment Source: Author calculations. 40 Findings from the Teacher Study TEACHERS’ CONTENT KNOWLEDGE EVALUATION The content knowledge evaluation results indicate that regardless of the grade and subject, teachers in Lebanon did not sufficiently master the content they teach. Teacher content knowledge evaluation is often linked to student outcome, as teachers cannot effectively teach a lesson to their students when they do not have sufficient knowledge. Teachers with better knowledge of the subject they teach offer a higher level of value added instruction to help students learn topics from a more in-depth perspective. FIGURE 2.12. AVERAGE SCORE STATISTICS FOR TEACHER CONTENT KNOWLEDGE EVALUATION—SECTION 1 Arabic, Grade 4 47% Arabic, Grade 7 60% Math in French, Grade 4 63% Math in English, Grade 4 63% English, Grade 4 75% Math in English, Grade 7 78% French, Grade 7 78% French, Grade 4 78% Math in French, Grade 7 79% English, Grade 7 80% 0 10 20 30 40 50 60 70 80 90 100 Percent of correct answers Source: Author calculations. Figure 2.12 shows the average score for the first section questions, which included mostly short answers. The outcome was below what is expected of teachers across all subjects and grades. For example, a low percentage of Grade 4 Arabic teachers (average of 47 percent correct answers) got correct answers, indicating that on average each teacher was only able to provide correct answers to around one-half of the questions in the evaluation test. With the exception of Grade 7 Arabic teachers, who scored 60 percent on average, the rest of the teachers in Grade 7, who taught English, French, and math, scored an average of 80 percent in their assessment for the first section of the evaluation. 41 For the second part of the teacher content evaluation, as shown in figure 2.13, the teachers were (1) asked to provide a correct answer to an incorrect response from a student test; (2) justify the reason behind the student mistake or confusion; and (3) provide a teaching remediation to help the student understand the underlying concept. Compared to the first section of content evaluation, fewer teachers were able to follow a coherent instructional process to provide correct answers to all parts of the questions in section 2. Less than 33 percent of the teachers were able to answer the follow-up questions correctly when asked to provide a justification of the source of student confusion and a guiding solution and/or a remedy to alter the way they teach. FIGURE 2.13. AVERAGE SCORE STATISTICS FOR TEACHER CONTENT KNOWLEDGE EVALUATION—SECTION 2 French, Grade 4 59% English, Grade 7 44.4% French, Grade 7 42.4% Math in French, Grade 4 31.9% Arabic, Grade 4 31.3% English, Grade 4 27.2% Arabic, Grade 7 26.5% Math in English, Grade 4 26.4% Math in French, Grade 7 23.4% Math in English, Grade 7 14.2% 0 10 20 30 40 50 60 70 80 90 100 Percent of correct answers Source: Author calculations. 42 TEACHER CONTENT KNOWLEDGE EVALUATION—BY SCHOOL SHIFT The teacher content knowledge evaluation results indicate a significant difference between first-shift and second-shift teachers regardless of the subject and grade taught. Second- shift public school teachers performed significantly lower on their evaluation tests than did first-shift teachers in most of the subjects. As an example, figure 2.14 indicates that Grade 7 French second-shift teachers scored significantly lower in 70 percent of their assessments compared to teachers who taught French in first-shift schools. FIGURE 2.14. TEACHER EVALUATION SCORE IN GRADE 7 FRENCH LANGUAGE—BY SHIFT 100.0 100.0% 100.0% 97.8% 90.0 86.7% 86.7% 86.7% 86.7% 80.0 77.8% 77.8% 70.0 Percent of assessments 68.9% 68.9% 66.7% 66.7% 66.7% 66.7% 60.0 60% 50.0 53.3% 40.0 44.4% 30.0 33.3% 33.3% 33.3% 33.3% 33.3% 33.3% 31.1% 20.0 10.0 0.0% 0.0 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 French, Grade 7, first shift French, Grade 7, second shift Source: Author calculations. TEACHER EVALUATION—BY SCHOOL TYPE The content knowledge evaluation analyses showed no statistically significant difference in evaluation scores of teachers in private and public schools across all grades and subjects. It is important to indicate that private school teachers (both fee-based and free private) performed better in their evaluation scores as compared to public school teachers; however this difference was not observed to be significant. CLASSROOM OBSERVATION OUTCOMES (USING CLASS TOOL) Similar to content knowledge evaluation findings, teachers in Lebanon follow steady and fixed teaching practices in their classrooms regardless of the subject and grade they teach, when observed live (using the Classroom Assessment Scoring System [CLASS] tool). While some students periodically shared their ideas or gave responses, generally most Grade 4 43 and Grade 7 students in Lebanon appeared to be in a receptive rather than an active mode. Teachers rarely followed students’ lead in the classroom and did not welcome students’ ideas and opinions. The low-performing teachers (around 73 percent of the teachers observed) only offered information with little to no variety in delivery strategies or materials (figure 2.15). Around 26 percent of the teachers scored medium-high according to the CLASS scoring system discussed earlier, and even those teachers were not able to deliver teaching practices that meet the higher range of CLASS instructional support domain standards, instead falling into the low to mid range of this domain. During live classroom observation, even the best and worst performing teachers seem to fail to deliver teaching practices that meet a high range of instructional support domain standards. They both fall within the low to mid range in this domain. Only a small number of teachers (fewer than 4 percent) were seen to maximize student engagement through clear presentation of key ideas reflected on by students. These teachers also focused on encouraging a deep understanding of content through the provision of meaningful, interactive discussions and an explanation of broad organizing ideas that were consistently linked to students’ prior knowledge in ways that clarified misunderstandings. In those select classrooms, students were consistently engaged in extended opportunities to use higher order thinking and independently solve or reason through an open-ended task, requiring them to select, utilize, or apply existing knowledge and skills. Unfortunately, these teachers still remain the exception rather than the norm in Lebanon. FIGURE 2.15. CLASS DOMAIN ANALYSIS SCORE, LEBANON Grade 4 Instructional support Classroom organization Emotional support 0 10 20 30 40 50 60 70 80 90 100 LOW 1 2 MODERATE 3 4 5 HIGH 6 7 Grade 7 Instructional support Classroom organization Emotional support 0 10 20 30 40 50 60 70 80 90 100 LOW 1 2 MODERATE 3 4 5 HIGH 6 7 Source: Author calculations. 44 CLASSROOM OBSERVATION—BY SUBJECT Teachers appeared inflexible, rarely following students’ leads or encouraging expression of their ideas and opinions. The majority of teachers (94 percent) rarely connected students’ practical experiences to the lesson. Class materials were not presented in such a way that they communicated relevance to students, nor did they help them understand the value or connection of the lesson to their current life experiences. In classroom settings, there were only limited opportunities for students to independently practice the skills relevant to lesson content. Further analyses of live classroom observation across subjects and grades indicated that teachers in Lebanon follow similar pedagogical practices (figure 2.16). Regardless of the subject taught in the classroom, teachers in both Grade 4 and Grade 7 were found to provide students with opportunities to apply knowledge and skills within familiar contexts; however, they did not provide them with opportunities for analyses and problem solving within novel contexts and/or without teacher support. FIGURE 2.16. CLASS SCORE—SUBJECT DOMAIN ANALYSIS, LEBANON Grade 4 7 6 5 5.50 5.46 5.37 5.37 5.41 4 3.89 3.86 3.9 3 3.71 3.7 3.28 3.01 3.01 2.89 2 2.9 1 0 Emotional support Classroom organization Instructional support Arabic language (n= 130) French language (n= 60) English language (n= 69) Math in French (n= 60) Math in English (n= 70) Grade 7 7 6 5.68 5.66 5 5.52 5.53 5.51 4 3.88 3.91 3.75 3.73 3.65 3 3.37 3.09 3.13 2.94 2.95 2 1 0 Emotional support Classroom organization Instructional support Arabic language (n= 107) French language (n= 49) English language (n= 56) Math in French (n= 49) Math in English (n= 57) Source: Author calculations. 45 CLASSROOM OBSERVATION—BY SHIFT There is a significant difference in the quality of teaching practices between first-shift and second-shift public school teachers, specifically in the quality of teacher feedback to students. Lack of student independence was mostly observed among teachers of Grades 4 and 7 in second-shift schools, indicating that students were not encouraged to think independently, evaluate, reflect on their own learning, or plan their own learning experiences. For instance, Grade 4 first-shift public school teachers were observed to mildly engage students in facilitating lesson content, whereas the vast majority of Grade 4 second-shift public school teachers (92 percent) were observed to communicate learning targets in a disorganized manner and only offered lesson contents in a single mode (for example, a lecture). Figure 2.17 indicates that only 10 percent of second-shift public school teachers scored medium-high, as compared to 30 percent of first-shift public school teachers. FIGURE 2.17. CLASS SCORE DISTRIBUTION IN LEBANON BY SHIFT Second-shift 29% 61% 10% First-shift 14% 55% 30% 1% 0 10 20 30 40 50 60 70 80 90 100 LOW 1 2 MODERATE 3 4 5 HIGH 6 7 Source: Author calculations. Furthermore, both Grades 4 and 7 second-shift public school teachers performed significantly lower than teachers at other schools on all dimensions of the instructional support domain discussed earlier in the CLASS tool description (figure 2.18). FIGURE 2.18. INSTRUCTIONAL SUPPORT DISTRIBUTION IN LEBANON BY SHIFT Grade 4 Instructional dialogue 2.28 3.04 Quality of feedback 2.48 3.33 Analysis and inquiry 1.84 2.39 Content understanding 2.77 3.56 Instructional learning format 3.25 3.82 0 1 2 3 4 5 6 7 Second-shift First-shift Grade 7 Instructional dialogue 2.36 3.07 Quality of feedback 2.61 3.31 2.09 2.42 Analysis and inquiry 3.03 3.64 Content understanding 3.4 Instructional learning format 3.78 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Second-shift First-shift Source: Author calculations. 46 CLASSROOM OBSERVATION—BY SCHOOL TYPE Teachers at fee-based and free private schools provided students with occasional opportunities to practice metacognition through reflection, self-evaluation, and planning. The quality of teaching at free private and fee-based private schools is similar. Even though teachers in fee-based private schools were observed to perform slightly better in the instructional support domain compared to their counterparts in free private schools, this difference was not observed to be statistically significant. While only 14 percent of public school teachers scored medium-high (a score of 5), 38 percent of teachers who taught at free private schools, and 43 percent of fee-based private school teachers scored medium- high (figure 2.19). Around one-third (27 percent) of public school teachers scored medium (between the ranges of 3 and 4 as described in CLASS scoring system earlier), while only 3 percent of fee-based private school teachers and 6 percent of free private school teachers scored medium. FIGURE 2.19. CLASS SCORE DISTRIBUTION IN LEBANON BY SCHOOL TYPE Fee-based private 3% 52% 43% 1% Free-private 6% 55% 38% public 1% 27% 58% 14% 0 10 20 30 40 50 60 70 80 90 100 LOW 1 2 MODERATE 3 4 5 HIGH 6 7 Overall, teachers in Lebanon follow fixed teaching practices in their classrooms throughout the school year. No evidence of significant improvement in quality of teaching outcomes was observed in the second visit to classrooms (the second visit was five to six months after the first visit). During both visits, students often showed passive rather than dynamic engagement in the classroom. Additionally, a majority of the teachers (90 percent) across Grades 4 and 7 in all subjects (languages and mathematics) did not exhibit a significant difference in the CLASS score compared to their content knowledge evaluation score, meaning that teachers in Lebanon, regardless of their level of knowledge and mastery of the curriculum, adopt similar instructional practices in classrooms. Most teachers (nearly 80 percent) did not encourage students to think, evaluate, or reflect on their own learning. Instruction was mostly presented in a rote manner with limited opportunities for students to engage in open-ended tasks, and the majority of surveyed students (nearly 90 percent) are not encouraged by their teachers to continuously apply previous knowledge and skills to new contexts during a lesson. 47 TEACHER POST OBSERVATION INTERVIEW FINDINGS TEACHERS’ PERCEPTION OF THE LEARNING ENVIRONMENT AND STIMULATION Classrooms appeared to offer a suitable environment in most schools (85 percent), although there were some schools without proper heating or electricity. Special Education Needs. Even though most teachers (81 percent) reported employing different instructional learning formats that adjust the pace of lessons to students with special needs, these descriptions were somewhat undermined by the low performance of the teachers in live classroom observations. Parents’ Support. Most public school teachers (85 percent) expressed their dissatisfaction with the degree to which parents were involved in their children’s education, such as frequency of participation—if any—and in school-related events and activities during a school year. The majority of the parents (94 percent) whose children attended second-shift schools were not involved at all in their children’s academic and social achievements. TEACHERS’ PERCEPTION OF CHALLENGES, SUCCESS, AND PROFESSIONAL DEVELOPMENT Educational Resources and Curriculum. The majority of observed classrooms (90 percent) used traditional resources such as textbooks or learning aids, and teachers did not have access to educational resources that employed information and communication technology. Second- shift teachers reported that schools did not provide them with the access to computers, tablets, or liquid crystal display projectors enjoyed by their colleagues who taught in first- shift schools. Learning Resources. Around 75 percent of the teachers expressed concern regarding students’ previous learning achievements or levels, which do not seem to be sufficient to give them the foundational skills needed to advance to higher grades. Most teachers (80 percent) expressed concern regarding access to learning resources (such as lab equipment, tablets, computers, projectors, etc.) needed to support their teaching. Teachers also stated that professional development training opportunities for creating stimulating classroom environments were insufficient. Most public school teachers (75 percent) requested training in the use of technology in the classroom, followed by support and training in the use of formative learning and assessment tools to enhance student learning, such as collaborative student projects, new methods in exams and quizzes, standardized tests, and so forth. There was a visible mismatch between teachers’ perception of the professional development support versus their performance in content evaluation and live classroom observations. Only 37 percent of the teachers expressed a need in professional development training that included in-depth understanding of the curriculum they taught. Based on the content knowledge evaluation findings, this need appears to be more alarming. 48 LESSON PLANS Preparation. On average, a typical teacher in Lebanon spends roughly 7 hours planning his/ her lessons and roughly 20 hours teaching during the week, which indicates that on average, for every hour of teaching, only 20 minutes is spent on planning and preparing a lesson. Additionally, on the day of the interview, around one-fifth of the teachers did not have an observable lesson plan.25 Content and Clarity. More than three-quarters (75 percent) of the observable plans were clear, organized, and easy to follow. However, more than one-half of the lesson plans did not include a review activity of what was taught in the previous lessons. The majority of the plans (76 percent) did not take into consideration that students were of different skill levels and learning abilities. Interactive methods in the lesson plans were not well structured. Less than 50 percent of the lesson plans included setting a homework assignment based on that lesson. Around 94 percent of lesson plans did not include the use of technology, particularly at second-shift public schools. Additionally, none of the lesson plans indicated a special activity/exercise for children with special needs who have learning difficulties/disabilities, nor was it observed in the lesson plans that the pace of the lesson would be adjusted for those individual students. Utilization. Most teachers (86 percent) did not utilize and implement their lesson plans in their classrooms. For instance, while the majority of teachers (94 percent) reported using discussions between the teacher and the students as an interactive method in their classrooms, only 43 percent of them included this activity in their lesson plans. TEACHERS’ PERCEPTION OF STUDENT ASSESSMENT Teachers (65 percent) did not utilize formative learning and assessment tools (such as student collaborative projects, poster presentation, or student self-evaluation techniques) that can foster students’ cognitive and critical thinking skills. Teachers also did not provide effective feedback and comments to increase student understanding of the lesson or curriculum. This result was also reflected in teacher content knowledge evaluation performance, where teachers were unable to identify students’ sources of mistakes and provide an appropriate teaching remediation. The majority of teachers (86 percent) of both grades four and seven failed to provide practical and critical thinking, and challenging questions for students in both languages and mathematics. Additionally, most teachers (close to 80 percent) did not provide additional assistance or hints in order to direct students to correct their mistakes. The majority of language teachers (86 percent) used short-answer, fill in the blank, and essay writing assessment questions in exams, homework assignments, and school projects, while mathematics teachers (83 percent) rarely utilized problem-solving questions requiring an analysis of graphs, charts, and tables that could stretch students’ critical thinking skills. 25 Observable lesson plans refer to the lesson plans prepared by the teacher and observed by the researchers after the classroom observation. 49 Findings from the Vulnerability Study The study survey was administered to over 1,800 Grade 4 and Grade 7 students26 in public schools or nongovernmental organizations (NGOs) across the eight governorates.27, 28 The study presented the analysis of specific factors associated with the learning experiences of vulnerable Lebanese children and refugee children in Lebanon. It further analyzed the factors that contributed to the dropout or persistence of these vulnerable children in schools in the country. The findings of the study provided a better understanding of the inequities in the Lebanese education system, particularly highlighting inequities due to age, gender, and socioeconomic status of vulnerable children. WHY DID SOME CHILDREN DROP OUT OF SCHOOL: PROTECTIVE AND RISK FACTORS IDENTIFIED IN STUDENTS’ SURVEYS The main risk factors associated with children struggling at school and potentially dropping out were age, gender, academic performance, behavioral problems, and bullying. In contrast, the school environment was the main protective factor that lessened children’s chances of dropping out. Poor academic performance was often associated with grade repetition and eventual dropout, which increased children’s vulnerability. The main factors correlated with academic performance were gender, age, and (public school) shift. Females showed higher academic achievement than males. Academic achievement was lower for older children, highlighting the importance of early intervention to support children who struggle at school. First-shift public school students showed higher academic achievement levels than those in the second-shift public schools. 26 It is important to note that the vulnerability study included a representative sample of Grades 4 and 7 students across eight different governorates in Lebanon. However, these students were not the same students targeted in Grades 4 and 7 in the other R4R studies. 27 The study targeted children from five cohorts of vulnerable children: (1) Syrian (displaced) children enrolled in the Lebanese school system (morning or afternoon shifts); (2) Syrian (displaced) children enrolled in nonformal education programs (usually at NGOs); (3) Syrian (displaced) children who are out of school (children who are not enrolled in any form of formal schooling, children who are enrolled only in nonformal schooling, or children who have dropped out of school [for more than 3 years] and are currently not enrolled in any form of schooling); (4) vulnerable Lebanese children enrolled in schools; and (5) Lebanese children who are out of school . 28 NGOs included in the study were organizations that provided services to refugees, among which were education services in the form of nonformal education. 50 The survey findings also showed that many vulnerable children were facing challenges of resilience and emotional distress, often suffering from behavioral and emotional disorders that can hamper their development as well as academic success. Children reporting severe social, emotional, and resilience challenges were at a higher risk of failing and dropping out of school. Children who suffered from behavioral and emotional disorders had negative perceptions of their school environment, which correlated with poor academic performance. Among all surveyed students, 63.5 percent reported having experienced bullying in school. Boys were more involved in bullying or being bullied than girls. Bullying among children varied according to gender, the role they played—whether they were being bullied or they were doing the bullying—their age, and the type of school they were enrolled in. From a social and emotional standpoint, children who were involved in bullying behaviors exhibited more hyperactivity, emotional symptoms, conduct problems, and peer problems. Children who reported not being involved in any type of bullying had better and more positive social-emotional attributes and skills. Also, the more positively children perceived their school environment, the less likely they were to be involved in bullying. Verbal bullying (40.19 percent), including spreading rumors (36.66 percent), was reported to be among the most common type of victimization used, followed by being bullied about one’s religion or sect (27.47 percent), and being rejected from a group (27.17 percent). In contrast to these behaviors, being bullied physically, threatened or forced to do things, and having money and other things taken away or damaged were found to be less prevalent. Additionally, interviewed children who had dropped out of school all reported experiencing violence at or on the way to the school. Many of the out-of-school children (OOSC) reported being bullied when they were at school and peers, school staff (including teachers), and other OOSCs inflicted the violence. Interviewees shared that the school did not address any of the bullying that took place at school, even in cases where violence and bullying were reported. They also mentioned violence between school shifts and among children of the same or different nationalities.29 In general, both Syrian and Lebanese children at a young age reported a positive perception of their school environment in both the first-shift and second-shift public schools. They felt safe and secure and reported being treated fairly by their teachers and classmates. Students who perceived their school positively tended to have lower levels of behavioral and emotional disorders and higher academic achievement. Female children had a more positive perception of their school environment than did males. They also received more social support than males from their teachers, peers, and friends in the form of emotional support, advice, encouragement, and companionship. Children reported diminishing satisfaction with their school environment as they grew older. They reported receiving lower levels of social support from teachers, peers, and friends; had fewer meaningful relationships with classmates; and experienced more dissatisfaction with teachers’ help in academic matters. A positive perception of the school environment was also associated with children, who reported having positive social and emotional attributes and skills. Violence was highlighted among the key issues within the school environment. Poverty was also a key characteristic (regardless of nationality) among most of the students participating in the study, particularly those who had dropped out of school. It was one of the main factors impacting the enrollment of children in formal education (and in nonformal educational institutions) as families struggled to provide some of the basic needs for their children. Many dropouts who were interviewed indicated that they were obliged to work in 29 Violence between school shifts refers to the time when children from the first-shift public school (morning shift) are leaving and children from the second-shift public school (afternoon) are entering. This is observed at public schools that run two shifts. 51 order to help their families withstand the pressure of the rising costs of living. Boys were at a higher risk of dropping out than girls. Almost all the male children who had repeated one or more grades worked in the summer—or through the entire academic year—to support their families. Poorer children were less likely to receive academic support at home—from parents or private tutors—when tackling schoolwork or struggling with it. Lack of diagnosis and support for those with learning difficulties30 and class repetition were other factors that put vulnerable children at risk of dropping out of school. Almost one-half of the students participating in the study suffered from learning difficulties. The majority of children who repeated a grade or dropped out were struggling in core learning areas: reading, writing (especially spelling), or mathematics in their native language (Arabic). However, none of these students had received any diagnostic assessment to identify their learning difficulties. Schools were not equipped to take on children with special educational needs because of the school’s inadequate infrastructure, or for lack of qualified teachers trained to handle the relevant situation. Data collected through the fieldwork showed that the lack of services for children with special education needs was a significant factor behind school dropout numbers. Furthermore, special education services were simply not available at public schools. Learning science and mathematics in a foreign language was associated with the likelihood of children struggling at school. Participating Lebanese and Syrian children reported that they had difficulties in learning foreign languages, whether English or French, and in some cases in learning Arabic. All these children said that if they had received academic support from the school or elsewhere, they would not have failed or repeated grades. Schools did not seem to have the administrative flexibility or resources to provide the academic support needed. WHY SOME VULNERABLE CHILDREN REMAINED AT SCHOOL: FACTORS LEADING TO RETENTION AT SCHOOL AND REDUCING VULNERABILITY Interviews with children from disadvantaged backgrounds who persisted at school, despite having some of the abovementioned risk factors, point to factors that support continued schooling: Higher aspirations and foreseeable opportunities. The majority of vulnerable children participating in the study that remained in school reported aspirations to at least finish Grade 7, so as to progress to vocational education or enroll in the army later.31 The families of these children also had higher aspirations and expectations and did not want them to drop out of school despite difficult circumstances. As for Syrians who persisted in school, they had hopes of going back to Syria and were among those refugees with better family and neighborhood circumstances. 30 Learning disabilities and learning difficulties are educational terms used in the UK and US. These two terms are often interchangeable when used in the context of health and social care for adults. Some people with learning disabilities prefer the term learning difficulties. In this report, we use the term “learning difficulties” (used in the UK) to describe difficulties in reading, writing, spelling, or mathematics. 31 Vulnerable Lebanese children, mainly males, aspired to join the army after completing middle school. 52 Integration into the first-shift public or private subsidized school. Almost all interviewed Syrian children who were succeeding in school were enrolled in first-shift public schools. These children had enrolled in these schools since Cycle 132 (beginning in Grade 1). If asked to move to the second-shift public schools, their families had chosen either to move them to another first-shift public school or enroll them in a private low-cost school, which reflected the negative perception of second-shift public schools. Children who were enrolled at private subsidized schools (run by a charity) made much better progress in school than those at second-shift public schools. Children described the environment of these schools as “safe” and the quality of learning as “good.” Joining an accelerated learning program. For refugee students who came to Lebanon and did not speak a foreign language, joining an intensive learning program allowed them to bridge the language gap and hence resume their normal academic course of study. A few of the interviewed children reported that joining such programs had helped them re-enroll in school. Academic support. The study found evidence of lack of academic support, particularly at the schools. Academic support outside of school was reported to be provided either by siblings at home, or by NGOs at which the children were enrolled, particularly among Syrian children. Additional academic support for children became even more scarce for older children, even among the Lebanese. However, if and when academic support was provided, Syrian children in particular reported that it had a significant effect, helping them to succeed and continue their education. NGOs provided a variety of educational programs such as literacy, remedial education, accelerated learning programs, intensive language support, homework clubs, and extracurricular activities. Extracurricular activities. Many of the children who were thriving at school had participated in extracurricular activities run either by the school or by NGOs. These programs were either academic (remedial) or psychosocial. 32 Cycle 1 includes Grades 1 to 3. 53 3. POLICY RECOMMENDATIONS Evidence from the reports provide a wealth of information on the current status of Lebanon’s bifurcated education system and across key education indicators. This information can be used to inform ongoing education reform measures as well as new sector planning. This next section uses the evidence from the R4R studies to propose a few, but not exhaustive, key policy options. The evidence presented in this report highlights similarities and discrepancies among the types of schools. For instance, in the teacher study, teachers’ weak performance in content knowledge evaluation coupled with the low quality of teaching practices were observed in both public and private schools. The discrepancies present areas of strengths among the schools as good practices and areas that may be further developed such as the need to increase the number of instructional days in public schools, or to ensure that all private schools have parents’ councils. These specific issues are identified in the discussion presented earlier. System Reform Recommendations Improve public school administration. Public school administration requires restructuring in order to ensure that more experienced school principals with higher academic qualifications are in place. It is vital that school principals receive increased and systematic professional development training opportunities. In order to further improve public school administration, it is also important to strengthen school-based management by elevating the role of the principal to a strategic decision-maker and agent of school change, and supporting a more decentralized approach to school management. Update the curriculum and improve teaching practices in order to improve students’ learning and align skills with the Lebanese economy’s needs. Curriculum reform to refocus the system on skills and learning has not been undertaken in the country since 1997. Most teachers surveyed in the teacher performance study reported that they were unable to cover all curricular content over the course of the school year, nor did the curriculum allow for knowledge and skills development beyond the textbooks. This was also confirmed by Lebanon’s poor Programme for International Student Assessment (PISA) and Trends in International Mathematics and Sciences Study (TIMSS) results. Student-centered teaching methods were not commonly observed in Lebanese classrooms, nor was active learning or collaboration among students. It is critical to focus on learning and conveying skills, and that will 54 necessitate revision of the curriculum, learning materials, and pedagogy. It is vital that efforts are made to ensure that teachers facilitate higher order thinking in their classrooms rather than introduce activities based on rote learning and memorization. If these efforts are built around a carefully revised curriculum, they are likely to improve the quality of instruction and student learning outcomes for all, and close the attainment gap between students. Furthermore, it is important to introduce social and emotional learning (SEL) programs at the basic level within the curriculum. Researchers have studied the impact of SEL programs on student learning.33 The inclusion of SEL programs within the Lebanese curriculum could yield important benefits, especially considering the psychological and emotional challenges that the large populations of vulnerable children in Lebanon are facing. Recommendations for Teacher Professional Development Reform Address the teacher’s specialization gap by allocating teachers who specialize in the subject they teach and consider a new hiring strategy which is more subject centered. Teachers who lack specialization in the subject they are teaching often do not master adequate content knowledge of the subject and therefore cannot effectively transfer that knowledge to students. This also includes pedagogical practices required for specific subjects. There is a pressing need to improve all teachers’ content knowledge and pedagogical skills, particularly public schools second shift teachers and Arabic content knowledge. Improve professional development opportunities for teachers with constant follow- up. The teacher performance study revealed that teachers could benefit from ongoing professional development and coaching opportunities. During post observation interviews, teachers asked to be provided with professional development courses that replicate their classroom environments. There is an overarching need to improve teaching practices through coaching, professional development opportunities with ongoing follow-up, individually tailored support to teachers that targets their specific skill level, improvement in their lesson planning development, and so forth. It may be difficult for teachers to apply techniques learned during training to their classrooms if these techniques cannot be adapted to their classroom environment, particularly in the absence of required educational materials. Teaching can be strengthened to better integrate and coordinate various functions to enhance teacher training with ongoing support. Unfortunately, teachers rarely received training on educational technology or how to integrate underperforming students into their classrooms and respond to their needs, despite the fact that public school teachers reportedly had the highest proportion of low performing students. Therefore, impactful improvements in student learning resulting from appropriate teacher training may in part be driven by a high degree of overlap with other interventions. Many of the successful instructional interventions are shown to be coupled with teacher training in how to employ new instructional methods in the classroom (McEwan 2015). The curriculum reform can also be considered as a stepping stone to include most updated pedagogical practice methods, as well as designing programs to improve teachers’ mastery of curriculum content. 33 A large meta-analysis (Durlak et al. 2011) measured the impact of 213 school-based, universal SEL programs involving more than 270,000 students from kindergarten all the way to high school. Compared to the controls, SEL participants demonstrated significantly improved social and emotional skills, attitudes, and behaviors, which were reflected in an 11 percent gain in achievement. 55 Improve quality of teaching and learning across all types of schools and shifts at public schools. The studies clearly showed that there was a difference between the quality of teaching at first-shift public schools and the quality at second-shift public schools. Teachers expressed their need to be provided with appropriate resources in their classrooms (e.g., lab equipment, books, digital resources, etc.). Policy options and interventions should focus on significant improvement in the quality of teaching practices and on methods to achieve a more equitable distribution of teaching and learning resources among the various school types. Additionally, it is critical that appropriate and professional development of teachers is provided in order to support underperforming students. Encourage collaboration among teachers. About one-half of private school teachers reported collaborating with other teachers within the same school between one to three times a week, and sometimes as often as almost every day, while most of the public school teachers reported collaborating between twice a year to twice per month at best. Such collaboration can create a teacher learning community that encourages peer mentoring and coaching among teachers within the school. Long-term peer coaching and in- classroom training programs (including peer observations) could be initiated to help teachers write and share lesson plans that can be effectively used in their classrooms. Recommendations for Reform of the School Environment Create and enable safe and nurturing learning environments for students. The vulnerability study found that school violence is present in many schools despite published child protection policies in Lebanon. Improving the learning environment is critical for improving academic achievement and supporting the development of social and emotional skills, as children with a positive perception of their school tended to show superior academic attainment. As a result, there is a pressing need to introduce anti-bullying intervention programs that tackle violence toward children. A reporting system should be introduced whereby teachers are held liable for any violations, while ensuring that parents and children feel safe to report such violations. Finally, key school personnel should be trained on preventing and responding to such violations. Improve physical and information technology infrastructure as well as other school facilities across school types . This entails making instructional materials available to teachers, particularly computers, smartboards, and so forth. The recent transition to distance learning due to the effect of the global pandemic presents a pressing need and an opportunity to invest in technology in education. Furthermore, it is critical for schools to ensure connectivity to facilitate the use of technology during teaching. Many teachers in public schools reported not having access to essential learning and instructional materials. It is equally important to ensure that teachers are trained on the effective use of this equipment and materials for instructional purposes. 56 Recommendations for Student-Centered Support Encourage upward learning mobility. This remains a challenge among the underperforming students who mostly do not catch up, and this challenge is further observed among older children who seem more likely to repeat a grade or drop out. Programs designed to focus on their needs are key for equitable learning growth. Furthermore, early intervention programs are critical to help identify difficulties among children for the design of remedial courses and academic support programs. Relationships between parents, schools, teachers, and the community could be strengthened, and a school culture toward promoting parental involvement and shared responsibility to improve learning should be created, rather than holding schools solely accountable. For instance, schools could provide families with information related to creating a supportive learning environment at home and establish effective school-to- home and home-to-school communications. Provide support and transition programs, particularly for children with special education needs, low performing students, and students from low socioeconomic backgrounds. More than 84 percent of surveyed students in the vulnerability study reported that teachers did not help them when they needed help. In view of the financial constraints faced by families today, particularly vulnerable families whose children are enrolled in public schools, extra academic support must be made widely available for those children who need it. As for the children with learning difficulties, it is essential to introduce professional academic support services for children with special education needs. It is equally important to provide teachers with the necessary training on addressing these children’s needs. Teachers must focus on underperforming students to avoid the risk that they will drop out of education, as the results show these students continue to perform poorly across survey rounds. 57 REFERENCES Abdul-Hamid, Husein, and Mohamed Yassine. 2020. Political Economy of Education in Lebanon: Research for Results Program. International Development in Focus.World Bank: Washington, DC. Bruns, Barbara, and Javier Luque. 2015. Great Teachers: How to Raise Student Learning in Latin America and the Caribbean. World Bank: Washington DC. Cadima J., Leal T., Burchinal M. 2010. “The quality of teacher-student interactions: Associations with first graders’ academic and behavioral outcomes.” Journal of School Psychology 48: 457–482. CERD (Lebanon, Center for Educational Research and Development). 2018. TIMSS 2015—Grade 8 National Report. CRDP: Lebanon, Beirut. CERD. (Lebanon, Center for Educational Research and Development). 2020. Statistical Bulletin 2019–2020. CRDP: Lebanon, Beirut. Coflan, Andrew Moore, Andy Ragatz, Amer Hasan, and Yilin Pan. 2018. “Understanding effective teaching practices in Chinese classrooms: evidence from a pilot study of primary and junior secondary schools in Guangdong, China.” World Bank Policy Research Working Paper No. 8396. Washington, DC. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., and Schellinger, K. 2011. “The impact of enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions.” Child Development, 82, 405–432. El Deeb, Sarah. 2021. In crisis-struck Lebanon, school year is gripped by chaos. AP News. Gajderowicz, Tomasz, and Maciej Jakubowski. Forthcoming. “PISA In-Depth Analysis of Results for Jordan and Lebanon (draft).” Washington, DC: World Bank. Goe, Laura, Kietha Biggers, and Andrew Croft. 2012. Linking Teacher Evaluation to Professional Development: Focusing on Improving Teaching and Learning. Policy Brief, National Comprehensive Center for Teacher Quality, Washington, DC. Hamre, Bridget, Bridget Hatfield, Robert Pianta, and Faiza Jamil. 2014. “Evidence for general and domain-specific elements of teacher–child interactions: associations with preschool children’s development.” Child Development 85(3): 1257–1274. Hu, Bi Ying, Xitao Fan, Chuanhua Gu, and Ning Yang. 2016. “Applicability of the Classroom Assessment Scoring System in Chinese preschools based on psychometric evidence.” Early Education and Development 27, no. 5 pp. 714- 734. 58 Malmberg, L.-E., & Hagger, H. 2009. “Changes in student teachers’ agency beliefs during a teacher education year, and relationships with observed classroom quality, and day-to-day experiences.” British Journal of Educational Psychology 79(4): 677–694. McEwan, Patrick. 2015. “Improving Learning in Primary Schools of Developing Countries: A Meta-Analysis of Randomized Experiments.” Review of Educational Research, 85, 353–394. 10.3102/0034654314553127. Pakarinen, E., Lerkkanen, M., Poikkeus, A., Kiuru, N., Siekkinen, M., Rasku- Puttonen, H., & Nurmi, J. 2010. “A validation of the Classroom Assessment Scoring System in Finnish kindergartens.” Early Education & Development 21: 95–124. Pianta, Robert C., and Bridget Hamre. 2009. “Conceptualization, measurement, and improvement of classroom processes: standardized observation can leverage capacity.” Educational Researcher 38(2): 109–119. UIS (UNESCO Institute for Statistics). 2018. Data for Sustainable Development Goals. UNESCO, Paris. UNESCO (United Nations Educational, Scientific, and Cultural Organization). 2018. Global Education monitoring report, 2019: Migration, displacement and education: building bridges, not walls. UNESCO, Paris. UNHCR (United Nations High Commissioner for Refugees). 2019. Vulnerability Assessment of Syrian Refugees in Lebanon. UNHCR, Beirut. World Bank. 2017. Lebanon Education Public Expenditure Review 2017. World Bank: Washington, DC. World Bank. 2018. Expectations and Aspirations: A New Framework for Education in the Middle East and North Africa: Overview. World Bank: Washington DC. World Bank. 2019. Lebanon PISA 2018. World Bank: Washington, DC.