Teaching at the Right Level Experience from Nepal Karthika Radhakrishnan, Uttam Sharma, and Seema Gupta Acknowledgements The report was prepared by Karthika Radhakrishnan, Uttam Sharma, and Seema Gupta. We thank Nepal’s Ministry of Education, Science and Technology, local governments, and community schools for close collaboration, Street Child for development of high-quality assessment tools and implementation, and Pratham for strong technical assistance. Special thanks to Sunil Poudel and Prativa Yadav who played a critical role in data collection. 2 Abstract From October 2021 to March 2022, the World Bank, in collaboration with the Ministry of Education, Science and Technology (MoEST), led an in-school pilot to implement Teaching at the Right Level (TaRL). The World Bank implemented the pilot in partnership with Street Child Nepal and Aasaman Nepal, with technical support from Pratham Foundation. In Nepal, TaRL was piloted in three local governments (LGs) —two rural and one urban—across three provinces. The pilot included a total of 2,163 students in Grades 4 –5 from 64 public schools in three LGs. To identify the learning levels of students, a baseline assessment was conducted using the TaRL tools before teaching began. The main outcome measures were children’s performance in the Nepali language and math tests. Children were grouped according to their learning levels and periodic assessments were conducted on a biweekly basis to measure the change of learning levels. Drawing from the assessment results collected in the Nepal’s TaRL baseline and endline surveys, this report presents the improvement made in literacy and mathematical skills of children who participated in the pilot. Baseline assessment results are compared with endline assessment results to assess the improvements made by children in their learning levels. The study provided strong evidence for the effectiveness of the TaRL model in raising basic reading and mathematics skills for primary school children. 3 Executive Summary Overview The World Bank, in collaboration with the Ministry of Education, Science and Technology (MoEST), led an in-school pilot literacy program called Teaching at the Right Level (TaRL), which ran from October 2021 to March 2022, and aimed at improving foundation skills in reading and mathematics for Grade 4 and Grade 5 children. At baseline, 52 percent children were unable to read basic paragraphs and comprehend stories, 66.26 percent children were not able to solve basic subtraction problems. After the intervention took place, the learning outcome of the students improved significantly. Around 80.61 percent children were able to read at least a paragraph in Nepali language, and 79.8 percent children were able to solve at least subtraction problems in math. Overall, the study provided strong evidence for the effectiveness of the TaRL model in raising basic reading and mathematics skills for primary school children. If this model is scaled up, it can have massive impacts on students’ learnings. Teaching at the Right Level (TaRL) TaRL is an evidence-based educational approach, developed by an Indian nongovernmental organization (NGO) Pratham, that helps school-going children at the primary level develop basic language and mathematics skills. The TaRL model assesses children’s learning levels using simple one-on-one tests. Children are then grouped based on their learning level, rather than their grade or age. In this model, regardless of the age or grade, teaching takes place at the level of the child. Teachers are trained to use interactive techniques to teach to the level of each group, rather than classroom-style teaching. As children’s learning proficiency improves, they can be moved to more advanced groups that match their skills. In March 2020, Nepal announced school closures in response to the COVID-19 pandemic. Since then, school closures varied at the local government (LG) level and there was a pattern of schools opening and closing intermittently between December 2020 and February 2022. Learning was expected to fall further as a result of COVID-19 shutdowns. A recent study shows that nearly 35 percent of Grade 5 students could 4 not do two-digit addition with carryover.1 For Nepal, the Learning Adjusted Years of Schooling (LAYS) was 7.2 years before the pandemic.2 Even conservative estimates suggest that LAYS will likely drop to 6.3 years—a 0.9 year drop.3 Given the learning losses caused by COVID-19-related school closures and economic shocks, Nepal has to move quickly to support learning recovery. One reliable approach to learning recovery is to implement the TaRL intervention in classrooms to fast-track learning recovery. TaRL Pilot in Nepal In Nepal, TaRL was piloted in three LGs—two rural and one urban—across three provinces. The pilot included a total of 2,163 students in Grades 4–5 from all 64 public schools in three LGs. To identify the learning levels of students, a baseline assessment was conducted using the TaRL tools before teaching began. The main outcome measures were children’s performance in the Nepali language and math tests. Children were grouped according to their learning levels and periodic assessments (baseline, midline, and endline) were conducted to measure the change of learning levels. In each of these assessments, children are marked at the highest level of proficiency that they can comfortably attain. Re-grouping of children was done on a continuous basis based on teachers' informal observations of the child’s progress. Before TaRL teaching began, only 6.8 percent children could solve division problems or more and 26.94 percent could at most solve a subtraction problem, and a large proportion of children could recognize at the most two-digit numbers (37.41 percent) or one-digit numbers (23.96 percent), or could not recognize any number at all (4.89 percent). This means that at baseline, 66.26 percent children were not able to solve subtraction problems and 93.2 percent were not able to solve division problems. At the same time, nearly 70 percent of children were not able to read a simple Grade-2 level story. After the intervention took place, the learning outcome of the students in terms of reading and math improved significantly. The proportion of children reading a story without making more than three mistakes has doubled in the endline from 29.16 percent to 60.48 percent. For math, at baseline only 33.74 percent of children were able to solve at least 1 Radhakrishnan, Karthika, Noam Angrist, Peter Bergman, Claire Cullen, Moitshepi Matsheng, Anusha Ramakrishnan, Shwetlena Sabarwal, and Uttam Sharma. 2021. Learning in the Time of COVID-19 : Insights from Nepal. World Bank, Washington, DC. 2 World Bank. 2018. “The Human Capital Project.” World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/30498 License: CC BY 3.0 IGO.” 3 Sharma, U., M. Sherpa, and K. Radhakrishnan. 2021. “Learning Loss as a Result of COVID 19 in Nepal.” Unpublished Manuscript. World Bank. 5 subtraction, while at endline 79.7 percent children could solve these questions, with highest jump seen for children who are able to at least solve division problems now (from 6.8 percent to 40.64 percent). The larger improvement at the highest proficiency for both reading and math is seen for Grade 5 children as compared to Grade 4. Not much significant difference is seen between improvements in proficiency levels of males and females indicating that TaRL provides a level playing field for both genders. Leraning outcomes at the beginning and end Learning outcomes at the beginning and end of pilot (Reading) of pilot (Math) 6.8 29.16 26.94 40.64 60.48 18.86 14.61 37.41 39.12 20.13 29.07 13.24 23.96 15.94 4.89 3.86 0.44 BASELINE (%) ENDLINE (%) BASELINE (%) ENDLINE (%) Beginner Reading Letter Reading Word Beginner One digit Two Digit Subtraction Division Since the LGs come under different jurisdictions, the total number of weeks of teaching children varied across LGs due to school closures as a result of COVID-19 Omicron variant. Results suggested that having higher attendance in school is associated with larger improvements in proficiency for both math and reading. Comparison of LG level learning outcomes show that Dhading, that rolled TaRL for the maximum number of days, is the best performing LG in terms of improvement in learning outcomes. Dhading showed 34 percentage points improvement in the proportion of children that could read Nepali stories or more as compared to 30.75 percentage points for Dhanusha and 29.33 percentage point increase for Salyan. Similarly, for proficiency in math Dhading showed close to 40 percentage point jump in the proportion of children that could solve division problems at endline as compared to 33.71 and 27.67 percentage point increase for Dhanusha and Salyan, respectively. Even though Dhanusha and Salyan had huge scope for increasing the proportion of children in high categories of proficiency, fewer days offered under TaRL could have resulted in limited increase in learning for these LGs. 6 Percentage increase in reading proficiency of Percentage increase in math proficiency of the the highest level (reading story) highest level (division) 80 60 60 34 29.33 40 31.32 40 30.75 33.71 39.9 33.84 20 20 34.63 40.3 27.67 29.16 18 9.03 2.45 7.68 6.8 0 0 Dhanusha Salyan Dhading Total Dhanusha Salyan Dhading Total Baseline Increase from baseline Baseline Increase from baseline Results also indicate that improvement is not seen only for average and above average children, but also for weaker children that were at a poor proficiency level at baseline. In reading, students who were at the beginner level have the least number of students (4.71 percent) that did not show any improvement. About 34.12 percent and 24.71 percent students from those who could not even recognize Nepali letters at baseline are reading words (moved two levels up) and paragraphs (moved three levels up) at endline, respectively. Similarly, of those who could just read Nepali letters at baseline 39.50 percent and 20.84 percent are reading paragraphs (two levels up) and stories (three levels up) at endline, respectively. Similarly for math, more than 60 percent of the weaker students have improved by two or more levels. About 32.38 percent moved two levels up, 23.81 moved three levels up, and 3.81 percent moved four levels up. Of those who could just recognize one digit in math at baseline, 44 percent can now solve subtraction problem and 9 percent can solve division problems. To understand the perceptions of teachers and mentors and scalability of TaRL project in Nepal, a separate survey was conducted for teachers and mentors. In general, mentors and teachers were satisfied with the program and convinced that TaRL method can substantially improve a child’s learning capabilities, especially those who are weak in learning levels. The main concern that most of the teachers and mentors reported has been the irregularity of children’s attendance in school that would hinder their learning. Timely availability of the teaching and training manuals, increased duration of training, and more time allotted for teaching every day were some of the suggestions that teachers had for better and effective implementation of TaRL. 7 Next Steps Overall, the study provided strong evidence for the effectiveness of the TaRL model in raising basic reading and mathematics skills for primary school children. If this model is scaled up, it can reach thousands of children who are in school but not learning foundational skills. The next step will be to implement a phased scale-up across the country while ensuring low cost and maintaining effectiveness at larger scale. Such an intervention has the potential to be beneficial for primary-age children throughout Nepal who, despite having access to school, are not learning foundational reading and numeracy skills. 8 Introduction Despite rapid gains in school enrolment over the past several decades, learning levels do not show corresponding gains, particularly in developing countries. Policy makers and practitioners over the years have focused on ‘Schooling for all’ or ‘Right to Education’. Globally, efforts have been made to design an age/grade-appropriate curriculum, overlooking the fact that not all students in one grade can engage with the curriculum in the same manner. Nepal has made impressive gains in education access and gender parity, across all levels. Between 2010 and 2020, gross enrolment rate at the Early Childhood Development/Pre-primary Education level increased from 66.2 percent to 87.6 percent, net enrolment rate (NER) at the basic level increased from 83.2 to 94.7 percent, and NER at the secondary level increased from 24.0 to 50.9 percent. Nepal has also made commendable progress along the gender dimension and has achieved gender parity in basic and secondary education. While access has improved, learning levels remain low and are projected to decline further as a result of school closures related to COVID-19 and climate-induced disasters. The National Assessment of Student Achievement (NASA) carried out by the Education Review Office (ERO) measures students’ pe rformance against the learning outcomes specified in the curriculum. Learning outcomes as measured by NASA are low across all levels and showed a declining trend in certain subjects even before the COVID-19 pandemic. In 2018, only 28 percent of Grade 5 students demonstrated grade-appropriate skills and knowledge in mathematics, and only 45 percent performed at the proficient or advanced level in Nepali. 4 In March 2020, Nepal announced school closures in response to the COVID-19 pandemic. Since then, school closures varied at the LG level and there was a pattern of schools opening and closing intermittently between December 2020 and February 2022. Learning is expected to fall further as a result of COVID-19 shutdowns. A recent study shows that nearly 35 percent of Grade 5 students could not do two-digit addition with carryover.5 For Nepal, the Learning Adjusted Years of Schooling (LAYS) was 7.2 years before the 4 Education Review Office, Ministry of Education, Science and Technology, Nepal. 2019. National Assessment of Student Achievement 2018: Main Report. Kathmandu. https://www.ero.gov.np/upload_file/files/post/1623747102_275787545_NASA%202018%20Report%20English.pdf 5 Radhakrishnan, Karthika, Noam Angrist, Peter Bergman, Claire Cullen, Moitshepi Matsheng, Anusha Ramakrishnan, Shwetlena Sabarwal, and Uttam Sharma. 2021. Learning in the Time of COVID-19 : Insights from Nepal. World Bank, Washington, DC 9 pandemic.6 Even conservative estimates suggest that LAYS will likely drop to 6.3 years —a 0.9 year drop.7 Given the learning losses caused by COVID-19-related school closures and economic shocks, Nepal has to move quickly to support learning recovery. One reliable approach to learning recovery is to implement the Teaching at the Right Level (TaRL) intervention in the classrooms to fast-track learning recovery. The TaRL approach brings a change in how children are taught in the school. It is an evidence-based educational approach that helps school-going children at the primary level develop basic language and mathematics skills. The TARL model starts by assessing children’s learning levels using simple one -on-one tests. Children are then grouped based on their learning level, rather than their grade or age. Teachers are trained to use interactive techniques to teach to the level of each group, rather than classroom-style teaching. As children’s learning proficiency improves, they can be moved to more advanced groups that match their skills. The TaRL approach in varied forms is implemented in many African and Asian countries. In Uttar Pradesh and Haryana in India, TaRL significantly improved the learning outcomes of children. 8 Similarly, countries such as Pakistan, Botswana, Nigeria, and so on showed positive improvement in learning levels of children from Grades 3 to 5. Nepal’s TaRL Pilot The World Bank, in collaboration with the Ministry of Education, Science and Technology (MoEST), led an in-school pilot to implement TaRL, which ran from October 2021 to March 2022. 9 The pilot was implemented by the World Bank in partnership with Street Child Nepal and Aasaman Nepal with technical support from the Pratham . TaRL was piloted in three LGs across three provinces out of seven in Nepal (Table 1). LGs were selected purposively based on ownership/interest and geographical topology. Two of the LGs were rural 6 World Bank. 2018. “The Human Capital Project.” World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/30498 License: CC BY 3.0 IGO.” 7 Sharma, U., M. Sherpa, and K. Radhakrishnan. 2021. “Learning Loss as a Result of COVID 19 in Nepal.” Unpublished Manuscript. World Bank. 8 Banerjee, Abhijit, Rukmini Banerji, James Berry, Harini Kannan, Shobhini Mukerji, and Michael Walton. 2016. “Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of 'Teaching at the Right Level' in India.” SSRN Electronic Journal 10: 2139/ssrn.2846971. 9 Includes time for preparatory activities, for example, training of master trainers and teachers. Implementation commenced in December 2021. 10 municipalities, and one was an urban municipality. Of the three provinces, two provinces selected are classified as lagging provinces10 in Nepal. The pilot included a total of 2,163 students in Grades 4–5 from all 64 public schools in the three LGs (see Table 2). Table 1: Name of the LGs Name of the LGs Name of the Province Ganeshman Charnath (Urban) Madhesh (lagging) Siddhakumakh (Rural) Karnali (lagging) Tripurasundari (Rural) Bagmati The pilot project started with six days of virtual ‘Training of Trainers on TaRL’ provided to seven Street Child Master Trainers by Pratham from October 25 to October 31, 2021. Following the training, the Master Trainers conducted ten days practice classes in one selected school of their respective locations to strengthen their own capacity, familiarize themselves with issues that could arise in a classroom setting, and understand its application in school. The Master Trainers then trained a total of 118 schoolteachers and 22 external volunteer teachers and twelve mentors who would implement TaRL in schools. The mentors, four per location, were onboarded to provide support to teachers and volunteers during implementation. At least one mentor per location was hired through the LG education system while others were hired by the project. The mentors hired by the project were partner personnel from the local communities who spoke the local dialect and either had some level of exposure to TaRL or other education projects. The mentors who were onboarded from the LG education system were retired teachers who had a proficient level of understanding of the school systems and curriculums. The volunteer teachers were also from the local communities with prior experience in education projects. The training at Tripurasundari Rural Municipality and Siddha Kumakh Rural Municipality were conducted from November 21 to November 28, 2021, whereas training at Ganeshman Charnath Municipality was conducted from November 22 to November 29, 2021. These three trainings ran parallel to each other. The Master Trainers were divided into three groups to run these trainings with onsite support from Pratham. The pilot was implemented in a two-pronged approach: one led by schoolteachers in 46 schools and the other led by volunteer teachers in 18 schools. The schools were divided into these two wings considering criteria such as students-teacher ratio and remoteness of location. In addition, school level/LG level core groups were 10 Lower literacy rates. 11 formed to support the implementation of the program. Mentors supported the teachers and volunteers by providing support on data collection, interpretation and real-time feedback during the implementation which highlighted focus strategies to support students with the lowest learning levels. Periodic one-to-one assessment was done, followed by regular monitoring and mentoring by mentors to strengthen capacity of teachers and volunteers. The pilot was initially planned to be implemented for a period of 10 weeks between December 2021 and February 202211. However, schools in Nepal were closed due to the Omicron wave in January 2022 for a month and hence the pilot was extended to March 2022 to complete the 10-week intervention, including time for baseline, midline, and endline assessments. Joint monitoring visits were conducted by stakeholders including representatives from the World Bank, MoEST, ERO, Center for Education and Human Resources Development (CEHRD), Curriculum Development Center (CDC), Foreign, Commonwealth and Development Office (FCDO), and LG representatives. To identify the learning levels of students, a baseline assessment was conducted using the TaRL tools before teaching began. The main outcome measures were children’s performance in the Nepali language and math tests. The Nepali reading test assesses children’s reading ability in terms of the following classifications: beginner (cannot recognize letters yet), letter (can recognize at least 4 out of 5 letters), word (can read at least 4 out of 5 words), paragraph (can read at most simple Grade 1 level sentences), and story (child can read fluently a simple story of Grade 2 level or more). The math test assesses children’s math level in terms of the following classifications: beginner (cannot identify single-digit numbers yet), one-digit number recognition (can recognize one-digit numbers at the most), two-digit number recognition (can recognize two-digit numbers but could not solve subtraction), subtraction (can solve simple subtraction but could not solve division), and division (could at least solve division question correctly). In each of these assessments, children are marked at the highest level that they can comfortably attain. Children were grouped according to their learning levels, and periodic assessments (baseline, midline, and endline) were 11The academic year was to start in mid-June 2021; however, schools remained closed until August 2021 due to a surge in Delta variant cases. 12 conducted to map progress of the children.12 Re-grouping of children was done on a continuous basis based on teacher's informal observations of the child’s progress. . The collected data was updated to the system on a real-time basis using mobile-based applications so that project managers and technical experts could review the data and provide support to the mentors. This was done by identifying low-performing schools and students, and enabling mentors to provide focused support to them. Key Findings Drawing from the assessment results collected in the Nepal’s TaRL baseline and endline surveys, this report presents the improvement made in literacy and mathematical skills of children who participated in the pilot. Baseline assessment results are compared with endline assessment results to assess the improvements made by children in their learning levels. Before the pilot was rolled out, all children were assessed on their levels in math and Nepali language through a baseline assessment (Table 2). Overall, for math, only 6.8 percent students could solve division problems or more and 26.94 percent could at most solve a subtraction problem, and a large proportion of children could recognize at most two-digit numbers (37.41 percent) or one-digit numbers (23.96 percent), or could not recognize any number at all (4.89 percent). This means that at baseline, 66.26 percent children were not able to solve subtraction problems and 93.2 percent were not able to solve division problems. At baseline, although Grade 5 children showed higher proficiency in math than Grade 4, the proportion of children not able to solve basic two-digit subtraction is still close to 60 percent (73.57 percent for Grade 4). At the same time, nearly 70 percent of children were not able to read a simple Grade 2 level story, 75 percent in Grade 4 ,and 66 percent in Grade 5. Table 2: Reading and Numeracy Proficiency by Grade at Baseline (in percentage) Math Proficiency Reading Grade 4 Grade 5 Total Grade 4 Grade 5 Total Beginner 7.13 2.71 4.89 Beginner 10.74 5.91 8.3 One digit 28.32 19.71 23.96 Letter 32.71 25.48 29.07 12 Children from Grades 4 and 5 were grouped together in some schools based on class size. 13 Two Digit 38.12 36.71 37.41 Word 15.57 13.66 14.61 Subtraction 22.18 31.59 26.94 Paragraph 16.35 21.32 18.86 Division 4.26 9.28 6.8 Story 24.63 33.62 29.16 Total # 1015 1032 2047 Total # 1011 1035 2047 After the intervention took place the learning outcomes of the students in terms of reading and math improved significantly. This is in accordance with many other pilots conducted using the TaRL approach in other parts of the world.13 Tables 3a and 3b present the change in reading and numeracy proficiencies for children of both Grades 4 and 5 before and after the program was rolled out. The tables also show proficiency achieved in both reading and math halfway through the pilot. The results show that a considerable proportion of children for both math and reading proficiency showed a significant increase at higher proficiency levels and decrease in lower proficiency level. Table 3a: Percentage of Students in Each Category of Proficiency, by Grade in Reading Grade 4 Grade 5 Total Baseline Midline Endline Baseline Midline Endline Baseline Midline Endline Beginner 10.74 2.07 0.79 5.91 1.55 0.19 8.3 1.81 0.49 32.71 20.79 5.81 25.48 15.99 5.52 29.07 18.37 5.67 Letter 15.57 21.28 14.58 13.66 14.34 11.92 14.61 17.78 13.24 Word Paragraph 16.35 18.62 23.35 21.32 16.38 16.96 18.86 17.49 20.13 Story 24.63 37.24 55.47 33.62 51.74 65.41 29.16 44.55 60.48 1,015 1,015 1,015 1,032 1,032 1,032 2,047 2,047 2,047 Total # Table 3a shows a drastic fall in the proportion of those who were at the beginner level and those who could just recognize Nepali letters from baseline to endline, from 8.3 percent to 0.49 percent and 29.07 percent to 5.67 percent, respectively. The proportion of children reading a story without making more than three mistakes have doubled in the endline from 29.16 percent to 60.48 percent. Table 3a also shows that for lower levels of proficiency and the highest level of proficiency, significant gains are seen at midline itself. For example, the proportion of children that were able to recognize only Nepali letter dropped from 29.07 percent at baseline to 18.37 percent at midline, followed by 5.67 percent at endline. Similarly, the proportion of children able to read Nepali story or more without making more than three mistakes 13 Duflo, E., J. Berry, S. Mukerji, and M. Shotland. 2015. A Wide Angle View of Learning: evaluation of the CCE and LEP Programs in Haryana, India, 3ie Impact Evaluation Report 22. New Delhi: International Initiative for Impact Evaluation (3ie). 14 increased to 44.55 percent at midline from 29.15 percent at baseline, eventually increasing to 60.48 percent at endline. The results show significant improvement in reading levels from baseline to endline, more so for Grade 4 students than Grade 5. This could be due to the fact that there was more scope for Grade 4 students for improvement as they were at a much lower level of proficiency than Grade 5 students before TaRL. Large improvement is seen for learning in math as well. Table 3b shows that at baseline only 33.74 percent of children were able to solve at least subtraction, while at endline 79.7 percent children can solve these questions, with the highest jump seen for children who are able to at least solve division problems now (from 6.8 percent to 40.64 percent). Significant improvement in math proficiency also started to show at midline followed by a larger increase at endline. For example, the proportion of children at least solving math division problems increased from 6.8 percent at baseline to 17.9 percent at midline, eventually increasing to 40.64 percent at endline. Comparing improvement in math proficiency levels between Grade 4 and Grade 5 students, it is found that Grade 4 students show significant improvements in the proportion of children solving subtraction at most (from 22.18 percent to 42.57 percent) and at least division (from 4.26 percent to 33.56 percent) problems, whereas Grade 5 children show a large improvement in solving at least division problems (9.28 percent to 47.54 percent) with a little increase in proportion of children solving at most subtraction problems (31.59 percent to 35.75 percent). Table 3b: Percentage of Students in Each Category of Proficiency, by Grade: Numeracy Grade 4 Grade 5 Total Midlin Baselin Baseline Midline Endline Baseline Endline Midline Endline e e Beginner 7.13 1.88 0.69 2.71 0.87 0.19 4.89 1.37 0.44 One digit 28.32 14.36 4.85 19.71 7.92 2.9 23.96 11.1 3.86 Two Digit 38.12 36.83 18.32 36.71 27.63 13.62 37.41 32.18 15.94 Subtraction 22.18 36.04 42.57 31.59 38.84 35.75 26.94 37.46 39.12 Division 4.26 10.89 33.56 9.28 24.73 47.54 6.8 17.9 40.64 Total # 1,010 1,010 1,010 1,035 1,035 1,035 2,047 2,047 2,047 Analyzing the improvement by gender, it is seen that generally females performed better than males at baseline in terms of reading. Table 4a shows that 31.33 percent females as compared to 26.75 percent males could read a story or more in Nepali, and 19.29 percent females could at most read a paragraph as compared to 18.41 percent males at baseline. At higher levels of proficiency, such as reading stories and 15 paragraphs, females perform better. The proportion of females in basic reading proficiency, such as just recognizing letters and words is lower as compared to males. While 15.97 percent males were in the category of reading words, the corresponding figure was 13.36 for females. This trend continued at endline as well with females performing better than males at the higher level of proficiency in reading. At endline, 62.18 percent females could read a Nepali story or more against 58.60 percent males. Table 4a: Learning Outcomes by Gender: Reading (in percentage) Male Female Total Baseline Endline Baseline Endline Baseline Endline Nepali Reading Beginner 8.34 0.41 8.28 0.56 8.3 0.49 Nepali Reading Letter 30.52 5.70 27.75 5.64 29.07 5.67 Nepali Reading Word 15.97 15.16 13.36 11.48 14.61 13.24 Nepali Reading Paragraph 18.41 20.14 19.29 20.13 18.86 20.13 Nepali Reading Story 26.75 58.60 31.33 62.18 29.16 60.48 Total # 983 983 1,063 1,063 2,047 2,047 In terms of assessment results for mathematics, at baseline males perform better than females at higher proficiency levels (Table 4b). At baseline, 29.54 percent males could solve at most a subtraction problem and 7.61 percent could solve division problems or more as compared to 24.55 percent and 6.04 percent females, respectively. Both males and females show high improvement at endline. Around 41.93 percent males could solve division problems or more at endline as against 39.47 percent females. Table 4b: Learning Outcomes by Gender: Numeracy (in percentage) Male Female Total Baseline Endline Baseline Endline Baseline Endline Math Beginner 4.06 0.41 5.67 0.47 4.89 0.44 Math One digit 21.42 3.55 26.25 4.15 23.96 3.86 Math Two Digit 37.36 15.94 37.49 15.96 37.41 15.94 Math Subtraction 29.54 38.17 24.55 39.94 26.94 39.12 Math Division 7.61 41.93 6.04 39.47 6.8 40.64 Total # 985 985 1,059 1,059 2,047 2,047 A separate analysis was conducted, and presented in Tables 5a and 5b, for males and females by grade. It is seen that for reading there is a large improvement in learning level, both for males and females, with the largest increase seen for males in Grade 5 at reading story level (29.84 percent at baseline to 63.51 percent 16 at endline). The second largest increase is seen in learning for females in Grade 4 at reading story level (from 25.43 percent at baseline to 57.12 percent at endline). Females in both Grades 4 and 5 are more proficient in reading story than males, both at baseline and endline. About 67.16 percent females in Grade 5 could read stories or more without making more than three mistakes as compared to 63.51 percent males during the endline. Similarly, 57.12 percent females could read stories or more at endline as against 53.59 percent males in Grade 4. Table 5a: Learning Outcomes by Gender and Grade: reading (in percentage) Grade 4 Grade 5 Male Female Male Female Baseline Endline Baseline Endline Baseline Endline Baseline Endline Beginner 9.86 0.6 11.57 0.95 6.85 0.20 5.04 0.19 Reading Letter 34.09 5.13 31.50 6.45 27.02 6.25 24.07 4.85 Reading Word 17.04 16.02 14.23 13.28 14.92 14.31 12.50 9.70 Reading Paragraph 15.40 24.63 17.27 22.2 21.37 15.73 21.27 18.10 Reading Story 23.61 53.59 25.43 57.12 29.84 63.51 37.13 67.16 Total # 487 487 527 527 496 496 536 536 Unlike reading, males perform better than females in mathematics, except for subtraction level in Grade 5 where females outperform males by almost five percentage points (33.33 percent in baseline to 38.01 percent in endline). Again, there is a large improvement in learning level, both for male and females with the largest increase seen for males in Grade 5 at division level (10.38 percent at baseline to 48.30 percent at endline), and females in Grade 5 at division level (8.24 percent to 46.82 percent). Males in both grades are more proficient in solving higher difficulty level questions (subtraction and division) than females, both at baseline and endline. About 48.30 percent males in Grade 5 could solve at least division problems as compared to 46.82 percent females. Similarly, 35.33 percent males could at least solve division problems in Grade 4 as against 32 percent females. Taking the highest two categories of proficiency (subtraction and division) together, we see a large number of Grade 5 children reaching the highest two categories as compared to Grade 4 children. Also, females outperform males in Grade 5 if the highest two categories of proficiency are considered together. About 81.6 percent of the males in Grade 5 can solve at least subtraction problems at endline as compared to 85 percent females. This is despite a lower share of females in Grade 5 in these categories at baseline (44.31 percent males and 37.64 percent females). On the other 17 hand, females underperform males by almost 5 percentage points in Grade 4 as 73.90 percent females are in the highest two categories of proficiency as compared to 78.5 percent males in Grade 4. Irrespective of the grade, the increase in the proportion of females who can do at least subtraction is more than males. While there was an increase of 47.19 percentage points for females in Grade 5 as compared to 37.32 percentage points increase for males, an increase of 50.47 percentage points for females in Grade 4 is seen as compared to 48.7 percentage points for males. Table 5b: Learning Outcomes by Gender and Grade: Math Grade 4 Grade 5 Male Female Male Female Baseline Endline Baseline Endline Baseline Endline Baseline Endline Math Beginner 5.58 0.6 8.57 0.76 2.59 0.20 2.81 0.19 Math One digit 26.03 3.93 30.29 5.71 16.97 3.19 22.28 2.62 Math Two Digit 38.64 16.94 37.71 19.63 36.13 14.97 37.27 12.36 Math Subtraction 25 43.18 19.62 41.90 33.93 33.33 29.40 38.01 Math Division 4.75 35.33 3.81 32 10.38 48.30 8.24 46.82 Total # 484 484 525 525 501 501 534 534 As mentioned earlier, TaRL’s pilot in Nepal was conducted in three LGs: Tripurasundari, Ganeshman Charnath, and Siddhakumakh of Dhading, Dhanusha, and Salyan districts, respectively. Since the LGs come under different jurisdictions, the total number of weeks of teaching children varied across LG due to school closures as a result of COVID-19 Omicron variant. In all, Salyan had on an average 45 days of TaRL teaching as compared to Dhanusha that had 60 days of teaching on an average and Dhading that had 78 days of teaching. The difference in the number of days taught is also reflected in the improvements in learning levels by LGs. Figures 1a and 1b below present LG-wise proportion of children in the highest category of learning for reading and math, respectively, at baseline, and the percentage point increase in this proportion at endline. Dhading had TaRL rolled out for the maximum number of days, and Salyan had it for the least number of days. Comparison of LG level learning outcomes show that Dhading is the best performing LG in terms of improvement in learning outcomes followed by Dhanusha, and Salyan is the worst performer. Dhading showed 34 percentage point improvement in the proportion of children that could read Nepali stories or 18 more as compared to 30.75 percentage points for Dhanusha and 29.33 percentage point increase for Salyan. Similarly, for proficiency in math Dhading showed close to a 40 percentage point jump in the proportion of children that could solve division problems at endline as compared to 33.71 and 27.67 percentage point increase for Dhanusha and Salyan, respectively. Figure 1a: Learning Outcomes by LG: Reading Percentage increase in reading proficiency of the highest level 80 (reading story) 60 34 29.33 31.32 40 30.75 20 34.63 40.3 29.16 18 0 Dhanusha Salyan Dhading Total Baseline Increase from baseline Figure 1b: Learning Outcomes by LG: Math Percentage increase in math proficiency of the highest level (division) 50 40 30 33.71 39.9 33.84 20 27.67 10 9.03 7.68 6.8 0 2.45 Dhanusha Salyan Dhading Total Baseline Increase from baseline Even though we see a substantial change in the learning outcomes from baseline to endline for all LGs, the proportion of children with high proficiency in math and reading vary vastly across LGs, both at baseline and endline. Tables 6a and 6b report the learning outcomes at baseline and endline by LG for reading and numeracy, respectively. As mentioned earlier, Dhading is the best performer in terms of increase in the proportion of children at the highest proficiency level, both for math and reading, it is also one of the best performing LGs if we look at baseline and endline separately. At baseline, Dhading had the highest proportion of children (40.3 percent) who could read a story or more without making more than three 19 mistakes as compared to Salyan (34.63 percent) and Dhanusha with just 18 percent of children at the same level. At endline, the proportion increased to 74.3 percent for Dhading as compared to 63.96 percent for Salyan, and 48.75 percent for Dhanusha. For math, Dhanusha had a greater proportion of children at division level than Dhading at baseline (9.03 percent and 7.68 percent respectively), yet the proportion of children solving division problems in Dhading (47.58 percent) was more than Dhanusha (42.74 percent) at endline. Even though Dhanusha and Salyan had huge scope for increasing the proportion of children in high categories of proficiency, fewer days offered under TaRL could have resulted in limited increase in learning for these LGs. Table 6a: Reading Proficiency by LG at Baseline and Endline, by LGs (in percentage) Dhanusha Salyan Dhading Baseline Endline Baseline Endline Baseline Endline Beginner 13.33 0.8 6.71 0.18 2.49 0.33 Reading Letter 37.24 6.04 27.39 9.89 18.74 1.16 Reading Word 13.67 18.11 13.6 11.48 16.92 7.79 Reading Paragraph 17.77 26.31 17.67 14.49 21.56 16.42 Reading Story 18 48.75 34.63 63.96 40.3 74.3 Total # 878 878 566 566 603 603 Table 6b: Math Proficiency by LG at Baseline and Endline, by LGs Dhanusha Salyan Dhading Baseline Endline Baseline Endline Baseline Endline Beginner 7.2 0.57 5.43 0.35 1 0.33 One digit 27.54 3.31 33.1 8.58 10.02 0.17 Two Digit 31.89 14.4 41.68 25.57 41.4 9.02 Subtraction 24.34 38.97 17.34 35.38 39.9 42.9 Division 9.03 42.74 2.45 30.12 7.68 47.58 Total # 875 875 571 571 599 599 Tables 7a and 7b provide evidence of change in learning levels by grade for each LG. It is seen that for the highest level of reading skills, that is, story level, Grade 4 children in Dhading show the maximum improvement (increase of 37.25 percentage points), whereas improvement is more for Grade 5 students in Dhanusha and Salyan as compared to Dhading, although there is not much difference as the increase in the proportion of Grade 5 children that could read Nepali story ranges from 30.0 to 32.5 percentage points for 20 all LGs. For other levels of learning, Grade 4 children in Dhanusha show maximum change (an increase of 17.75 percentage point). Table 7a: Improvement in Learning Outcomes by Grade and LGs: Reading (in percentage) Grade 4 Grade 5 Dhanusha Salyan Dhading Dhanusha Salyan Dhading Nepali reading beginner −17.08 −6.62 −2.34 −7.85 −6.46 −1.97 Nepali reading letter −36.63 −20.22 −18.46 −25.64 −14.97 −16.72 Nepali reading word 6.97 −2.21 −11.74 1.84 −2.05 −6.56 Nepali reading paragraph 17.75 2.2 −4.7 −0.92 −8.17 −5.58 Nepali reading story 28.99 26.84 37.25 32.57 31.64 30.82 445 272 298 433 294 305 Total # Table 7b: Improvement in Learning Outcomes by Grade and LGs: Math (in percentage) Grade 4 Grade 5 Dhanusha Salyan Dhading Dhanusha Salyan Dhading Math Beginner −9.73 −6.91 −1.03 −3.46 −3.38 −0.32 Math One digit −27.61 −28.00 −12.97 −20.78 −21.29 −6.86 Math Two Digit −13.57 −12.73 −35.84 −21.48 −19.25 −29.08 Math Subtraction 22.63 24.36 13.31 6.47 12.16 −6.86 Math Division 28.28 23.27 36.52 39.26 31.76 43.14 Total # 444 272 298 433 294 305 For math, as seen earlier, Grade 5 students in all LGs perform better at higher levels of learning and show more improvement than Grade 4 students (Table 7b). Table 7b also shows that there is a significant change in the proportion of children at math subtraction level for Grade 4 in all the LGs. Grade 4 also showed a large incease in the proportion of children solving division problems but the increase is more for Grade 5 children. Dhading showed the maximum increase of 43.14 percentage points in the proportion of children in Grade 5 solving division problems as compared to 39.26 and 31.76 for Dhanusha and Salyan, respectively. For Grade 4 too, the maximum increase in the proportion of children solving division problems is seen in Dhading (36.52 percentage points) as compared to 28.28 in Dhanusha and 23.27 in Salyan. Grade 4 children show large improvements in both subtraction and division levels, whereas most of the improvement for Grade 5 children is seen at division level. As mentioned before, there is variation across LGs in terms of the number of days TaRL was offered in schools. At the same time, within LGs there are variations across schools in terms of number of days 21 teaching was conducted and days the children attended school. It is interesting to see how attendance in class has an impact on the learning levels of children. Table 8 shows the proportion of children at different levels of attendance categories by LGs. Table 8: Attendance by LGs (in percentage) Attendance Dhading Dhanusha Salyan Total less than 50% 0.16 4.72 4.3 3.3 50–75% 40.10 44.32 10.78 34.23 75–100% 59.74 50.96 84.91 62.42 It is found that more than 96.7 percent children across schools attended school for more than 50 percent of the classes, 34.23 percent attended 50–75 percent classes, and 62.42 attended 75 –100 percent classes. Salyan, which is not the top performer LG showed the maximum (84.91 percent) number of children attending more than 75 percent of the classes as compared to 59.14 percent in Dhading and 50.96 percent in Dhanusha. The poor performance in Salyan could be on account of lower number of classes conducted in the LG as compared to the other two. This indicates that not just attendance but the number of days of TaRL teaching offered also has an impact on the proficiency levels. Table 9 shows that Salyan has the least number of days attended by children out of all LGs. About 42.38 percent of students in Salyan attended just 30–40 days of schooling as compared to only 4.79 percent in Dhanusha and none in Dhading. Similarly, 46.06 percent students in Salyan attended 40–50 days of TaRL teaching as compared to 32.83 percent and 4.52 percent in Dhanusha and Dhading, respectively. More than 95 percent children in Dhading attended more than 50 days of TaRL teaching as compared to 61 percent in Dhanusha and just 2.8 percent in Salyan. Table 9: Number of Days TaRL Classes Attended, by LGs (in percentage) Number of Days Dhading Dhanusha Salyan Less than 20 days 0 0.58 1.93 20-30 days 0 1.17 6.83 30-40 days 0 4.79 42.38 40-50 days 4.52 32.83 46.06 50-60 days 64.82 60.16 2.80 60-70 days 30.65 0.47 0 Total # 597 856 571 22 Table 10a provides evidence of the relationship between school attendance and improvement in reading proficiency levels of children at endline. Studying the relationship between the number of days classes attended by children at lower levels, including those at beginner and letter level (at the time of baseline), and improvement they made at endline, we found that those who were beginners at baseline in reading and attended more than 75 percent of the classes did not remain at beginner level at the endline, and 25.5 percent reached the highest level of learning as compared to 18.18 percent and 13.52 percent of children attending less than 50 percent and 50–75 percent classes, respectively. Also, the proportion of beginners (39.26 percent) that have moved three levels up (to the reading paragraph level) attended more than 75 percent of the classes. At the same time, 71.43 percent of those who could only recognize Nepali letters at the time of baseline remained at the same level if they attended less than 50 percent of the classes. Almost 8.14 percent reached the highest level of learning if they attended more than 75 percent of classes as compared to less than 7.06 percent for those with less than 75 percent attendance. It is interesting to see that although the proportion of children reaching reading paragraph level at endline is more for those who attended 50–75 percent classes than those attending more than 75 percent, 9.41 percent children also fell back to beginners’ level and 30.59 percent stayed at reading letter level if they had 50–75 percent attendance. In total, 72 percent of those at recognizing letter level at baseline moved up levels with more than 75 percent attendance as compared to 60 percent for those with 50 –75 percent attendance. Table 10a: Reading Proficiency by Level of Attendance Beginner at Baseline Could only recognize letters Attendance Attendance Reading Proficiency at endline 0–50 50–75 75–100 0–50 50–75 75–100 Beginner 0 0.41 0.29 0 9.41 0 Letter 18.18 11.07 9.17 71.43 30.59 27.91 Word 36.36 36.07 25.79 14.29 27.06 41.86 Paragraph 27.27 38.93 39.26 14.29 25.88 22.09 Story 18.18 13.52 25.5 0 7.06 8.14 Total # 7 85 85 22 244 338 For math, as presented in Table 10b, 30.61 percent of those at beginner level at baseline and who attended school for more than 75 percent of the classes are able to solve subtraction problems at endline compared to 17.31 percent for those who attended 50–75 percent classes. Moreover, 13.46 percent of those who had 50–75 percent attendance stayed at beginner level as compared to zero for those who had more than 75 23 percent attendance. Around 42.28 percent of children who could recognize one-digit numbers at baseline and who attended more than 75 percent classes have reached the subtraction level, 11.74 percent have reached the highest level of learning in math as compared to 4.84 percent for those who attended 50 –75 percent classes. Results suggested that having higher attendance in school is associated with larger improvements in proficiency for both math and reading. Table 10b: Numeracy Proficiency by Level of Attendance Beginner at Baseline Could only recognize one digit Math Proficiency 0–50 50–75 75–100 0–50 50–75 75–100 Beginner 25 13.46 0 0 0 0 One digit 50 28.85 34.69 25 10.22 8.39 Two Digit 0 34.62 32.65 31.25 37.10 37.58 Subtraction 25 17.31 30.61 37.5 47.85 42.28 Division 0 5.77 2.04 6.25 4.84 11.74 Total # 4 52 49 16 186 294 It is interesting to see not just the difference in the proportion of children in each category of learning between baseline and endline, but also to see how far children have gone over and above their learning level at baseline. After the implementation of TaRL, the improvement in level of learning for students can be classified into the following values: 0, stays same; 1, moved to the next level; 2, moved two levels up; 3, moved 4 levels up. Of course, those who were at the highest difficulty level of learning at baseline do not have any next level to reach. Tables 11a and 11b provide the change in the level of learning for children based on their learning level at baseline. Table 11a shows that weaker students do show a lot of improvement in reading proficiency. In reading, students who were at the beginner level show the least number of students (4.71 percent) that did not show any improvement. About 34.12 percent and 24.71 percent students from those who could not even recognize Nepali letters at baseline are reading words (moved two levels up) and paragraphs (moved three levels up) at endline, respectively. Similarly, of those who could just read Nepali letters at baseline, 39.50 percent and 20.84 percent are reading paragraphs (two levels up) and stories (three levels up) at endline, respectively. 24 Table 11a: Changes in the Level of Reading Proficiency from Baseline to Endline for Each Category at Baseline (percentage) Level of Change One level No One Level Two Three levels Four down Change Up levels up up levels up Total # Beginner 0 4.71 28.82 34.12 24.71 7.65 170 Baseline Reading Letter 0.34 10.08 29.24 39.50 20.84 n.a. 595 Reading Word 2.34 10.03 32.11 55.52 n.a. n.a. 299 Reading Paragraph 2.33 8.03 89.64 n.a. n.a. n.a. 386 Reading Story 1.34 98.6 n.a. n.a. n.a. n.a. 597 Table 11b: Changes in the Level of Numeracy Proficiency from Baseline to Endline for Each Category at Baseline Level of Change One level One Level Two levels Three levels Four levels down No Change Up up up up Total # Beginner 0 7.62 32.38 32.38 23.81 3.81 100 Baseline One digit 0 9.6 37.2 44.2 9 n.a. 490 Two Digit 0.13 14.49 53.11 32.4 n.a. n.a. 765 Subtraction 1.09 27.22 72.78 n.a. n.a. n.a. 551 Division 2.88 97.12 n.a. n.a. n.a. n.a. 139 In math (Table 11b), similar to reading, students who were at the beginner level show the least number of students (7.62 percent) that did not show any improvement from baseline. More than 60 percent of the weaker students have improved by two or more levels. About 32.38 percent moved two levels up, 23.81 moved three levels up, and 3.81 percent moved four levels up. Of those who could just recognize one-digit numbers in math at baseline 44 percent can now solve subtraction problems and 9 percent can solve division problems. Comparison between top and low performing schools While there are variations across LGs in terms of improvement in levels of learning there are greater variations across schools in terms of their performance. Using the teacher and mentor survey data and assimilating the information with the performance of the schools, we found that good performing schools 25 have teachers with more number of years of experience.14 Over 55 percent of teachers in weaker schools have less than five years of teaching experience as compared to 22 percent in strong schools. Four out of nine teachers in weak schools found problems in class material as against 44.4 percent in strong schools. Nearly 78 percent teachers in low performing schools found it difficult to manage seating arrangement as compared to 22 percent in strong schools. Apart from these, there are no significant differences between the characteristics of teachers in weak and strong schools. Effect of sudden school closure on student learning Since there was a break in studies due to the sudden closure of schools as a response to the COVID outbreak after the midline survey was conducted, a midline retest was done with 235 students based on 10 percent sample selected from each class after the schools reopened. It was expected that the learning levels of students would have fallen as there was a break in teaching and learning. Using the midline retest data and comparing them to the original midline (Table 12), it is found that 7.7 percent of the students reported a reduced learning level in reading, whereas 11.64 percent students reported reduced learning levels in math as compared to the midline assessment conducted just before schools were shut down. At the same time, approximately 14.5 percent students showed increased learning levels in the retest for reading as compared to 17.24 percent increased learning levels for math. Suspension of in-classroom teaching in schools after the lockdown led to concerns about consequences for students’ learning, especially the academically weaker students. The midline retest revealed, as reported in Table 12, that 34.29 percent of the children at lower levels in reading just before schools were shut have fallen one level down after the midline re-assessment. Similarly, for math 8.93 percent have fallen one level down. It is also worth reporting that a large number of weaker students either showed no change (22.86 percent) in reading capabilities between the assessments conducted just before and after the lockdown, or showed improvement (42.86 percent). At the same time for math, 50 percent showed no change and 41.07 percent showed improvement before and after the school closure. 14 Based on comparison of 9 teachers from good schools and 9 teachers from weak schools, in total 10 schools (5 good and 5 poor). 26 Table 12: Difference in Learning Outcomes Between Midline and Midline Retest Reading Numeracy Reading Numeracy All Children Weaker Children N Percent N Percent N Percent N Percent Two levels down 1 0.43 2 0.86 0 0 0 0 One level down 17 7.26 25 10.78 12 34.29 5 8.93 Remained same 182 77.78 165 71.12 8 22.86 28 50 Increased one level up 30 12.82 36 15.52 11 31.43 19 33.93 Increase two levels up 4 1.71 4 1.72 4 11.43 4 7.14 Table 13a and Table 13b compare the learning outcome at midline conducted before the schools were shut and midline survey conducted after schools were reopened. Out of 234 retested children, 158 were already at the highest level of reading at the first midline. Of those, only three children dropped to reading paragraph level at the second midline survey. Similarly, out of 41 children who were are reading paragraph level at the first midline, 19 stayed at the same level and 19 moved up to reading story level at retest. Larger consequences were seen for children who were at a lower level of proficiency at the first midline. For example, out of 18 children who were at recognizing word level at the first midline, seven moved down to letter level, and out of 16 that were at letter level five moved down to the beginner level at the retest. This indicates the vulnerability of weaker children to disruptions in studies. Math, on the other hand, showed significant changes for children who were at one-digit level at the time of the first midline survey. Out of 13 who were at one-digit level, 3 fell down to the beginner level at the re-test, and only 2 out of 43 who were at two-digit level fell one level down to one-digit level. A very positive takeaway from the exercise of comparing midline and midline retest is that, although many students stayed at the same level of learning at the second midline, some students did show improvement. For example, 19 out of 41 children who were at paragraph level at first midline moved up to story level at the second midline. Four out of 18 that were at reading word level moved to paragraph and two moved to story level. Similarly 17 out of 84 who were at subtraction level moved to division level in math, and 17 out of 43 who were at two-digit level moved to subtraction level. This could be the result of teaching methods at TaRL that focus on making children develop foundational skill. 27 Table 13a: Number of Children at Different Learning Outcomes of Reading at Midline and Midline Retest Midline Retest Beginner Letter Word Paragraph Story Total Beginner 0 1 0 0 0 1 Original Midline Letter 5 3 6 2 0 16 Word 0 7 5 4 2 18 Paragraph 0 1 2 19 19 41 Story 0 0 0 3 155 158 Total 5 12 13 28 176 234 Table 13b: Number of Children at Different Learning Outcomes of Math at Midline and Midline Retest Midline Re-Test Beginner One digit Two Digit Subtraction Division Total One digit 3 6 2 2 0 13 Original Midline Two Digit 0 2 22 17 2 43 Subtraction 0 1 6 60 17 84 Division 0 0 1 14 77 92 Total 3 9 31 93 96 232 The results from the endline were reassuring as with few more weeks of teaching the proportion of students in weaker categories for both reading and math 15 had significantly fallen. In the endline, none of the children remained at beginner category, both for math and reading. Table 14 reports the learning outcomes for all assessments conducted on 235 children who were retested after the schools reopened. Even though there was a slight increase in the proportion of children in the beginner category after the retest, it is found that after the TaRL teaching is completed none of the children that were re-assessed after midline belonged to beginner category in both math and reading. 15 Numbers are only for those who were retested after the midline, approximately 235 children. 28 Table 14: Learning Outcomes for Retested Children Reading Math Baseline Midline Retest Endline Baseline Midline Retest Endline Beginner 3.48 0.43 2.13 0 Beginner 0.43 0 1.28 0 Letter 13.04 6.84 5.11 1.31 One digit 13.48 5.6 3.83 0 Word 7.83 7.69 5.53 5.68 Two Digit 29.57 18.53 13.19 6.99 Paragraph 18.7 17.52 11.91 8.73 Subtraction 38.26 36.21 40.43 26.2 Story 56.96 67.52 75.32 84.28 Division 18.26 39.66 41.28 66.81 Total # 230 234 235 229 Total # 230 232 235 229 The study provided strong evidence for the effectiveness of the TaRL model in raising basic reading and mathematics skills for primary school children. If this model is continued, it can have massive impacts on students’ learnings. Findings from teachers and mentors survey To understand the perceptions of teachers and mentors and the scalability of the TaRL project in Nepal, a separate survey was conducted for teachers and mentors. The survey also helped understand how TaRL was implemented in schools. A total of 119 teachers and 11 mentors were surveyed from 64 schools that were selected for the pilot. Most of the teachers that were trained for TaRL were regular teachers and only 13.45 percent were volunteer teachers. Only 11 percent teachers had master’s degree and 36 percent had bachelor’s degree, the remaining teachers had education below graduate level. Almost all teachers reported receiving six to seven days of TaRL training. Almost 75 percent of the teachers thought that the number of days training provided was inadequate, yet they were happy about the quality of training received. No particular association between the education level of the teachers and their satisfaction for the number of days training provided was seen. Almost all teachers were satisfied with the training method (99 percent). The teachers were asked about the challenges they encountered while conducting TaRL methods in classroom. Around 55.5 percent teachers said they faced unavailability of teaching material and 71 percent complained of not getting the teaching manual on time. Furthermore, 35 percent teachers also complained 29 of facing difficulty in conducting classes in a chronological manner as taught to them at the time of training, and 36 percent teachers had trouble implementing seating arrangement in the class. Nevertheless, in comparison to the traditional method of teaching, 70 percent of the teachers found TaRL a better teaching method. The remaining teachers were indifferent to the method of teaching. As far as teachers’ perception regarding the effectiveness of TaRL on children’s learning levels is concerned , 99 percent of teachers found TaRL very effective. At the same time, an impressive 97 percent teachers reported that they would continue using TaRL methods even after the program ended. About 74 percent teachers believed that this program would benefit more than 75 percent of the children. Of course, there were some teachers who had apprehensions about the program mainly due to irregularity in attendance of some children and extremely poor learning levels of some. This could be the reason why approximately 35 percent teachers suggested to have TaRL lessons only for weaker students and not the entire class. About 52 percent of the teachers also suggested to have TaRL classes after school hours and not as part of regular school. In terms of the skills gained by teachers through TaRL teacher training and classroom teaching all teachers reported to have gained immensely from the program. Overall satisfaction from the program is apparent from the teacher’s survey. Timely availability of the teaching and training manual, increased duration of training, and more time allotted for teaching everyday were some of the suggestions that teachers had for better and effective implementation of TaRL. Some of the teachers also suggested to have this program implemented at the beginning of the year, and also at lower primary levels so that fundamental problems can be corrected at an early age. Since mentors were also heavily involved in training the teachers, and overseeing how the program was being implemented, 11 mentors were also interviewed to gauge their understanding on teachers’ quality and their own perception about TaRL project. In terms of education level attained, nine out of 11 mentors had bachelor’s degree and two were below graduation level. Mentors also felt that the total duration of training teachers was less and needs to be increased for better skill upgradation. Like teachers, lack of training material and unavailability of manual were other concerns raised by mentors. In general, according to mentors, most of the teachers liked the TaRL program. However, they raised concerns about older 30 teachers who are close to their retiring age being less motivated for the program. Nevertheless, mentors too were convinced that the TaRL method can substantially improve a child’s learning capabilities. Conclusion The results from midline and endline assessments are very impressive - both male and female students showed huge improvements in foundational reading and numeracy skills. Weaker students who were at the beginner level in both reading and numeracy skills during baseline showed the greatest improvement by endline. In general, teachers were also confident that TaRL would improve the learning capabilities of children. About 98 percent teachers agreed that TaRL is going to improve the learning levels of children, and an impressive 96 percent said they will continue using this method even after the pilot was over when there would be no mentors to supervise their work. In terms of the skills gained by teachers through TaRL teacher training and classroom teaching, all teachers reported to have gained immensely from the program. Mentors also had positive reviews about TaRL, and they were convinced that teachers as well as children are going to benefit immensely from TaRL. The main concern that most of the teachers and mentors reported has been the irregularity of children’s attendance in school that would hinder their learning. Timely availability of the teaching and training manual, increased duration of training, and more time allotted for teaching everyday were some of the suggestions that teachers had for better and effective implementation of TaRL. Results suggested that having higher attendance in school is associated with greater improvements in proficiency for both math and reading. In two LGs that had huge scope for increasing the proficiency of children, fewer number of days offered under TaRL could have resulted in limiting the increase. Similarly, the one-month disruption in studies as a result of the third wave of COVID-19 appeared to disproportionately affect the weaker children negatively. Overall, the study provided strong evidence for the effectiveness of the TaRL model in raising basic reading and mathematics skills for primary school children. If this model is scaled up, it can have massive impacts on students’ learnings. The next step will be to implement a phased scale-up across the country while 31 ensuring low-cost and maintaining effectiveness at larger scale. Such an intervention will be beneficial for children who have access to school but not to learning foundational reading and numeracy skills. 32