Timor Leste 2011 EGMA: Baseline Report M IN IS ÃO TÉ AÇ RIO DA EDUC THE WORLD BANK Timor Leste 2011 EGMA: Baseline Report ABSTRACT In 2011, the Timor-Leste Ministry of Education, with assistance from the World Bank and Ausaid, conducted the first Early Grade Mathematics Assessment (EGMA) survey in Timor-Leste. More than 1200 students in 65 schools were surveyed in Grades One, Two and Three. The analysis of the EGMA 2011 Timor-Leste baseline survey calls for an immediate response to safeguard the educational future of young Timorese students. Although students perform well in basic, “Phase Zero” mathematics skills such as oral counting and correspondence counting, “Phase One” skills (number identification) shows signs of slow gains. Later manipulative and calculation skills, i.e. those in “Phase Two” and “Phase Three” (quantity discrimination, missing numbers, word problems and arithmetic) are poorly understood and used by Timorese students. After three years of schooling, Grade Three students can only answer 46% of simple subtraction problems correctly on average and only 72% of addition problems -all of which should have been understood by the end of Grade One. The low ability of Timorese students to handle basic numeracy puts in doubt their ability to cope with an increasingly stringent curriculum in later years. Language use was one of the most concerning aspects of mathematics education revealed by the survey. Although the main language of the classroom is Tetum, students’ mathematics textbooks are in Portuguese. Some students did not speak enough Tetum or Portuguese to complete the survey without aid of a translator for their local languages. This linguistic diversity within the classroom indicates that further research into how Timorese students learn and in which languages is required. In terms of factors that showed positive association with early mathematic abilities, the participation in daily mathematics lessons, working with others in those lessons and doing homework were all associated with statistically significant improvements in mathematics outcomes at the 95% level. i Timor Leste 2011 EGMA : Baseline Report ii Acknowledgements The 2011 Timor-Leste Early Grade Mathematics Assessment (EGMA) survey is the result of the intention of the Ministry of Education of Timor-Leste to improve numeracy skills of the students in the education system, with recognition that the early years of education is the key period for foundational learning. The report was prepared by Steph de Silva (Econometrician, World Bank), Bronwyn McNamee (Education Specialist within the ESSP project) and Luc Gacougnolle (Education Economist, World Bank). As of 2011, Kashif Saeed, Simao do Rosario and members of the Ministry of Education EMIS team provided assistance in school sampling, data cleaning and data entry. The report benefitted from significant comments, editing and additions by Barbara Thornton (International Development Consultant) and Myrna Machuca-Sierra (Education Specialist, World Bank). Wendi Ralaingita of Research Triangle Institute (RTI) generously directed the authors to research and reports of other countries where EGMA has been implemented and Yasmin Sitabkhan (RTI) provided helpful review comments on the final draft report. The 2011 Timor-Leste EGMA was implemented as a direct result of commitment, encouragement and support, as of 2011, from Joao Cancio Freitas (Former Minister of Education), Paulo Assis Belo (Former Vice Minister of Education), Domingos de Deus Maia (Former Director General of School Administration, Innovation and Curriculum Development), Raimundo Neto (Former Director of Curriculum and Assessment), Alfredo de Araujo (Director of Basic Education). As of 2011, Filomena Sequeira (Directorate of Curriculum and Assessment), José Antonio Cardoso (Chief of Department of Assessment in the Directorate of Curriculum and Assessment), Fernando Mouzinho Gama (Directorate of Curriculum and Assessment) and Pedro Laurentino da Silva (ESSP advisor) formed the team for the initial adaptation and trial of the survey. The authors would especially like to thank and commend the EGMA core team of trainers, mentors and monitors from the Directorate of Curriculum and Assessment. Regional and District education personnel, school directors and teachers are commended and thanked for their generous and willing cooperation in terms of liaising and making preparation for the school visits by the enumerators and EGMA mentors. The authors also wish to commend and express gratitude to the enumerators and supervisors who carried out the survey implementation professionally and efficiently and within the set time frame. The geography of Timor-Leste presents many challenges for travel, particularly in remote areas. These people performed with a determination and cheerfulness that made the 2011 Timor-Leste EGMA possible: Afriano Lopo Nono, Baltazar Pereira, Belita Lopes, Bendito José Casimiro, Benita Pereira, Brigildo Frans Xavier, Carlito Paixao Neno, Casimiro Sousa de Araujo, Deodata Gomes, Felisberto Pereira, Fidelia de Fatima Soares Correia, Januario Afonso Amaral, Januario da Costa, Januario Nunes Viegas, Julio Conceição de Araujo, Julio Ximenes Ratuk, Leoneto de Araujo Freitas, Lourdes Maria, Lourença Lalisuk, Luis Martins, Luzinha Gomes, Manuela Esposta, Maria de Fatima Soares, Natalina de Sousa Pinta, Paulino de Araujo, Supriyati da Costa Abi and Virgilio de Araujo. Finally and significantly, acknowledgement must be given to the 1226 Timorese students who willingly participated in the survey. Obrigado barak. iii Contents Abstract.....................................................................................................................................................................................................................................i Acknowledgements..........................................................................................................................................................................................................iii Contents....................................................................................................................................................................................................................................iv List Of Figures........................................................................................................................................................................................................................v List Of Tables...........................................................................................................................................................................................................................v Executive Summary...........................................................................................................................................................................................................vi Background......................................................................................................................................................................................................................vi Factors Associated With Better Numeracy Outcomes...........................................................................................................................ix Recommendations......................................................................................................................................................................................................ix Chapter 1: Introduction...................................................................................................................................................................................................1 Chapter 2: Context, Survey Design And Implementation.........................................................................................................................3 Early Mathematics Learning And Egma.........................................................................................................................................................4 Chapter 3: Instrument Design, Sample Design And Execution..............................................................................................................7 Instrument Design And Adaptation.................................................................................................................................................................7 Enumerator Selection And Responsibilities, Training And Field Work.........................................................................................8 The Unexpected Consequences Of Working In The Field..................................................................................................................8 Reliability Of The Instrument.................................................................................................................................................................................9 Sample Design...............................................................................................................................................................................................................10 Chapter 4: Sub-Test Results...........................................................................................................................................................................................11 Sub-Test One: Oral Counting.................................................................................................................................................................................11 Sub-Test Two: Correspondence Counting....................................................................................................................................................12 Sub-Test Three: Number Identification...........................................................................................................................................................13 Sub-Test Four: Quantity Discrimination..........................................................................................................................................................14 Sub-Test Five: Missing Numbers..........................................................................................................................................................................15 Sub-Test Six: Word Problems.................................................................................................................................................................................16 Sub-Test Seven: Addition.........................................................................................................................................................................................17 Sub-Test Eight: Subtraction....................................................................................................................................................................................18 Conclusion........................................................................................................................................................................................................................19 Chapter 5: Language And Mathematics...............................................................................................................................................................21 Languages Used By Enumerators......................................................................................................................................................................22 Language And Performance.................................................................................................................................................................................23 Chapter 6: Factors Associated With Better Numeracy Outcomes.........................................................................................................27 Chapter 7: Conclusions And Recommendations............................................................................................................................................31 References:...............................................................................................................................................................................................................................35 Annex One: School And Student Specific Factors..........................................................................................................................................37 Annex Two: Egma Instrument.....................................................................................................................................................................................51 Annex Three: Word Problems Per-Item Analysis..............................................................................................................................................53 iv LIST OF FIGURES Figure 1: Oral Counting....................................................................................................................................................................................................11 Figure 2: Sub-test 2, Correspondence Counting.............................................................................................................................................12 Figure 3: Sub-test 3, Number Identification........................................................................................................................................................13 Figure 4: Sub-test 4, Quantity Discrimination...................................................................................................................................................14 Figure 5: Sub-test 5, Missing Numbers..................................................................................................................................................................15 Figure 6: Sub-test Six, Word Problems....................................................................................................................................................................16 Figure 7: Sub-test 7, Addition.......................................................................................................................................................................................17 Figure 8: Sub-test Eight: Subtraction.......................................................................................................................................................................18 LIST OF TABLES Table 1: Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part One.....................................................vii Table 2: Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part Two.....................................................vii Table 3: Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part Three..................................................viii Table 4: Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part Four....................................................viii Table 5 Main Education Indicators............................................................................................................................................................................4 Table 6: Phases of Early Mathematics Development.....................................................................................................................................6 Table 7: EGMA Sub-tests and Phases of Early Mathematics Learning................................................................................................7 Table 8: Cronbach’s Alpha, Complete Sample...................................................................................................................................................9 Table 9: Cronbach’s Alpha, Sample with Zero-Scoring Students Removed....................................................................................9 Table 10: Correlation Between Sub-tests..............................................................................................................................................................10 Table 11 Gender and Grade Distribution..............................................................................................................................................................10 Table 12: Sub-test One, Oral Counting...................................................................................................................................................................12 Table 13: Sub-test Two, Correspondence Counting......................................................................................................................................13 Table 14: Sub-test Three, Number Identification.............................................................................................................................................14 Table 15: Sub-test Four, Quantity Discrimination............................................................................................................................................15 Table 16: Sub-test Five, Missing Numbers............................................................................................................................................................16 Table 17: Sub-test Six, Word Problems...................................................................................................................................................................17 Table 18: Sub-test Seven, Addition...........................................................................................................................................................................18 Table 19: Sub-test Eight, Subtraction......................................................................................................................................................................19 Table 20: Frequency of Enumerator and Child Languages Used; Oral Counting .......................................................................22 Table 21: Frequency of Enumerator and Child Languages Used; Correspondence Counting...........................................22 Table 22: Frequency of Enumerator and Child Languages Used; Numbers Identified............................................................22 Table 23: Frequency of Enumerator and Child Languages Used; Words Problems...................................................................22 Table 24: Oral Counting Results by Language...................................................................................................................................................24 Table 25: Correspondence Counting Results by Language......................................................................................................................24 Table 26: Number Identification Results by Language................................................................................................................................25 Table 27: Missing Numbers Results by Language ..........................................................................................................................................25 Table 28: Word Problem Results by Language .................................................................................................................................................25 Table 29: School and Student-Specific Variables ............................................................................................................................................28 Table 30: Effects of School and Student-Specific Variables on Average Scores Part A.............................................................29 Table 31: Effects of School and Student-Specific Variables on Average Scores Part B.............................................................29 Table 32: Per-item Analysis, Sub-test 6...................................................................................................................................................................67 v Executive Summary Background In 2011, the Ministry of Education of Timor-Leste carried out an Early Grade Mathematics Assessment (EGMA) under the World Bank funded Education Sector Support Project1 and with technical support from the World Bank and and Australia’s Program for International Development (AusAID). The purpose of the survey was to set up a baseline for early mathematics skills in Timorese schools. Overall, survey results show cause for concern. Although the most basic mathematics skills, oral counting and correspondence counting, show healthy levels of attainment in the grades tested; other skills, such as quantity discrimination and simple calculations lag significantly behind curriculum expectations. Tables 1-4 give a comparison between curriculum expectations in Timor-Leste and the average performance of students who participated in the EGMA survey. Despite EGMA tasks are arranged as a series of sub-tests in approximately the order of skill required; this does not imply a pre-determined order as students acquire different skills at different rates. Overall, Timorese students demonstrated relatively good ability in the first two sub-tests in EGMA involving number counting (table 1). In oral counting –i.e. the ability to recite numbers in order from memory-- the average Grade 1 student was able to count to 57 in a minute, while Grade 2 students were able to count, on average, to 66 and Grade 3 students to 72. Although average Grade 1 results are above curriculum expectations, Grade 2 and 3 results fall behind. Timorese students also performed strongly in correspondence counting (i.e. counting objects individually). The average Grade 1 student counted 75 objects, Grade 2 students counted 93 objects on average, and Grade 3 students 96 objects. These results are either in line with curriculum expectations or exceed them. Timorese students demonstrated worrisome progress when asked to identify numbers (i.e. students are asked to recognize the printed form of a number ranging from a single digit to four digits) and to discriminate between quantities (i.e. students were required to identify the bigger of two numbers). On average, Grade 1 students were able to recognize 25% of the numbers presented. There is a substantial increase on performance in Grade 2 when students are able to recognize, on average, 65% of the numbers presented; and Grade 3 students are able to recognize 80% of the numbers shown. Student performance on quantity discrimination showed a similar pattern: on average, Grade 1 students identified the bigger of two numbers correctly 54% of the time, Grade 2 students 73% of the time and Grade 3 students 85% of the time. The next skill level is the basic manipulative skills, which allow a student to use numbers to solve both abstract and real-world problems. Table 3 gives the results for these sub-tests. Grade 1 students were able to answer just 19% of the questions correctly in the sub-test on missing numbers, while Grade 2 and 3 students were able to answer only 36% and 50% correctly on average, respectively. Students fared slightly better with word problems with Grade 1 students scoring an average of 28% while Grade 2 students scored 48% and Grade 3 students 62%. The final sub-set of skills is the skills necessary to make a beginning to formal calculation: basic arithmetic skills of addition and subtraction. Although not all of the number combinations tested were expected of Grade 1 at the time of testing (see Table 4 for details), results indicate students start from a low average base and progress slowly in this vital set of skills. Grade 1 students were able to answer only 21% of the addition and 12% of the subtraction questions correctly. Grade 2 students were only able to answer 53% and 30% of addition and subtraction questions correctly, while Grade 3 students answered just 72% and 46% correctly. The highest proportion of autostop cases was found in the numeracy skills requiring higher level reasoning: word problems, addition and subtraction. 1 IDA Grant No. H583-TP vi TABLE 1. Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part One2 Curriculum expectations (based on half-way through school year) EGMA Average Results Mathematics EGMA Task in relation to the tasks Notes Skill Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3 Oral counting: Recite the number names in Up to 20 Up to 100 Beyond 57 66 72 Highest Count, from order, starting at 1, in preferred 100 number memory, in language for counting. Child students were sequence, starting counts until he cannot count on average from 1. further, makes an error, or time able to count expires. The child is then asked how many circles he counted. Timed (to 60 seconds) as a measure of fluency. Rational Counting: Point and count 30 circles in a Up to 20 Beyond 30 Beyond 30 75 93 96 Average Count showing 1:1 5X6 array. number of correspondence items students Timed (to 60 seconds) as a and knowing the were able to measure of fluency. Stop when last number said count without child can’t count further or is the number mistake. at point of error, including that represents counting one circle twice or the total of the two or more circles as one. After collection (the counting, the child is asked how quantity counted many circles he/she counted. TABLE 2. Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part Two Curriculum expectations (based on half-way through school year) EGMA Average Results Mathematics EGMA Task in relation to the tasks Notes Skill Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3 Number Identify numbers ranging from Up to 20 Up to 100 Greater 35% 65% 80% Overall score identification: single digit to 4 digits. Move than 1000 Naming of to next question after 5 second numbers pause. Stop after 3 consecutive errors. Timed (to 60) as a measure of fluency seconds. Quantity Point to the number of the pair Up to 20 Up to 100 Greater 54% 73% 85% Overall score Discrimination: that is greater. Pairs of numbers than 1000 Make comparisons range from single digit to 4 about lower/ digits. higher position in counting order or lesser/ greater in quantity represented by the number. 2 It is worth noting that some curriculum expectations are measured in different units to those in the EGMA survey. Comparisons here are not straightforward vii Timor Leste 2011 EGMA : Baseline Report TABLE 3. Timor Leste EGMA Survey and Corresponding Curriculum Outcomes Part Three Curriculum expectations (based on half-way through school year) EGMA Average Results Mathematics EGMA Task in relation to the tasks Notes Skill Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3 Missing Numbers: Tell the number that is missing Up to 20 Up to 100 Beyond 19% 36% 50% Overall score Discern the from the sequence. Move to 100 pattern of next question after 10 second progression from pause. Stop after 3 consecutive one number errors. to another, in a sequence of Relevant Relevant Relevant numbers and Questions Questions Questions determine the in sub- in sub- in sub- number that is test: test: 4,5,9 test: 7,10 missing from the 1,2,3,6,8 sequence. Word Problems: Solve one step word problems Questions Questions Questions 28% 47% 62% Overall score Apply involving addition or subtraction 1,2 3,4 3,4 mathematics with numbers to 10. concepts and skills in real- world problems, presented orally. TABLE 4. Leste EGMA Survey and Corresponding Curriculum Outcomes Part Four Curriculum expectations (based on half-way through school year) EGMA Average Results Mathematics EGMA Task in relation to the tasks Notes Skill Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3 Addition: 4+5 *   21% 53% 72% Overall score Understand the 8+2 *   meaning of and complete addition 20+4   algorithms. 13+12   11+9 *   Subtraction: 9-5 *   12% 30% 46% Overall score Understand 10-8 *   the meaning of and complete 24-4   subtraction 25-13   algorithms. 20-9 *   Key:  Student expected to process calculation * First half of Grade 1 year, orally presented only viii Executive Summary Factors Associated with Better Numeracy Outcomes A short socio-economic and classroom practices survey accompanied the EGMA survey. In it students were asked to self-report on a number of student- and classroom-specific factors. Several of these were associated with positive, significant outcomes on the EGMA survey: • Students participating in daily mathematics lessons is associated with increases in average scores of between 8 and 17% depending on the sub-test. These sub-tests included number identification, missing numbers, addition and subtraction. • Students working with others during lessons is associated with increases in average scores of between 6 and 8% depending on the sub-test. These sub-tests included number identification, missing numbers, word problems and subtraction. • Students completing mathematics homework is associated with sub-test scores of between 12 and 23% higher depending on the sub-test. These sub-tests included number identification, missing numbers, word problems, addition and subtraction. • The household ownership of a television was a relevant socio-economic indicator and was associated with average scores of around 7% higher than those who did not live in a household with a television. This was the case for the sub-tests on missing numbers and subtraction. Language was found to be a substantial factor associated with performance but the evidence does not support it being considered a causative factor in performance. Students answering in Portuguese were associated with higher scores in some sub-tests and students answering in Bahasa Indonesian were associated with higher scores in others. Research into language practice in the classroom is needed before any conclusions can be drawn from this, particularly in light of the results from the EGRA survey findings on this same subject of language. Overall, the short socio-economic survey was not able to offer substantial guidance on future interventions in part due to the age of the respondents and in part due to its short length. However, classroom practices such as a daily mathematics lesson, working with others and homework had significant, positive associations with mathematics outcomes. A television at home had a positive, significant association with some sub-test results, but rather than a causal link it is likely that it is an instrument for socio-economic status. Recommendations Survey results call for immediate action to ensure that all students in east Timor have access to a numerate future. Specific recommendations to be considered are as follows: Language: • Research is urgently required into language practices in the classroom. Timor Leste is a linguistically rich country and students answered some sub-tests in the EGMA survey in up to three languages, indicating a substantial exposure to language in their community. In order to understand how and why students are learning, it must be known in what language and combination of languages they are actually learning in, rather than supposed to be learning according to curriculum mandates and workbooks provided. Conceptual: • Further research is recommended into the confidence and skills of teachers and students to use effectively the current mathematics student workbooks. These are currently used as textbooks and in Portuguese language. ix Timor Leste 2011 EGMA : Baseline Report • Further research into students’ ability to use the workbooks is required. Based on the findings of the 2011 EGRA (Early Grade Reading Assessment) Survey, and the poor facility of students in the Portuguese language there is substantial cause for concern that students may not be able to use these materials effectively. The ownership of mathematics texts was only self-reported by students at around 80%, despite a national distribution and the ownership of the text was not associated with a statistically significant increase in performance. These results suggest that research into the use, distribution and implementation of these resources is warranted. • The establishment of mathematics benchmarks to monitor classroom improvements. These benchmarks can be used to assess students at-risk of falling further behind by a process of continuous assessment. It is important to consider that initial standards should be considered temporary reference standards since not enough is currently known about when and at what rate Timorese children should progress in learning mathematics. In this sense, benchmark standards should not be seen as high-stakes but an essential piece to monitor mathematics progression in the classroom. One way of doing this would be to use the percentage of zero-score students in selected sub-tests as a marker and track reductions in the shares at least biannually. Monitoring achievements over time will eventually provide more information on the rate and the way in which average mathematics skill develops in Timor. A modified version of the test could be used to screen students during the school year in need of additional support. Teaching , Teachers and the Classroom: • Support for implementing and sourcing mathematics resources in a students’ main language of instruction is suggested. In the light of evidence that supports early learning in the child’s first language, it would follow that priority should be given to the production of mathematics textbooks/workbooks and stimulus pictures in mother tongue. • It is recommended that training of teachers, both pre-service and in-service, be designed to improve pedagogical skills in deepening conceptual understandings of mathematics. It is recommended that Escola Basica Deputy Directors and school inspectors be involved in parallel professional development. • It is highly recommended teachers receive continued ongoing support in using the stimulus, display and manipulative materials that were distributed to all schools in 2010 and were further augmented in 2012. Pedagogy that supports deep understanding through participation, manipulation and interaction is most effective for mathematics learning and teachers need to be supported in achieving this. • Ongoing support in mathematics program planning, teaching methodology and formative assessment is essential. The future of Timorese numeracy is in the hands of its teachers. Improvements in Timorese children’s mathematical skill depends not only on classroom resources but on effective understanding of how to implement curriculum, resources and assessment. • It is recommended that the Ministry should leverage on the national distribution of materials and the EGMA results to identify grade-specific skills that teachers will need pedagogical methods to convey content knowledge to students. Without the pedagogical understanding of how to convey content to students, knowledge transmission cannot occur efficiently. • Research into the confidence and skills of teachers to interpret and teach according to the current mathematics curriculum and teacher guide for mathematics is required. Modified versions of the Classroom Observational Snapshot tool used in other Pacific countries and tailored to the specific needs of the Timorese education system will provide information on the skills of teachers, while teacher surveys will provide elucidation on the thoughts, concerns, backgrounds and skills of these vital components of the Timorese education system. x Executive Summary • Curriculum and teacher support to provide further opportunities for students to work in groups with each other is recommended. Students who engaged in group activities for mathematics learning had scores significantly higher than those who did not. Opportunities to engage in this process provide the foundation not only for improving numerical skills, but gaining language and social skills that will benefit Timorese students. However, teachers need to be supported in their efforts to engage their students in meaningful group activities and this should be a priority in future curriculum and lesson planning. • Teacher encouragement and support to engage in mathematics learning on a daily basis is required. Students who were engaged in mathematics on a daily basis had a significantly higher score in the sub-tests examined than those who did not. Mathematics is a cumulative skill set, each concept building on those learnt previously. Especially in early grades, students are learning patterns of thinking that will be a valuable life-long asset. A healthy 91% of students self-reported that they engaged in mathematics learning daily, however, that leaves a large proportion of students who do not and this statistic may over-report the reality. Ensuring that each child has the opportunity to build on their foundational skills in a consistent daily manner should be a focus of lesson planning and teacher training. Parents and the Community: • Promoting parent education and involvement is needed, not only to interest parents in their children’s mathematics education and build accountability into the Timorese education system at the village level. It is also needed to build parents’ confidence, extending their knowledge and strategies that will enable them to engage with and assist their young children in mathematics. Only a single student out of the 1226 surveyed reported that they received assistance with their mathematics homework. Parental involvement and engagement in mathematics is a key method of improving outcomes for Timorese students. • The evidence in the report suggests that providing opportunities for students to engage in meaningful learning at home will result in benefit to students. Students who reported doing mathematics homework had scores significantly higher than their counterparts. However, this should not be construed as an argument for large amounts of rote-learning to be sent home with children on a regular basis. The importance of parental engagement has been outlined above, however when only a single student in more than 1200 reported having assistance with their homework expectations of Timorese parents must be realistic. On a related note, 39% of students reported that their parents sell things in the market. This statistic indicates that a basic level of numeracy is prevalent in the community and that the community understands the need for numeracy skills. Providing children with the opportunity to engage in active learning at home through meaningful homework activities will have several effects: to encourage and cement concepts learnt in school, to engage parents and the community with the importance of numeracy as a concept and to increase children’s’ confidence in their own mathematical abilities. Examples of such activities may include assisting parents at market or in shopping, making change, counting and arranging items in groups in the home or market, correlating and observing sports scores or engaging in word problems with the assistance of every day objects (a skill many Timorese students struggled with). Encouraging parents to involve their children with every day numeracy opportunities may help demystify mathematical knowledge. xi Timor Leste 2011 EGMA : Baseline Report xii Chapter 1 Introduction In 2011, an Early Grade Mathematics Assessment (EGMA) was carried out in Timor Leste. Results are concerning and call for immediate action. Although student fluency in both oral and rational counting meets curriculum expectations in Grade 1 (though not in Grade 2 or 3), the more formal skills required for more complex numeracy such as quantity discrimination, addition and subtraction show very poor levels of attainment, although there is slow average progression as students advance by grade. The report takes the following format. Chapter 2 will discuss the EGMA survey and the context of Timor Leste. Chapter 3 will discuss the implementation, design and sampling of the survey. Chapter 4 will present the results of the sub- tests and Chapter 5 will discuss the effect of language on student performance. Chapter 6 will analyze the factors associated with student outcomes. Chapter 7 concludes with recommendations for the future of mathematical education in Timor. 1 Timor Leste 2011 EGMA : Baseline Report 2 Chapter 2 Context, Survey Design and Implementation A decade after the restoration of independence, Timor-Leste is no longer a post-conflict least-developed country. It is now considered a country with a medium human development standing of 120 of the 169 countries in the 2010 Global Human Development Report (Timor-Leste Human Development Report, 2011). The rapid development of Timor’s economy and democracy is matched by the expansion of its education system: particularly in the early grades. Population growth is high and near-universal enrollment has been achieved. Net enrolment figures in primary education have increased from 68% to more than 90% from 2004/05 to 2011. There were rapid reductions in grade-to-grade dropout from 11-12% in 2008-09 to 3-4% in 2010. As identified in key sector diagnostics such as the Early Grade Reading Assessment (EGRA) and the National Education Strategic Plan itself, the sector continues to face major challenges in the quality and efficiency of education, including continued high levels of repetition (see Table 5). 3 Timor Leste 2011 EGMA : Baseline Report TABLE 5. Main Education Indicators 2004-05 2005-06 2006-07 2007-08 2008-09 2010 2011 Primary Total enrollment 157,516 169,384 189,398 206,476 218,674 230,496 238,936 Gross intake rate (%) 94.4 92.7 117.0 133.2 126.4 144.7 121.1 Gross enrollment rate (%) 99.6 102.7 109.5 113.8 116.1 127.4 129.1 Net enrollment rate (%) 68.0 70.7 75.0 80.3 84.6 93.0 94.3 Primary Completion rate (%) - - 86.3 92.7 80.2 77.8 76.9 Repetition rate (%) 15.0 13.5 13.9 19.4 17.6 17.9 - Pre-Secondary Total enrollment 60,610 60,618 61,270 No reliable Data Gross enrollment rate (%) 80.1 77.1 75.3 Total Basic Education Total enrollment 279,284 291,114 300,206 No reliable Data Gross enrollment rate (%) 105.8 112.3 112.8 Secondary Total enrollment 22,874 28,379 29,409 No reliable Data Gross enrollment rate (%) 47.7 57.9 59.4 Enrolment rates in 2010 and 2011 use 2010 Census data of school age children as denominator while earlier indicators are based on 2004 population projection. Source: Timor-Leste Ministry of Education EMIS 2011 Timor Leste faces challenges in ensuring access to quality education to all children. Gross enrolment rates are above 100%, indicating that the country has the physical capacity to provide educational services to all children of school age. Meanwhile, net enrolment rates remain below 100%, pointing at part of the school age population that remains excluded. Other challenges facing the system include institutional capacity, efficiency, with high repetition and drop-out rates, and quality of education and learning outcomes. To tackle these challenges, the government of Timor-Leste has developed and launched in November 2010 its National Education Strategic Plan (NESP). Amongst its key Priority Programs is “Improving Teaching Quality”, with the specific short-term objective of “Improving the quality of education by substantially increasing the quality of teaching”. However, as the NESP recognizes, the country lacks regular, standardized, practical tools to establish baselines and targets, assess progress and target interventions in this endeavour of better quality of education. In response to this, the Ministry of Education, with financial support from the Education For All Fast Track Initiative (EFA- FTI, now Global Partnership for Education), provided stimulus and manipulative materials for mathematics. These were targeted for supporting the teaching of base competencies in the mathematics curriculum in the early primary grades, were supplied to all primary schools. A further scaling up of these materials is taking place at present with the delivery period likely to take place in mid 2012. It is in this context of an increased focus on monitoring of education quality in early grades, that the Ministry of Education conducted a first Early Grade Mathematics Assessment (EGMA) in 2011. Early Mathematics Learning and EGMA It is clear that not all children learn mathematics using the same progression or at the same pace. For the purposes of this report we are arranging some of the basic skills of early mathematics learning into four phases, each of which requiring further skill or knowledge and each of which a child will progress through individually. Children may not necessarily learn in the same process or order. Table 6 gives these four phases of early mathematics development. 4 Context, Survey Design and Implementation TABLE 6. Phases of Early Mathematics Development Phase Name Key Skills Phase Zero Development Counting: reciting numbers from memory, object counting, numeral recognition Phase One Discernment Discernment between actual quantities (groups) and patterns Phase Two Manipulation Ability to use numbers in every day life to solve simple problems Phase Three Calculation Ability to use numbers to perform simple abstract calculations Phase Zero we can describe as a nascent Phase of developing numeracy. Children learn rote, oral counting (e.g. starting from one and continuing in order). Children recognize patterns in the ordering of numbers. They then learn to apply this ordering of numbers to objects in correspondence counting: the ability to number objects individually without repetition, missing objects or numbers and to know how many objects there are in a group. Repeated practice, through song, through number games, through other classroom activities results in a child’s automaticity in these early concepts of mathematics: an important step if students are to apply them to later Phases of mathematics development. In this developmental Phase, students begin to recognize numerals in print and to identify them with meaning. Using the fundamental skills learnt in Phase Zero, most children proceed on to the next phase. This is a discernment phase. Children begin to discern the differences between groups in a mathematical way: more and less, many and few, bigger and smaller. They begin to discern numerical patterns and anticipate them. Phase Two is a manipulative phase. Children learn to manipulate mathematical concepts. Understanding the concept of numerical patterns, they learn to manipulate them and to solve simple problems in the context of their every day lives. Phase Three is when children develop the ability to make formal calculations using mathematical formulae. The Early Grade Mathematics Assessment (EGMA) instrument was originally designed by the Research Triangle Institute (RTI) in 2008. The EGMA is an individual survey administered as an interview and is suitable for students whose reading abilities are not yet well-developed enough for them to sit a formal test. It gives information on accuracy and fluency of skills and is based on these earliest Phases of mathematical development and the foundational conceptual understandings, which are a topic of instruction in schools. The EGMA interview takes about 20 minutes per child to administer. EGMA has been adapted and implemented in a number of developing countries with the aim of providing baseline data for mathematics achievement in the early grades. It requires adaptation in each country it is used to reflect the standards and contextual setting for the particular country in which it is to be implemented. In order to determine the fluency with which a student was able to complete a task, a measure of their automaticity, some of the sub-tests were timed. The purpose of this is to determine the level at which a student has internalized the skills: an important component for performing higher order skills at later Phases of mathematical development. The timed data was intended to be used to compare students by “per minute” measures. However, there were unexpected problems in the field and this will be discussed in the next chapter. The Timor Leste EGMA presented students with 8 tasks. Table 7 presents the tasks and corresponding Phases. The three tasks in Phase Zero were oral counting, correspondence counting and number identification. Oral counting required students to count from one as high as they could in one minute, in order. Students were stopped when they had made a mistake and were asked to count in the language they were most familiar with. The second sub-test, correspondence counting, presented students with a page of circles, evenly spaced and ordered as a grid. Students were asked to count as many circles as they could in one minute. Students were asked to count in the language they were most familiar with and they were stopped when they made a mistake. The number identification sub-test (Sub-test 3) presented students with a page of numbers and asked to identify as many as they could in 60 seconds. They were stopped if they had three consecutive incorrect answers or paused for more than five seconds. 5 Timor Leste 2011 EGMA : Baseline Report TABLE 7. EGMA Sub-tests and Phases of Early Mathematics Learning Phase Name EGMA Sub-test Phase Zero Development Sub-tests 1, 2 and 3: Oral counting, correspondence counting, number identification Phase One Discernment Sub-tests 4 and 5: Quantity Discrimination and Missing Numbers Phase Two Manipulation Sub-test 6: Word Problems Phase Three Calculation Sub-tests 7 and 8: Addition and Subtraction The sub-tests corresponding to Phase One were Sub-tests 4 and 5. Sub-test four was untimed. Students were presented with ten sets of two numbers and asked to identify the bigger of the two in each case. Children had access to counters as visual and tactile stimuli, if they wished. The enumerator provided two examples on how to use them in the preamble to the sub-test. The fifth sub-test presented students with a series of numbers, one of which was missing. They were asked to identify the missing number. The sub-test was untimed and there were ten items in total. Phase Two aimed to identify children’s skills in manipulating numbers. The sixth sub-test asked students a series of simple word problems pertaining to familiar manipulations in numbers they might make in their everyday lives. One such question was, “Maria has four bananas. Her mother has five more. How many do they have now?” The questions were read to the child in the language most familiar to the child, although the default language was Tetum. There were four items in total. Phase Three aimed to identify children’s skills in formal arithmetic. Sub-test 7 presented students with 5 addition problems presented to the child in print. Sub-test 8 presented students with 5 subtraction problems presented in print. These arithmetic problems were simple one and two-digit problems. Students had access to counters as visual and tactile stimuli if required and two examples before each sub-test to show them how to use them. The children had two minutes to complete each sub-test and were stopped if they made three consecutive errors. 6 Chapter 3 Instrument Design, Sample Design and Execution Instrument Design and Adaptation In April 2011, initial adaptation of the EGMA design commenced. Eight volunteer university students from Dili trialed the instrument and advised on the clarity of instructions, usage of symbols to assist in quick referral to interview instructions, cultural consistency and wording, layout of the interview protocol, ease of administration with students, timing for administration of the interview, usage and placement of the manipulative materials and minimizing chances of errors in recording. Following agreement from the Ministry to proceed with the EGMA program, several workshops with representatives from curriculum and assessment directorate took place. The instrument was trialed with 24 school students in a Dili school. Adapting the EGMA instrument gave consideration to both the design and individual tasks. The test items were adapted with a view to their consistency with the expectations of the Mathematics Curriculum of Timor-Leste. Word problems were adapted to ensure the content was suitable to the context and the language structure easily accessible to children of ages in Grades 1, 2 and 3. The linguistic environment of Timor Leste is complex and required some careful design. The protocol was produced in Tetum, the national language and one required to be used for 75% of contact hours in Grades 1 and 2 by the syllabus. The NESP and Mother Tongue policy indicate that a children’s first language is to be the language of instruction in the early years of primary school. Enumerators (interviewers) were instructed to use local language or call an assistant for translation where necessary if the initial introductory conversation with the child revealed that the local language would be preferable to optimize the child’s full understanding of the tasks. However for the tasks where numbers and mathematical operations were read to the child, the language used would be Portuguese, consistent with language used in teacher in-servicing and current mathematics student work books. Interpretation of these higher-order mathematical tasks should be considered in the light of this policy. Although students should be familiar with the mathematical concepts in Portuguese language, the EGRA 2011 Report indicated that many students struggle with the most basic understanding of oral Portuguese. This may explain some of the poor results exhibited by students and further suggests that research is required into the languages used for instruction. 7 Timor Leste 2011 EGMA : Baseline Report A second instrument was developed to accompany the EGRA survey. The student context questionnaire gathered information about conditions at home and school. But also about classroom practices that may impact on learning, as well as the status and impact of national initiatives implemented in schools. ‘Home’ questions included those related to assistance at home with mathematics homework, nutrition (daily breakfast), money-handling experiences of the family and media inputs for the family via television and radio. ‘School’ questions related to the students’ participation in pre-school, self-images as mathematics learners, frequency of mathematics lessons, experience with text/workbooks, use of manipulative materials in learning mathematics, teacher pedagogical practices (group work for learning), homework and the school feeding program. The initial Ministry of Education working group adapted the student questionnaire to reflect the current Ministry initiatives that the members considered should be positively impacting on development of student competencies. These included the Ministry of Education’s provision of text books and manipulative materials for mathematics, the promotion of collaborative group learning during teacher in-services and the school feeding (“merenda escolar”) program. The protocol was further adapted with input from members of the Directorate of Curriculum and Assessment who were delegated as associates in the training, mentoring and monitoring of the enumerators. Enumerators also provided input during their training program, which resulted in the final instrument. Due to this adaptation, the results are uniquely for Timor-Leste and not intended for international comparison. Both strengths and weaknesses indicated in the findings should be viewed primarily as opportunities for review of and planning for Ministry of Education policies and practices. Enumerator selection and responsibilities, training and field work Enumerators were required to have completed secondary school and to have commenced tertiary education in Dili. The enumerator teams contained at least one member who was fluent in the local language(s) in the respective allocated sub-districts for interviewing. There were 28 enumerators selected. The enumerators participated in 6 days of training in Dili. The training period included practice interviews in Dili schools and was fundamental to the ensuring efficiency of the EGMA implementation. Practice was followed by input from trainers and the Ministry training and monitoring team. The training period concluded with certification of interviewers as suitable for the EGMA implementation. Following training as enumerators, seven interview teams were formed, with three enumerators and a designated supervisor in each team. In every school visited, three enumerators each conducted an interview with one boy and one girl from each of Grades 1, 2 and 3. Each team was responsible for interviews in two or three Sub Districts over a two-week period. The Unexpected Consequences of Working in the Field Unfortunately, on analysis of the data, it was found that the timing data, which is used for determining student fluency, was unreliable. Out of range or unobservable responses were seen to be above 1%, which suggests that unidentifiable errors were also unacceptably prominent in the data. As a result and to ensure accuracy, the decision was taken to analyze the data as total scores rather than as fluency measures as intended by the EGMA survey. 8 Instrument Design, Sample Design and Execution Reliability of the Instrument In order to determine the reliability of the instrument, Cronbach’s alpha was calculated. As a rule of thumb, the minimum acceptable coefficient is 0.7. Table 8 gives the results for the entire sample and the instrument appears reliable with an overall coefficient of 0.89. However, large numbers of zero-scoring students can inflate measures of reliability such as Cronbach’s alpha and in some sub-tests, zero scoring students were in excess of 20%. Table 9 gives the reliability statistics for the sample where students scoring zero on one or more sub-tests were removed. The reliability of the instrument is still very good with a Cronbach’s alpha of 0.86. TABLE 8. Cronbach’s Alpha, Complete Sample Item N Sign Item-test Item-rest Inter-item correlation Alpha Numbers Counted 1226 + 0.71 0.62 0.51 0.88 Circles Counted 1224 + 0.56 0.43 0.56 0.90 Number Identification (as a percentage) 1224 + 0.88 0.83 0.47 0.86 Quantity Identification (as a percentage) 1225 + 0.76 0.67 0.50 0.88 Missing Number (as a percentage) 1221 + 0.77 0.68 0.50 0.87 Word Problems (as a percentage) 1213 + 0.75 0.66 0.50 0.88 Additions (as a percentage) 1219 + 0.86 0.81 0.47 0.86 Subtractions (as a percentage) 1208 + 0.73 0.63 0.51 0.88 Test Scale 0.50 0.89 TABLE 9. Cronbach’s Alpha, Sample with Zero-Scoring Students Removed Item N Sign Item-test Item-rest Inter-item correlation Alpha Numbers Counted 1225 + 0.73 0.62 0.43 0.84 Circles Counted 1223 + 0.62 0.43 0.49 0.87 Number Identification (as a percentage) 1182 + 0.86 0.79 0.39 0.81 Quantity Identification (as a percentage) 1210 + 0.77 0.65 0.41 0.83 Missing Number (as a percentage) 1007 + 0.77 0.66 0.42 0.84 Word Problems (as a percentage) 965 + 0.63 0.48 0.46 0.86 Additions (as a percentage) 861 + 0.81 0.70 0.42 0.84 Subtractions (as a percentage) 632 + 0.72 0.60 0.44 0.85 Test Scale 0.43 0.86 9 Timor Leste 2011 EGMA : Baseline Report Correlation between sub-tests is given in Table 10. Most sub-tests are well correlated with one another, however the second sub-test, correspondence counting or “circles counted” has a low degree of correlation with the other sub- tests. Possible reasons for this are given in Chapter Four. TABLE 10. Correlation Between Sub-tests Number Quantity Missing Word Numbers Circles Additions Subtractions All Identification Identification Number Problems Counted Counted (as %) (as %) (as %) (as %) (as %) (as %) Numbers Counted 1 Circles Counted 0.43 1 Number Identification (as %) 0.64 0.45 1 Quantity Identification (as %) 0.46 0.29 0.68 1 Missing Number (as %) 0.46 0.29 0.65 0.53 1 Word Problems (as %) 0.41 0.29 0.58 0.49 0.52 1 Additions (as %) 0.50 0.35 0.74 0.64 0.62 0.66 1 Subtractions (as %) 0.37 0.23 0.53 0.46 0.54 0.56 0.68 1 Overall, despite some difficulties with the linguistic diversity of Timor and execution of the timed sub-tests, the instrument is reliable and offers an interesting insight into the learning of mathematics in the early grades of Timor Leste. Sample Design The sample was a stratified random sample using the EMIS schools list as a sample frame. The sample was stratified by district and school type. All 13 districts were represented in the sample. The target population was students in Grades 1-3 in schools implementing the national mathematics curriculum. Each grade sampled around 400 students and 1226 students in 65 schools were sampled, including 73 students from the piloting of the instrument. Gender parity was close to unity. Table 11 gives the sample frequencies by grade and gender. TABLE 11. Gender and Grade Distribution Gender Gender Total 1 2 3 Male 206 203 200 609 Female 206 201 210 617 Total 412 404 410 1,226 10 Chapter 4 Sub-test Results This chapter will present sub-test results by grade, by gender and overall. Special attention should be paid not just to average progression, but also median progression. Students who are particularly able can inflate average scores, but the median gives the score of the student on the 50th percentile (the middle student) and is a good counterbalance to this. Sub-test One: Oral Counting In this sub-test, students were asked to begin at one and count onwards as high as they could in one minute. As discussed in the previous chapter, timing data was dropped and the scores are analyzed in their raw form.Table 12 gives the results of this sub-test. Overall, results show average grade progression that is statistically significant. Only one student in Grade One was unable to count at all. On average, students in Grade One were able to count to 39 and the median student was also able to count to 39. Grade Two students were able to count to 56 on average and the median child counted to 57. Grade Three students were able to count to 66 on average and the median child in Grade Three counted to 69. Boys and girls performed very similarly and there were no significant differences between them. Figure 1 gives a graphical presentation of these results. FIGURE 1: Oral Counting Grade 3 Grade 2 Grade 1 0 10 20 30 40 50 60 70 80 Median Numbers counted in one minute Mean 11 Timor Leste 2011 EGMA : Baseline Report TABLE 12. Sub-test 1, Oral Counting Grade Mean SD Min Max Median N LCL UCL 1 39 20 0 100 39 412 37 40 2 56 20 1 100 57 404 54 57 Overall 3 66 20 1 100 69 410 65 68 Total 53 23 0 100 56 1226 52 55 1 39 20 1 100 39 411 37 41 Zero Scores 2 56 20 1 100 57 404 54 57 Removed 3 66 20 1 100 69 410 65 68 Total 54 23 1 100 56 1225 52 55 1 39 20 0 100 39 206 37 42 2 54 19 9 100 56 201 51 57 Girls 3 67 19 7 100 70 210 65 70 Total 54 23 0 100 56 617 52 55 1 38 21 3 100 35 206 35 41 2 57 21 1 100 57 203 54 60 Boys 3 65 21 1 100 69 200 63 68 Total 53 24 1 100 56 609 51 55 Sub-test Two: Correspondence Counting In this sub-test students were asked to count a series of printed circles on a page: as many as they could in one minute. Table 13 gives the results of this sub-test. Like sub-test 1, students showed good results with Grade 1 students on average able to count 25 circles, Grade 2 students, 29 circles and Grade 3 students, 29 circles also. The median student in each grade counted 30 circles. Only one student was unable to count any circles. Although there is not evidence of strong grade progression in this sub-test, this may be due to the nature of its presentation, rather than a particular lack in correspondence counting skills in older students. There were 30 circles on the page presented to students. Although some students counted up to 60 circles successfully (the page twice), it is possible some enumerators stopped students after they completed the page or that some students may have made a mistake when re-starting the page to count for the second group of 30 circles and been stopped. This may also account for this sub-test’s relatively weak correlation with the other sub-tests discussed in Chapter Three. Figure 2 gives the frequency distribution of the sub-test results overall. As can be seen, an unexpectedly disproportionate number of children were stopped, or stopped, at exactly 30 circles (over 78% of students). Less than 1% of students continued counting beyond 30 circles. FIGURE 2: Sub-test 2, Correspondence Counting 80 70 60 50 Percent 40 30 20 10 0 0 20 40 60 Circles counted 12 Sub-Test Results TABLE 13. Sub-test 2, Correspondence Counting Grade Mean SD Min Max Median N LCL UCL 1 25 8 0 60 30 410 24 26 2 29 4 5 46 30 404 28 29 Overall 3 29 3 2 31 30 410 29 30 Total 28 6 0 60 30 1224 27 28 1 25 8 1 60 30 409 24 26 Zero Scores 2 29 4 5 46 30 404 28 29 Removed 3 29 3 2 31 30 410 29 30 Total 28 6 1 60 30 1223 27 28 1 26 8 1 60 30 206 25 27 2 29 5 5 46 30 201 28 29 Girls 3 29 3 2 30 30 210 29 30 Total 28 6 1 60 30 617 27 28 1 24 8 0 40 30 204 23 26 2 29 4 5 30 30 203 28 29 Boys 3 29 3 5 31 30 200 29 30 Total 28 6 0 40 30 607 27 28 Sub-test Three: Number Identification In this sub-test, students were given 12 numbers and asked to identify them. The numbers ranged from one figure up to four figures. The numbers were 2, 5, 9, 13, 10, 18, 65, 50, 97, 104, 468, 6430. Table 14 presents the results of this sub- test as a percentage of correct answers. Evidence of grade progression is strong. Grade One students recognized, on average, 35% and the median child 33%. Grade Two students recognized 65% on average and the median student recognized 75%. Grade Three students recognized 80% on average and the median child recognized 83%. Figure Three presents the results of this sub-test. Only 3% of students were unable to answer this sub-test. Most of these students were in Grade One, where 9% of students did not recognize any numbers and only 1.2% in Grade Two. However, the fact that 9% of Grade One students unable to recognize any of 12 numbers is concerning. Boys scored consistently higher than girls in all grades, however this difference is not significant. FIGURE 3: Sub-test 3, Number Identification Grade 3 Grade 2 Grade 1 0% 20% 40% 60% 80% 100% Median % of numbers correctly identified Mean 13 Timor Leste 2011 EGMA : Baseline Report TABLE 14. Sub-test 3, Number Identification Grade Mean SD Min Max Median N LCL UCL 1 35% 23% 0% 100% 33% 410 33% 37% 2 65% 23% 0% 100% 75% 404 63% 67% Overall 3 80% 17% 0% 100% 83% 410 79% 82% Total 60% 28% 0% 100% 67% 1224 58% 62% 1 38% 21% 0% 100% 33% 373 36% 40% Zero Scores 2 66% 22% 0% 100% 75% 399 64% 68% Removed 3 80% 17% 0% 100% 83% 410 79% 82% Total 62% 26% 0% 100% 67% 1182 61% 64% 1 34% 22% 0% 100% 33% 206 31% 37% 2 63% 23% 0% 100% 67% 201 59% 66% Girls 3 80% 18% 0% 100% 83% 210 77% 82% Total 59% 28% 0% 100% 67% 617 57% 61% 1 36% 24% 0% 100% 33% 204 33% 39% 2 67% 22% 0% 100% 75% 203 64% 70% Boys 3 81% 16% 0% 100% 83% 200 79% 83% Total 61% 28% 0% 100% 67% 607 59% 63% Sub-test Four: Quantity Discrimination In this sub-test, students were asked to identify the larger of two numbers. There were ten items in total and results are presented in Table 15 as a percentage. The complete sub-test is presented in Annex 2: EGMA Instrument. Results in this sub-test were showed reasonable grade progression. Grade One students scored an average of 54% and the median child 50%. Grade Two students scored an average of 73% and the median child 80%. Grade Three students scored an average of 85% and the median child in that grade scored 90%. The proportion of zero scoring students was very low at only 1.2%. Most of these were in Grade One where 3% of students were unable to answer any of the questions correctly. Boys performed consistently better than girls in this sub-test and there was a significant difference between the genders. The reasons for this are not clear and bear further investigation. Figure 4 presents the gender gap between students graphically. FIGURE 4: Sub-test 4, Quantity Discrimination Grade 3 Grade 2 Grade 1 0% 20% 40% 60% 80% 100% Boys % of items correctly identified Girls 14 Sub-Test Results TABLE 15. Sub-test 4, Quantity Discrimination Grade Mean SD Min Max Median N LCL UCL 1 54% 24% 0% 100% 50% 412 51% 56% 2 73% 23% 0% 100% 80% 403 70% 75% Overall 3 85% 17% 0% 100% 90% 410 84% 87% Total 70% 25% 0% 100% 80% 1225 69% 72% 1 55% 22% 0% 100% 50% 399 53% 57% Zero Scores 2 73% 23% 0% 100% 80% 402 71% 75% Removed 3 86% 16% 0% 100% 90% 409 84% 87% Total 71% 24% 0% 100% 80% 1210 70% 73% 1 51% 24% 0% 100% 50% 206 48% 54% 2 68% 24% 0% 100% 70% 200 65% 71% Girls 3 84% 18% 0% 100% 90% 210 81% 86% Total 68% 26% 0% 100% 70% 616 66% 70% 1 56% 23% 0% 100% 60% 206 53% 59% 2 77% 20% 0% 100% 80% 203 74% 80% Boys 3 87% 15% 0% 100% 90% 200 85% 89% Total 73% 24% 0% 100% 80% 609 71% 75% Sub-test Five: Missing Numbers This sub-test asked students to complete a series of missing numbers. Table 16 gives the results of this sub-test in percentage form. There were ten items overall. Grade progression begins from a low base. Grade One students score only 19% on average with the median child scoring 20%. Grade Two students score 36% on average and the median child 30%. Grade Three students score just 50% on average and the median Grade 4 child 50%. There was a high incidence of zero-scoring students in this sub-test with 18% of students overall unable to answer any questions. The highest proportion was, as expected in Grade 1 with 30% of students unable to answer. However, Grade 2 and 3 had substantial proportions of students scoring zero: 14% and 9% respectively. Boys scored marginally higher than girls in this sub-test, however, these results were not significant. Figure 5 gives the proportion of zero-scoring students by grade in graphical format. FIGURE 5: Sub-test 5, Missing Numbers Grade 3 Grade 2 Grade 1 0% 10% 20% 30% 40% 50% % of correct responses Proportion of zero scoring students Median Mean 15 Timor Leste 2011 EGMA : Baseline Report TABLE 16. Sub-test 5, Missing Numbers Grade Mean SD Min Max Median N LCL UCL 1 19% 18% 0% 100% 20% 407 17% 21% 2 36% 25% 0% 100% 30% 404 34% 39% Overall 3 50% 28% 0% 100% 50% 410 47% 52% Total 35% 27% 0% 100% 30% 1221 33% 36% 1 27% 16% 0% 100% 20% 286 25% 29% Zero Scores 2 42% 22% 0% 100% 40% 349 40% 44% Removed 3 55% 24% 0% 100% 60% 372 52% 57% Total 42% 24% 0% 100% 40% 1007 41% 44% 1 19% 18% 0% 100% 20% 203 16% 21% 2 35% 25% 0% 100% 30% 201 32% 39% Girls 3 49% 27% 0% 100% 50% 210 45% 53% Total 35% 27% 0% 100% 30% 614 32% 37% 1 19% 18% 0% 100% 20% 204 16% 21% 2 37% 25% 0% 100% 30% 203 34% 41% Boys 3 50% 28% 0% 100% 50% 200 46% 54% Total 35% 27% 0% 100% 30% 607 33% 37% Sub-test Six: Word Problems This sub-test presented students with 4 problems, administered orally. The problems are presented in the instrument, available in Annex 2. A complete item analysis is given in Annex 3. The problems were simple calculations such as the children may have been familiar with from their every day lives in household transactions. Table 17 presents the results of this sub-test in percentage form. Grade One students answered on average 28% of the problems correctly and the median student 25%; grade progression is very slow for such simple problems. Grade Two students answered just 47% of problems correctly on average and the median child 50%. Grade Three students answered just 61% on average correctly and the median child showed no progression from Grade Two, answering just 50% correctly. Although the average scores in each grade show significant differences, the lack of median progression is deeply concerning. Students unable to answer any questions correctly make up a substantive proportion of this sample, 20% overall. The majority are in Grade One where 40% of students are unable to answer any of the questions, while in Grade Two 17% are unable to answer any questions. Grade Three indicates that the bottom cohort of students is making some progress with only 5% of students unable to answer any questions correctly. Figure 6 gives these results graphically. Boys perform statistically significantly better than girls in this sub-test overall. By grade, the results are not significant in Grade One, but are significant in Grades Two and Three. FIGURE 6: Sub-test 6, Word Problems Grade 3 Grade 2 Grade 1 0% 10% 20% 30% 40% 50% 60% 70% 80% Numbers counted in one minute Proportion of zero scoring students Median Mean 16 Sub-Test Results TABLE 17. Sub-test 6, Word Problems Grade Mean SD Min Max Median N LCL UCL 1 28% 28% 0% 100% 30% 404 26% 31% 2 47% 29% 0% 100% 50% 400 44% 50% Overall 3 62% 27% 0% 100% 50% 409 59% 64% Total 46% 31% 0% 100% 50% 1213 44% 48% 1 47% 20% 0% 100% 50% 242 45% 50% Zero Scores 2 57% 22% 0% 100% 50% 333 54% 59% Removed 3 65% 24% 0% 100% 80% 390 62% 67% Total 58% 24% 0% 100% 50% 965 56% 59% 1 25% 26% 0% 100% 30% 201 21% 28% 2 43% 28% 0% 100% 50% 198 39% 47% Girls 3 57% 27% 0% 100% 50% 209 53% 60% Total 42% 30% 0% 100% 50% 608 39% 44% 1 32% 29% 0% 100% 30% 203 28% 36% 2 52% 30% 0% 100% 50% 202 48% 56% Boys 3 67% 27% 0% 100% 80% 200 63% 70% Total 50% 32% 0% 100% 50% 605 47% 53% Sub-test Seven: Addition This sub-test presented children with five simple addition problems (4+5, 8+2, 20+4, 13+12, 11+9) to be completed within two minutes. Students were stopped if they answered three items incorrectly. Table 18 presents the results of this sub-test. Grade progression is significant overall, but begins from a very low base. In Grade One, the average student answers 21% of the questions correctly, but the median child cannot answer any. In Grade 2 substantial gains are made. The average student answers 53% of the questions correctly and the median child 60%. In Grade 3, the average child can answer 72% of the questions correctly and the median child 80%. The proportion of zero-scoring students is substantial, 29% overall. Most of these are in Grade One, where 55% of students are unable to answer any questions. Despite the good average gains in Grade Two a concerning 23% of students, nearly one in four, are unable to answer any questions. In Grade 3, 10% of students are unable to answer any questions. Figure 7 gives these results graphically. Results on basic arithmetic problems raise substantial concerns about students’ ability to cope with the curriculum requirements in higher grades. Boys performed statistically significantly better than girls in this sub-test. FIGURE 7: Sub-test 7, Addition Grade 3 Grade 2 Grade 1 0% 20% 40% 60% 80% 100% % of correct items Proportion of zero scoring students Median Mean 17 Timor Leste 2011 EGMA : Baseline Report TABLE 18. Sub-test 7, Addition Grade Mean SD Min Max Median N LCL UCL 1 21% 29% 0% 100% 0% 409 18% 24% 2 53% 37% 0% 100% 60% 402 49% 56% Overall 3 72% 32% 0% 100% 80% 408 69% 75% Total 49% 39% 0% 100% 60% 1219 46% 51% 1 47% 25% 0% 100% 40% 183 44% 51% Zero Scores 2 68% 26% 0% 100% 80% 310 66% 71% Removed 3 80% 23% 0% 100% 80% 368 78% 82% Total 69% 28% 0% 100% 80% 861 67% 71% 1 18% 27% 0% 100% 0% 203 14% 22% 2 48% 37% 0% 100% 60% 200 43% 53% Girls 3 66% 34% 0% 100% 80% 209 62% 71% Total 44% 39% 0% 100% 40% 612 41% 47% 1 24% 30% 0% 100% 0% 206 20% 29% 2 58% 36% 0% 100% 60% 202 53% 63% Boys 3 78% 29% 0% 100% 80% 199 74% 82% Total 53% 39% 0% 100% 60% 607 50% 56% Sub-test Eight: Subtraction The final sub-test presented students with five simple subtraction problems (9-5, 10-8, 24-4, 25-12, 20-9) to be completed in two minutes. Table 19 gives the results of this sub-test. Grade progression is significant, but starts from a low base and is very slow. Grade One students are able to answer just 12% on average, while the median student can answer none. Grade Two students are able to answer 30% on average and the median student 20%. Grade Three students are able to answer 46% on average and the median student 40%. Boys performed significantly better than girls in this sub-test. The proportion of zero-scoring students is very substantial, 48% of students overall. A majority of Grade One students, 67% were unable to answer a single question in this sub-test and 46% of Grade Two students were still unable to answer any questions. Grade Three students presented the most worrying statistic with almost 1 in 3, 30% unable to answer any of the questions. Figure 8 presents this graphically. FIGURE 8: Sub-test 8: Subtraction Grade 3 Grade 2 Grade 1 0% 20% 40% 60% 80% % of correct items Proportion of zero scoring students Median Mean 18 Sub-Test Results TABLE 19. Sub-test 8, Subtraction Grade Mean SD Min Max Median N LCL UCL 1 12% 21% 0% 100% 0% 405 10% 14% 2 30% 35% 0% 100% 20% 398 27% 34% Overall 3 46% 38% 0% 100% 40% 405 42% 49% Total 29% 35% 0% 100% 20% 1208 28% 31% 1 38% 21% 0% 100% 40% 132 34% 41% Zero Scores 2 56% 28% 0% 100% 60% 215 53% 60% Removed 3 65% 28% 0% 100% 60% 285 62% 68% Total 56% 28% 0% 100% 60% 632 54% 59% 1 10% 20% 0% 100% 0% 202 7% 13% 2 25% 32% 0% 100% 0% 197 20% 29% Girls 3 39% 36% 0% 100% 40% 208 34% 44% Total 25% 32% 0% 100% 0% 607 22% 27% 1 14% 22% 0% 100% 0% 203 11% 17% 2 36% 37% 0% 100% 20% 201 31% 41% Boys 3 53% 38% 0% 100% 60% 197 48% 58% Total 34% 36% 0% 100% 20% 601 31% 37% Conclusion Although Timor Leste students demonstrate reasonable abilities to recognize and discern numbers with solid results and steady, though slow grade progression in oral counting, correspondence counting, number identification and quantity discernment; higher order early mathematics skills are lacking, progress from a low base and very slowly. The proportion of students with no demonstrable skills in these manipulative and arithmetic skills (missing numbers, word problems, addition and subtraction) is worryingly high and low levels of attainment even amongst those students demonstrating some skill puts in doubt their ability to cope with a more complex mathematical curriculum in later years. 19 Timor Leste 2011 EGMA : Baseline Report 20 Chapter 5 Language and Mathematics In the linguistically diverse environment of Timor Leste, curriculum suggests that students are taught in both Tetum and Portuguese to a ratio of 3:1 contact hours in each language respectively. The workbooks that were distributed as part of the national intervention in mathematical materials are in Portuguese. The EGMA instrument used in Timor Leste was primarily delivered in Tetum with sub-tests 4 (quantity discrimination), 7 (addition) and 8 (subtraction) delivered in Portuguese so as to remain in line with the material presented to students in the classroom. Enumerators had language support in the form of a translator or member of the team who was proficient in local language. These were called upon when in the conversation with the student preliminary to the survey it became clear that students needed to be tested in their local language. Information on languages used by enumerators and students was collected for sub-tests 1, 2, 3, 5 and 6. It is this information that is the subject of the analysis in this chapter. Some children and enumerators used multiple combinations of languages in each sub-test. Some children and enumerators used up to three languages for an individual sub-test and these different combinations are difficult to analyze individually with any clarity. To counteract this possible source of confusion, we have divided students and enumerators into groups: those who used Tetum in a sub-test and those who did not; those who used Portuguese in a sub-test and those who did not; those who used another language (Bahasa Indonesian or local languages) in a sub-test and those who did not. These form the basis of our comparisons. Given that some students and enumerators used multiple combinations of languages it is impossible to consider the language groups mutually exclusive and this should be borne in mind when interpreting the results. 21 Timor Leste 2011 EGMA : Baseline Report Languages Used by Enumerators The majority of the survey was in Tetum, however some parts were in Portuguese. The Portuguese-language sub-tests did not have accompanying language data collected. Enumerators were directed to use the language the student felt most comfortable with, information gathered during a preparatory conversation with the child to put him/her at their ease. Enumerator language information was collected for four sub-tests: oral counting, correspondence counting, number identification and the word problems. Comparing these to child language data collected for the same sub-tests we can see the relationships between child and enumerator conversations. These cross tabulatations are given in Tables 20-23. For example, in Table 20, we can see that in the oral counting sub-test, although the enumerator most often used Tetum (1052 surveys), the child did not respond in kind. Most students responded in Portuguese (1172 surveys). Only 73 students used another language than Tetum or Portuguese. In 50 of these cases, the enumerator did not respond in a third language. TABLE 20. Frequency of Enumerator and Child TABLE 21. Frequency of Enumerator and Child Languages Used; Oral Counting Languages Used; Correspondence Counting Child Child Enumerator No Tetum Tetum Total Enumerator No Tetum Tetum Total No Tetum 139 0 139 No Tetum 145 2 147 Tetum 1052 35 1087 Tetum 934 145 1079 Total 1191 35 1226 Total 1079 147 1226 Child Child Enumerator No Portuguese Portuguese Total Enumerator No Portuguese Portuguese Total No Portuguese 50 1172 1222 No Portuguese 151 1068 1219 Portuguese 1 3 4 Portuguese 2 5 7 Total 51 1175 1226 Total 153 1073 1226 Child Child No other Other No other Other Enumerator Total Enumerator Total language language language language No other language 998 50 1048 No other language 897 140 1037 Other language 155 23 178 Other language 142 47 189 Total 1153 73 1226 Total 1039 187 1226 TABLE 22. Frequency of Enumerator and Child TABLE 23. Frequency of Enumerator and Child Languages Used; Numbers Identified Languages Used; Word Problems Child Child Enumerator No Tetum Tetum Total Enumerator No Tetum Tetum Total No Tetum 152 1 153 No Tetum 149 13 162 Tetum 1015 58 1073 Tetum 393 671 1064 Total 1167 59 1226 Total 542 684 1226 Child Child Enumerator No Portuguese Portuguese Total Enumerator No Portuguese Portuguese No Portuguese 44 1174 1218 No Portuguese 478 741 1219 Portuguese 1 7 8 Portuguese 0 7 7 Total 45 1181 1226 Total 478 748 1226 Child Child No other Other No other Other Enumerator Total Enumerator Total language language language language No other language 827 215 1042 No other language 894 123 1017 Other language 147 37 184 Other language 57 152 209 Total 974 252 1226 Total 951 275 1226 22 Language and Mathematics Table 20 indicates that although the enumerator may ask the child a question in one language, the child will often choose to answer in a different language. This is especially true of sub-tests in which the child is asked a question in Tetum but may choose to respond in Portuguese. This is a direct contrast to the 2011 EGRA results, which indicated that children tested in Portuguese were unable to answer many, if any listening comprehension questions asked. It is likely a result of rote memorization of numbers and other numeracy concepts in Portuguese. This may be a direct result of the workbooks distributed and used extensively for mathematics teaching in Timorese classrooms. It may reflect the different strengths of the children’s vocabulary in the different languages at their command and their inherent facility at “switching” between languages in a multilingual environment. Language and Performance This section compares groups of students by the language(s) they answered the sub-tests for which there are language data available. Note that some students chose to use combinations of languages. These groups are not mutually exclusive and thus, the results should be interpreted with caution. Table 24 gives a comparison by language for the first sub-test, oral counting. It indicates that whilst there are no significant differences between students speaking Tetum and not speaking Tetum, between speaking Portuguese and not speaking Portuguese; there is a significant difference between students answering in another language and those who do not. That is, there are statistical differences between those students speaking in a third language and those who do not. Students who answered this particular sub-test with at least one other language than Portuguese or Tetum scored significantly higher than those who did not. This unexpected result required further exploration. Looking specifically at students who answered in Bahasa Indonesian (also in Table 24), it appears that these students have a substantially higher mean and median result than other students. This may be due to some unspecified socio-economic factor for which we do not have data. Table 25 gives the results of the second sub-test, correspondence counting, by language. Due to the unexpected enumerator behavior described in the last chapter where most students were stopped at the end of the page (rather than repeating it), the vast majority of students stopped counting at 30 and there are no significant differences between the languages used. Table 26 gives the results of the third sub-test, number identification, by language. It indicates that students who do not use Tetum have significantly better scores in this sub-test than those who do. Conversely (and not mutually exclusively) those who use Portuguese have significantly better results than those who do not. Other languages were not significant. This may indicate that those students able to understand Portuguese well enough to identify numbers are better able to absorb the mathematics curriculum which is, anecdotally at least, taught predominantly in Portuguese. This suggests that children unable to understand basic mathematics vocabulary in Portuguese are at risk of falling behind in the curriculum. Further research is required into how mathematics teaching is taking place in Timorese classrooms. The missing number sub-test had a similar outcome. Table 27 gives the results of this sub-test by language. Table 27 indicates that whilst other languages are not significant, students who answer this sub-test in Portuguese are associated with higher scores than those who do not. Table 28 gives the results for the word problems sub-test by language. Although none of the language variables are statistically significant, students answering in Portuguese have an average score higher than those who do not. Overall, it appears that amongst those skills for which language data was collected, language is an important factor associated with student performance. However, it should not be seen as a causative factor in student performance, but rather one associated with performance. In an educational environment as linguistically diverse as Timor Leste, language is complex. Without data on the use and purpose of each language and its frequency, it is impossible to determine how concepts, pedagogy and language combine in the classroom to promote learning. Given that these results diverge substantially from what might be expected based on the EGRA 2011 assessment, more research into actual classroom language practices and possible socio-economic correlations is urgently required. 23 Timor Leste 2011 EGMA : Baseline Report TABLE 24. Oral Counting Results by Language (Non-overlapping confidence intervals given in bold) Mean SD Min Max Median N LCL UCL No Tetum 54 23 0 100 56 1191 52 55 Tetum 51 27 2 96 51 35 42 60 Total 53 23 0 100 56 1226 52 55 No Portuguese 56 29 9 100 60 51 48 64 Portuguese 53 23 0 100 56 1175 52 55 Total 53 23 0 100 56 1226 52 55 No other language (includes Indonesian) 53 23 0 100 55 1153 52 54 Other language (includes Indonesian) 63 27 9 100 70 73 57 69 Total 53 23 0 100 56 1226 52 55 No Indonesian 53 23 0 100 55 1179 52 54 Indonesian 70 25 19 100 73 47 63 77 Total 53 23 0 100 56 1226 52 55 TABLE 25. Correspondence Counting Results by Language (Non-overlapping confidence intervals given in bold) Mean SD Min Max Median N LCL UCL No Tetum 28 6 2 60 30 1077 28 28 Tetum 26 8 0 30 30 147 25 27 Total 28 6 0 60 30 1224 27 28 No Portuguese 25 8 0 31 30 151 24 27 Portuguese 28 6 1 60 30 1073 28 28 Total 28 6 0 60 30 1224 27 28 No other language (includes Indonesian) 28 6 0 60 30 1037 27 28 Other language (includes Indonesian) 27 6 3 31 30 187 27 28 Total 28 6 0 60 30 1224 27 28 24 Language and Mathematics TABLE 26. Number Identification Results by Language (Non-overlapping confidence intervals given in bold) Mean SD Min Max Median N LCL UCL No Tetum 61% 28% 0% 100% 67% 1077 59% 62% Tetum 54% 32% 0% 100% 67% 147 49% 59% Total 60% 28% 0% 100% 67% 1224 58% 62% No Portuguese 50% 32% 0% 100% 50% 153 45% 55% Portuguese 61% 27% 0% 100% 67% 1071 60% 63% Total 60% 28% 0% 100% 67% 1224 58% 62% No other language 61% 28% 0% 100% 67% 1038 59% 62% Other language 57% 29% 0% 100% 67% 186 52% 61% Total 60% 28% 0% 100% 67% 1224 58% 62% TABLE 27. Missing Numbers Results by Language (Non-overlapping confidence intervals given in bold) Mean SD Min Max Median N LCL UCL No Tetum 36% 27% 0% 100% 30% 1075 34% 37% Tetum 30% 27% 0% 100% 25% 146 26% 35% Total 35% 27% 0% 100% 30% 1221 33% 36% No Portuguese 25% 25% 0% 100% 20% 152 21% 29% Portuguese 36% 27% 0% 100% 30% 1069 35% 38% Total 35% 27% 0% 100% 30% 1221 33% 36% No other language 36% 27% 0% 100% 30% 1034 34% 37% Other language 30% 26% 0% 100% 30% 187 27% 34% Total 35% 27% 0% 100% 30% 1221 33% 36% TABLE 28. Word Problems Results by Language (Non-overlapping confidence intervals given in bold) Mean SD Min Max Median N LCL UCL No Tetum 46% 31% 0% 100% 50% 1066 44% 48% Tetum 44% 32% 0% 100% 50% 147 39% 49% Total 46% 31% 0% 100% 50% 1213 44% 48% No Portuguese 41% 30% 0% 100% 50% 152 37% 46% Portuguese 46% 31% 0% 100% 50% 1061 45% 48% Total 46% 31% 0% 100% 50% 1213 44% 48% No other language 46% 31% 0% 100% 50% 1027 44% 48% Other language 45% 30% 0% 100% 50% 186 41% 49% Total 46% 31% 0% 100% 50% 1213 44% 48% 25 Timor Leste 2011 EGMA : Baseline Report 26 Chapter 6 Factors Associated with Better Numeracy Outcomes As an accompaniment to the EGMA survey, a brief socio-economic survey was undertaken at the same time in order to determine which school- and student-specific factors have an association with early mathematics outcomes. The contents of the socio-economic survey were necessarily brief and decided on by the ministry in accordance with the research questions that were of interest. Specifically, during 2011 and 2012 an intervention in the form of mathematics workbooks (in Portuguese language) and physical resources such as manipulative objects was rolled out nationally. The ministry was interested to know how these affected mathematics outcomes. When interpreting the results of this survey, it is important to note that the variables in this survey are self-reported by very young children. The results should be treated cautiously. Table 29 gives the basic statistics for the socio- economic indicators collected. Table 30 gives the relationships of the indicators to several key sub-tests of the EGMA survey. Significant variables are given in bold and the difference in average sub-test scores between groups with and without the indicators is also given. The significance level used in Table 30 is the 95% confidence level. Please note that whilst these indicators may have an effect on student performance, the results given in Table 30 should not be construed as causative but as associative. There are many factors that may be associated with student performance and these statistics given cannot account for them all. The figures in Table 30 should be used not as predictive measures but as measures of association. Five sub-tests were chosen to compare the association of the socio-economic factors chosen for inclusion in the survey. These were chosen partly for reasons of brevity and partly to avoid misleading results by including the early sub-tests (oral counting and correspondence counting), which were discussed at length in previous chapters. The sub-tests chosen concentrated on the skills with which students had the most difficulty; as these are the skills most likely to require intervention in the near future. These were: number identification, missing numbers, word problems, addition and subtraction. Full statistics are given in Annex One. 27 Timor Leste 2011 EGMA : Baseline Report Approximately 37% of students reported that they had attended preschool. Whilst this figure may seem quite high for a country of the development status such as Timor Leste, it should be noted that in other Pacific countries, the definition of “preschool” is an amorphous one. It can mean anything from an hour’s activities in the church hall on a weekly basis to a five-day per week specific early childhood development program. Its meaning to the young children answering the question is not defined. Attendance at preschool was not significantly associated with any of the sub-tests. Although students who self-reported as attending preschool had lower mean scores in each of the sub-tests than those who did not, these differences between estimated scores with and without attending preschool are very marginal and close to zero. Just over 91% of students self-reported that they had mathematics lessons daily. The statistic is encouraging, but should be treated with caution since it is possible some local translators used were the children’s schoolteachers. Also encouraging is the fact that students who self-reported they had daily mathematics lessons had significantly higher sub-test scores on average in all sub-tests examined excepting word problems. The sub-test average scores were between 8% and 17% depending on sub-test higher for those children who had daily mathematics lessons. Textbook ownership was high with 80% of students reporting they owned the text. However, given that the texts were distributed nationally, this figure may in fact be too low. Ownership of the text conferred no significant advantage to students over those who did not: although students with the textbook had on average higher sub- test scores than those who did not, these were not significantly different from zero at the 95% confidence level. Students also reported high degrees of using objects in mathematics lessons (75%) and working with others (83%). Whilst both items had positive associations with better sub-test scores on average, only working with others was significant for the number identification, missing numbers and word problems sub-tests. Over 90% of students reported doing homework, but only one student in the entire sample reported having help with homework. This suggests that parental involvement with mathematics in early grade education is minimal or non-existent. Doing homework had a significant, positive effect on all sub-test scores with increases ranging from between 12-23%, TABLE 29. School and student-specific variables, descriptive statistics Factor Proportion SD Attended preschool 37% 48% Maths Daily 91% 29% Textbook 80% 40% Uses Objects 75% 43% Works with others 83% 37% Does homework 90% 30% Has help with Homework 0% 3% Sells things 39% 49% Breakfast that morning 89% 32% Breakfast Daily 92% 28% Merenda Escolar 95% 21% TV at home 41% 49% Radio at Home 56% 50% 28 Factors Associated with Better Numeracy Outcomes TABLE 30. Effects of School and Student-Specific Variables on Average Scores Part A Subtest Numbers Identified Missing Numbers Word Problems Addition Subtraction Item Sign and Difference Sign and Difference Sign and Difference Sign and Difference Sign and Difference Significance at Mean Significance at Mean Significance at Mean Significance at Mean Significance at Mean Attended - -0.027 - -0.018 - -0.035 - -0.026 - -0.008 Preschool Maths Daily + 0.159 + 0.079 + 0.066 + 0.167 + 0.080 Owns Text + 0.102 + 0.079 + 0.066 + 0.167 + 0.080 Uses Objects + 0.024 + 0.008 + 0.009 + 0.020 + 0.015 Works with + 0.078 + 0.056 + 0.071 + 0.020 + 0.057 Others Does + 0.179 + 0.121 + 0.141 + 0.226 + 0.141 Homework Child Sells in - -0.011 - -0.010 - -0.026 - -0.004 + 0.005 Marketplace Child ate breakfast that + 0.051 + 0.018 + 0.018 + 0.010 + 0.052 morning TABLE 31. Effects of School and Student-Specific Variables on Average Scores Part B Subtest Numbers Identified Missing Numbers Word Problems Addition Subtraction Item Sign and Difference Sign and Difference Sign and Difference Sign and Difference Sign and Difference Significance at Mean Significance at Mean Significance at Mean Significance at Mean Significance at Mean Child Eats Breakfast + 0.048 + 0.015 + 0.010 + 0.026 + 0.024 Everyday Merenda + 0.079 + 0.065 + 0.057 + 0.094 + 0.096 Escolar Family Owns + 0.036 + 0.071 + 0.001 + 0.052 + 0.070 TV Family Owns + 0.033 + 0.032 + 0.017 + 0.045 + 0.039 Radio 29 Timor Leste 2011 EGMA : Baseline Report It was considered possible that the many Timorese children whose parents sell goods in the market place may have access to a means of numeracy education outside of school. Nearly 40% of students (38.5%) indicated that they engaged in this practice. However, there was no significant association with learning outcomes. Childhood nutrition is known to be an important determinant of educational outcomes. Students were asked if they had eaten breakfast that day (89% had eaten breakfast), if they ate breakfast everyday (92%) and about school participation in the school feeding program merenda escolar (95%). The fact that fewer students had eaten that morning than eat breakfast every day can be ascribed to two things: either a random quirk from small children self- reporting, or that some children felt compelled to reply in the affirmative for reasons of pride. There is a statistically significant difference between the proportions of students who eat breakfast every day and those who had eaten that morning. That is, students self-reported incorrectly to a significant degree. This result urges caution in ascribing reliability to this section of the data. Furthermore, although all three nutritional variables had a positive association with sub-test average scores: none of them were statistically significant at the 95% level. Lastly, students were asked about two consumer durables in their home: the presence of a TV (41%) and/or a radio (56%). Both of these had a positive association with student outcomes. Having a television in the home was significantly associated with positive outcomes in both the missing numbers and subtraction sub-tests at the 95% level and may reflect a positive socio-economic association with education. Another possibility is that the presence of a television in the home may offer exposure to languages used in the classroom whicb may not be in regular use in the home. Overall, the short socio-economic survey was not able to offer substantial guidance on future interventions in part due to the age of the respondents and in part due to its short length. However, classroom practices such as a daily mathematics lesson, working with others and homework had significant, positive associations with mathematics outcomes. A television at home had a positive, significant association with some sub-test results, but rather than a causal link it is likely that it is an instrument for socio-economic status. 30 Chapter 7 Conclusions and Recommendations The analysis of the EGMA 2011 Timor Leste baseline survey calls for an immediate response to safeguard the educational future of young Timorese students. Although students perform reasonably in basic, “Phase Zero” mathematics skills such as oral counting, correspondence counting and number identification; this may be in part due to a misinterpretation of the correspondence counting sub-test by many enumerators. Higher order discernment skills show signs of slow gains. Later manipulative and calculation skills, those in “Phase Two” and “Phase Three” (missing numbers, word problems and arithmetic) are poorly understood and reproduced by Timorese students. After three years of schooling, Grade Three students can only answer 46% of simple subtraction problems correctly on average and only 72% of addition problems: all of which should have been understood by the end of Grade One. The lack of facility for basic numeracy skills sets in doubt Timorese students’ ability to cope with an increasingly stringent curriculum in later years. Language use was one of the most concerning aspects of mathematics education revealed by the survey. Although the main language of the classroom is Tetum, students’ mathematics textbooks are in Portuguese. Some students did not speak enough Tetum or Portuguese to complete the survey without aid of a translator for their local languages. This linguistic diversity within the classroom indicates that further research into how Timorese students learn and in which languages is required. Survey results call for immediate action to ensure that all students in east Timor have access to a numerate future. Specific recommendations to be considered are as follows: Language: • Research is urgently required into language practices in the classroom. Timor Leste is a linguistically rich country and students answered some sub-tests in the EGMA survey in up to three languages, indicating a substantial exposure to language in their community. In order to understand how and why students are learning, it must be known in what language and combination of languages they are actually learning in, rather than supposed to be learning according to curriculum mandates and workbooks provided. 31 Timor Leste 2011 EGMA : Baseline Report Conceptual: • Further research is recommended into the confidence and skills of teachers and students to use effectively the current mathematics student workbooks. These are currently used as textbooks and in Portuguese language. • Further research into students’ ability to use the workbooks is required. Based on the findings of the 2011 EGRA (Early Grade Reading Assessment) Survey, and the poor facility of students in the Portuguese language there is substantial cause for concern that students may not be able to use these materials effectively. The ownership of mathematics texts was only self-reported by students at around 80%, despite a national distribution and the ownership of the text was not associated with a statistically significant increase in performance. These results suggest that research into the use, distribution and implementation of these resources is warranted. • The establishment of mathematics benchmarks to monitor classroom improvements. These benchmarks can be used to assess students at-risk of falling further behind by a process of continuous assessment. It is important to consider that initial standards should be considered temporary reference standards since not enough is currently known about when and at what rate Timorese children should progress in learning mathematics. In this sense, benchmark standards should not be seen as high-stakes but an essential piece to monitor mathematics progression in the classroom. One way of doing this would be to use the percentage of zero-score students in selected sub-tests as a marker and track reductions in the shares at least biannually. Monitoring achievements over time will eventually provide more information on the rate and the way in which average mathematics skill develops in Timor. A modified version of the test could be used to screen students during the school year in need of additional support. Teaching , Teachers and the Classroom: • Support for implementing and sourcing mathematics resources in a students’ main language of instruction is suggested. In the light of evidence that supports early learning in the child’s first language, it would follow that priority should be given to the production of mathematics textbooks/workbooks and stimulus pictures in mother tongue. • It is recommended that training of teachers, both pre-service and in-service, be designed to improve pedagogical skills in deepening conceptual understandings of mathematics. It is recommended that Escola Basica Deputy Directors and school inspectors be involved in parallel professional development. • It is highly recommended teachers receive continued ongoing support in using the stimulus, display and manipulative materials that were distributed to all schools in 2010 and were further augmented in 2012. Pedagogy that supports deep understanding through participation, manipulation and interaction is most effective for mathematics learning and teachers need to be supported in achieving this. • It is recommended that the Ministry should leverage on the national distribution of materials and the EGMA results to identify grade-specific skills that teachers will need pedagogical methods to convey content knowledge to students. Without the pedagogical understanding of how to convey content to students, knowledge transmission cannot occur efficiently. • Ongoing support in mathematics program planning, teaching methodology and formative assessment is essential. The future of Timorese numeracy is in the hands of its teachers. Improvements in Timorese children’s mathematical skill depends not only on classroom resources but on effective understanding of how to implement curriculum, resources and assessment. • Research into the confidence and skills of teachers to interpret and teach according to the current mathematics curriculum and teacher guide for mathematics is required. Modified versions of the Classroom Observational Snapshot tool used in other Pacific countries and tailored to the specific needs of the Timorese education system will provide information on the skills of teachers, while teacher surveys will provide elucidation on the thoughts, concerns, backgrounds and skills of these vital components of the Timorese education system. 32 Conclusions and Recommendations • Curriculum and teacher support to provide further opportunities for students to work in groups with each other is recommended. Students who engaged in group activities for mathematics learning had scores significantly higher than those who did not. Opportunities to engage in this process provide the foundation not only for improving numerical skills, but gaining language and social skills that will benefit Timorese students. However, teachers need to be supported in their efforts to engage their students in meaningful group activities and this should be a priority in future curriculum and lesson planning. • Teacher encouragement and support to engage in mathematics learning on a daily basis is required. Students who were engaged in mathematics on a daily basis had a significantly higher score in the sub-tests examined than those who did not. Mathematics is a cumulative skill set, each concept building on those learnt previously. Especially in early grades, students are learning patterns of thinking that will be a valuable life-long asset. A healthy 91% of students self-reported that they engaged in mathematics learning daily, however, that leaves a large proportion of students who do not and this statistic may over-report the reality. Ensuring that each child has the opportunity to build on their foundational skills in a consistent daily manner should be a focus of lesson planning and teacher training. Parents and the Community: • Promoting parent education and involvement is needed, not only to interest parents in their children’s mathematics education and build accountability into the Timorese education system at the village level. It is also needed to build parents’ confidence, extending their knowledge and strategies that will enable them to engage with and assist their young children in mathematics. Only a single student out of the 1226 surveyed reported that they received assistance with their mathematics homework. Parental involvement and engagement in mathematics is a key method of improving outcomes for Timorese students. • The evidence in the report suggests that providing opportunities for students to engage in meaningful learning at home will result in benefit to students. Students who reported doing mathematics homework had scores significantly higher than their counterparts. However, this should not be construed as an argument for large amounts of rote-learning to be sent home with children on a regular basis. The importance of parental engagement has been outlined above, however when only a single student in more than 1200 reported having assistance with their homework expectations of Timorese parents must be realistic. On a related note, 39% of students reported that their parents sell things in the market. This statistic indicates that a basic level of numeracy is prevalent in the community and that the community understands the need for numeracy skills. Providing children with the opportunity to engage in active learning at home through meaningful homework activities will have several effects: to encourage and cement concepts learnt in school, to engage parents and the community with the importance of numeracy as a concept and to increase children’s’ confidence in their own mathematical abilities. Examples of such activities may include assisting parents at market or in shopping, making change, counting and arranging items in groups in the home or market, correlating and observing sports scores or engaging in word problems with the assistance of every day objects (a skill many Timorese students struggled with). 33 Timor Leste 2011 EGMA : Baseline Report 34 REFERENCES Berch, D. B. (2005). Making sense of number sense: Implications for children with mathematical disabilities. Journal of Learning Disabilities, 38(4), 333–339. Clarke, B., Baker, S., Smolkowski, K., & Chard, D. J. (2008). An analysis of early numeracy curriculum-based measurement: Examining the role of growth in student outcomes. De Silva, S., Guacgonolle, L. (2012). The Timor Leste 2011 EGRA: Tetum Pilot Results. The World Bank. EGMA Malawi – Student stimulus sheets. Version 5.0, Malawi, October 2010. Retrieved from https://www. eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=307 (Retrieved April 2011) EGMA Rwanda - Primary 4 – Enumerator Protocol – Pilot Instrument – January 2011. Retrieved from https://www. eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=307 (Retrieved April 2011) EGMA Rwanda - Primary 4 – Student Sheets – Pilot Instrument – January 2011. Retrieved from https://www. eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=307 (Retrieved April 2011) Eddata II. Early Grade Mathematics Assessment (EGMA): Administrator Instructions and Protocol. v.1 12/2008. Contract Number EHC-E-02-o4-00004-00 (Retrieved Oct. 2010) Garnett, K. Developing fluency with basic number facts: Intervention for students with learning disabilities. Learning Disabilities Research and Practice (1992), 7:210-216. Gervasoni, A. Extending Mathematical Understanding. Middle Years Specialist Teachers Manual. Australian Catholic University, Draft Edition, 2009. Glewwe, P & Kremer, M. (2005). Schools, Teachers and Education Outcomes in Developing Countries. Centre for International Development at Harvard University Working Paper No. 122. September 2005 Griffin, S. (2004). Teaching Number Sense. Educational Leadership, 61(5), 39 – 42 Griffin, S. & Case, R. (1997). Re-thinking the primary school math curriculum. An approach based on cognitive science. Issues in Education, 3(1) 1-49. Huayu Sun (2008). Chinese Young Children’s Strategies on Basic Addition Facts. Retrieved from http://www.merga. net.au/documents/RP602008.pdf (May 2011). Kilpatrick, J., Swafford, J. & Findell, Bradford. Mathematics Learning Study Committee. Adding It Up (Helping Children Learn Mathematics). National Research Council [NRC], Washington, 2001), Mansory, A. (2010). Do Children Learn in Afghan Schools? – Assessent of math and language achievements of students at the end of Grades 3 and 6 in SCA supported schools. Swedish Committee for Afghanistan (SCA), Education Technical Support Unit, Kabul, Afghanistan. Ministry of Education National Education Strategic Plan, 2011 – 2030, 2nd Draft. 35 Timor The Leste 2011 Timor-Leste 2011 EGMA : Baseline EGRA: Tetum Pilot Results Report Ministry of Education, Timor Leste, Deomocratic Republic of Timor-Leste, Dili. Ministry of Education National Education Policy, 2007 – 2012. Ministry of Education, Timor Leste, Democratic Republic of Timor-Leste, Dili. RTI International: Crouch, L & Reubens, A. (2009). Early Grade Mathematics Assessment (EGMA): A Conceptual Framework Based on Mathematics Skills Development in Children. RTI International, 2011. Core EGMA, January 2011 Retrieved from https://www.eddataglobal.org/documents/ index.cfm/Core%20EGMA%20%2D%20January%202011.pdf?fuseaction=throwpub&ID=305,https://www. eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=305 (April 2011) RTI International. 2009. Early Grade Reading Assessment Toolkit. Prepared by RTI for The World Bank, Office of Human Development. Available at https://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=149 (accessed September 03, 2010). RTI International (Prepared by Andrea Rubens). 2009. Preliminary Snapshot of some of the tasks in the Early Grade Mathematics Assessment (EGMA) Data. EdData II Technical and Managerial Assistance, Task Number 2, Contract Number EHC-E-00-04-00004-00, Strategic Objective 3, December 2009. RTI International (Prepared by Andrea Rubens and Tracy Kline). 2009. Pilot of the Early Grade Mathematics Assessment, Final Report, Dec 23, 2009. Prepared for review by the United States Agency for International Development under task order: EdData !! Technical and Managerial Assistance, Task Number 2, Contract Number EHC-É-02-04-00004-00, Strategic Objective 3, December 23, 2009. UNDP Timor-Leste Human Development Report 2011. World Bank (2010): An analysis of early grade reading acquisition in Timor Leste: Report of the Early Grade Reading Assessment (EGRA) 2009. Wright, R., Martland, J. & Stafford, A. (2000). Early Numeracy: Assessment for teaching and intervention. London: Paul Chapman Publishing 36 Annex 1: School and Student Specific Factors Numbers Identified % Mean SD Min Max Median N LCL UCL Did not attend Preschool 0.6103 0.2808 0 1 0.6667 765 0.5904 0.6303 Attended Preschool 0.5830 0.2850 0 1 0.6667 459 0.5569 0.6090 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Did not attend Preschool 0.4712 0.3169 0 1 0.5000 756 0.4486 0.4938 Attended Preschool 0.4360 0.3051 0 1 0.5000 457 0.4080 0.4640 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL Did not attend Preschool 0.3559 0.2726 0 1 0.3000 764 0.3366 Attended Preschool 0.3379 0.2688 0 1 0.3000 457 0.3132 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 Additions % Mean SD Min Max Median N LCL UCL Did not attend Preschool 0.4957 0.3929 0 1 0.6000 763 0.4678 0.5236 Attended Preschool 0.4702 0.3838 0 1 0.4000 456 0.4349 0.5054 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Did not attend Preschool 0.2977 0.3537 0 1 0.2000 755 0.2725 0.3230 Attended Preschool 0.2901 0.3392 0 1 0.2000 453 0.2588 0.3213 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 37 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL No Maths Daily 0.4553 0.2734 0 1 0.4167 110 0.4042 0.5064 Maths Daily 0.6144 0.2796 0 1 0.6667 1114 0.5980 0.6308 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL No Maths Daily 0.3977 0.2939 0 1 0.5000 110 0.3428 0.4526 Maths Daily 0.4640 0.3141 0 1 0.5000 1103 0.4454 0.4825 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL No Maths Daily 0.2773 0.2305 0 1 0.2000 110 0.2342 0.3204 Maths Daily 0.3563 0.2740 0 1 0.3000 1111 0.3401 0.3724 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL No Maths Daily 0.3339 0.3267 0 1 0.2000 109 0.2726 0.3953 Maths Daily 0.5011 0.3921 0 1 0.6000 1110 0.4780 0.5242 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL No Maths Daily 0.2224 0.2937 0 1 0.0000 107 0.1668 0.2781 Maths Daily 0.3019 0.3524 0 1 0.2000 1101 0.2811 0.3227 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 38 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Does not own text 0.5184 0.2879 0 1 0.5000 240 0.4820 0.5548 Owns text 0.6200 0.2778 0 1 0.7500 984 0.6026 0.6374 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Does not own text 0.4078 0.3191 0 1 0.5000 236 0.3671 0.4486 Owns text 0.4701 0.3102 0 1 0.5000 977 0.4506 0.4895 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Does not own text 0.3189 0.2696 0 1 0.3000 238 0.2847 0.3532 Owns text 0.3565 0.2713 0 1 0.3000 983 0.3395 0.3734 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Does not own text 0.3908 0.3758 0 1 0.4000 239 0.3431 0.4384 Owns text 0.5094 0.3895 0 1 0.6000 980 0.4850 0.5338 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Does not own text 0.2542 0.3149 0 1 0.0000 236 0.2141 0.2944 Owns text 0.3047 0.3553 0 1 0.2000 972 0.2824 0.3271 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 39 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL Does not use objects 0.5822 0.2904 0 1 0.6667 306 0.5497 0.6148 Uses objects 0.6060 0.2799 0 1 0.6667 918 0.5879 0.6241 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Does not use objects 0.4512 0.3055 0 1 0.5000 302 0.4167 0.4856 Uses objects 0.4602 0.3153 0 1 0.5000 911 0.4397 0.4807 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Does not use objects 0.3435 0.2773 0 1 0.3000 306 0.3124 0.3745 Uses objects 0.3510 0.2693 0 1 0.3000 915 0.3336 0.3685 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Does not use objects 0.4708 0.3807 0 1 0.4000 305 0.4281 0.5136 Uses objects 0.4912 0.3925 0 1 0.6000 914 0.4658 0.5167 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Does not use objects 0.2839 0.3300 0 1 0.2000 298 0.2464 0.3214 Uses objects 0.2985 0.3541 0 1 0.2000 910 0.2755 0.3215 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 40 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Does not work with others 0.5354 0.2950 0 1 0.5833 205 0.4950 0.5757 Works with others 0.6131 0.2784 0 1 0.6667 1019 0.5960 0.6302 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Does not work with others 0.3990 0.3091 0 1 0.5000 203 0.3565 0.4415 Works with others 0.4698 0.3124 0 1 0.5000 1010 0.4505 0.4891 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Does not work with others 0.3024 0.2711 0 1 0.3000 205 0.2653 0.3395 Works with others 0.3586 0.2704 0 1 0.3000 1016 0.3419 0.3752 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Does not work with others 0.4380 0.3761 0 1 0.4000 205 0.3866 0.4895 Works with others 0.4959 0.3917 0 1 0.6000 1014 0.4717 0.5200 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Does not work with others 0.2478 0.3151 0 1 0.0000 201 0.2042 0.2913 Works with others 0.3043 0.3539 0 1 0.2000 1007 0.2824 0.3261 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 41 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL Does not do homework 0.4387 0.2665 0 1 0.4167 121 0.3912 0.4862 Does homework 0.6178 0.2788 0 1 0.6667 1103 0.6013 0.6342 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Does not do homework 0.3313 0.3077 0 1 0.2500 120 0.2762 0.3863 Does homework 0.4719 0.3104 0 1 0.5000 1093 0.4535 0.4903 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Does not do homework 0.2405 0.2319 0 1 0.2000 121 0.1992 0.2818 Does homework 0.3611 0.2727 0 1 0.3000 1100 0.3450 0.3772 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Does not do homework 0.2826 0.3283 0 1 0.2000 121 0.2241 0.3411 Does homework 0.5086 0.3894 0 1 0.6000 1098 0.4855 0.5316 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Does not do homework 0.1678 0.2382 0 0.8 0.0000 118 0.1248 0.2108 Does homework 0.3086 0.3555 0 1 0.2000 1090 0.2875 0.3297 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 42 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Child does not sell 0.6042 0.2780 0 1 0.6667 752 0.5843 0.6240 Child sells items 0.5936 0.2901 0 1 0.6667 472 0.5674 0.6197 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child does not sell 0.4679 0.3063 0 1 0.5000 747 0.4459 0.4898 Child sells items 0.4421 0.3227 0 1 0.5000 466 0.4128 0.4714 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child does not sell 0.3529 0.2649 0 1 0.3000 751 0.3339 0.3718 Child sells items 0.3432 0.2812 0 1 0.3000 470 0.3178 0.3686 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child does not sell 0.4876 0.3809 0 1 0.6000 751 0.4604 0.5149 Child sells items 0.4838 0.4034 0 1 0.5000 468 0.4472 0.5203 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child does not sell 0.2930 0.3416 0 1 0.2000 746 0.2685 0.3175 Child sells items 0.2978 0.3590 0 1 0.2000 462 0.2651 0.3306 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 43 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL Child does not sell 0.6042 0.2780 0 1 0.6667 752 0.5843 0.6240 Child sells items 0.5936 0.2901 0 1 0.6667 472 0.5674 0.6197 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child does not sell 0.4679 0.3063 0 1 0.5000 747 0.4459 0.4898 Child sells items 0.4421 0.3227 0 1 0.5000 466 0.4128 0.4714 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child does not sell 0.3529 0.2649 0 1 0.3000 751 0.3339 0.3718 Child sells items 0.3432 0.2812 0 1 0.3000 470 0.3178 0.3686 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child does not sell 0.4876 0.3809 0 1 0.6000 751 0.4604 0.5149 Child sells items 0.4838 0.4034 0 1 0.5000 468 0.4472 0.5203 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child does not sell 0.2930 0.3416 0 1 0.2000 746 0.2685 0.3175 Child sells items 0.2978 0.3590 0 1 0.2000 462 0.2651 0.3306 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 44 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Child did not eat breakfast 0.5546 0.2751 0 1 0.5833 139 0.5088 0.6003 Child ate breakfast 0.6059 0.2832 0 1 0.6667 1085 0.5891 0.6228 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child did not eat breakfast 0.4424 0.3029 0 1 0.5000 139 0.3921 0.4928 Child ate breakfast 0.4600 0.3142 0 1 0.5000 1074 0.4412 0.4788 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child did not eat breakfast 0.3329 0.2497 0 1 0.3000 140 0.2915 0.3742 Child ate breakfast 0.3512 0.2739 0 1 0.3000 1081 0.3349 0.3676 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child did not eat breakfast 0.4771 0.3813 0 1 0.5000 140 0.4140 0.5403 Child ate breakfast 0.4873 0.3908 0 1 0.6000 1079 0.4640 0.5106 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child did not eat breakfast 0.2489 0.3165 0 1 0.0000 139 0.1963 0.3015 Child ate breakfast 0.3008 0.3518 0 1 0.2000 1069 0.2798 0.3219 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 45 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL Child does not eat breakfast everyday 0.5561 0.2617 0 1 0.5833 101 0.5051 0.6071 Child eats breakfast everyday 0.6040 0.2842 0 1 0.6667 1123 0.5874 0.6207 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child does not eat breakfast everyday 0.4485 0.3231 0 1 0.5000 102 0.3858 0.5112 Child eats breakfast everyday 0.4588 0.3120 0 1 0.5000 1111 0.4405 0.4772 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child does not eat breakfast everyday 0.3353 0.2574 0 1 0.3000 102 0.2853 0.3853 Child eats breakfast everyday 0.3504 0.2725 0 1 0.3000 1119 0.3344 0.3664 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child does not eat breakfast everyday 0.4627 0.3828 0 1 0.4000 102 0.3885 0.5370 Child eats breakfast everyday 0.4883 0.3903 0 1 0.6000 1117 0.4654 0.5112 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child does not eat breakfast everyday 0.2733 0.3246 0 1 0.2000 101 0.2100 0.3366 Child eats breakfast everyday 0.2968 0.3504 0 1 0.2000 1107 0.2762 0.3175 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 46 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Child does not participate in merenda 0.5249 0.2821 0 1 0.5000 57 0.4516 0.5981 escolar Child participates in merenda escolar 0.6038 0.2822 0 1 0.6667 1167 0.5876 0.6200 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child does not participate in merenda 0.4035 0.2825 0 1 0.5000 57 0.3302 0.4768 escolar Child participates in merenda escolar 0.4606 0.3141 0 1 0.5000 1156 0.4425 0.4787 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child does not participate in merenda 0.2875 0.2783 0 1 0.2500 56 0.2146 0.3604 escolar Child participates in merenda escolar 0.3521 0.2707 0 1 0.3000 1165 0.3366 0.3676 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child does not participate in merenda 0.3965 0.3955 0 1 0.4000 57 0.2938 0.4992 escolar Child participates in merenda escolar 0.4905 0.3889 0 1 0.6000 1162 0.4682 0.5129 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child does not participate in merenda 0.2036 0.3086 0 1 0.0000 56 0.1227 0.2844 escolar Child participates in merenda escolar 0.2993 0.3496 0 1 0.2000 1152 0.2791 0.3195 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 47 Timor Leste 2011 EGMA : Baseline Report Numbers Identified % Mean SD Min Max Median N LCL UCL Child's family does not own TV 0.5853 0.2802 0 1 0.6667 727 0.5649 0.6056 Child's family owns TV 0.6217 0.2851 0 1 0.7500 497 0.5967 0.6468 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child's family does not own TV 0.4577 0.3120 0 1 0.5000 721 0.4349 0.4805 Child's family owns TV 0.4583 0.3144 0 1 0.5000 492 0.4306 0.4861 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child's family does not own TV 0.3204 0.2624 0 1 0.3000 724 0.3013 0.3396 Child's family owns TV 0.3909 0.2786 0 1 0.4000 497 0.3665 0.4154 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child's family does not own TV 0.4651 0.3885 0 1 0.4000 728 0.4369 0.4933 Child's family owns TV 0.5173 0.3894 0 1 0.6000 491 0.4829 0.5518 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child's family does not own TV 0.2669 0.3373 0 1 0.0000 721 0.2422 0.2915 Child's family owns TV 0.3363 0.3601 0 1 0.2000 487 0.3044 0.3683 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 48 Attachment Numbers Identified % Mean SD Min Max Median N LCL UCL Child's family does not own radio 0.5816 0.2788 0 1 0.6667 534 0.5580 0.6053 Child's family owns radio 0.6144 0.2849 0 1 0.7500 690 0.5931 0.6356 Total 0.6001 0.2826 0 1 0.6667 1224 0.5842 0.6159 Word Problems % Mean SD Min Max Median N LCL UCL Child's family does not own radio 0.4486 0.3102 0 1 0.5000 530 0.4222 0.4750 Child's family owns radio 0.4652 0.3149 0 1 0.5000 683 0.4416 0.4888 Total 0.4580 0.3128 0 1 0.5000 1213 0.4404 0.4756 Missing Numbers % Mean SD Min Max Median N LCL UCL Child's family does not own radio 0.3310 0.2613 0 1 0.3000 532 0.3088 0.3532 Child's family owns radio 0.3631 0.2780 0 1 0.3000 689 0.3424 0.3839 Total 0.3491 0.2712 0 1 0.3000 1221 0.3339 0.3644 Additions % Mean SD Min Max Median N LCL UCL Child's family does not own radio 0.4609 0.3863 0 1 0.4000 532 0.4281 0.4937 Child's family owns radio 0.5057 0.3912 0 1 0.6000 687 0.4764 0.5349 Total 0.4861 0.3896 0 1 0.6000 1219 0.4643 0.5080 Subtractions % Mean SD Min Max Median N LCL UCL Child's family does not own radio 0.2728 0.3296 0 1 0.2000 525 0.2446 0.3010 Child's family owns radio 0.3119 0.3612 0 1 0.2000 683 0.2848 0.3389 Total 0.2949 0.3482 0 1 0.2000 1208 0.2752 0.3145 49 Timor Leste 2011 EGMA : Baseline Report 50 Annex Two: EGMA Instrument 51 Timor Leste 2011 EGMA : Baseline Report 52 Attachment M O IN Ã IS AÇ TÉ RIO DA EDUC Avaliasaun Matemátika ba Grau Sedu (Early Grade Mathematics Assessment - EGMA) Timor-Leste Instrusaun Sira ba Enumeradór 2° Esbosu JUNE 2011 53 Timor Leste 2011 EGMA : Baseline Report Instrusaun Jerál Importante maka estabelese uluk lai relasaun ne’ebé haksolok no hakmatek ho labarik sira ne’ebé ita atu avalia, liuhosi konversa dahuluk ne’ebé simples kona-ba asuntu sira ne’ebé labarik ne’e gosta. Labarik ne’e sei persebe avaliasaun ne’ebé tuirmai ne’e nu’udar jogu ida ne’ebé halo nia kontente duké hanesan situasaun ne’ebé ladi’ak bá nia. Konsente Verbál Lee tekstu iha kaixa laran ne’e ho klaru ba labarik ne’e: Molok atu hahu, ha’u hakarak dehan ha’u naran ba ó. Ha’u naran ............... Ha’u serbisu iha Ministériu Edukasaun. Ami hakarak atu hatene kona-ba oinsá labarik sira aprende matemátika. Ami hili ó tuir sorte, hanesan liuhosi rifa ka sortéiu. Ami presiza ó nia tulun ba ida ne’e. Maibé se ó lakohi atu hola parte mós laiha buat ida. Ita sei halimar jogu balun kona-ba sura no kona-ba númeru. Hodi uza tempu iha relójiu ne’e, ha’u sei haree ó sura to’o wainhira maka hotu. Ida ne’e LA’OS ezame no LA’OS atu halo ó pasa ba klase iha eskola. Ha’u mós sei husu ó kona-ba ó nia familia, hanesan buat hirak ne’ebé ó halo hamutuk ho ó nia familia iha uma. Ha’u sei LA hakerek ó nia naran atu nune’e laiha ema ida maka atu hetene katak ó nia resposta sira maka ne’e. Dala ida tan, se ó lakohi lalika hola parte iha hasoru malu ne’e. Wainhira ita hahú, se ó lakohi atu hatán pergunta ruma, laiha buat ida. Entaun, ó hakarak atu partisipa?  LOOS (se hakarak) Ó pronto atu hahú? (se la hetan konsente verbál, fó obrigadu ba labarik ne’e no muda fali ba labarik seluk, uza formuláriu ne’e nafatin) 54 Attachment ATIVIDADE 1: SURA   (segundu 60) A. Data ba avaliasaun: Tempu hahú avaliasaun: B. Entrevistadór nia naran/ Tempu remata avaliasaun: kodigu : C. Eskola nia NARAN: F. Nivel Grau Estudante nian: o 1 = Grau 1 o 2 = Grau 2 D. Kódigu Eskola: o 3 = Grau 3 E. Sub Distritu: o feto o mane Idade: ___________ F. Distritu: 55 Timor Leste 2011 EGMA : Baseline Report Ha’u hakarak atu ó sura. Ha’u sei uza   relójiu ne’e. Ha’u sei hatete wainhira • Se nia sura la loos maka hahú no wainhira maka para. • Se tempu (segundu 60) hotu ona. Hahú sura husi um (1) iha lingua ne’ebé ó prefere atu uza hodi sura. Ó pronto ona? Loos, hahú.  Haree ha’u sura. Um, dois, três. Entaun,  ha’u hakarak ó sura hanesan ha’u iha lingua ne’ebé ó prefere. Entaun, hahú: um, …  Númeru loos ikus ne’ebé labarik temi: [ ]  Tempu iha relójiu:  ____ segundu  Lingua saida maka ita boot uza ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ]  Se ita boot uza lingua seluk ba atividade ne’e, indika lingua saida maka uza ne’e:  Lingua saida maka labarik uza liuliu ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ] Hili ida deit ho marka [X]  Se labarik uza lingua seluk ba atividade ne’e, indika lingua saida maka uza ne’e: 56 Attachment ATIVIDADE 2-1: SURA. 1:1 KORESPONDÉNSIA  SURAT TAHAN 2-1 EZEMPLU PRÁTIKA Tau SURAT TAHAN 2-1 ho sírkulu haat iha labarik nia oin. Hatudu ita boot nia liman husi karuk bá kuana iha sírkulu sira. Iha sírkulu balu iha ne’e. Ha’u hakarak ó atu hatudu ba sírkulu ho ó nia liman-fuan no sura  sírkulu sira ne’e. Hatudu ba sírkulu dahuluk ho ita-boot nia liman-fuan. Hahú iha ne’e no sura sírkulu sira ne’e. Wainhira labarik remato ho sura husu nia lalais: Sírkulu hira maka ó sura ona?  Ne’e loos, 4. Hatudu bá kada sírkulu ho ita boot nia liman-fuan no sura : Um, dois, três, quatro. Iha sírkulu quatro.  Mai ita halo fali jogu seluk. ATIVIDADE 2-2:  SURAT TAHAN 2-2 SURA 1:1 KORESPONDÉNSIA Tau SURAT TAHAN 2-2 ho sírkulu 30 iha labarik  nia oin. Hatudu ita boot nia liman husi karuk bá • Se labarik sura la loos kuana iha sírkulu sira nia leten. • Se nia sura síkulu ida dala rua, ka rua dala ida • Se tempu (segundu 60) hotu ona. Sírkulu sira seluk maka ne’e.  Ha’u hakarak ó atu hatudu ba sírkulu sira ho ó nia liman-fuan no sura sírkulu sira. Hatudu ita-boot nia liman-fuan ba sírkulu sira.  Hahú iha ne’e no sura sírkulu sira. Loos, hahú.  Wainhira labarik remato ho sura husu nia lalais:  Sírkulu hira maka ó sura ona?  Sírkulu ikus ne’ebé sura loloos: [ ]  Númeru sírkulu sira ne’ebé labarik dehan nia sura ona: [ ]  Tempu iha relójiu:  ____ segundu  Lingua saida maka ita boot uza ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ]  Se ita boot uza lingua seluk ba atividade ne’e, indika lingua saida maka uza ne’e:  Lingua saida maka labarik uza liuliu ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ] Hili ida deit ho marka [X] 57 Timor Leste 2011 EGMA : Baseline Report ATIVIDADE 3:  SURAT TAHAN 3-1  (segundu 60) IDENTIFIKASAUN NÚMERU SIRA  SURAT TAHAN 3-2 Tau SURAT TAHAN 2-2 ho sírkulu 30 iha labarik  nia oin. Hatudu ita boot nia liman husi karuk bá • Se labarik halo sala 3 tuituir malu kuana iha sírkulu sira nia leten. • Se tempu (segundu 60) hotu ona. Númeru balun maka ne’e.  Ha’u hakarak ó atu hatudu ó nia liman-fuan ba númeru ida-idak no dehan mai ha’u númeru saida maka ne’e.  Ha’u sei uza relójiu no sei fó hatene ó • Se labarik para iha númeru ida durante segundu 5 wainhira maka haú no wainhira maka nia laran. remata. Hatudu ho ó nia liman-fuan bá númeru primeiru.  Hahú iha ne’e. Loos, hahú. ... Se labarik para iha númeru ida durante segundu 5 nia laran – hatudu ho ó nia liman-fuan bá númeru tuirmai. Númeru ne’e, númeru ne’e saida? Kontinua bá SURAT TAHAN 3-2  LOOS LALOOS, LAIHA LOOS LALOOS, LAIHA RESPOSTA RESPOSTA 1 2 7 65 2 5 8 50 3 9 9 97 4 13 10 104 5 10 11 468 6 18 12 6,430  Total resposta ne’ebé loos: [ ]  Tempu iha relójiu:  ____ segundu  Lingua saida maka ita boot uza ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ]  Se ita boot uza lingua seluk ba atividade ne’e, indika lingua saida maka uza ne’e:  Lingua saida maka labarik uza liuliu ba atividade ne’e? Tetun Português Indonesia Seluk [ ] [ ] [ ] [ ] Hili ida deit ho marka [X]  Se labarik uza lingua seluk ba atividade ne’e, indika lingua saida maka uza ne’e: 58 Attachment ATIVIDADE 4A:   KOMPARASAUN KUANTIDADE   EZEMPLU PRÁTIKA P1, P2 objetu konta nian EZEMPLU PRÁTIKA P1. Uza objetu balun hodi kria grupo rua. Grupu karuk iha objetu 5 no grupo kuana iha objetu 2. Wainhira ita- boot kria hotu ona grupu sira, hatudu grupu sira ho ita-boot nia liman-fuan.   Haree bá grupu rua ne’e. Dehan mai ha’u ida ne’ebé maka boot liu?  Hatudu ho ó nia liman-fuan bá grupu ne’ebé boot liu.  Hatudu grupu 5. Grupu ne’é mak boot liu . 5 boot liu 2. EZEMPLU PRÁTIKA P2. Kria grupo rua. Grupo karuk iha objetu 1 no grupu kuana iha objetu 3. Wainhira ita-boot kria hotu ona grupu sira, hatudu grupu sira ho ita-boot nia liman-fuan.   Haree bá grupu rua ne’e. Dehan mai ha’u ida ne’ebé maka boot liu?  Hatudu ho ó nia liman-fuan bá grupu ne’ebé boot liu. Hatudu grupu 3. Grupu ne’é mak boot liu . 3 boot liu 1.  ATIVIDADE 4:  SURAT TAHAN 4-1  KOMPARASAUN KUANTIDADE  SURAT TAHAN 4-2 Tau objetu sira bá sorin no tau SURAT TAHAN 4-1  iha labarik nia oin. •Se labarik halo sala 3 tuituir malu. Agora, mai ita koko atu uza númeru balun.  Haree bá númeru sira ne’e. Dehan mai ha’u ida ne’ebé maka boot liu.  Kontinua bá SURAT TAHAN 4-2 •Se labarik la responde hafoin segundu 5 liu ona.  RESPOSTA LOOS LOOS LALOOS, LAHATENE, LAIHA RESPOSTA 1 4 2 (4) 2 7 8 (8) 3 14 17 (17) 4 19 18 (19) 5 40 96 (96) 6 79 70 (79) 7 32 36 (36) 8 65 56 (65) 9 145 163 (163) 10 1,400 1,235 (1,400) Total resposta ne’ebé loos: [ ] 59 Timor Leste 2011 EGMA : Baseline Report ATIVIDADE 5:  SURAT TAHAN 5-P  NÚMERU FALTA  SURAT TAHAN 5-1  SURAT TAHAN 5-2 EZEMPLU PRÁTIKA P2. Tau SURAT TAHAN 5-P iha labarik nia oin. Hatudu ita-boot nia liman husi karuk bá kuana. Hatudu ho ita-boot nia liman-fuan bá liña.  Númeru balun maka ne’e. Um, dois, três … Númeru saida maka mai iha ne’e?  Ne’e loos, QUATRO. Mai ita kontinua nafatin. Ne’e, númeru QUATRO maka ne’e. Sura hamutuk ho ha’u.  Hatudu ho ita-boot nia liman-fuan bá kada númeru. Um, dois, três, QUATRO . Ne’e, númeru QUATRO maka ne’e. Mai ita kontinua. Tau SURAT TAHAN 5-1 iha labarik nia oin. Hatudu  ita-boot nia liman husi karuk bá kuana bá • Se labarik halo sala 3 tuituir malu. pergunta. Númeru sira seluk maka ne’e.  Hatudu ho ita-boot nia liman bá liña.   Dehan mai ha’u númeru saida maka mai iha • Se labarik la responde hafoin segundu 10 liu ona. ne’e. Kontinua kada pergunta. Bá kada pontu  pergunta iha liña nia okos no ho liña no hatete: Dehan mai ha’u númeru saida maka mai iha ne’e. Kontinua bá SURAT TAHAN 5-2  RESPOSTA LOOS LOOS LALOOS, LAHATENE, LAIHA RESPOSTA 1 4 5 6 7 7 2 8 9 10 11 11 3 17 18 19 18 4 89 90 91 89 5 20 30 40 40 6 4 3 2 1 1 7 100 200 300 400 400 8 2 4 6 8 8 9 10 15 20 25 20 10 500 400 300 200 300 Total resposta ne’ebé loos: [ ] 60 Attachment ATIVIDADE 6:   PROBLEMA SIRA    Lapis, surat tahan, no objetu konta nian EZEMPLU PRÁTIKA P2. Tau SURAT TAHAN 5-P iha labarik nia oin. Hatudu ita-boot nia liman husi karuk bá kuana. Hatudu ho ita-boot nia liman-fuan bá liña.  Númeru balun maka ne’e. Um, dois, três … Númeru saida maka mai iha ne’e?  Ne’e loos, QUATRO. Mai ita kontinua nafatin. Ne’e, númeru QUATRO maka ne’e. Sura hamutuk ho ha’u.  Hatudu ho ita-boot nia liman-fuan bá kada númeru. Um, dois, três, QUATRO . Ne’e, númeru QUATRO maka ne’e. Mai ita kontinua. Muda dook husi livriñu pájina estudante nian.  • Se labarik halo sala dala 2 tuituir malu. Ha’u iha problema balun ne’ebé ha’u  hakarak ó atu rezolve. Iha ne’e sasán sira balun ne’ebé bele tulun ó atu sura.  Hatudu lapis, surat tahan, no objetu konta nian  bá lababarik. • Se labarik la responde ba pergunta hafoin segundu 10 liu ona, repete pergunta dala ida, hein to’o segundu 20, Ó bele uza sasán sira ne’e se ó hakarak,  maibé la obriga ó. Rona ho atensaun no muda bá pergunta tuirmai. didi’ak. Se ó hakarak, ha’u bele repete fali pergunta. Mai ita hahú. RESPOSTE RESPOSTA LOOS LALOOS, Labarik ne’e: LABARIK LOOS LAHATENE, NIAN LAIHA Liman Uza objetu Halo kalkulasaun RESPOSTA fuan konta nian eskrita 1 Maria iha hudi-tasak fuan 4. Nia mamá iha  9 (4) fuan 5 tán. Hudi-tasak hamutuk hotu fuan hira maka agora sira na’in rua iha? 2 José iha hudi-tasak 3 (17) fuan 6. Nia hán tiha hudi-tasak fuan 3. Hudi-tasak fuan hira maka agora hela? 3 Diva iha lapis 8. Pedro 5 (96) iha lapis 3. Diva lori lapis hira mak liu Pedro? 4 Ha’u iha rebusada 6 (36) fuan 3. Ha’u hakarak fó rebusada ida-ida bá ha’u nia kolega na’in 9. Ha’u presiza hira tan?  Total resposta ne’ebé loos: [ ] 61 Timor Leste 2011 EGMA : Baseline Report ATIVIDADE 7-P:  SURAT TAHAN 7-P ADISAUN NO SUBTRASAUN EZEMPLU PRÁTIKA P1 & P2    Lapis, surat tahan, no objetu konta nian Tau objetu sira iha labarik nia sorin. Tau surat-tahan SURAT TAHAN 7-P iha labarik nia oin. Se labarik la responde loloos bá pergunta P1, repete pergunta dala ida, hein to’o segundu tolu, no pasa bá pergunta tuirmai. EZEMPLU PRÁTIKA P1.  Agora, ita atu rezolve problema balun kona-bá adisaun no subtrasaun Hatudu bá ezersísiu P1. (1+2 = ___) Um mais dois iguala ... ?  LOOS [ ] LALOOS, LAHATENE, LAIHA RESPOSTA [ ]  Ne’e loos, um mais dois iguala três. Um mais dois igual três.  Uza kontadór sira no lee ezersísiu, dudu kontadór bá labarik. Ne’e, ne’e maka um. Hafoin, dudu kontadór rua bá labarik. Ne’e, ne’e maka dois. Sura kontadór tolu ho lian moos. Um mais dois iguala três. Mai ita halo tán bá problema seluk. EZEMPLU PRÁTIKA P2.  Hatudu bá ezersísiu P2. (3–2 = ___) Três menos dois igual ... ?  LOOS [ ] LALOOS, LAHATENE, LAIHA RESPOSTA [ ]  Ne’e loos, três menos dois igual um ... ? Nia resposta maka ida.  Uza kontadór sira no lee ezersísiu, dud kontadór tolu bá labarik. Ne’e, ne’e maka três. Hafoin, hasai tiha rua husi kontadór tolu ne’ebé ita-boot tau ona iha labarik nia oin. Menos dois. Sura kontadór ho lian moos. Um. Três menos dois igual um. Mai ita kontinua. 62 Attachment ATIVIDADE 7-1:  SURAT TAHAN 7-1  (minutu 2) ADISAUN    Lapis, surat tahan, no objetu konta nian Tau SURAT TAHAN 7-1 iha labarik nia oin.  • Se labarik halo sala dala 3 tuituir malu. Hatudu ho ita boot nia liman-fuan bá ezersísiu • Se tempu (minutu 2) hotu ona. adisaun nian dahuluk iha surat-tahan laran.   Hein to’o segundu 20, no muda bá pergunta tuirmai.  Iha koluna RESPOSTE LABARIK NIAN hakerek númeru ne’ebé labarik temi. Se labarik la fó reposta to ka nia la hatene, hakerek ( - ). RESPOSTE RESPOSTA LOOS LALOOS, Labarik ne’e: LABARIK LOOS LAHATENE, Liman Uza objetu Halo NIAN LAIHA fuan konta nian kalkulasaun RESPOSTA Lee ba labakik … eskrita 1 4+5 = Quatro mais dois igual… ?  9  2 8 +2= 10 Oito mais dois igual… ?  3 20+4= 24 Vinte mais quatro igual …?  4 13+12 = 25 Treze mais doze igual … ?  5 11+9= 20 Onze mais nove igual … ?   Total resposta ne’ebé loos: [ ]  Tempu iha relójiu:  ____ : _____  Se ita boot uza lingua seluk (ne’ebe laós Portugues) bá pergunta ne’e, indika lingua saida mak ita boot uza: 63 Timor Leste 2011 EGMA : Baseline Report ATIVIDADE 7-2:  SURAT TAHAN 7-2  (minutu 2) SUBTRASAUN    Lapis, surat tahan, no objetu konta nian Tau SURAT TAHAN 7-2 iha labarik nia oin.  • Se labarik halo sala dala 3 tuituir malu. Hatudu ho ita boot nia liman-fuan bá ezersísiu • Se tempu (minutu 2) hotu ona. adisaun nian dahuluk iha surat-tahan laran.   Hein to’o segundu 20, no muda bá pergunta tuirmai.  Iha koluna RESPOSTE LABARIK NIAN hakerek númeru ne’ebé labarik temi. Se labarik la fó reposta to ka nia la hatene, hakerek ( - ). RESPOSTE RESPOSTA LOOS LALOOS, Labarik ne’e: LABARIK LOOS LAHATENE, Liman Uza objetu Halo NIAN LAIHA fuan konta nian kalkulasaun RESPOSTA Lee ba labakik … eskrita 1 9-5 = Nove menos cinco igual… ?  4  2 10-8= 2 Dez menos oito igual… ?  3 24-4= 20 Vinte quatro menos quatro  igual …? 4 25-12 = 13 Vinte cinco menos doze igual ..?  5 20-9= 11 Vinte menos nove igual … ?   Total resposta ne’ebé loos: [ ]  Tempu iha relójiu:  ____ : _____  Se ita boot uza lingua seluk (ne’ebe laós Portugues) bá pergunta ne’e, indika lingua saida mak ita boot uza: 64 Attachment REMATA ONA EZERSÍSIU EGMA – HAKEREK TEMPU REMATA NIAN IHA PÁJINA 1 MOLOK MUDA BÁ KESTIONÁRIU. KESTIONÁRIU Ita besik atu hotu ona! Agora iha pergunta oituan tán de’it kona-ba ó nia esperiénsia ho  Matemátika. 1 Molok tama Primeiro Ano ó tama LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] eskola TK ka lae? 2 Iha eskola, imi hetán disiplina LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] Matemátika loron-loron? 3 Ó sente Matemátika difisil ka fasil? DIFISIL [ ] FASIL [ ] LAIHA RESPOSTA [ ] 4 Iha eskola, ó uza livru tekstu LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] Matemátika? 5 Iha eskola dala ruma uza objetu sira LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] hodi aprende no resove problema Matemátika? 6 Iha eskola, dala ruma servisu LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] hamutuk iha grupu hodi aprende no resove problema Matemátika? 7 Ó nia profesór/a fó Traballu Para LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] Kaza Matemátika bá imi? SE LAE BA PERGUNTA 9. 8 Ema ruma tulun ó ho nia Traballu LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] Para Kaza? 9 Ó nia ama ka apa fa’an sasaan LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] (kios/loja/merkadu)? 10 Iha ó nia uma iha televisaun ka lae? LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] 11 Iha ó nia uma iha radio ka lae? LOOS [ ] LAE [ ] LAIHA RESPOSTA [ ] LAHATENE [ ] Remata ona! Ha’u kontente teb-tebes. Agora ó bele fila hirak bá loos  klase. Halo favór labele ko’alia ho labara sira seluk ne’ebé atu mai. SE ITA-BOOT HAKARAK, HAKEREK ITA-BOOT NIA KOMENTÁRIU IHA NE’E BÁ PREUKUPASAUN KONA-BÁ LINGUA NO PROBLEMA SIRA SELUK RUMA NE’EBÉ ITA-BOOT HETAN IHA IMPLEMENTASAUN EVALUASAUN IDA NE’E NIA LARAN 65 Timor Leste 2011 EGMA : Baseline Report 66 Annex Three: Word Problems Per-Item Analysis Sub-test 6 asked the student to answer four word problems. The problems were in Tetum and the text is given in Annex 2. An English translation of each of the problems is given as follows. 1. Maria has 4 bananas. Her mother has 5 bananas. How many do they have in total now? 2. Jose has 6 bananas. He eats 3 bananas. How many are left now? 3. Diva has 8 pencils. Pedro has 3 pencils. How many pencils does Diva have more than Pedro? 4. I have 3 sweets [bananas]. How many more bananas do I need if I want to give one sweet [banana] to my 9 friends. Table 29 gives the percentage of students able to answer each item in this sub-test correctly. Problem 2 was answered correctly most often, which was surprising given that most students found the subtraction sub-test 8 more difficult than sub-test 7 (addition). Problem 1, an addition problem was answered correctly by 64% of students overall. Problem 3, was answered by only 20% of students while Problem 4 was answered by only 27% of students. These were also subtraction problems but required a more complex reasoning. TABLE 29. Per-item Analysis, Sub-test 6 Grade Statistics Problem 1 Problem 2 Problem 3 Problem 4 Mean 0.4078 0.4854 0.1044 0.1117 1 SD 0.4920 0.5004 0.3061 0.3153 N 412 412 412 412 Mean 0.6683 0.7599 0.1881 0.2624 2 SD 0.4714 0.4277 0.3913 0.4405 N 404 404 404 404 Mean 0.8341 0.9000 0.3024 0.4220 3 SD 0.3724 0.3004 0.4599 0.4945 N 410 410 410 410 Mean 0.6362 0.7145 0.1982 0.2651 Total SD 0.4813 0.4518 0.3988 0.4416 N 1226 1226 1226 1226 67 Timor Leste 2011 EGMA : Baseline Report 68 69 Timor Leste 2011 EGMA : Baseline Report 70