Indonesia
MAIN REPORT
A Video Study of Teaching Practices in
TIMSS Eighth Grade Mathematics
Classrooms
Understanding What Teaching Practices are Used,
Why They are Used and How They Relate to Student Learning
÷
+ 5
1
Main Report
KEMENTERIAN
PENDIDIKAN NASIONAL i
The Word Bank Office Jakarta
Indonesia Stock Exchange Building, Tower II, 12th floor
Jl. Jend. Sudirman Kav. 52-53
Jakarta 12190
Tel : (+6221) 5299-3000
Fax : (+6221) 5299-3111
http://www.worldbank.org/id
Published in December 2015
Indonesia – A Video Study of Teaching Practices in TIMSS Eight Grade Mathematics Classrooms: Understanding What Teaching Practices are Used,
Why They are Used and How They Relate to Student Learning is a product of the staff of the World Bank. The findings, interpretation, and conclusions
expressed herein do not necessarily reflect the view of the World Bank, its board of Executive Directors, officers or any of its member countries.
The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information
shown on any map of this work do no imply any judgment on the part of the World Bank concerning the legal status of any territory or the
endorsement or acceptance of such boundaries.
The work is funded by the Government of the Kingdom of the Netherlands under the Dutch Education Support Program (DESP) Trust Fund. DESP
provides support to the Government of Indonesia through the World Bank for the purpose of developing policies, studies, and programs that help
the government achieve its education strategic plan.
Indonesia
A Video Study of Teaching Practices
in TIMSS Eighth Grade Mathematics
Classrooms
Understanding What Teaching Practices are Used,
Why They are Used and How They Relate
to Student Learning
MAIN REPORT
May 27, 2015
Table of Contents
TABLE OF CONTENTS
Acknowledgements......................................................................................................................xiii
List of Abbreviations....................................................................................................................xvi
Preface on the Indonesia Video Study Series.................................................................................xviii
Executive Summary......................................................................................................................xx
Study Background................................................................................................................................... xx
Key Findings on Teacher Knowledge....................................................................................................... xxi
A Bigger Toolbox:......................................................................................................................... xxi
Better Tools: ................................................................................................................................. xxi
Key Findings on Teaching Practices......................................................................................................... xxii
Interaction.................................................................................................................................... xxii
Questioning.................................................................................................................................. xxii
Methods....................................................................................................................................... xxiii
Use and Approach for Mathematical Problems............................................................................. xxiii
Key Findings on Trends in Teaching Practices......................................................................................... xxiv
Key Findings on the Beliefs-Knowledge-Practices Relationship............................................................... xxvi
Implications: Relevance in the Context of Recent Policies on Teaching Practices and
the 2013 Curriculum.................................................................................................................... xxvi
Learning Approach....................................................................................................................... xxvi
Teaching Approach....................................................................................................................... xxvi
The 2013 Curriculum in the Context of Mathematics.................................................................. xxvi
Promising Approach, but a Big Gap to Fill................................................................................... xxvi
Implications: The Challenge of Changing Practices................................................................................. xxvii
Effective Change in Practices Requires Realigning Teacher Knowledge and Beliefs........................ xxvii
Implications: Relevance for the Teacher Professional Management System.............................................. xxviii
Effective Use of Competency Testing Requires Political Will…..................................................... xxix
…As Well as a Reliable and Effective Instrument.......................................................................... xxix
Continuous Professional Development (CPD).............................................................................. xxix
From Research to Action: A CPD Module Based on Study Findings....................................................... xxx
An Innovative Pilot as One Potential In-service Approach............................................................ xxx
Pre-service Training Must Also be in Line with the New Approaches............................................ xxxi
Overall Key Messages.............................................................................................................................. xxxi
Main Report v
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Background and Context.............................................................................................................1
1.1 Teacher Reform in Indonesia........................................................................................................ 3
1.2 Mathematics Teaching in Indonesia.............................................................................................. 4
1.3 Video Study of TIMSS Classrooms in Indonesia........................................................................... 5
Overview of Design, Methods and Analytical Approach...............................................................7
2.1 Introduction................................................................................................................................. 8
2.2 Objectives..................................................................................................................................... 8
2.3 Research Questions....................................................................................................................... 9
2.4 Why a Video Study?..................................................................................................................... 10
2.5 Capturing Trends Over Time........................................................................................................ 10
2.6 Conceptual Framework................................................................................................................. 12
2.6.1 Incorporation of Student Learning into the Analysis......................................................... 15
2.7 General Methodological Aspects................................................................................................... 15
2.7.1 Mixed Methods Approach................................................................................................ 15
2.7.2 Unit of Study and Analysis............................................................................................... 16
2.7.3 Sampling.......................................................................................................................... 16
2.8 Instruments.................................................................................................................................. 19
2.9 Case Studies.................................................................................................................................. 20
2.9.1 Length and Scheduling of Visit ....................................................................................... 22
2.10 Linking Components.................................................................................................................... 22
2.10.1 Overview of Relationship Mapping.................................................................................. 22
2.10.2 Analytical Steps Framework.............................................................................................. 24
2.11 Macro to Mezzo to Micro Analysis................................................................................................ 25
2.12 Regression Modelling Approach.................................................................................................... 27
2.12.1 Selection of Control Variables........................................................................................... 27
2.12.2 Progressive Building-Up of the Model.............................................................................. 28
2.12.3 Presentation of Results in Relationship Matrices............................................................... 29
2.12.4 Data Mining for Patterns.................................................................................................. 30
2.12.5 Confounding Factors........................................................................................................ 31
Teaching Practices .......................................................................................................................33
3.1 Proportions, Trends and Cross-Country Comparisons.................................................................. 34
3.2 Relationships between Teaching Practices and Learning Outcomes............................................... 39
vi Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Table of Contents
Mathematical Knowledge in Teaching.........................................................................................41
4.1 Teacher Subject and Pedagogy Competency Assessment............................................................... 44
4.2 Teacher Mathematical Scenarios Interview.................................................................................... 45
4.2.1 Summary of Case Studies on the Mathematical Scenarios................................................. 46
4.3 Linking MKiT Assessment and Scenarios Results......................................................................... 48
4.4 Relationship of Teacher Knowledge with Student Learning Outcomes.......................................... 48
4.5 Key Points on Teacher Knowledge................................................................................................ 50
Teacher Beliefs on Teaching and Learning....................................................................................51
5.1 Teacher Mathematical Beliefs Survey............................................................................................ 53
5.2 Case Study Interviews on Teacher Mathematical Beliefs................................................................ 56
5.3 Linkages Between Beliefs Survey and Beliefs Interview.................................................................. 58
5.4 Beliefs Reconceptualised............................................................................................................... 59
Exploring the Beliefs-Knowledge-Practices Relationship..............................................................61
6.1 The Beliefs-Knowledge Link......................................................................................................... 63
6.1.1 Macro View: Linking the Beliefs Survey and Teacher Assessment Results.......................... 63
6.1.2 Case Study Insights on the Beliefs-Knowledge Relationship............................................. 70
6.2 The Beliefs-Practices Link............................................................................................................. 72
6.2.1 Exploring Patterns in Beliefs-Practices.............................................................................. 73
6.2.2 Beliefs Theme: The Role of the Teacher............................................................................ 75
6.2.3 Beliefs Theme: Tools and Resources, Management and Evaluation................................... 77
6.2.4 Beliefs Theme: Practices for and Processes of Learning...................................................... 78
6.2.5 Case Study Teacher Insights on Beliefs-Practices........................................................................... 80
6.2.6 Key Concluding Points on Beliefs-Practices...................................................................... 84
6.3 The Knowledge-Practices Link...................................................................................................... 85
6.3.1 Linking Practices with MKiT Assessment Scores.............................................................. 86
6.3.2 Use of Practices in Relation to Knowledge Indicators....................................................... 87
6.3.3 Case Study Relationships.................................................................................................. 89
6.4 Key Concluding Points on Relationships...................................................................................... 90
Main Report vii
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A New Conceptual Model of Beliefs-Knowledge-Practices for Indonesia’s Teachers.....................93
7.1 Representing Beliefs, Knowledge and Practices as Discs................................................................ 94
7.2 Disc Size Significance.................................................................................................................... 94
7.3 Disc Interaction............................................................................................................................ 95
7.4 Gravitational and Repelling Forces within the Model.................................................................... 97
7.5 The Cultural Dimension as a Key Driver...................................................................................... 99
7.6 Application of the Model to Specific Instances.............................................................................. 100
7.6.1 Large Dissonant Versus Large Congruent Zone................................................................ 100
7.6.2 Comparison of Higher and Lower Knowledge Teachers.................................................... 101
7.6.3 Key Conjectures from the Model...................................................................................... 103
Themes on Teaching Practices and Policies..................................................................................105
8.1 How Teacher Knowledge is Enacted in the Classroom.................................................................. 106
8.2 Teacher-Student Interaction.......................................................................................................... 113
8.2.1 What is a Student-Centered Approach?............................................................................ 113
8.2.2 Interaction and Recent Policies on Teaching Practices....................................................... 114
8.2.3 Interaction of Teachers and Students................................................................................ 114
8.2.4 What Type of Interaction Really Matters?......................................................................... 118
8.3 Teaching Practices Encouraged over the Past Decade.................................................................... 120
8.4 The National Exam and Other Influences on Teaching Practices................................................... 126
8.4.1 What Do the Trends Tell Us about Teacher Reform and Other Influencing Factors?......... 126
Overarching Conclusions from the Study Findings .....................................................................133
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module.........137
10.1 Using Key Study Findings to Identify Areas for Improvement....................................................... 138
10.2 An Innovative Approach to CPD through the Use Of Video........................................................ 138
10.2.1 In-On-In Approach.......................................................................................................... 139
10.2.2 Elpsa Framework.............................................................................................................. 139
10.2.3 Incorporation of Videos into the Module......................................................................... 140
10.2.4 Facilitators as an Integral Part of Session Activities............................................................ 140
10.3 Implementation in Teacher Working Groups................................................................................ 141
10.4 Impact on Teacher Beliefs, Knowledge and Practices..................................................................... 142
10.5 Key Lessons Learned and Considerations...................................................................................... 143
Content........................................................................................................................................ 143
Implementation............................................................................................................................ 144
Effectiveness................................................................................................................................. 145
Scaling Up and Sustainability........................................................................................................ 146
10.6 Wider Policy Implications Emerging from the Pilot...................................................................... 146
viii Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Table of Contents
References....................................................................................................................................149
Appendices...................................................................................................................................163
Coding Layers and Definitions................................................................................................................ 164
Graphical Mapping of Findings for Each Stage of Analysis...................................................................... 167
Distribution of Responses for Each Statement in the Mathematical Beliefs Survey.................................. 168
Likert Scale Responses (Scale Of 1-5) .......................................................................................... 168
Frequency of Teaching Practices in the Teacher’s Lessons............................................................... 170
Ranking Responses....................................................................................................................... 171
Summary of Teacher Responses to Mathematical Scenarios..................................................................... 172
Technical Report as a Separate Appendix................................................................................................. 173
Main Report ix
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
INDEX OF FIGURES
Figure 1: Estimated Impact of High vs. Low Performing Teachers on Student Achievement................... 2
Figure 2: Concept of Measuring Trends in Teaching Over Time............................................................. 11
Figure 3: Framework for Analysis of Teacher Practices, Beliefs and Knowledge....................................... 12
Figure 4: Mapping of Instruments by Full Sample vs. Case Study Sample.............................................. 20
Figure 5: Data Shown as Levels of Perspective for MKiT and Mathematical Beliefs................................ 23
Figure 6: Four Stages of Analysis: (1) Individual; (2) Component; (3) Cross-component;
and (4) Full-component........................................................................................................... 24
Figure 7: Breakdown of Macro-Mezzo-Micro Approach to Analysis....................................................... 26
Figure 8: Example of Coding of Regression Results to Indicate Relationships......................................... 29
Figure 9: Proportional Breakdown of Lesson Time................................................................................. 34
Figure 10: Duration for Different Activities in Indonesia and Other Countries........................................ 35
Figure 11: Time for Different Learning Activities in Mathematics Lessons............................................... 35
Figure 12: Percentage of Time for Public Interaction and Private Interaction............................................ 36
Figure 13: Public (Whole Class) Interaction Breakdown: Teacher and Student Involvement..................... 36
Figure 14: Average Number of Words Spoken by Teacher and Students During the Lesson...................... 37
Figure 15: Mathematical Time Divided into Problem vs. Non-Problem Segments (Percent of total)......... 38
Figure 16: Scatter Plot of Teacher Subject and Pedagogy Scores on the Teacher Assessment...................... 44
Figure 17: Relationship between Teacher Knowledge and Student Learning............................................. 49
Figure 18: Likert Scale Statements with Large Majority of Responses being Either Positive or Negative.... 53
Figure 19: Distribution of Frequency Scale Responses in the Mathematical Beliefs Survey....................... 55
Figure 20: Frequency Scale Results on a Sample of Teaching Practices...................................................... 55
Figure 21: Proportions of the Relationship between Beliefs Questions and Knowledge............................. 63
Figure 22: Beliefs Statements Having a Relationship with Both Subject and Pedagogical Knowledge........ 65
Figure 23: Beliefs Statements Having a Relationship with Subject But Not Pedagogical Knowledge......... 66
Figure 24: Examples of Beliefs Statements with a Relationship with Pedagogical But Not
Subject Knowledge.................................................................................................................. 66
Figure 24: Example of Beliefs Statement with Subject and Pedagogy MKiT Scores in
Opposite Directions................................................................................................................ 67
Figure 26: Beliefs Statements where MKiT and Student Learning have Opposite Relationships............... 68
Figure 27: Scatter Plot of Teacher Subject and Pedagogy Scores on the Teacher Assessment...................... 71
Figure 28: Matrix with Beliefs Role of the Teacher related to Actual Practices........................................... 75
Figure 29: Matrix with Beliefs on Teaching Approach, Assessment, and Resources Related to
Actual Practices........................................................................................................................ 77
Figure 30: Matrix with Beliefs on Learning Related to Actual Practices..................................................... 78
Figure 31: Teaching Practices of Teachers in Relation to Their Overall MKiT Score................................. 87
Figure 32: Conceptual Model of Beliefs-Knowledge-Practices Interaction................................................ 94
Figure 33: Comparison of Teachers Reacting to Influences....................................................................... 96
Figure 34: Large Dissonant vs. Large Congruent Zone............................................................................. 98
Figure 35: Illustrative Contrasting Example between Higher and Lower Knowledge Teachers.................. 99
x Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Index of Figures
Figure 36: Distribution of Scores of Mathematics Teachers in UKG......................................................... 104
Figure 37: Integrated Framework for the Teacher Professional Management System................................. 109
Figure 38: Proportions of Whole-class vs. Group and Seatwork................................................................ 113
Figure 39: Whole-class Work is on the Increase, while Group Work is Declining..................................... 113
Figure 40: Ratios of Words from the Teacher for Every Word from a Student........................................... 116
Figure 41: Proportion of Lesson Time by Methods Used.......................................................................... 120
Figure 42: Problem and Question Contexts.............................................................................................. 120
Figure 43: Theoretical Model Showing Alignment of Teaching Practices with Knowledge and Beliefs....... 123
Figure 44: Proportion of Mathematics Time by Structure of Interaction................................................... 125
Figure 45: Proportion of Whole Class Time by Interaction Type.............................................................. 125
Figure 46: Proportion of Teaching Methods and Approaches Used........................................................... 126
Figure 47: Proportion of Mathematics Time as Problem vs. Non-problem Time...................................... 127
Figure 48: Proportion of Problem Work as Involving Routine versus Non-Routine Problems................... 128
Figure 49: Proportion of Problem Work as Involving Mathematical Language vs. Real World Problems... 128
Figure 50: Differences in Teaching Practices of Certified Teachers vs. Uncertified Teachers in
Indonesia TIMSS Video Study of Mathematics for the 2011 Phase......................................... 129
Figure 51: Example of ELPSA for the Topic of Right Triangles................................................................. 138
Main Report xi
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
INDEX OF TABLES
Table 1: Research Data, Sources and Sample Grouping......................................................................... 19
Table 2: Case Study Activities and Their Purpose.................................................................................. 21
Table 3: Variables Used in Various Layers for in the Regression Analysis............................................... 28
Table 4: Sources of Data for MKiT analysis.......................................................................................... 44
Table 5: Beliefs Lenses and Description of Categories Used in the Analysis........................................... 52
xii Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Acknowledgements
ACKNOWLEDGEMENTS
This report is the result of a joint program between the World Bank and the Ministry of Education and Culture
(MOEC). The report was authored by Andy Ragatz under the operational leadership of Susie Sugiarti and overall
guidance of Susiana Iskandar and Ratna Kesuma, but it is just one piece of an extensive two-phase study that
involved a large team and support from many government officials and stakeholders.
The original design of the study was done by Dr. Frederick Leung (University of Hong Kong). The design was based
heavily on extensive work done for the report, Teaching Mathematics in Seven Countries: Results from the TIMSS
1999 Video Study (Hiebert et al, 2003) in order to allow for comparison of Indonesia’s results to those of other seven
other countries that conducted similar studies. Subsequent components were added to the 2011 phase.
The implementation was led by Susie Sugiarti, with a core team that consisted of senior mathematics instructors
from the Center for Development and Empowerment of Mathematics Teacher and Education Personnel (P4TK),
the Educational Quality Assurance Institution (LPMP), and some junior secondary schools. Team members
included Adi Wijaya, Budiharjo, Erwin Roosilawati, Al Krismanto, Rachmadi Widdiharto, Puji Iryanti, Reubun,
Suhendro, Sri Yuniati, Suwarkono, Sri Yuniarti, Ade Sunawan, and Isfarudi. The study team tirelessly analyzed and
reanalyzed the videos to provide the essential coding results, and produced initial analysis and qualitative studies
that contributed to this report. Aria Jalil was also a key author of various pieces and conducted the case studies with
Andy Ragatz.
The MOEC was heavily involved in all stages. Fasli Jalal (former Directorate General of Quality Improvement of
Teacher and Education Personnel) provided the vision and initial provisions to launch the study. Baedhowi (former
Directorate General for Quality Improvement of Teacher and Education Personnel), Sumarna Surapranata (Director
General of Teacher and Education Personnel Development), Totok Suprayitno (Head of Agency of Research and
Development), Nizam (Head of Center of Education Assessment) provided supports for the dissemination of the
study findings through various events. Bastari, Burhanudin Tola, Nugaan Yulia Whardani Siregar and Hari Setiadi
(former Heads of Center of Education Assessment) supported the development of student and teacher assessment
instruments and access to TIMSS schools. Abi Sujak, Unifah Rosyidi and Dian Wahyuni (officials of former Agency
of Human Resource Development on Education and Culture and Education Quality Assurance) not only provided
leadership and guidance, but also continually participated in and supported the activities. Other technical team
members are listed on the next page.
The report benefited from the excellent comments and advice of several World Bank experts, including Barbara
Bruns (peer reviewer), Helen Craig (peer reviewer), Muna Salih Meky (peer reviewer), Susiana Iskandar, Ratna
Kesuma, Samer Al-Samarrai, Tazeen Fasih and Cristobal Ridao-Cano, as well as production support from Santi
Sugiarti Santobri.
But the biggest contributors to this effort were the over 200 teachers who welcomed us into their classroom and
participated in the program.
Main Report xiii
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Members of Technical Team of MOEC TIMSS Video Study 2011
Center of Education Assessment (Puspendik)
School sampling, student and teacher test instrument development, socialization and instrument piloting, pre-test and
teacher assessment, post-test:
1. Rumondang Purwati (Coordinator/Contact Person)
2. Erika Afiani (Contact Person for TIMSS Score results)
3. Ainun Salim
4. Rahmah Zulaiha
5. Rustomo
6. Rina Rosdiani Patria
7. Lilis W
8. Devi
9. Siti Nadhiroh
10. Fahmi
11. Fitri
12. RR Herlina
13. Malik Azis
14. Zul Arsiah
15. Aris Subranto
16. Benny Widaryanto
17. Rahmawati
18. Nonny S
19. Deddy Rokhidin
Directorate General for Quality Improvement of Teacher and Education Personnel/Agency of Human
Resource Development on Education and Culture and Education Quality Assurance
1. Gogot Suharwoto
2. Emy Fitriani
3. Fitriadi
Student pre-test and teacher assessment and post-test (Administrator test)
1. Fitriadi
2. Laura Rossiana
3. Yusrizal
xiv Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Acknowledgements
Directorate for Management of Junior Secondary Education
Student pre-test and teacher assessment and post-test (Administrator test)
1. Evi Maharani
2. Henny Harsian
Videotaping
1. Juandanilsyah
2. Nyoman Ekasanagiri
Education Quality Assurance Institute & Education Office (Non Core Team)
Post test (Administrator test)
1. Soko B. Oswaldus (Pengawas Dinas Pendidikan Ende, NTT)
2. Drs. Ola Sanga Yohanes, MM (Widyaiswara LPMP NTT)
3. Baharudin (Widyaiswara, LPMP Sulawesi Selatan)
4. Amir Daud (Widyaiswara, LPMP Sulawesi Selatan)
5. Hadi (Widyaiswara, LPMP Sulawesi Tengah)
6. Lapahisa (Widyaiswara, LPMP Sulawesi Tenggara)
7. Bobi Abdul Ghani (Widyaiswara, LPMP Gorontalo)
8. Kasim (Widyaiswara, LPMP Kalimantan Barat)
9. Husein Siregar (Widyaiswara, LPMP Kalimantan Timur)
Main Report xv
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
LIST OF ABBREVIATIONS
AC Action-related Competencies
BK Basic Knowledge
BKP Beliefs, Knowledge and Practices
CCK Common Content Knowledge
CPF Connectionist-Progressive-Facilitator (used to describe teachers tending to hold the combined beliefs
categorizations)
CK Content Knowledge
CS Case Study (often used as, for example, CS1 representing Case Study 1)
CLCC Creating Learning Communities for Children (a UNICEF program in Indonesia)
COACTIV Professional Competence of Teachers, Cognitively Activating Instruction, and the Development of
Students’ Mathematical Literacy (study)
CTL Contextual Teaching and Learning (which refers to an Indonesian policy initiative)
IEA International Association for the Evaluation of Educational Achievement
KCS Knowledge of Content and Students
METE Mathematics Education Traditions in Europe (project)
MKiT Mathematical Knowledge in Teaching. (Also often referred to as Mathematical knowledge for teaching,
and combined Content Knowledge and Pedagogical Content Knowledge)
xvi Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
List of Abbreviations
MOEC Ministry of National Education and Culture in Indonesia (previously MONE: Ministry of National
Education)
MT21 Mathematics Teaching in the 21st Century (project)
PCK Pedagogical Content Knowledge
PGCE Post Graduate Certificate in Education
PISA Programme for International Student Assessment
RC Reflective Competencies
RQ Research Question
SCK Specialized Content Knowledge
TEDS-M Teacher Education and Development Study in Mathematics (cross-country study)
TIMSS Trends in International Mathematics and Science Study
TTI Traditional-Transmissionist-Instructor (used to describe teachers tending to hold the combined beliefs
categorizations)
WB World Bank
Main Report xvii
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
PREFACE ON THE INDONESIA
VIDEO STUDY SERIES
This report is one in a series stemming from the Indonesia Trends in International Mathematics Study (TIMSS)
Video Study project, a research program to examine in extensive detail the practices of mathematics teachers in
eighth grade classrooms across Indonesia. More than 200 mathematics teachers participated in the study from 2007
to 2011. The study involved in-depth research based on quantitative and qualitative data analysis into teaching
methods and behaviors in the classroom, and the impact of these various teaching methods on student learning
outcomes. The main findings from the study were later used to develop a pilot training module for mathematics
teachers to use in regular workshops as part of their continuous professional development (CPD).
The study itself was comprehensive and highly complex. The analysis utilized multiple frameworks and sub-
frameworks. The dynamic interactions analysis involved literally thousands of regressions and connected qualitative
data from interviews of various types with quantitative data involving video coding of teaching practices, teacher
competency testing, surveys, and student pre- and post-tests.
Because of the study’s complexity, providing details of all the findings while simultaneously conveying key messages
in a digestible manner would simply not have been effective; the messages would become lost in the depths of the
technical analysis. For this reason it was decided to separate most of the technical analysis from the main report. A
Technical Report has been developed to provide the important details of the design, analysis and general findings.
This Main Report, on the other hand, does not attempt to provide all details, but rather just key aspects of the
design and analysis. Only key findings are presented and they are grouped in a thematic manner. For readers
interested in details on the design, instrument development, implementation and results, the Technical Report will
be most relevant. For readers interested in a summary of key findings, the Main Report will be of more interest.
In addition, a series of briefs and infographics has been created to cover specific topics that would make the study
results accessible to different audiences. Finally, a videograph was created that provides an animated story about the
video study, how it relates to key policies and how the CPD module was developed based on the study findings to
support strengthening of teaching practices.
xviii Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Preface on the Indonesia Video Study Series
Main Report Technical Report
Background and context Full literature review of international research related to the
components of the study
Brief overviews of international research related to various In-Depth description of design, methods and analytical approach
components of the study
Overview of Design, Methods and Analytical Approach Full results of all analysis
Select results on teaching practices, beliefs and teacher Detailed presentation of results including step-by-step
knowledge
Select results Full description of new conceptual model for the dynamic beliefs-
knowledge-practices relationship
Full description of new conceptual model for the dynamic Very little policy discussion
beliefs-knowledge-practices relationship
Thematic presentation of key teaching practices, trends and Full appendices of instruments, case study profiles, results used for
policy implications pattern analysis
Very few appendices; references instead to the Technical Report
Summary of pilot teacher continuous professional development
module that put the study findings into practice
Main Report xix
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
EXECUTIVE SUMMARY
Indonesia is a committed participant in the Trends in International Mathematics and Science Study (TIMSS),
the Program
for International Student Assessment (PISA) and the Progress
in International Reading Literacy
Study (PIRLS) international student examinations and is one of
the few non-OECD (Organisation for Economic
Cooperation and Development) countries to participate so fully.
Indonesian student performance in these examinations has been relatively low, even when taking socio-economic
levels into account. For example, 8th grade mathematics in Indonesia ranked 39 out of 43 participating countries in
TIMSS 2011, and its score was more than one standard deviation below the international average. While the results
are useful in providing an indication of Indonesia’s relative standing in student achievement and its progress over
time, the real challenge is to take the next step and translate the results into an understanding of the factors driving
student learning outcomes and what might be done to enhance student achievement in Indonesia.
STUDY BACKGROUND
The Indonesia TIMSS Video Study of 8th-grade mathematics teachers was a collaborative study between the Ministry
of Education and Culture (MOEC) and the World Bank under the Dutch Education Support Program (DESP).
The study aimed to determine not only what takes place in the classroom, but also gain a better understanding of
why it takes place and how teaching practices relate to student learning. Detailed coding of videotaped lessons for
200 teachers provided insights on classroom activities. The use of video also allowed for in-depth explorations on
key practices such as the handling of student misconceptions and the use of questioning.
The study is one of the most wide-reaching and in-depth studies of classroom practices in the world. It followed
a coding methodology used by seven countries in the 1999 TIMSS video study, allowing for cross-country
comparisons. It also included two phases (2007 and 2011), allowing for analysis of trends in practices. The study
goes beyond what happens in the classroom, also attempting to understand why it happens through exploration of
the relationship of teaching practices with teacher mathematical knowledge and mathematical beliefs. Practices are
also explored in relation to student learning to understand what may be most effective in the Indonesia context. The
research was later turned into action, with the findings and video footage used to produce a course module for use
in Indonesia’s mathematics teacher working groups.
xx Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
KEY FINDINGS: TEACHER KNOWLEDGE
Of all education system variables, teacher knowledge emerged as having the strongest relationship with student
learning outcomes. While the positive relationship itself is not surprising, the study uncovered the important ways
in which knowledge plays a role and how it is employed in the classroom. It can be summarized as teachers having
a bigger toolbox, as well as better tools.
A BIGGER TOOLBOX:
Higher knowledge teachers tended to utilize a greater range of practices. Techniques such as investigation, open-
ended questioning, the use of mathematical language and symbols and the use of non-routine problems and
applications tended to be found more in classrooms with higher knowledge teachers. This appears to be in part
because the techniques themselves require a greater amount of subject mastery to be conducted effectively. For
example, open-ended questioning requires that the teacher develop questions that could be answered in a variety
of ways and tend to be more complex. When in the classroom, the teacher must then be prepared for the different
possible responses of students, have contingencies for the different responses and possibly also involve higher-order
thinking.
BETTER TOOLS:
While higher knowledge teachers tended to have a bigger toolbox, they also tended to have better tools. Further
in-depth qualitative analysis uncovered important differences in the ways in which higher knowledge teachers used
the same practices (giving them better tools). Teachers with higher knowledge tended to:
1. Make fewer mathematical errors
2. Use more accurate language and be more concise in their explanations
3. Come up with new problems that were not originally in the lesson plan
4. Make a greater attempt to probe student thinking
5. Be more able to spot student misconceptions and effectively correct them
6. Involve a greater range of students, including those who might not provide correct answers
7. Demonstrate more confidence and command in the classroom
8. Stimulate more teacher-student interaction, with the interaction involving more higher order thinking
9. Use questioning more and encouraging students to think and share ideas rather than just follow instructions
10. Make connections between procedures and concepts
Main Report xxi
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
KEY FINDINGS: TEACHING PRACTICES
Teaching practices were explored mainly through the coded video data to determine proportions of lesson time for
each practice and counts of frequency of use. These practices were also examined from a qualitative standpoint to
better understand how teachers actually utilized these practices. The relationship of these practices were also related
to student learning outcomes to provide an indication of whether they might be effective in the Indonesian context,
but with an understanding that its effectiveness is highly contextual and dependent on the teacher and the way in
which he/she actually enacts the practice in the classroom.
INTERACTION
One of the key focus areas for the Video Study was the interaction that takes place between teachers and students.
Whole-class time tended to be dominated by a teacher-centered approach, with 74 percent of time being teacher-
only. Teacher-student interaction in whole class time was only 11 percent, but had a positive relationship with
student learning whereas teacher-only time had a negative relationship.
For group work and seatwork, an important finding is that students whose teachers spent a higher proportion of time
monitoring and facilitating had higher learning outcomes. These findings of the positive relationship of teacher-
student interaction across whole-class, group work and individual seatwork highlight the critical importance that
teacher-student interaction plays in Indonesia’s mathematics classrooms for student learning. They also indicate that
a student-centered approach can be effective in the Indonesian context, while the teacher-centered approach found
in most classrooms tends to be less effective.
QUESTIONING
Questioning can be a powerful instrument in a mathematics teacher’s repertoire of practices, serving various
purposes in defining the dynamics of the classroom and developing student learning. An in-depth exploration of
how teachers use questioning revealed that the questioning tends to be superficial, with nearly 90 percent of student
responses involving a single word and very rarely were the questions involving higher order thinking or explanation
of reasoning. The use of open-ended questions was rare, with nearly all questions being closed. Teachers also tended
to use pseudo questions or testing questions, very rarely using the more effective questioning techniques of genuine
questions or directing and provoking questions.
xxii Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
METHODS
Tied to the fact that a teacher-centered approach tended to dominate in most classrooms, most lesson time involved
exposition and lecturing, making up 61 percent of mathematics lesson time. The more enquiry and real-world
context methods of investigation and hands-on practical work tended to be very rare, making up only 1 percent and
7 percent, respectively. In terms of relationships with student learning outcomes, exposition/lecturing had a negative
relationship whereas investigation and problem-solving had a positive relationship.
USE AND APPROACH FOR MATHEMATICAL PROBLEMS
There was little variation in the approaches used for mathematical problems, which tended to be nearly all routine
and closed, and involving mathematical language and symbols. Problems that were non-routine, open or involving
real-world contexts were extremely rare. In terms of relationships with student learning outcomes, non-routine and
open problems and problems using the context of mathematical language and symbols had a positive relationship.
The negative relationship of student learning outcomes with real-world contexts was surprising, but also may have
had to do with the fact that the student tests were based on Indonesia’s National Exam and TIMSS, which tend to
focus on procedures and formulas rather than applications and real-world contexts.
KEY FINDINGS: TRENDS IN TEACHING PRACTICES
Trends in teaching practices between 2007 and 2011 indicate a shift to a more teacher-centered, procedural, routine
approach. Teacher-only time as a proportion of whole-class time increased from 59 to 64 percent, while the teaching
method of exposition/lecturing as a proportion of mathematics time increased from 52 to 61 percent. Group work
decreased from 21 percent of lesson time to 11 percent. New content increased from 47 percent of lesson time to
58 percent. The proportion of problem work involving non-routine problems (vs. routine) decreased from 12 to 4
percent, while problems involving real-world contexts (vs. mathematical language and symbols) decreased from 11
to 7 percent. In problem set-up, use of procedures increased from 40 to 64 percent, while stating a concept dropped
from 54 to 30 percent.
What might be driving these shifts? Influences are complex and cannot be narrowed down to a couple of factors.
Still, there do seem to be some factors that help explain the trends. One appears to be that the annual National
Exam (Ujian Nasional, or UN) taken by all primary and secondary students is having greater influence on teaching
practices and that teachers are adapting their teaching practices to “teach to the test”. In 2007 and 2011 teachers
were asked what factors influenced their teaching. The results indicate that the National Exam had greater influence
in 2011 than 2007. Regression analysis indicates that teachers who used more teacher-centered approaches such as
exposition, closed-questioning and routine problems also tended to say that the National Exam played a greater role
in their selection of practices.
Main Report xxiii
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Another likely factor seems to be the curriculum. Similar to the National Exam, the results indicate that the
curriculum had greater influence in 2011 than 2007. Regression analysis indicates that teachers who spent more
time on new content and used more exposition also tended to say the curriculum had greater emphasis on their
practices.
The teacher reform that effectively began in 2006 was a main focus of this study, with the first phase of the study
in 2007 being a useful baseline of teaching practices. Certification, the key program of the reform, was examined
as a possible influencing factor. Interestingly, there were almost no statistically significant differences between the
teaching practices of certified and uncertified teachers when controlling for other factors. The only difference was
that certified teachers tended to have a larger proportion of whole-class time. There was also no difference in the
student learning outcomes of certified vs. uncertified teachers.
Upgrading may have had an effect, with students of teachers holding a four-year degree tending to have better
learning outcomes. For practices, teachers with four-year degrees tended to use more open-ended questioning, and a
higher proportion of group work, and were less likely to use procedural approaches to problems, all of which actually
go in the opposite direction of the trends. While the teacher reform may be having some effect, these changes are
not clear when looking at 8th grade mathematics classrooms and do not appear to explain the identified trends.
KEY FINDINGS: THE BELIEFS-KNOWLEDGE-PRACTICES RELATIONSHIP
The study is, at its heart, an exploration of interplay among three components: teaching practices, teacher
mathematical beliefs and mathematical knowledge in teaching. Knowledge and beliefs are two widely researched
areas in relation to their relationship with teaching practices.
The knowledge relationship with respect to teaching practices was already touched upon. For beliefs, teachers who
tended to have a transmissionist or traditional belief orientation tended to also use more teacher-centered and routine
practices. Their students also tended to have lower learning outcomes compared with teachers with a connectionist
orientation. What was of more interest, though, is that teachers whose students tended to have the highest learning
outcomes had what could be described as a very flexible belief system. They actually argued against there being
“best practices”, instead saying that what is best practice is highly contextual, depending on factors such as the
achievement level and learning affinities of the students, the topic taught and the stage of the lesson. For example,
while exposition may not be the most effective approach in general, these teachers recognized value in using it at
key stages of the lesson and while investigation may be a positive approach, it is not applicable for every topic. Their
flexible beliefs allowed them to adapt their practices to the context and they had a bigger toolbox of practices at
their disposal.
Through a process of “data mining” by using regressions to explore thousands of combinations of belief statements,
practices and teacher knowledge, relationships were identified. In-depth case studies and qualitative analysis were
also used to understand their connections. Through this process a new conceptual model of the beliefs-knowledge-
practices relationship was developed as a way of thinking about effectiveness of teaching.1
1
The BKP (Beliefs-Knowledge-Practices) Congruency Model was developed as part of a doctoral thesis by Ragatz (2013).
xxiv Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
Conceptual Model of Beliefs-Knowledge-Practices Interaction
PRACTICES USED
(actions)
Dissonant Zone
Kn ence
Ab
ne
ow Zo
s
sen lief
Zo
led ne
Ab Be
ce
ge
Congruent Zone
(practices-knowledge-beliefs
all aligned in congruence)
Kn
s
rac o
ow d no
tice
in p ed t
an
led t u
sed t ti
ge sed
r u ut no
sep in
dg ld, b
ara pra
te
wle he
eo
fro tices
kno eliefs
m
c
be
Practice
B
KNOWLEDGE BELIEFS
lie
fs
(tools) Absence Zone (perceptions)
The following are key points of the model:
• Teachers are most effective when the practices they use align with their knowledge (they know from a subject
and pedagogical standpoint how to use the practice) and beliefs (they believe the practice is effective in the
teaching-learning process), represented by the “Congruent Zone”, where there is full alignment.
• Teachers are least effective in using practices that do not align with their knowledge or beliefs. This is
represented by the “dissonant zone”. It is an unnatural state that may occur when teachers are required due
to curriculum, school policies or other factors that oblige the teacher to teach in a way she or he would not
choose on their own.
• Effectiveness of a practice (e.g., inquiry-based learning) is relative and contextual. Whereas it can be an
effective method for a teacher who has the proper knowledge and beliefs, it can be an ineffective practice for
a teacher who does not have this knowledge and/or beliefs.
This model was developed largely as a tool for discussion and simulation in relation to Indonesia’s implementation
of the 2013 curriculum, which requires teachers to utilize very different practices from those currently used, as
evidenced in the Video Study.
Main Report xxv
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
IMPLICATIONS: RELEVANCE IN THE CONTEXT OF RECENT POLICIES ON
TEACHING PRACTICES AND THE 2013 CURRICULUM
Policies on teaching practices over the past decade have tended to encourage progressive practices focused on active
learning with greater student interaction and collaboration as well as an emphasis on enquiry-based learning,
reasoning, innovation, creativity and higher order thinking. The 2013 curriculum places even greater emphasis on
these practices, requiring a distinct shift in the roles of students and teachers and a change in teaching methodology.
LEARNING APPROACH
The new learning approach encourages students to seek out and make observations. Students are directed to
formulate the problem (ask), not just to solve the problem (answer). They are expected to think analytically (make
a decision), rather than just in a mechanistic manner (routine). Students are also expected to work together and
collaborate in solving problems.
TEACHING APPROACH
This more open and student-centered teaching/learning paradigm requires that the teacher become more of a guide
and facilitator than simply the provider of information through one-way teaching. Classroom practice must foster
independent learning, observation, questioning, processing, making judgments, communicating and creating.
THE 2013 CURRICULUM IN THE CONTEXT OF MATHEMATICS
For mathematics education, this new approach emphasizes the use of investigative and practical (hands-on) teaching
approaches with more of a connection to real-world contexts. The lesson activities should encourage higher-order
thinking through techniques such as the use of open-ended questioning, non-routine problems, and the application of
various enquiry methods. Teachers are expected to use a student-centered learning paradigm with classroom practice
to foster independent learning, observation, questioning, processing, making judgments, communicating and creating.
PROMISING APPROACH, BUT A BIG GAP TO FILL
Such findings of the Video Study demonstrated that teachers will need to teach in a very different style than currently
tends to be used and that there is a large gap to fill. In nearly every case, the practices encouraged in the 2013
curriculum are the ones rarely used. The student-centered learning approach encouraged in the 2013 curriculum
is clearly in contrast to existing practices. While problem-solving was used 22 percent of the time, approaches of
practical work and investigation, which align with the new curriculum, were rare. Open-ended questioning, non-
routine problems and real-world contexts were particularly rare. Overall, the results indicate there is a significant
gap between practices used in 2011 and what is expected through the 2013 curriculum.
xxvi Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
IMPLICATIONS: THE CHALLENGE OF CHANGING PRACTICES
Changing practices is notoriously difficult, as can be seen in previous Indonesia curriculum reforms. The 1984
curriculum reform was specifically designed for student-centered learning, but subsequent studies indicate that the
teacher-centered approach held strong. Teachers take the information and interpret it in the context of their existing
beliefs and environmental situation. With weak or limited training, teachers will be even less likely to change their
practices as expected. With limited support the teachers will often not implement in the intended manner, possibly
with detrimental results. Change requires a long-term, consistent process.
EFFECTIVE CHANGE IN PRACTICES REQUIRES REALIGNING TEACHER KNOWLEDGE AND
BELIEFS
A useful model for understanding teachers’ use of practices is to consider two major influences: their subject and
pedagogical knowledge, and their beliefs of how students learn and how to best teach. Teachers will attempt to
align their practices to their knowledge and beliefs. The Video Study results indicate that it is when the practices are
aligned with knowledge and beliefs that teachers are most effective. When they have to utilize practices for which
they do not have sufficient mathematical or pedagogical knowledge or that do not align with their beliefs on the
teaching-learning process, then they will not be effective. For example, a teacher who is forced to use a student-
centered approach where they are a guide and facilitator rather than an instructor will struggle if he/she does not
know how to use the approach and/or he/she does not believe this is the way students learn.
Theoretical model showing alignment of teaching practices with knowledge and beliefs
PRACTICES
Dissonant Zone
PRACTICES
Congruent Congruent
Zone Zone
KN
O
KN
S
W
EF
OW
LE
LI
BE
DG
LE
E
S
DG
EF
E
LI
BE
Teacher using many practices not aligned Teacher whose practices are mostly
with her knowledge set or beliefs aligned with her knowledge set or beliefs
Main Report xxvii
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The implications for the 2013 curriculum, as represented in this model, are crucial. If teachers do not have sufficient
knowledge in the new approaches and they do not believe they are effective approaches then teachers will do one of
two things: (1) they will revert back to the practices that do align with their knowledge and beliefs; or (2) they will
continue to use the practices, but use them ineffectively. This second result could even be more detrimental (in the
short term) than if teachers simply use the practices for which they are capable.
In order for teachers to effectively implement the new curriculum with the expected practices, teachers need
continual, periodic support to progressively build the knowledge and beliefs necessary to utilize the new practices.
IMPLICATIONS: RELEVANCE FOR THE TEACHER PROFESSIONAL
MANAGEMENT SYSTEM
MOEC has undertaken the massive task of developing a Teacher Professional Management System (TPMS), where
the key elements of this system form an “integrated framework” of requirements. The system makes the teacher
directly accountable for both work performance through annual appraisal (PKG) and professional development
through continuous professional development (CPD). The test of minimum standard in professional and pedagogic
competency (UKG) must also be passed in order to progress between the professional levels and the rate of progression
depends on a credit point total based on the PKG (annual performance-based teacher appraisal) and CPD.
The integrated framework of the TPMS
What a teacher does
Feedback based on observation
Performance
Appraisal
Continuous
Competency Professional
Test Development
What a teacher knows Tools for improvement
Identi cation of knowledge areas for improvement Tailored plan based on identi ed needs
xxviii Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
The results of the Video Study highlight just how important teacher knowledge is in relation to student learning
outcomes. The UKG can serve important purposes related to teacher management, including the hiring of high-
quality teachers, identifying areas of improvement for incumbent teachers, and providing the appropriate support
to improve competencies. The results can also be used to reward teachers who demonstrate high competencies or
significant improvements:
• Hiring of teachers: Only teachers who can demonstrate a threshold mastery or strong subject and pedagogical
knowledge in the UKG should be hired.
• Professional Development support of teachers: The UKG is already incorporated into the MOEC’s TPMS, and
can be used to identify a teacher’s strengths, as well as areas for improvement. The areas for improvement can
then be addressed through the teacher’s annual professional development and results plan.
• Progression and promotion: The TPMS also uses the UKG results for teacher progression and promotion.
The thresholds have not yet been established, but if the results are to encourage teachers to improve their
knowledge then the thresholds must be meaningful and teachers must meet the competency requirements in
order to be promoted.
EFFECTIVE USE OF COMPETENCY TESTING REQUIRES POLITICAL WILL…
If utilized properly, the UKG can serve as a tool to help teachers identify areas for improvement, set goals and
targets and seek out ways to improve their competencies. But to encourage improvement in competencies the UKG
must also set thresholds and ensure teachers reach those thresholds. This will require strong political will, but if this
will can be mustered it can play an important role in strengthening teacher competencies and improving student
learning outcomes in the long term.
…AS WELL AS A RELIABLE AND EFFECTIVE INSTRUMENT
If the UKG is to fulfill its role in the above-mentioned areas, the instrument (and the way it is administered) must
be seen as a valid, reliable and credible measure of teacher competency. An invalid tool could lead to poor decisions,
poor outcomes and a frustrated teaching workforce. The assessment items must be validated thoroughly and be
based on subject and pedagogical knowledge that are truly relevant for teachers for each subject and school level.
Investment in ensuring a valid, reliable and credible instrument will ensure all stakeholders accept it.
CONTINUOUS PROFESSIONAL DEVELOPMENT (CPD)
One of the key roles of the UKG and the performance appraisal is as a feedback mechanism. By identifying areas
for improvement, teachers can then seek out professional development activities that will address these areas. This
requires teachers to have awareness of, and access to, a wide range of materials that are relevant to the identified
improvement areas. This involves the cataloging and mapping of materials available from teacher training centers,
district/city governments, third-party vendors and online resources. Teachers should have clear support for CPD,
Main Report xxix
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
including a way of receiving credit for their efforts. There needs to be support of multiple avenues for receiving CPD,
from teacher working groups (KKG/MGMP) to online courses (for example, the MOEC’s mathematics education
teacher training centers [P4TK], the Open University [Universitas Terbuka, or UT], and teaching training colleges
[LPTKs]), and central and local government provision of CPD opportunities. A clear strategy and options will be
critical for the TPMS to function optimally and truly improve teacher competency and performance.
FROM RESEARCH TO ACTION: A CPD MODULE BASED ON STUDY FINDINGS
The insights gained through this study highlight the importance of strengthening both teacher-subject and
pedagogical knowledge, and also where teaching practices should be enhanced. The study is also particularly relevant
in the context of the practices now being promoted in the 2013 curriculum reform.
A natural extension of the study was to incorporate the lessons learned into modules that teachers could use in their
CPD activities. The over 600 hours of video footage captured natural classroom time, providing useful real-life
examples of what teachers do in their classrooms. This unique and invaluable data provided a perfect platform upon
which to develop teacher professional development activities.
AN INNOVATIVE PILOT AS ONE POTENTIAL IN-SERVICE APPROACH
For teachers already in the system, a consistent, periodic support approach is required in order to build the necessary
skills required for the new curriculum. This support can come from a variety of avenues, including traditional
training, and online training. One of the most relevant methods for Indonesia’s large number of teachers spread out
across many islands is through teacher working groups (KKG-MGMP).
A teacher CPD course was developed based on the findings of the Video Study. The course utilized innovative
approaches to simultaneously strengthen both subject and pedagogical knowledge with a focused goal of changing
teacher behavior.
• It simultaneously builds subject and pedagogical knowledge.
• It uses an approach of Experience, Learning, Picture, Symbol and Application (ELPSA) that links mathematics
teaching practices to the new curriculum.
• It uses video clips from the Video Study as artefacts for real-life classroom examples and tools for discussion.
• It is delivered in a scaffolded manner, with activities involving a process of “in-on-in” where concepts and
approaches are learned first in a teacher working-group session, then practiced in the classroom, and finally
discussed and reflected upon with other teachers in the subsequent working-group session.
• It is designed to change beliefs and mindset through discussion, practicing and reflection.
• It uses innovative techniques like teachers videotaping their own classrooms.
xxx Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Executive Summary
The impact evaluation results indicate that the module did in fact change teacher beliefs about the teaching-learning
process and strengthened teacher subject and pedagogical knowledge. More importantly, though, there are also
indications of teachers changing their practices in the classroom.
P4TK Matematika has converted the module into an online environment with an attempt to retain the interactive
and discussion-oriented philosophy. This virtual KKG/MGMP approach has the possibility of reaching a large
number of teachers across Indonesia.
PRE-SERVICE TRAINING MUST ALSO BE IN LINE WITH THE NEW APPROACHES
The coursework for students at teacher training colleges (LPTKs) must be designed to align with the 2013 curriculum
and ensure upon graduation they are prepared to utilize the new teaching and learning approaches. Of course this
has to be supplemented by a strong, solid induction program at their respective schools.
OVERALL KEY MESSAGES
1. Many of the practices encouraged in the past decade and in particular the 2013 curriculum have a positive
relationship with student learning outcomes, indicating that proper implementation of the new approach
should improve student learning.
2. BUT the practices encouraged in the 2013 curriculum are rarely used, indicating that it will require a major
effort to shift in terms of methods and practices. Teachers will need strong support to understand and properly
utilize the practices encouraged in the new curriculum.
3. The strong, positive relationship between teacher subject and pedagogical knowledge, and student learning
outcomes, indicates it is a critical element in teacher effectiveness. But the national teacher competency
test indicates teacher subject and pedagogical skills are generally very low. Support to in-service teachers to
improve their knowledge and targeted hiring with teacher competency as a criteria would improve the quality
of the teacher workforce.
4. Teacher-student interaction is an important element of student learning. Students in classrooms with greater
teacher-student interaction have better learning outcomes. This holds true in whole-class, group work and
individual seatwork situations. Teacher-centered practices such as teacher-only time lecturing tend to have a
negative relationship with student learning outcomes.
5. The trends in teaching practices from 2007 to 2011 indicate decreases in many practices that have a positive
relationship with student learning outcomes. The National Exam and curriculum have increased in influence
according to teachers and appear to be playing a role in some of the trends seen.
6. Teachers require support if they are to change their practices. Results from a pilot geometry module
implemented in KKG/MGMP indicate that continual support through “in-on-in” activities (see explanation
above) with practical applications appear to be an effective way of helping to change teacher beliefs, improve
teacher knowledge and change practices in the classroom.
Main Report xxxi
CHAPTER 1
BACKGROUND AND
CONTEXT
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Over recent years, Indonesia has achieved significant success in facilitating a rapid increase in the rate of
enrolment in junior and now senior secondary education. However, despite this increased rate of enrolment, a
large proportion of students in Indonesia still have low levels of literacy and cognitive skills. The performance
of Indonesian students in international assessments indicates relatively low levels of student achievement, even
when taking socio-economic factors into account. In the area of mathematics, Indonesia ranked 36th out of 48
participant countries in the 2007 Trends in International Mathematics and Science Study (TIMSS), with its score of
397 being more than one standard deviation below the international average (Mullis, Martin and Foy, 2008); In
2011 Indonesia’s score declined by 11 points (Mullis, Martin and Foy, 2012) and ranked 38 out of 42 participant
countries. In the 2006 Programme for International Student Assessment (PISA), Indonesia had a score of 391 and
ranked 51st of the 57 participant countries in mathematics (OECD, 2007); in the 2009 PISA Indonesia’s score
decreased 20 points and ranked 68 of 74 countries (Fleischman et al., 2010). In 2012 Indonesia’s score increased 4
points, but ranked 64 of 65 countries.
Figure 1: Estimated Impact of High vs. Low Performing Teachers on Student Achievement
100th
percentile
Student performance on Standardized Exam
90th percentile
53 percentile
point di erence
50th
percentile
37th percentile
0th
percentile
Age 8 Age 11
Source: Sanders and Rivers (1996) on the Tennessee Value-Added Assessment System (TVAAS)
International evidence shows that students who are taught by effective teachers dramatically outperform
those with ineffective teachers. There is a significant body of evidence to show that the quality of teachers plays
a key role in determining student learning outcomes (e.g. Hattie, 2003; Bruns and Luque, 2014; Hanusheck and
Rivkin 2010). How much difference does a good teacher make compared to a bad teacher in terms of facilitating
student learning outcomes? In a seminal study, Sanders and Rivers (1996) conducted research into the Tennessee
Value-Added Assessment System (TVAAS) to attempt to determine an answer to this question. The study found that if
eight-year old students with average levels of scholastic performance are taught by teachers of varying qualities, their
levels of achievement diverge dramatically. Specifically, one group of students was taught by high ability teachers (in
the top 20%), while the other group was taught by low ability teachers (in the bottom 20%) over a 3-year period.
At the end of the three years, the levels of performance of the two respective groups had diverged by more than 53
2 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Background and Context
percentile points. Thus, by age 11, on average, students taught by teachers in the upper group would be scoring in
the 93rd percentile, while those taught by teachers in the lower group would be scoring in the 37th percentile. The
study also found that students with lower levels of achievement benefit most significantly from having high ability
teachers. More recently, Hanusheck and Rivken (2010) found similar impact, where students with a weak teacher
may master 50 percent or less of the curriculum for that grade while students with a good teacher get an average
gain of one year and students with great teachers advance 1.5 grade levels or more.
Cross-country comparative studies have highlighted deficiencies in the level of teacher quality and teacher
support in Indonesia. For example, in a study conducted by Martin and Mullis (2006) to compare the quality
of teachers in Indonesia, Malaysia and Singapore in the area of mathematics in TIMSS 2003, it was found
that Indonesian students actually received a greater number of hours of instruction in mathematics than their
counterparts. However, their teachers had a lower average level of formal education; were less likely to have a degree
in mathematics; and received less professional support to improve content knowledge and teaching skills.
Cultural context plays a critical role in determining what is effective for student learning outcomes. Teaching
practices that are effective in one country may not lead to the same outcomes in another country. In the TIMSS
1999 Video Study (Hiebert et al, 2003), it was found that the nature of instructional practices in different countries
varied significantly, with no direct correlation between the method adopted and student learning outcomes. For
example, Japan and Hong Kong SAR both achieve high rankings in the TIMSS exam. While they both achieve
high rankings, the teaching practices and classroom environment in the respective countries vary significantly. This
highlights the need to understand the cultural context in order to determine the level of teaching effectiveness. Thus, in
the case of Indonesia, it is important to understand what teaching practices are effective within the Indonesian context.
1.1 TEACHER REFORM IN INDONESIA
In recent years, Indonesia has implemented major reforms to improve the quality of its teachers. In December
of 2005 a major law on teachers was passed, which has become known as the Teacher Law. A cornerstone of this
reform process is a policy requiring all teachers to have a 4-year degree and to undergo a process of certification
by 2015. This has had major implications for in-service teachers, with 95% of all primary teachers, 45% of all
junior secondary teachers, and 29% of all senior secondary teachers holding a lower level of qualification than that
prescribed at the time the policy was promulgated (Ragatz and Chen, 2009). Teachers receive a bonus that effectively
doubles their salary after passing through the certification process. This has significant financial and budgetary
implications. However, there has been a great deal of criticism of the certification program. These criticisms relate
both to how the process attempts to measure teacher ability and to whether certification in any form will actually
result in improvements to teacher quality or teacher performance.
A central goal of the Teacher Law has been to upgrade the quality of teachers and their performance in the
classroom. Of course, the ultimate measure of success for this initiative is improved student learning outcomes.
As stated previously, Indonesia participates in a number of important comparative international studies, such as
the IEA TIMSS and PIRLS and the OECD PISA studies. By determining the level of performance of Indonesian
students compared to their counterparts elsewhere over the years, these studies provide a relatively objective of the
change in outputs of the system.
Main Report 3
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
However, while the ultimate measure of success is improved student outcomes, a more direct measure is the
degree to which the initiative has improved levels of teacher competence in the classroom. Therefore, a study
that measures these levels will provide an important indicator to determine the effect of the Teacher Law.
The focus of Indonesia’s reform initiative has been on improving the level of teachers’ qualifications. However,
there is increasing recognition that determining the means to improve the quality of teaching requires an
understanding of what takes place in the classroom. Discussions on measures to improve teacher quality must
be informed by empirical evidence of what constitutes effective teaching and learning in the Indonesian context. To
date, very little such research has been conducted. As a result, important policy decisions are often being made on
the basis of anecdotal evidence or studies conducted in other countries.
1.2 MATHEMATICS TEACHING IN INDONESIA
The research on mathematics teachers and teaching practices in Indonesia is relatively sparse. However, the
limited research that has been conducted provides some perspective on teaching practices. Somerset (1997)
described the teaching-learning process as being almost always based on traditional, teacher-centered approaches.
Hendayana et al. (2003) agree, stating that teachers tend to dominate mathematics lessons, rather than letting
students learn. On the basis of their observation of lessons, they found that activities tended to involve students
copying notes and listening to the teachers, with limited interaction between students and a large proportion of
students appearing to be disengaged from the process. According to this research, the approach to teaching geometry
is usually highly theoretical, with many abstract concepts and formulas being introduced without much attention
being paid to aspects such as logic, reasoning, and understanding (Karnasih and Soeparno, 1999; Soedjadi, 2000).
Fauzan et al. state that in general, students are passive throughout the lesson; ‘chalk and talk’ is the preferred
teaching style; there is an emphasis on factual knowledge; questions tend to require only single word responses,
often provided in chorus; there is a lack of learning questioning; only correct answers are accepted and acted upon;
and whole-class activities often involve writing, with no hands-on work carried out (2002).
On the other hand, in a recent OECD report, Indonesia was ranked as one of the highest countries in an
index of innovation of education for mathematics and science. To determine each educational system’s top
innovations in pedagogic and organizational practices, data from three international education datasets – Trends in
International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS),
and the Programme on International Student Assessment (PISA) – were analyzed to identify the areas in which
each education system has demonstrated emerging or changing organizational and pedagogic practices over a
specific period. Major changes in pedagogic and organizational changes were considered innovations (regardless of
direction).2
Indonesia’s top organizational innovations included: (i) more use of assessments; (ii) more use of assessment
data in secondary education; (iii) more public posting of secondary achievement data; (iv) more parental service on
secondary school committees; and (v) more teacher observations of secondary school science classrooms.
2
It is important to note that while the term “innovation” generally has a positive connotation, for this study it refers to any changes, but it
could be argued that some of these changes may not necessarily be going in a positive direction. It is also important to note that these are
changes as specified in teacher and student surveys, rather than observed changes.
4 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Background and Context
Indonesia’s top pedagogical innovations included: (i) more use of textbooks as primary resources in secondary
school science; (ii) more grouping by ability in secondary education; (iii) more relating of secondary school lessons
to everyday life; (iv) more use of answer explanation in secondary mathematics; and (v) more observation and
description in secondary school science lessons (OECD 2014).
1.3 VIDEO STUDY OF TIMSS CLASSROOMS IN INDONESIA
An increasingly popular approach for the analysis of teacher behavior and instructional practices is through
the use of video. In the TIMSS 1999 Video Study (Hiebert et al., 2003), it was found that instructional practices
in classrooms across countries varied significantly, with each country having its own unique “lesson signature.” Even
among high achieving countries, the lesson signature differed significantly, suggesting that there is no single correct
method of teaching mathematics. Due to social, economic and cultural complexities, teaching practices that may be
effective in one country will not necessarily be effective in another. It is important to understand how students learn
within a country’s own unique contexts and to determine the most appropriate forms of teaching in those contexts.
With financial support from the Royal Government of the Netherlands, the Ministry of National Education
(MOEC) and the World Bank conducted a two-phase Video Study to examine and assess the practice of
mathematics teachers in Indonesian classrooms. The study involved the production of video footage of 100
classes that participated in the 2007 TIMSS international assessment, and of another 104 classes that participated
in the 2011 assessment. The design of the Video Study, including the method for coding videos, is based upon the
methodology used by a TIMSS 1999 Video Study of seven countries (Hiebert et al., 2003).
An initial report was produced on the basis of the 2007 phase of the study. This report examined teaching
practices in Indonesia and compared them to practices in the seven countries that participated in the TIMSS 1999
Video Study (Hiebert et al., 2003). One of the most striking findings from the study was how few words both
teachers and students uttered throughout the period of the class. Indonesia also stood out in terms the proportion
of class time dedicated to specific activities. Amongst the comparator countries, on average, at least 96% of class
time was dedicated to actually teaching mathematics. By contrast, in Indonesia, the proportion of class time spent
on actually teaching mathematics was only 89%. A significantly higher proportion of the lesson time was spent
on organizational work (8%) and on non-mathematics activities (3%) than in other countries. The proportion
of time spent on teaching mathematics was also broken down into problem-solving and non-problem-solving
time. In Indonesia, 76% of the time spent on teaching mathematics was devoted to problem-solving, while in
the comparator countries, this proportion ranged from 81% to 96%. In Indonesia, teachers also set relatively few
independent problems,3 with 3.3 problems on average per lesson. In the comparator countries, the number ranged
from 3 to 13, although Indonesian students did tend to spend more time on average on each independent problem,
with this time running to an average of 6.6 minutes compared to a range 3 to 5 minutes in the comparator countries
(Leung and Ragatz, 2010).
3
Counts of words spoken focused on only full-class interaction (rather than group or individual work) and were standardized to indicate the
number of words spoken over 50 minutes. Indonesian teachers spoke fewer than half the number of words as in other countries, with only
2,633 words in an average lesson compared to 5,198–5,902.
Main Report 5
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Following the 1999 TIMSS study (Hiebert et al., 2003), the study identified a “lesson signature” on the basis of the
pattern of practices typically used in the classroom. Typically, classes were segmented into three stages, these being:
(i) the introduction stage; (ii) the development stage; and (iii) the closing stage. The introduction stage typically involved
the review of homework from the previous class. This was followed by the development stage, which introduced new
content. Teachers typically began this stage by motivating students with an exhortation on the importance of the
material in the lesson, followed by questions related to the prerequisite knowledge required to engage with the new
material. During the closing stage, teachers summarized the day’s lesson, sometimes with a certain level of student
involvement, and assigned tasks for home work. There was a great deal of consistency, with nearly every observed
lesson following this pattern (Leung and Ragatz, 2010).
The coding structure used in the study enabled the identification of an initial profile of what takes place in the
classroom in terms of the time spent on different activities, approaches to problems, and teaching methods
used. However, it must be acknowledged that there are some limitations to this approach. The indicators are
low inference. As noted by Watson (2008) regarding the TIMSS Video Study approach used to identify mathematical
coherence, rationale and complexity: “…these methods did not give access to what was coherent or how it was
achieved” (p. 129). She adds that: “There are studies that probe ‘typicality’ in mathematics classrooms, and studies
that show how certain modes of teaching make a difference to students’ learning, but nothing that looks at how
mathematical ideas unfold in successful classrooms of all kinds.” (p. 130). This point highlights the need to probe
beyond the TIMSS Video Study codes in order to uncover the deeper influences of teaching practices on levels of
student attainment.
Another limitation is that the video analysis provided insights into what takes place in the classroom, without
explaining why it takes place. Understanding the underlying motivations of teachers would provide a critical
foundation upon which to gain a deeper understanding of what takes place in the classroom. The 2011 phase of
the study was modified to enable a deeper exploration into the motives of teachers for their teaching practices.
Within educational research, two areas have been identified as being particularly important in understanding the
motivations and influences behind teaching practices in mathematics education. These are teacher mathematical
beliefs and mathematical knowledge in teaching.
6 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 2
OVERVIEW OF DESIGN,
METHODS AND
ANALYTICAL APPROACH
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
2.1 INTRODUCTION
This study was intended to examine the mathematics teaching-learning process in the Indonesian context
and to determine why it takes place, how it has changed over time, and how it relates to student learning
outcomes. The following section on the methodology of the research first defines its conceptual framework and
specifies its objectives. It then elaborates on the research design, first by explaining key general methodological
issues, and then by explaining the purpose, design and implementation of the instruments. The study data has three
levels, with a large sample of more than 200 teachers (100 in 2007 and 105 in 2011) and a smaller sample of 10
case study teachers. In addition there is an in-depth examination of three of the case studies. As will be explained in
this chapter, these three levels are referred to as the Macro, Mezzo and Micro levels.
The unique iterative, two-phase design of the Video Study has allowed it to evolve in ways that allowed for
modifications to its approach and objectives on the basis of its initial findings.
2.2 OBJECTIVES
The purpose of the Video Study is to monitor the level of success of the initiatives implemented through the
Teacher Law and the teacher reform effort. The data collected is intended to enable stakeholders to gain a better
understanding of teaching practices in the Indonesian classroom, to see how these practices have changed over time,
and how this change relates to changes in policy. In addition, it is also intended to generate insights into how the
changes impact levels of student achievement.
A secondary purpose of the project is to build the capacities of relevant personnel in Indonesia. BERMUTU
is a World Bank project intended to support the Indonesian Government’s initiatives under the Teacher Law. As
such, the expectation is that the Video Study should be conducted mainly by personnel in Indonesian Government
agencies, with the World Bank primarily playing an advisory and supportive role. This will not only enhance
ownership of the project by local officials and educators, it will hopefully also enable them to build their capacities
to use videos as a research tool.
A third purpose of the study is to document teaching practices over time and to integrate the use of video
into teacher professional development activities. As a by-product of the Video Study, the collected video data
may be used for studying aspects of Indonesian classrooms other than those on which this Video Study focuses.
The classroom videos may also be used to produce an archive for future use, for the development of teacher training
materials and other purposes. It has also been suggested that the Video Study could be expanded for use in teacher
professional development activities. The BERMUTU project supports teacher working groups, which consist of
teachers from 6-10 neighboring schools, with members of these groups meeting regularly to conduct professional
development activities. These teacher working groups could provide an ideal environment for the use of video as a
means for self-assessment and for improving teacher quality.
A fourth purpose of the study is to go beyond what takes place in the classroom to address the issue of why
it takes place. The analysis from the initial phase focused mainly on what takes place. However, it also raised
new questions related to why certain practices are used. An understanding of why certain practices are used in the
classroom is critical for the development of programs and policies intended to facilitate changes in teacher behavior
8 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
and to improve teacher quality. A key additional component of the study in the 2011 phase is the incorporation
of components to explore teacher mathematical beliefs and teacher mathematical knowledge in teaching (MKiT). In
addition, the second phase included in-depth qualitative components, including the in-depth case studies.
To summarize, the objectives of the study are as follows:
1. To characterize classroom teaching-learning behavior in the context of the stated intention of the curriculum
and with reference to classroom characteristics in other countries;
2. To compare baseline data collected in 2007 with data collected in 2011 to identify emerging trends in
teaching practices;
3. To understand why teachers use specific practices, particularly in relation to key influencing factors such a
teacher’s individual subject and pedagogical knowledge and teacher mathematical beliefs on teaching and
learning, as well as outside influencing cultural and system factors;
4. To understand the relationship between classroom teaching-learning behavior and student learning outcomes
and to determine which teaching methods are more effective;
5. To produce an archive of classroom videos for the purposes of research and to facilitate teacher development;
6. To develop the capacities of relevant personnel.
2.3 RESEARCH QUESTIONS
The overarching aim of the study was to establish a foundational understanding of mathematics teaching in
the Indonesian context. On the basis of this foundation, the study aimed to explore how teaching practices relate
to student learning outcomes. The study was intended to determine not only what takes place, but also why it takes
place. While the teaching-learning process is the focal point of the study, the analysis also attempts to determine
what shapes teachers’ strategies and practices.
Characteristics of the teaching-learning process
1. How well are teachers prepared for their teaching?
2. What mathematics contents are covered in the lessons?
3. How competent are teachers in teaching mathematics?
4. How are the lessons structured and how is time managed during the lessons?
5. What types of mathematics problems do students solve?
6. How are mathematics problems solved?
7. What teaching strategies do the teachers use?
8. What are types of questions do teachers ask?
9. How do teachers assess students’ learning?
10. Are teachers motivated to improve their teaching skills?
11. What learning resources are used for supporting teaching and learning?
12. What are the profiles of the teachers?
13. What are students’ attitudes toward mathematics?
14. Which of the characteristics above have positive or negative relationships with high student achievement?
Main Report 9
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Characteristics of the Beliefs-Knowledge-Practices relationship
1. What is the relationship between teaching practices, teacher mathematical beliefs, and mathematical
knowledge in teaching?
2. How does the teaching-learning process ensue within Indonesian eighth grade mathematics classrooms in
terms of teaching practices and student responses?
3. What are the mathematical beliefs and orientations of Indonesia’s eighth grade mathematics teachers?
4. What is the level and variation of Mathematical Knowledge in Teaching (MKiT) of Indonesia’s eight grade
mathematics teachers?
2.4 WHY A VIDEO STUDY?
In recent years, the use of videos has been shown to be a highly effective means of studying classroom
activities and the classroom practices of teachers (Hiebert et al, 2003). In parallel with the implementation
of the BERMUTU project, the World Bank and the MOEC decided to conduct a Video Study4 to gain a better
understanding of the level of competence of teachers in the classroom. By deliberate design, the Video Study is
linked with Indonesia’s participation in TIMSS 2007 and 2011. Indonesia’s participation in these two rounds of
the TIMSS provided a golden opportunity to monitor the effect of the Teacher Law by exploring the relationship
between teachers’ classroom performance and their learning outcomes, as measured by the results of the TIMSS
process in 2007 and 2011.
In this Video Study, to determine the effectiveness of the initiatives implemented through the Teacher Law,
the subject of mathematics at the eighth grade level was chosen as an example of teaching practices more
generally. The process of teaching mathematics in a selected sub-sample of classes that participated in 2007 was
studied by extensive videotaping of lessons conducted in the classroom. The process was repeated in 2011, with the
results of the study being linked students level of achievement as measured by TIMSS in both cases. Since the study
attempted to track changes over time, exact replication was an essential feature, with the core activities conducted
in 2007 being reiterated exactly in 2011.
2.5 CAPTURING TRENDS OVER TIME
A central aim of the study was to analyze changes in teaching practice over time and to determine the effect
of these changes on levels of student achievement. As stated previously, the Teacher Law establishes minimum
academic and professional standards for teachers. The assumption is that teachers with a higher level of qualifications
and who participate in various professional development activities mandated by the Teacher Law will provide a
higher quality of teaching, which will lead to improved student learning outcomes. Accordingly, this Video Study
is based on the conceptual framework represented in the diagram below:
4
Arguments for the use of videos in studying classroom teaching over the use of other means such as questionnaires and live observation
in studying classroom teaching were presented in the report of the Pilot of this Video Study by Asrijanty et al (August 2006) and are not
repeated here.
10 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
Figure 2: Concept of Measuring Trends in Teaching Over Time
2007 2011
Teacher Law:
More quali ed
sets minimum requirements
teaching force
for teachers
Video Study Video Study
(Phase 1) (Phase 2)
Teachers’ Classroom Improved Teachers’
Instructional Practices Classroom
(base-line) Instructional Practices
Student Achievement Improved Student
(base-line: TIMSS 2007) Achievement
(TIMSS 2011)
As this diagram makes clear, the Video Study was conducted in the context of the implementation of the
Teacher Law to determine its level of effectiveness. Thus, phase one of the study documents and assesses teachers’
classroom instruction practices in 2007. This was the first time that a representative documentation of teaching in
the Indonesian classroom had been conducted. A comparison of the output of this study with that of similar studies
in other countries will facilitate an understanding of instructional practices in the current Indonesian classroom
and enable a determination of how they differ from practices elsewhere. More importantly, the output of this study
provides base-line data to determine the extent to which practices had changed by 2011, when the second phase of
the study was conducted. By 2011, it could be assumed that the measures mandated by the Teacher Law will have
impacted a portion of the teachers in the sample. A comparison of the teaching practices of these teachers with those
of: (i) similar teachers in 2007; and (ii) teachers in 2011 who have not yet met the new criteria established by the
Teacher Law, will provide a very good indication of the effects of the Teacher Law.
The framework is based upon the hypothesis that both in 2007 and 2011, there would be a relationship
between teachers’ classroom practices and student learning outcomes, as measured by the TIMSS scores. For
the 2007 phase of the study, this will provide valuable input regarding the appropriateness of the curriculum and
the kind of instructional practices necessary to enhance students’ level of achievement. A similar statistical analysis
in 2011 would provide a means to determine the effectiveness of the changes mandated by the Teacher Law.
Main Report 11
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
2.6 CONCEPTUAL FRAMEWORK
A conceptual framework for analysis was developed to examine what takes place in Indonesia’s mathematics
classrooms, why it takes place and how it relates to student learning. This framework was inspired primarily by the
work of Askew et al (1997), although it also incorporates many of the concepts discussed in the literature review
included in the technical report. While the framework is teacher-centric, this does not imply that a student-centered
framework, which has been used by a number of researchers, including Clarke et al. (2006) is not relevant. However,
for scoping purposes, this research focuses specifically on teachers and teacher practices.
Figure 3: Framework for Analysis of Teacher Practices, Beliefs and Knowledge
Classroom
Environment In uences
Lesson
Segment Purpose
System
Cultural Teacher’s and Policy
In uences Practices In uences
Mathematical
Mathematical Beliefs
Knowledge in
and Orientation
Teaching (MKiT)
Knowledge of How to teach How students
Subject learn to do
knowledge teaching mathematics
approaches mathematics
What it is a
Knowledge mathematics-
of capable
students student
Sources: The model is an adaptation of various models on teaching practices, teachers’ beliefs and knowledge, mainly from Askew et al. (1997), but also including: Aubrey
(1993); Bennett, Summers and Askew (1994); Lerman (1990); Thompson (1984), Lindmeier and Ufer (2010) and Schoenfeld (1989, 2010). Contextual factors are influenced by
multiple sources, including: Skott (2001, 2009), Beswick (2005, 2007), Andrews (2007b, 2010), Sullivan and Mousley (2001), and Correa et al. (2008).
The entities within the above framework can be separated into two distinct types: (i) the triad formed through
the connection of beliefs, knowledge and practices;5 and (ii) the contextual influencing factors. In the above
diagram, the triad is depicted in the foreground. This triad is the direct focal point for analysis, while the contextual
influencing factors can be seen in the background. These contextual influencing factors play an indirect role in how
the beliefs, knowledge and practices are formed and ultimately enacted in the classroom. Each of the entities within
the framework is discussed below.
5
The terms “beliefs”, “knowledge” and “practices” are shortened terms used throughout this text for “mathematical beliefs”, “mathematical
knowledge in teaching” and “teaching practices” respectively.
12 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
The framework revolves around Teachers’ practices. According to this framework, teachers’ practices drive what
takes place in the classroom and how student learning takes place. But this begs the question: How do teachers
determine what practices to use? The hypothesis is that two key background dimensions influence teaching practices,
these being: (i) teachers’ mathematical beliefs and orientation; and (ii) mathematical knowledge in teaching.
An important aspect of the research design is its recognition that beliefs, knowledge and practices do not exist
in a vacuum, but rather develop and evolve due to contextual influencing factors. The factors seen as being most
influential are represented in the framework and range from very event-specific to much more general and longer-term.
Teaching practices in particular are not static, but are instead dynamically shaped by the situation of the moment. The
teacher may plan a lesson and have ideas of the teaching practices to employ, but each lesson contains instances where
the teacher must evaluate the context and make teaching decisions in the moment (Schoenfeld, 1989; Thompson,
1985; Handel, 2003). The context of events is therefore considered to be of crucial importance in the analysis.
The background bubbles of lesson segment purpose, classroom environment, cultural influences and system and
policy influences are intended to illustrate their important role in shaping what ultimately takes place in the
classroom. The four factors include:
• Lesson Segment Purpose: A teacher’s practices are likely to be tailored to the lesson segment purpose. For
example, one layer of the overall video study coding structure divides lessons into review of previous material,
introduction of new material, practice and assessment. How might practices such as questioning, handling of
misconceptions, use of mathematical problems and teaching strategies differ based on these segments? A
teacher may choose to use a different type of questioning in each of these instances.
• Classroom Environment: The classroom context is also likely to influence the practices a teacher chooses. A
teacher’s approach is not static and is likely be adjusted to fit student needs. Factors like student background,
ability level, willingness to participate and others are likely to be taken into account by teachers. A classroom
in, for example, a rural, high poverty community presents a very different environment from an urban,
wealthy community. Understanding how teachers see their environment and context may provide insights
into why certain teaching practices are utilized.
• System and Policy Influence: Policies may also directly or indirectly play a role in classroom activity. Indonesia
has promoted certain policies relating to mathematics education and education in general. The goals and policy
lingo used at times emerged in teacher interviews and at times surfaced in classroom activities. It is certainly
not always clear whether their appearance is directly due to the policies themselves, so drawing a link between
policy and practice must be approached with extreme caution. At the same time it is important to consider
how the teachers internalize the policies in terms of interpretation and their willingness to follow them.
Some of the more prominent recent policy goals in Indonesia relate to: (1) student-centred learning;
(2) “active and joyful learning” which encourages participation and creation of an enjoyable, fun
atmosphere; and (3) contextual teaching and learning (CTL), which encourages the use of real-life
examples to help students see how mathematics applies to their everyday life. How might the teachers
interpret the policies and how might the teacher choose to incorporate them into the classroom or not?
Main Report 13
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
One aspect that is not explicitly stated in this framework but is found in other important frameworks
such as that of Bruns and Luque (2014) is teacher motivation. In their education results chain
and reduced form education production function, teacher skill and motivation along with teacher
content knowledge drive teacher classroom practice. Vegas and Umansky (2005) identify with three
broad classes of incentives to motivate teachers: (1) professional rewards; (2) accountability pressure;
and (3) financial incentives. The concept of motivation is particularly relevant for this study in
considering whether teacher certification and doubling of salary may have an influence on teaching
practices. The motivation and influence of the teacher reforms are explored in detail within this report.
• Cultural Influence: Indonesia is a large and culturally diverse country, so any attempt to use blanket
descriptions of cultural traits would be dangerous. Still, there are certain cultural tendencies that may be
taken into account when attempting to understand what takes place in the classroom. Aspects like authority
and hierarchy, individual vs. societal balance, when and how it is appropriate to speak in public situations,
and socio-economic factors may shape teaching practices. Also, culture is something that can vary from
community to community. For example, an education expert stated that rural communities in Indonesia tend
to hold what might be seen as more traditional role of a student within the classroom as quiet, disciplined and
listening to the teacher (with one rural school having a sign with the phrase “Quiet students, happy teacher”
possibly providing an insight into what is expected of students), while in urban environments many parents
push the schools to develop communication, debate, and leadership skills for their children and there is an
expectation from the principal, teachers, parents and students themselves that students should speak up in
class and make persuasive arguments rather than following the teacher’s lead (Jalil, 2011). Again, blanket
statements should be avoided, but these cultural aspects can help explain what takes place in the classroom.
The positioning and layering of these bubbles within the framework are intentional. The layers are intended
to illustrate how direct the influencing factor is, starting with most direct and moving to successively more and
more indirect. It begins with the lesson segment purpose, followed in order by classroom environment, system
and policy influences and finally cultural influences. Regarding the positioning, it is hypothesized that cultural
influences, while touching all aspects of the model, tend to most influence teacher’s beliefs relatively more than other
components, while system and policy influences, also touching all aspects, tend to play more of a role in teacher’s
MKiT. While the influences are important to keep in mind, they will not be examined explicitly on their own, but
rather incorporated into the examination of beliefs-knowledge-practices.
The model recognizes that there are important factors that operate in the background to influence a teacher’s
decision on what practices to use. These come from system and policy influences such as the curriculum. The
model is based on the assumption that practices in classroom lessons are the major factor influencing student
learning outcomes.6
6
It is possible that students also learn from influences outside the classroom, including tutoring, parental involvement and opportunities to
use mathematics in everyday situations. Data related to these aspects were captured in the student questionnaire and controlled for when
performing analysis.
14 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
The study is about interactions and relationships, with the model highlighting the interactions between beliefs,
knowledge and practices. In Figure 3, the relationships are illustrated through the arrows connecting the various
components. It is hypothesized that the direction of the flow is not one way, but that there is typically a primary
direction. As can be seen with the Teachers’ practices component, the arrows indicate that the relationship between
this component and the other components are reciprocal. However, the thicker arrows indicate the hypothesized
primary direction of flow. For example, much of the research on teachers’ beliefs tends to emphasize that a change in
beliefs leads to a change in practices (e.g. Swann, 2007; Arvold and Albright, 1995). However, there have also been
findings to indicate that the flow can go in the opposite direction, with a change in practices leading to a change in
beliefs (e.g. Hoyles, 1992; Sullivan and Mousley, 2001).
2.6.1 INCORPORATION OF STUDENT LEARNING INTO THE ANALYSIS
Beliefs, knowledge and practices become more relevant if they can be examined in the context of student
learning outcomes. A measure of student learning outcomes was incorporated into the design of the research by
using the results of mathematical assessments that were conducted for all students of the teachers participating in
the study. The tests were conducted both at the beginning and the end of the eighth grade year. Student learning
outcomes are often used in studies to determine the effectiveness of aspects of the teaching process. In the case of
this study, these outcomes were used not only to gain a sense of the effectiveness of teaching processes, but primarily
as a contextual factor. For example, the student entry achievement level, as determined by the pre-test, is useful for
understanding the nature of the environment in which teachers are operating. A teacher of a class of students with
a high average entry achievement level may use a very different approach than a teacher with students with a low
average entry achievement level. It is also useful to understand how students progressed over the course of the year,
as it provides an indicator of how the various factors studied might have played a role in student learning. However,
this study does not have the goal of identifying a specific list of practices that teachers should use or a set of beliefs
they should have. Full descriptions of each component in the model can be found in the Technical Report.
2.7 GENERAL METHODOLOGICAL ASPECTS
2.7.1 MIXED METHODS APPROACH
While the first phase of the study had some qualitative elements, it mainly involved quantitative methods.
In an effort for the second phase of the study to go beyond the coding of videos to gain a deeper understanding
of specific practices and to determine why teachers use the practices, it became clear that a greater emphasis on
qualitative methods was required.
The study used a mixed methods approach involving both quantitative and qualitative elements. The aim
was for these elements to complement one another and to create positive synergies. Denscombe (2013, page
270) notes that the mixed methods approach has emerged as a “third paradigm” for social research and that: “It has
developed a platform of ideas and practices that are credible and distinctive and that mark the approach out as a
viable alternative to quantitative and qualitative paradigms.” Johnson and Onwuegbuzie (2004, page 14) argue that:
“A key feature of mixed methods research is its methodological pluralism or eclecticism, which frequently results in
superior research (compared to monomethod research).”
Main Report 15
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
For the second phase, the research was designed on the assumption that a mixed methods approach involving
both qualitative and quantitative approaches would enable a better understanding of the teaching practices
of eighth grade mathematics teachers in Indonesia. Thus, in addition to the full sample of 104 teachers (with a
combined total of more than 3,500 students), case studies of eight teachers were conducted. A study based on the
full sample can help to create a somewhat generalizable7 picture of teaching practices (with recognized limitations).
However, practices that can feasibly be examined for such a large data set has many inherent limitations, including
the need to use low inference data and the inability to generate critical insights into why teachers use the practices
and how they make their decisions. The case studies of eight teachers allows for a more in-depth exploration of
teacher thinking and an examination of the rationale for the decisions they make related to teaching practices. It also
facilitates a more in-depth and higher-inference analysis of the practices. The case studies have greater value if what
emerges from these studies can be related to the larger sample in order to understand the results in a broader context.
2.7.2 UNIT OF STUDY AND ANALYSIS
The most applicable unit of analysis for the purposes of this study is the classroom. The objective is to characterize
mathematics teaching practices in Indonesia’s classrooms and to identify which of the classroom teaching practices
are related to high student achievement. The purpose is not to characterize or evaluate the performance of individual
teachers. Thus, individual teachers are not the unit of analysis in this study. Of course, using individual teachers as
the unit of analysis would have the advantage of enabling a more sensitive analysis. However, to do this, it would be
necessary to videotape multiple lessons by the same teachers to obtain a reliable measure of the performance of these
individual teachers. In addition, to study changes in performance over two points in time, from 2007 to 2011, it
would be necessary to trace the teachers so that they could be videotaped for a second time, four years later. This would
obviously be logistically impractical. As an alternative, this study utilized the same methodology as that adopted for
the TIMSS 1999 Video Study, in which a representative sample of teachers was chosen, with one lesson per teacher
being videotaped. Following this methodology, no conclusions can be drawn regarding the level of performance of
individual teachers. Rather, the aim is to draw conclusions regarding the level of performance of teachers as a whole.
This method also enables a comparison between groups of teachers with particular characteristics, such as teachers
with undergraduate qualifications compared to those without such qualifications, over the two points in time.
2.7.3 SAMPLING
The Video Study used a random sub-sample of the schools selected for the TIMSS sample, which itself was
selected on the basis of a rigorous sampling methodology. A representative sample is necessary in order to be
able to generalize the findings to the population under study. There are at least two aspects of representativeness
in a Video Study: (i) whether the teachers chosen are representative of teachers in the general population; and (ii)
whether the lessons videotaped are representative of the teaching of the teachers concerned.
7
The sample is based on the TIMSS sample, which is intended to provide a generalizable view for Indonesia, but limitations are recognized
and will be treated with care.
16 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
Sample of Video Study teachers
The selection of a random sub-sample of the TIMSS schools is intended to ensure that the selected teachers are
representative of teachers in the population. Since the goal is to establish a relationship between the characteristics
of classroom teaching with student learning outcomes, as measured by TIMSS, the sample for the Video Study
should be linked to the sample of the eighth grade classrooms in the TIMSS study. To keep the scale of the study
manageable while ensuring representativeness, 100 classrooms and their teachers were randomly selected from the
TIMSS sample. Following the practice of the TIMSS 1999 Video Study, one lesson per classroom was videotaped
for the purposes of coding and analysis. By randomly selecting a subset from this overall sample, the Video Study
sample was intended to be as nationally representative as possible.
The Indonesian TIMSS sample was itself selected in two stages. First, a random sample of 150 schools was
drawn from the target population of schools in the country, using a PPS method.8 Two stratifications were used in
the sample: type of school (public or private) and quality of school (based on the national test scores: high, average,
and low). One eighth grade class from each selected school was randomly drawn for the purposes of the study, with
all students in the chosen classes being asked to take the test and to complete the student questionnaire. Following
the practice of TIMSS, the sub-sample for the Video Study was also drawn from the main sample set using a PPS
method.
This method was utilized at two points in time, both in 2007 and in 2011. In each case, 100 teachers were
selected. In 2007, there were 150 classrooms in the sample set, while in 2011 there were 174. After the 100
schools were selected, the teachers of the eighth grade classes selected for the TIMSS were invited to participate
in the Video Study. The sample of teachers chosen to participate in the Video Study was randomly selected from
the 174 classrooms in 154 schools that participated in the 2011 TIMSS.9 The TIMSS sample followed a rigorous
methodology established and was conducted by IEA to ensure national representativeness. By randomly selecting
a subset from this overall sample, the Video Study sample was also intended to be as nationally representative as
possible.10
8
PPS stands for “Probability Proportional to Size”. The sample is chosen in such a way that the probability of a school being chosen is
proportional to the number of grade 8 students in the school.
9
There are 19 large schools that have two classrooms included in the sample.
10
One important note on the approach is that the study favored breadth over depth in its approach of including a large sample. Whereas some
studies have a smaller sample but examine a large number of lessons, this study aimed to get a large number of teachers, but filmed only two
lessons and coded one. This has methodological limitations that are discussed later. Additional depth was added through multiple filmings
of ten case study teachers.
Main Report 17
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Sample of case study teachers
The eight case study teachers were selected through a targeted approach, with an attempt to include teachers
who were most likely to provide insights into which teaching practices might be related with positive student
learning outcomes. A rich set of data was available for teachers that participated in the 2007 Video Study phase,
including classroom, school, student, community and teacher background variables, as well as TIMSS and school-
level national assessment results. This data allowed for the selection of a sample of teachers that the data indicated
were effective teachers. This selection was made purely on the basis of objective data and results, with the data from
the videos not being used. This use of this data was avoided in order to ensure that teachers were selected on the basis
of their results, rather than on any preconceived notions of what teaching practices might be effective.
The selection methodology was based on the actual versus predicted performance of each teacher’s students,
after controlling for key variables such as community poverty level, parent education level, etc. The top
teachers, whose students performed higher than predicted, were selected. It is important to note that it was relative
rather than absolute performance that mattered. Thus, the final sample selection included teachers from poor, rural,
under-resourced schools whose students did not score as well as those in other schools, but who performed better
than could be predicted on the basis of their circumstances.
In the selection of teachers, it was decided that five teachers would be selected from the 2007 sample based
on their relative effectiveness. However, another five teachers were selected from the 2011 sample on the basis of
variables of interest to MOEC and World Bank. Schools were chosen based on community poverty levels, urban-
rural mix, student pre-test results and teacher assessment results.
Initial analysis11 of the pre- and post-test results indicated that four of the teachers in the sample recorded
significantly higher than average student learning outcomes. However, four teachers recorded only average
or slightly above average levels of performance, while two teachers recorded lower than average levels of
performance. This altered the analysis approach somewhat, as the case studies could not be utilized to the extent
originally intended to examine common practices amongst the relatively more effective teachers. Still, the mix of
high, average and low-performing teachers enabled many useful comparisons, as will be discussed in later sections
of this report.
11
Only part of the teacher, student, school, and community background information was available for the initial analysis. This data is
important for more concise and accurate comparisons so student results are compared to others in similar contexts. This will be done, but the
initial results do indicate a mixed group of teachers and this is unlikely to change.
18 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
2.8 INSTRUMENTS
The study was highly complex, with multiple components and a range of different types of data. The following
section provides a brief overview of the multiple instruments used in the research.
Table 1: Research Data, Sources and Sample Grouping
Full Case Studies
Data 2007 2011 Source Type
Sample Only
LESSON OBSERVATION
Video of classroom sessions X X Study team Videotaping X
Transcripts of videotaped lessons X X Study team Transcripts X
Observation data sheets X X Study team Lesson data X
Coding of activities/practices following 1999 TIMSS X X Study team Coded data X
coding scheme
Video of case study teachers X Study team Videotaping X
Stimulated recall of case study X Study team Recorded X
Interview
TEACHER MATHEMATICAL KNOWLEDGE IN TEACHING
Subject knowledge competency assessment X Puspendik Assessment X
Pedagogic knowledge competency assessment X Puspendik Assessment X
Mathematical Scenarios X Study team Recorded X
Interview
TEACHER MATHEMATICAL BELIEFS AND ORIENTATION
Beliefs survey X Study team Questionnaire X
Beliefs interviews X Study team Recorded X
Interview
BACKGROUND SURVEYS
School survey X X IEA**, Study Questionnaire X
team
Teacher survey X X IEA**, Study Questionnaire X
team
Student survey X X IEA**, Study Questionnaire X
team
STUDENT LEARNING
TIMSS Mathematics test X X IEA Assessment X
Student pre- and post-tests X Puspendik Assessment X
** The original instruments from IEA/TIMSS were modified with additional questions specifically for the Video Study. These surveys were given separately from the IEA visit
that took place in May 2011. For the final analysis, data was available from both the IEA and Video Study surveys.
Main Report 19
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The figure below maps the instruments from a participant-specific point of view. All instruments are at least indirectly
connected, but in order to ensure clarity, only the primary links are displayed. The diagram highlights the fact that
for every set of teacher data at the full-study level of 104 teachers, there is related data at the case-study level. Each
of these instruments is elaborated in this section, providing details on the purpose, design, the analytical approach
and links to other data, together with milestones and issues related to the design and implementation of the study.
Figure 4: Mapping of Instruments by Full Sample vs. Case Study Sample
Student
Assessment
(pre- and post-tests)
and Students
All Teachers
Mathematical Beliefs Teaching Practices MKiT Context/Background
Teacher Subject
Teaching Practices Teacher, student
Beliefs Survey and Pedagogy
(video analysis) and school surveys
Assessment
Studies
Case
Teacher General
Beliefs Interview Stimulated Recall Teacher MKiT Background
Interviews Scenarios
Interview
2.9 CASE STUDIES
The case studies were intended to go beyond the what of teaching practices to explore the why. The full-sample
portion of the study provided extremely useful data for quantitative analysis and for painting a picture of teaching
practices in Indonesia’s eighth grade mathematics classrooms. However, while the overall study could help answer
what takes place, it could not answer why it takes place. In addition, the quantitative data from surveys could
only provide simple and somewhat superficial information, without enabling much deep insight. The case study
component was intended to address this gap by not only examining what takes place, but extending the analysis to
examine why certain teaching practices are used, including motivations and factors that influence choices made by
teachers in the teaching process. The case studies are by no means simply a supplement to the larger study. Rather,
it is the interactions between the quantitative and qualitative aspects of the study that enable us to draw a more
complete picture of mathematics teaching practices.
The case studies involved week-long visits to allow for immersion into each individual teacher’s environment,
with ample opportunity for interviews and observations. Although it is an exaggeration to consider the week-long
field visits to the case study teachers as ethnographies, the concept captures the spirit of the case study approach. A
foundational concept was that the visits should not simply be treated as a series of interviews to gather information,
20 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
but rather an immersion in the teacher’s environment to the fullest extent possible over the week spent at the school.
Thus, the extent of this immersion varied at each school, depending on the schedule and willingness to participate of
the teacher and other members at the school, with some teachers fully dedicating themselves to the week-long visit,
while others preferred to focus only on the specific activities related to the case study. In the cases where the teachers
were less accessible, an attempt was made to interact with other teachers and the headmaster.
Design Based on Conceptual Framework
The case study design is based on the framework described in Section 2.6. The conceptual framework relies heavily
on the case studies to provide details for all examined components. In particular, it relies on these case studies to
provide insights into why teachers use their chosen teaching practices. The case study visits were planned with
the purpose of obtaining a detailed account of the teacher’s perspectives and skills from a variety of angles, with
the focus areas being: (i) teaching practices; (ii) teachers’ Mathematical Knowledge in Teaching (MKiT); and (iii)
teachers’ mathematical beliefs. Based on these dimensions, the formal activities listed in Table 2 below were created
for the case study visits.
Table 2: Case Study Activities and Their Purpose
Activity Description/Purpose
General Interview with A relatively unstructured interview designed to obtain important background information on
Headmaster the school, parents, community and the headmaster him/herself. This interview also served the
purpose of making the headmaster comfortable with presence of the researchers.
General Interview with Teacher This interview was intended to gather important background information on the teacher, but
its primary goal was to get the teacher comfortable with the method of being recorded in an
interview and to establish a cordial relationship with the teacher. Questions were related to the
teacher’s experiences with mathematics from his/her time as a student up through becoming a
teacher.
Observation and video recording All lessons during the week that the teacher had for the given class involved in the study (e.g.
of lessons class 8.C) were observed and videotaped with a single digital video recorder at the back of the
classroom. Teachers typically had other eighth grade classes, but only the sampled class was
videotaped.
Stimulated Recall Interviews In these interviews, the teacher watched videos of his/her class and provided insights on what
took place in up to three classes. The first interview was designed to be teacher-driven, with
the researchers asking the teacher to identify any key moments or interesting insights. The
researchers remained relatively quiet in an attempt to avoid influencing the teacher regarding
what to focus on. The second and third interviews involved the researchers first analyzing the
videos to identify events of interest and then developing specific questions for the teachers.
Mathematical Scenarios These interviews involved the construction of five different classroom scenarios designed to elicit
Interview responses to enable the teacher to demonstrate aspects of their MKiT.
Mathematical Beliefs Interview These interviews consisted of two stages. The first stage involved a general discussion regarding
mathematical beliefs, including where the teacher developed his/her beliefs about mathematics
and how they may have evolved over time, as well as specific topics relating to (i) the origin
and essence of mathematics; (ii) how students learn mathematics; and (iii) how to best teach
mathematics. The second stage of the interview involved the teacher responding to descriptions
of different mathematical orientations.
Main Report 21
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
2.9.1 LENGTH AND SCHEDULING OF VISIT
The case studies involved visits of one week in each case. Various factors were taken into account when determining
the length of the case study visits, with the primary consideration being how to ensure the core information desired
could be gathered in the time allowed. However, other factors, including both fixed and variable cost, the burden
on the teacher and school, and the value of time, were taken into consideration. It was determined that a shorter
visit would be ineffective, because teachers tend to be less open and possibly even less honest during initial contact
with researchers. The researchers needed to build a relationship with the teachers before putting them through
challenging assessment-oriented tasks and discussing sensitive topics. Therefore, the visit was structured specifically
to enable first building a relationship with the teacher and school, followed by increasingly difficult or sensitive
tasks.
Every attempt was made to ensure the visit was as unobtrusive as possible. It was recognized that a week-long
visit with multiple recording sessions and interviews is a burden on both the teachers and their students. While
disruption was inevitable, the scheduling of activities was extremely flexible and always built around the schedule
and needs of the teacher.
2.10 LINKING COMPONENTS
An overarching aim of this research was to explore the relationships between interactions, practices, mathematical
beliefs and knowledge, as well as student learning outcomes. The following is a description of the analytical approach
for establishing these connections and exploring the relationships between them.
2.10.1 OVERVIEW OF RELATIONSHIP MAPPING
The multiple sources of data can be seen as providing slices of perspective. The diagram in Figure 5 below
illustrates that data for both MKiT and mathematical beliefs was gathered at four different levels, with each of
these levels providing a different perspective through which to inspect the component. This diagram is somewhat
hierarchical in the sense that it begins with the most concrete possible situation (the actual practices used in lessons)
and moves a step away to stimulated recall, in which the teacher gives a description or explanation of specific events
within the concrete situation. While this is “actual” in the sense that it relates to actual situations experienced by
the teacher, it is also a post-hoc construction by the teacher at the time of the interview. It is possible that the
teacher may intentionally or unintentionally craft an explanation that does not fully reflect the understanding or
thought process of the teacher at the time of the event. The next step away from the concrete comes through focused
explorations. The MKiT scenario uses hypothetical situations, while the beliefs interview explores the mathematical
beliefs in a more abstract, general sense. The survey and assessment do not necessarily represent a further step away
in the hierarchy, but rather provide information to complement the interviews in a standardized fashion that can
also provide a more generalized context, given that the full set of 100 teachers also participated in these activities.
22 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
Figure 5: Data Shown as Levels of Perspective for MKiT and Mathematical Beliefs
MKiT Beliefs
Practices used Video Video
Enacted/Actual
in lessons Analysis Analysis
Teacher explanation Stimulated Stimulated
of the practices Recall Recall
Evoked/Espoused
Focused explorations Scenarios Beliefs
(general/hypothetical) Interview Interview
Standardized Teacher Beliefs
measures Assessment Survey
• Actual practices: This level examines what teachers actually do in the classroom in search of aspects related to
MKiT and beliefs. From a mathematical beliefs perspective, this is often referred to as “enacted beliefs.” For
MKiT, this represents the knowledge the teacher evokes and applies in the classroom for the given situation.
There are many environmental and circumstantial factors that may influence the teacher’s actions. These may
not be visible or understood by the observer. What actually takes place should be interpreted with caution,
given the limitations that the researcher has in interpreting MKiT and beliefs of a teacher through observed
practices.
• Teacher explanation of practices: Through stimulated recall interviews, certain moments in the lesson
are discussed with the teacher, to enable the teacher to provide his or her perspective on the event. The
researchers typically did not ask questions directly related to the teacher’s beliefs or MKiT, but statements
made by the teachers often gave indications related to their beliefs and, to a lesser extent, provided insights
into their MKiT.
• Focused explorations: The MKiT Scenario and Mathematical Beliefs interviews explored the concepts much
more directly than the stimulated recall interviews. However, they did so with a certain degree of separation
from the actual situations recorded in the videotaped lessons. Beliefs were discussed in general terms,
although teachers would often contextualize their statements or relate them to situations they experienced.
The Mathematical Scenarios interview presented hypothetical classroom situations to demonstrate aspects
of MKiT through their identification of issues, their thought process and strategies they developed. This
demonstration of MKiT may or may not capture what teachers would do in an actual classroom situation,
where many other complexities and contextual considerations may be involved.
• Standardized measures: The Teacher Assessment and beliefs survey can be seen as scaled and more easily
cross-comparable aspects of MKiT and mathematical beliefs.
Main Report 23
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Each level provides a different perspective. It is through the triangulation (or in this case, “quad-angulation”) of
the angles that a well-rounded picture of the teachers’ MKiT and beliefs are formed and the role they play in the
classroom. The design of the conceptual framework is based on the theory that a teacher’s MKiT and beliefs are
important influencing factors in their teaching practices. The Standardized measures and Focused explorations levels
provide a picture through espoused statements and evoked knowledge; the video level shows actual practice; and the
analysis is a constant back-and-forth between actual and espoused/invoked aspects in more hypothetical situations.
The stimulated recall not only provides detail through elaborated explanations, but also enables the teacher to
provide important insights into why there may be discrepancies between the actual and hypothetical/external.
2.10.2 ANALYTICAL STEPS FRAMEWORK
The analysis can be seen as a four-stage process, with each subsequent stage building upon the results of the previous
stage. This progressive analysis is depicted in Figure 6 below.
Figure 6: Four Stages of Analysis: (1) Individual; (2) Component; (3) Cross-component; and (4) Full-component
PRACTICES
Video
Teaching practices used in the lessons
A
Stimulated recall
D How teacher describes events within the E
lessons and explains decision/actions
G
MKiT MATHEMATICAL BELIEFS
Scenarios Beliefs Interview
Demonstrated knowledge of students, Espoused beliefs through general
knowledge of practices and subject knowledge discussion and beliefs situations
F
B C
Teacher Assessment Beliefs Survey
Subject and pedagogical knowledge test scores Questions covering nature of mathematics,
how students learn, how to teach and
teacher’s own practices
Stage 1: Individual Instrument Analysis
The data collected for each instrument is first analyzed on its own, with the method of analysis being dependent on
the type of data collected. The results are summarized with the aim of being to enable a comparison with the results
of the other instrument within in the component.
24 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
Stage 2: Component Analysis (A, B and C)
The Points A, B and C indicate comparative analysis that takes place within components. Each component has two
data sources and the cross-instrument analysis is mainly concerned with identifying consistencies and discrepancies
between the two sources of data and with combining the different aspects provided in order to generate a more
complete picture of Beliefs, Knowledge and Practices.
Stage 3: Cross-Component Analysis (D, E and F)
The next stage was to relate the three components to one another. The Points D, E and F indicate analysis across
components. The questions that drive this relationship analysis relate to consistency, flow of influence and
effectiveness, as follows:
• Are there consistencies in what is espoused or demonstrated outside the classroom (hypothetical scenarios)
and what is enacted or evoked in the classroom?
• In the case of identified inconsistencies, what might explain those inconsistencies?
–– Are there common factors that may explain why some teachers are more or less consistent than others?
–– Are the inconsistencies isolated incidents or are they part of a larger pattern?
–– To what extent could the inconsistencies be due to researcher interpretation?
• How might the components be influencing one another and what is the dynamic of the influence?
–– The clearest hypothesis, as demonstrated in previous studies, is that Beliefs and MKiT influence Practices,
but is the flow primarily one-way or is it bi-directional and/or cyclical?
• Might the inconsistencies in any way impact the effectiveness of teaching practices?
Stage 4: Full-Component Analysis (G)
This final stage, depicted at Point G, could be considered part of Stage 3, in that it is a form of cross-component
analysis. However, it involves taking a summary view of the cross-component results to determine how all three
components interact as a whole.
2.11 MACRO TO MEZZO TO MICRO ANALYSIS
The original analytical approach anticipated using the full set of 104 teachers to generate a general, high-level
perspective and case study teachers to generate a more in-depth perspective. The sample size of eight case study
teachers was chosen to ensure the inclusion of a good range of situations to enable meaningful comparisons. It also
included a buffer in case some case study teachers dropped out of the study.
All ten teachers did end up making themselves available for the field visits. However, various complications
arose in cases CS1 and CS8. In the case of CS1, the beliefs interview was not recorded due to a technical issue. In
the case of CS8, the teacher was busy at the time of the visit and unable to allocate sufficient time for some of the
interviews. While she had agreed to participate, she was clearly uncomfortable with the interviews and provided
short and somewhat incomplete responses. It was decided that these case studies would be dropped from the
analysis.
Main Report 25
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Initial analysis of the eight remaining case studies provided a rich range of data for the purposes of comparison.
However, when attempting to formulate detailed descriptive accounts for each case study, it became clear that the
large number of cases made the descriptions unwieldy and difficult to follow when moving from stage to stage of
the analysis steps framework (see Section 2.10.2). In order to address this issue, a Macro-Mezzo-Micro approach
was chosen. The eight cases provided a rich range of data and it certainly made sense to keep the analysis. However,
in order to make the detailed descriptions more meaningful and to ensure that the additional in-depth analysis
remained focused, a subset of three case study teachers was selected. The number of teachers and the general
presentation approach used for each level is shown in Figure 7.
Figure 7: Breakdown of Macro-Mezzo-Micro Approach to Analysis
MACRO MEZZO MICRO
Full set of 104 teachers 8 Case Study Teachers 3 Case Study Teachers
Survey, coding and Initial analysis for all More in-depth
assessment data only components descriptions
No individual teacher Summary results Cases threaded through
descriptions of non- presented each stage of analysis
case study teachers Some speci c examples
drawn upon
The Micro cases were selected on the basis of the results of the initial Mezzo analysis. All the Micro cases
included interesting material for analysis and important lessons could have been learned from each through deeper
focused analysis, making it difficult to reduce the number. A primary criterion at the initial research design stage
was to attempt to determine what makes for effective teaching in Indonesia. Thus, student learning outcomes, as
determined by pre-test and post-test results, were a leading factor in the selection process. A clear choice was CS9,
because this teacher’s students recorded the largest learning gains out of the full 104 sample of teachers. Therefore,
to gain an understanding of what might make teachers effective, CS9 could provide important insights. She was
also one of the highest scorers in the MKiT assessment, in both the subject and pedagogy sections. This case study
had the potential to enable an investigation into the relationship between knowledge and teacher effectiveness. CS9
had similar characteristics to CS6 and CS4 in terms of the school environment, relatively high MKiT scores, and
strong student learning outcomes. Thus, these cases were dropped from the Micro analysis in order to ensure a wide
range of teacher types. CS3 had some interesting characteristics, in that his subject score was the highest of the full
sample of teachers, but the pedagogy score was closer to average. In this case, the students’ learning outcomes were
slightly below average, providing a contrast to CS9. Finally, CS7 had a relatively high pedagogy score, but a below
average subject score. This teacher also taught in a poor rural area environment, but recorded above average student
learning outcomes. It was decided that this case study had the potential to provide insights into effective teaching
and learning in this context.
26 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
As the Micro analysis progressed, it became clear that at times the other Mezzo teachers needed to be brought in for
certain pieces of the analysis. A gap that emerged was that none of the three selected Micro teachers had a strong
transmissionist beliefs orientation. As this became an important focus of the analysis, CS2 was often brought in to
provide additional insights.
2.12 REGRESSION MODELLING APPROACH
An overarching goal of the study was to understand how teaching practices, teacher mathematical beliefs and
mathematical knowledge in teaching relate to one another and how they in turn relate to student learning
outcomes. For the quantitative analysis, a regression modeling approach was used to analyze the data from the
surveys, assessments and video coding produced for the full sample of 104 teachers. The analysis was conducted to
examine patterns from the regressions. In order to ensure consistency throughout the study and to make it easier to
interpret the information, the results of the regression are all presented in a standardized manner. The logic behind
the results presentation and interpretation are provided below.
2.12.1 SELECTION OF CONTROL VARIABLES
The results of regressions are presented in terms of relationships of multiple two-variable (e.g. variable x
and variable y) combinations. For example, the regression would show the relationship between (x) teachers’
use of expositional teaching in relation to (y) teachers’ score on the MKiT assessment. However, when producing
those results, many student, family, classroom, teacher, school and community variables are controlled for. In the
selection of variables, a reference point had to be used. Thus, student learning outcomes, based on the post-test
score of each student with their pre-test score being controlled for, were used as the reference point. The selection of
variables was implemented by asking the question: “Which student, home, classroom, school, teacher background
and community variables are related to student learning?”
The first step was to determine basic correlations for each variable from the student, teacher and school
surveys. For example, simple correlations indicated that student learning outcomes were related to the level of
poverty of the community; the educational level of the student’s parents; whether the teacher was a civil servant; the
frequency of bullying in school; and many other variables. In the case of all of these factors, there was a statistically
significant correlation. These variables were then combined using an iterative process of stepwise regression analysis
until a set was developed of the most important variables to control for when comparing practices, beliefs and
knowledge variables. These variables are listed in Table 3 below.
Main Report 27
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Table 3: Variables Used in Various Layers for in the Regression Analysis
Groupings Variables
Student Student pre-test score
Student age
Hours of homework per week
Extent to which homework done with peers
Hours spent reading for enjoyment per week
Hours of tutoring in mathematics outside the classroom per week
Student’s perception of mathematics as relevant for future work
Student home Whether the home has a computer
Extent to which Indonesian is spoken at home (vs. a local dialect)
Average education level of parents
Classroom and School School size
School type (public or private)
Whether the school is a madrasah (religious school)
Resource measure –materials
Resource measure – calculator
Level of involvement of the principal (as specified by the teacher)
Level of involvement of visiting supervisor (as specified by the teacher)
Frequency of students skipping class (as specified by the principal)
Extent to which vandalism is a problem (as specified by the principal)
Community Community poverty level (percent poor)
Community size
Teacher background Years of experience teaching
Whether teacher is a civil servant
Whether teacher is female
Whether teacher is school-hired
Whether teacher participates in a teacher working group
Amount of training received in the last year
Teacher quality Teacher degree type
Teacher education level
Whether teacher is certified
Teacher MKiT assessment score (not included in all regressions)
For the purposes of scope, these variables are treated as control variables only, with no discussion on how these
variables change on the basis of the practices, beliefs and knowledge variables used.
2.12.2 PROGRESSIVE BUILDING-UP OF THE MODEL
The regressions began with simple correlations. They then progressed with a small number of variables, which
were progressively built up with more variables. At first, no control variables were used; then student, school
and community variables were added; then teacher background variables were added; and finally teacher quality
variables were added. Such analysis may be useful for probing how relationships might change based on aspects of
the teacher. However, for the purposes of scope in this study, it was decided that the results from only the final, fully
built model would be used.
28 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
2.12.3 PRESENTATION OF RESULTS IN RELATIONSHIP MATRICES
The results from the regressions analysis provide an extensive amount of information, such as coefficient size.
However, for the purposes of this analysis, a single result was the focus of attention, this being the statistical
significance level of the relationship between the two variables of interest. In the case of these regressions, it is the
t-statistic. It provides a simple answer to the question: “Is there a statistically significant relationship between variable
x and variable y?” This could be applied for multiple combinations of variables for beliefs, knowledge and practices.
The benefit of focusing on the statistical significance is that it allows for pattern analysis. However, it is
limited in its interpretation, as this study recognises. It is certainly true that the coefficient, which is the constant
(a) in the formula y = ax + b, is of particular importance because it represents the rate of change of one variable (y)
as a function of changes in the other (x). A particularly small coefficient would indicate that even if the relationship
is statistically significant, the change itself is minor. However, for the purposes of this analysis, what is of interest is
the determination of whether relationships exist and detection of patterns over multiple combinations. No claims
are made regarding terms of magnitudes or how much of a difference these relationships would make in terms of
dimensions such as student learning outcomes.
A simplified illustration of a matrix is provided in Figure 8 below. It shows two beliefs statement variables and
three MKiT assessment score variables. The first statement: “Mathematics is a creative subject like arts / music”
shows that there is a positive relationship at the 5% level (95% confidence) with the teacher MKiT score. This
means both move in the same direction and that as teachers’ MKiT score increases, there is a tendency towards an
increased level of agreement with the statement. When breaking town the MKiT score into subject and pedagogy
sections, there is no relationship between subject score and the statement. However, there is a positive relationship
at the 1% level, indicating a very strong relationship between the statement and the pedagogy score. On the other
hand, the statement: “Telling the children the answer is an efficient way of facilitating mathematics learning” has
no relationship with MKiT and subject scores. However, it has a negative relationship with the pedagogy score at
the 1% level. This indiates that as the teacher pedagogy score increases, there is a tendency for the respondent to
disagree with the statement.
Figure 8: Example of Coding of Regression Results to Indicate Relationships
Pedagogy
Subject
Beliefs Statements and Stated Use of Practices
MKIT
Mathematics is a creative subject like arts/music 2
Telling the children the answer is an e cient way of facilitating their mathematics learning -1 -1
POSITIVE relationship and statistically signi cant at 1% level (99% con dence level)
POSITIVE relationship and statistically signi cant at 5% level (99% con dence level)
POSITIVE relationship and statistically signi cant at 10% level (99% con dence level)
NEGATIVE relationship and statistically signi cant at 10% level (99% con dence level)
NEGATIVE relationship and statistically signi cant at 5% level (99% con dence level)
NEGATIVE relationship and statistically signi cant at 1% level (99% con dence level)
Main Report 29
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The color coding for the cells is based on the t-statistic.12 While initially the numbers themselves were displayed
in the cells, this resulted in a lack of clarity. Since the approach to the analysis is to identify relationships and
patterns, it was determined that using color coding, without providing numbers, was more appropriate. An example
of the regression that created the above result for the combination of the first statement and MKiT can be found in
the technical report.
2.12.4 DATA MINING FOR PATTERNS
The data mining concept posits that within large datasets made up of many variables, there often are hidden
gems of data that cannot easily be uncovered using simple analysis or by approaching the data with specific
hypotheses.13 Hand (2007) defines data mining as the discovery of interesting, unexpected or valuable structures
in large datasets and notes two different aspects to this definition. One concerns large-scale, global structures, with
the aim being to model the shapes, or features of the shapes, of distributions. The other concerns small-scale, local
structures, with the aim being to detect these anomalies and to determine if they are real or chance occurrences.
Data mining uses algorithms designed to exploit relationships in the data, searching for unexpected but
important pieces of information. A classic example used to describe the method and benefits of data mining is
one in which Walmart analyzed data of purchases to uncover an unexpected relationship between beer and diapers
(Chan and Wong, 1991). This connection was surprising, but through additional analysis it was discovered that
young fathers tended to be the ones making the beer and diaper purchase combination. The explaination goes that
when young fathers are sent out on an errand to buy diapers, often on weekends, they would purchase a six-pack of
beer to reward themselves for their heroic efforts. This information could then be used to place beer next to diapers
in their stores, resulting in a dramatic sales increase.14
While important results may emerge by mining the data in search of relationships and patterns, there are
also inherent deficiencies in this approach. The approach is weak if the analysis stops at the stage of identifying
relationships and blindly draws conclusions on the basis of the correlation. Rather, data mining is an initial step
that must be followed by posing conjectures and conducting additional analysis to understand why the relationship
might exist.
12
Although the 10% level (90% confidence) is not considered sufficient to meet the threshold of being considered statistically significant,
these instances are still identified in with very light colorings in order to identify those that are approaching but do not quite reach statistical
significance.
13
As a preface, I previously worked as a consultant designing databases for corporations. In 1998 the concept data mining became was
emerging as new analytical tools and methods allowed for more powerful and accurate analysis. It continues to be a common analytical
method, particularly in large businesses and government organizations.
14
It should be noted that there are many variations on this story and there is also a question as to whether it is true or a legend.
30 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overview of Design, Methods and Analytical Approach
The analysis performed in this study uses the data mining concept to analyse thousands of variable combinations
through regressions, with a common set of control variables to identify statistically significant relationships.
These relationships are then analyzed for common patterns and groupings. For example, there may be many
statistically significant positive relationships between combinations of beliefs-practices regarding beliefs related to
the teacher’s role as a guide and practices related to teachers’ use of the investigative method. This combination
might be expected and could be used as evidence to strengthen the hypothesis. However, there may also be cases
where unexpected relationships emerge, either with combinations where there is no hypothesized relationship or,
even more interesting, where the hypothesized relationship goes in the opposite direction to that expected.
2.12.5 CONFOUNDING FACTORS
An important limitation to recognize in this regression approach is that of confounding factors. The regressions
may produce relationships between the independent variable and a dependent variables that are estimated incorrectly
because the estimates fail to account for confounding factors. The incorrect estimations could suffer from omitted-
variable bias. As an example that will be discussed later, there is a strong relationship between teacher subject
knowledge and student learning outcomes. While it is logical and likely that the teacher’s knowledge would play a
role in making him or her be more effective, it may also be the case that teachers with higher knowledge tend to be
more motivated, driving them to study harder in pre-service training and seeking out in-service opportunities and
other information to build their skills. This motivation would also play a role in the effort they put into planning
and executing their lessons. The higher learning outcomes could therefore be in part due this motivation, but it is
being captured through the teacher’s higher knowledge scores.
This limitation is recognized and attempts were made to not simply take the regression results as given. A great
deal of effort was made to question the regression results and combine them with more qualitative analysis in order
to provide a more complete picture. With the teacher knowledge example videos were examined to understand how
knowledge is enacted in the classroom. Case studies were used to probe knowledge from various angles in order to
probe connections between knowledge, beliefs, practices and learning. While the confounding factors cannot be
eliminated, it was hoped that this process would get closer to understanding relationships.
Main Report 31
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
32 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 3
TEACHING PRACTICES
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
This chapter presents a brief summary of the study’s findings on proportions, frequencies, trends, cross-
country comparisons and relation to student learning outcomes. The practices are grouped as follows: (i)
lesson structure; (ii) lesson content; and (iii) instructional practices. The Lesson Structure focuses on the length of
the lessons; how much time is dedicated to mathematics, non-mathematics and mathematics organization activities;
the purposes of various segments; and the type of interaction that takes place. The Lesson Content section focuses on
the mathematics content of the lessons and assesses the complexity of problems and whether the problems involved
applications or proofs. The Instructional Practices section focuses on how the mathematics problems were presented
and addressed; the opportunities of teachers and students to talk; and resources used during the lesson.
Complete analysis of each teaching practice, including full definitions; graphical representations of proportions
of time dedicated to each practice; and comparisons with the seven other countries that participated in the
1999 TIMSS Video Study, can be found in the full Technical Report. This Main Report summarizes the key
findings, with more in-depth discussion in relation to practice and policy themes in chapter 8.
3.1 PROPORTIONS, TRENDS AND CROSS-COUNTRY COMPARISONS
Lesson Structure
There was a large variation in the length of the mathematics classes videotaped in Indonesia. The average
duration of classes in Indonesia ranged from 39 to 97 minutes, with an average class length of 70 minutes. This
period of duration is much longer than the seven comparator countries. However, the actual total time dedicated to
mathematics classes each week is estimated to be an average of 140 minutes, which is lower than the other countries,
which tend to have more lessons each week.
Indonesia’s classes tended to spend a relatively larger portion of class time on non-mathematical and
mathematical organization activities. The proportion of time spent on mathematical activities was on average
89% in 2007, declining to 86% in 2011. By comparison, the average proportion of time spent on mathematical
activities amongst the comparator countries ranged from 95% to 98%.
Figure 9: Proportional Breakdown of Lesson Time
Indonesia 2007 86 10 4
Indonesia 2011 89 8 3
Japan 98 11
Czech Republic 98 11
Switzerland 97 21
Hong Kong 97 21
United States 96 3 1
Netherlands 96 2 2
Australia 95 4 1
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mathematics Mathematics Organization Non-Mathematics
34 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teaching Practices
In terms of lesson segment purpose, the most common use of mathematics time was to introduce new content.
On average in 2011, 59% of time was dedicated to new content, 26% to practice, and 15% to review. The trend
indicates that there was a large increase in time dedicated to new content and a corresponding decrease in practice.
In comparison to other countries, the most striking difference is how much more time is dedicated to review, with
Indonesia are spent relatively less time on review and more time in practice than did classes in the comparator
countries.
Figure 10: Duration for Different Activities in Indonesia and Other Countries
Indonesia 2007 11 48 41
Indonesia 2011 15 59 26
United States 53 23 25
Switzerland 35 40 25
Netherlands 39 34 27
Japan 24 60 16
Hong Kong SAR 24 39 37
Czech Republic 58 22 20
Australia 39 33 28
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Review New Content Practice
Instructional Practices
By far the most common teaching strategy utilized in Indonesian classrooms was exposition. Indonesia’s lessons
tend to be teacher-centered, with 61% of time on average being dedicated to exposition or lecturing; 22% to problem
solving; 9% to discussion; and 7% to practical work. Only 1% of the time was dedicated to investigation. In terms
of trends, the biggest change was the increase in exposition from 53% to 61%, while discussion, investigation and
practical work all saw slight decreases.
Figure 11: Time for Different Learning Activities in Mathematics Lessons
2007 13 53 3 10 21
2011 9 61 1 7 22
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Discussion Exposition Investigation Practical work Problem solving
Note: The instructional practices layer was not part of the 1999 TIMSS video study, so no cross-country comparison can be made.
Main Report 35
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Classroom Interaction
Indonesian classes fell in the middle of the range compared to other countries in terms of whole-class (public)
versus group and seatwork (private) interaction, with 64% of class time involving whole class interaction and
36% involving private interaction. The proportion of time spent on whole class interaction increased in 2011
increased from the figure of 57% recorded in 2007. By contrast, teachers in Hong Kong SAR were at one extreme,
with 80% of class time being spent on public interaction, while the Netherlands was at the opposite extreme, with
only 44% of time being allocated for this purpose.
Figure 12: Percentage of Time for Public Interaction and Private Interaction
Indonesia 2007 59 41
Indonesia 2011 64 36
Hong Kong SAR 80 20
Czech Republic 79 22
United States 68 32
Japan 66 34
Switzerland 55 45
Australia 52 48
Netherlands 44 55
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Public (whole-class) interaction Private (group work and seatwork) interaction
Of whole-class interaction time, in Indonesian classrooms, most of the time was spent on teacher-only
interaction, with these interactions typically involving lecture type presentations. The teacher-only (lecture)
was the most common form of public interaction, making up 74% of all public interaction in 2011. This was also a
large increase from the proportion of time spent on this type of activity in 2007, when the proportion stood at 61%.
Figure 13: Public (Whole Class) Interaction Breakdown: Teacher and Student Involvement
2007 61 21 18
2011 74 11 15
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Teacher only Teacher and student Student only
On the other hand, teacher-student interaction made up a small proportion of whole-class time, with this
proportion declining over the two points in time. In 2011, the proportion of time spent on this type of interaction
was only 11%, down from the figure of 21% recorded in 2007.
36 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teaching Practices
The number of words spoken by both teachers and students in Indonesia’s classrooms was significantly less
than in other countries. The average number of words spoken by the teacher in a lesson (standardized to 50
minutes) was 3,243. At the extremes, some teachers spoke only 813 words, while others spoke up to 5,687 words.
By contrast, the average number of words spoken by students in a lesson was 509.
Figure 14: Average Number of Words Spoken by Teacher and Students During the Lesson
7,000
6,000 5,798 5,902
5,360 5,452 5,536
5,148
5,000
4,000
3,243
3,000
2,000
1,016 1,018
1,000 766 824 810
509 640
0
Indonesia Netherlands Switzerland Czech Australia Hong Kong United
Republic States
Average Teacher Words Average Student Words
Indonesian teachers rarely spoke in long sentences. Fifty-six percent of all sentences spoken by teachers consisted
of 1–4 words, which was a much higher proportion than that recorded in other countries, which ranged from 18%
to 29%. The proportion of sentences consisting of 24 words or more was very low in Indonesia, at 3%. In other
countries, this proportion ranged from 25% to 41%.
Indonesian Students also rarely spoke in long sentences. Seventy-nine percent of all sentences spoken by
Indonesian students consisted of 1–4 words In Indonesia, which was significantly higher proportion than that
recorded in other countries, which ranged from 66% to 77%. The number of sentences consisting of 5–9 words
in Indonesia was 12%, significantly lower than in other countries, where the figure ranged from 23% to 34%. The
proportion of sentences consisting of 9 words or more in Indonesia was 9%, compared to a range of from 4% to
9% in other countries.
The Role of Mathematical Problems
As in other countries, most of the lesson time in Indonesia was spent on solving mathematical problems,
either by the teacher or by the students. But in Indonesian classrooms, a significantly larger proportion of time
was spent on non-problem segments compared to other countries. Indonesia spent much more time on non-
problem segments than in other countries, with 76% dedicated to problem work in 2007, with this figure declining
to 64% in 2011. Of the seven comparator countries, the next lowest proportion was 82% in Japan, while the
highest was 96% in the Netherlands.
Main Report 37
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Figure 15: Mathematical Time Divided into Problem vs. Non-Problem Segments (Percent of total)
Indonesia 2007 76 24
Indonesia 2011 64 36
Japan 82 18
Hong Kong SAR 85 15
Czech Republic 85 15
Australia 85 15
Switzerland 87 13
United States 89 11
Netherlands 96 4
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Problem Non-problem
In comparison with other countries, Indonesian classes addressed fewer independent problems, but spent
more time on solving them. In Indonesia, most classes addressed only two independent problems per lesson,
compared to an international average of 3.3 problems. The number of independent problems solved over the course
of each videotaped lesson varied from one to 10. At one extreme, Japanese classes addressed an average of three
independent problems per lesson, but spent 15 minutes on each problem. At the other extreme, classes in the Czech
Republic addressed an average of 13 independent problems, but spent only four minutes per problem. Only in
Japanese classrooms was more time spent per independent problem.
On average, Indonesian classes addressed very few problems of high complexity. However, there were a larger
proportion of problems of medium complexity than in other countries. The proportion of problems that were
of low complexity ranged from 17% to 77%, with problems of average or medium complexity ranging from 22%
to 45% of the total, while high complexity problems ranged between 6% and 39% of the total. The problems
solved in Indonesian classrooms were generally of a lower level of complexity, with the average percentage of low
level problems being 57% and of medium complexity being 40%. Only 3% of the problems were of a high level
complexity.
Indonesia had very few problems that involved applications. Applications are often presented using verbal
descriptions, graphs or diagrams, rather than just mathematical symbols. Applications are important because they
require students to make decisions about how and when to use procedures they may have already learned and
practiced. In this sense, applications are, by definition, more conceptually demanding than routine exercises. The
average proportion of problems which included applications in Indonesia was 16%, compared to a range of 34%
to 74% in other countries.
Relative to other countries, Indonesian teachers stated the concepts for problems to a greater extent, while they
used procedures to a lesser extent. When comparing Indonesia to other countries, it can be seen that Indonesian
teachers stated concepts much more frequently than did their counterparts in other countries, with a rate of 35%
compared to a range of 5% to 24%. Indonesian teachers made relatively less use of procedures, with a rate of only
41%. They made connections relatively frequently, with a rate of 24%.15
15
Note: The number of problems differs significantly from the amount of time for each type. Make connections was only 3.6% of the total
time while state concepts was 60% and use procedures was 35%
38 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teaching Practices
Indonesian teachers tended to use the problem context of real-life situations relatively less frequently than
did teachers in other countries. In Indonesia, the average proportion of problems involving a real world context
was 12%, while in other countries, the figure ranged from 9% to 42%. On the other hand, 88% of the problems in
the Indonesian classroom involved mathematical language and symbols only, compared to a range of 40% to 89%
in other countries.
Tools and resources used
The use of physical tools in Indonesia was noted in more than half the classrooms. Physical tools in the
mathematics learning process include tools for measuring (e.g., rulers, protractors), compasses, visual aids tools,
tiles, three-dimensional and two-dimensional geometry models, etc. Essentially, physical tools are tools that can
be manipulated by teachers and students. Sixty three percent of the lessons in the sample used physical tools. The
average number of problems which required the use of physical tools was 3.7.
The use of real objects was relatively more prevalent in Indonesia than in other countries. Approximately 28%
of the lessons in the sample used real objects compared to a range of 4% to 21% in other countries.
The percentage of lessons in Indonesia in which a calculator was used was very small, occurring in only 13%
of schools in the sample. This was in great contrast to the practice in other countries, where the rate of usage could
be up to 91%. This situation in Indonesia was due to policies set by schools or teachers of not permitting the use of a
calculator in the mathematics learning process, the intention being to prepare students for the national mathematics
examination, during which students are not permitted to use calculators.
3.2 RELATIONSHIPS BETWEEN TEACHING PRACTICES AND LEARNING
OUTCOMES
Regression analysis allows for the detection of relationships between various teaching techniques and student
math scores. For the 2011 phase, tests were given to students at the beginning and end of the 8th grade year to
measure student learning outcomes for the year. Relationships cannot be interpreted to show cause and effect. Still,
after controlling for key student, household, class, school, and demographic characteristics, statistically significant
relationships emerge between teaching practice and student learning outcomes and provide useful initial insights
into what may be relatively more effective teaching practices in Indonesia.
It should be emphasized upfront that these relationships were seen as simply a first step in understanding
how teaching practices relate to student learning; the relationships tell only part of the story. The effectiveness
of practices is not simply the proportion of time a teacher spends using a specific practice, but also how the teacher
uses it that will determine its effectiveness. The practices were also examined in a qualitative manner and considered
in the context of the classroom environment and teacher characteristics.
Main Report 39
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The following results emerged:
• A constant theme that emerges from the regression results is that classes with higher student-teacher
involvement (e.g. student-teacher interaction in whole class settings and teacher monitoring and participation
in group work and individual seatwork) had higher test scores. Traditional teacher lecturing, on the other
hand, tended to have a negative relationship.
• Teacher involvement in activities, either through monitoring or direct teacher-student interaction, tended to
have a positive relationship compared to student-only interaction (students working alone during seat work
and group work and whole class time where the teacher was not actively involved).
• For teaching approaches, classes where investigative learning and practical activities were used more often
tended to have higher learning outcomes whereas classes with more expositional teaching tended to be negative.
• The proportion of time on task tended to be important, where classes spending a larger proportion of time
on mathematics tended to have higher learning outcomes, while classes with a higher proportion on non-
mathematics (off-task) tended to have lower learning outcomes.
• Although it was rarely used, assessment time and assessment-related activities such as quizzes had a positive
relationship to test score.
• The percent of time spent on problem math (as opposed to non-problem math) had a positive relationship with
student learning outcomes.
• The process of setting up a problem with use of a procedure had a negative relationship with student
achievement. Making a connection, on the other hand, is not as common, but it has a positive relationship
with student learning outcomes.
• Students in classes where a larger number of proofs were introduced tended to have higher student learning
outcomes.
• Classes where the set-up of problems used math language tended to have higher student learning outcomes
than those that tended to use setup with real life context.
• Classes where open-ended questioning rather than closed questioning tended to have higher learning outcomes,
but the practice is rare.
• Classes where non-routine problems were used tended to have higher learning outcomes, but the practice is
rare.
• Use of tools and resources could play a role, where the use of projectors tended to have a positive relationship
with math scores, while use of textbooks tended to have a negative relationship.
The results in this brief overview on proportions, trends and relationships to student learning outcomes will be
explored in more detail through later sections and presented in a more thematic approach in section 8 where
conclusions are drawn and policy implications are considered.
40 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 4
MATHEMATICAL
KNOWLEDGE IN TEACHING
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Across multiple studies, the level of teachers’ knowledge has been found to be one of the most significant
components in relation to student learning outcomes. Although there is no strong, consistent correlation between
observable teacher characteristics such as age, degrees, certification status and years of experience, with effectiveness,
teacher knowledge is consistently correlated to student learning outcomes. Bruns and Luque (2014) note that while
relatively few studies have been able to examine the academic ability of teachers in service by testing them directly,
the studies that have done so find positive correlations between teachers’ test scores and those of their students
(Marshall and Sorto 2012; Hanushek and Rivkin 2006; Hill, Rowan, and Ball 2005; Eide, Goldhaber, and Brewer
2004).
The significance of levels of teacher knowledge as a factor in determining student learning outcomes appears
to be even stronger in developing countries. A review conducted by Bruns and Luque (2014) of the evidence
from developing countries on how different educational inputs affect educational outcomes found that teachers’
knowledge of their subjects is one of the few variables consistently correlated with student learning outcomes.
The review involved 20 different studies of the impact of teachers’ knowledge, as measured by test scores, on their
students’ learning performance. All of these studies found a positive correlation, with 13 finding a significantly
positive correlation. The researchers concluded that of all the education inputs that have been evaluated, including
school infrastructure, learning materials, teacher and principal characteristics, and school organization, only three
are consistently and significantly correlated with learning outcomes. These are (i) teacher knowledge; (ii) teacher
absence rates; and (iii) the availability of desks. (Glewwe et al. 2011)
The importance of the teacher’s level of knowledge in the teaching-learning process has long been recognized.
Recent research in mathematics teaching has begun to uncover the complexity of this knowledge. Since the mid-
1980s, attempts have been made to investigate the concept of what, for the purposes of this research, will be referred
to as Mathematical Knowledge in Teaching (MKiT). Various theoretical models have been developed to describe the
domains of mathematical knowledge needed for teaching. Many studies have also explored the interactions between
various knowledge domains, with some being topic-specific (e.g. geometry, algebra, etc.), while others focus on the
classroom and cultural contexts within which knowledge may play different roles.
With a few notable exceptions,16 research on teacher knowledge before the mid-1980s tended to focus on
general aspects of teaching, with subject matter being seen as little more than context. Shulman noted a gap in
the examination of content and its role in instruction, labeling this the “missing paradigm” in research on teaching
and teacher knowledge (Shulman, 1986). Shulman is often credited with opening the doors to a new area of
research on teacher knowledge through his proposed special knowledge domain that he and his colleagues termed
Pedagogical Content Knowledge (PCK). Shulman noted that that there is content knowledge unique to teaching, a
kind of professional knowledge that is specific to a particular subject matter. Shulman argued that: “The currently
incomplete and trivial definitions of teaching held by the policy community comprise a far greater danger to good
education than does a more serious attempt to formulate the knowledge base.” (Shulman, 1987, p. 20) added
that: “Pedagogical content knowledge is the category most likely to distinguish the understanding of the content
specialist from the pedagogue” (Shulman, 1987, p. 8).
16
For example, two notable early theories that began to distinguish different types of mathematical knowledge include Schwab (1978) and
Skemp (1976). Schwab made a distinction between substantive and syntactic knowledge, arguing that teachers need to understand both the
content of their subject discipline and how that content is organized (Schwab, 1978). Skemp made a distinction of a relational (knowing both
what to do and why) vs. instrumental (rules without reasons) understanding of mathematical ideas (Skemp, 1976).
42 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Mathematical Knowledge in Teaching
Ball, Thames and Phelps (2008) credit Shulman and his colleagues with another important contribution for
representing content understanding as a special kind of technical knowledge that is critically important to the
teaching profession. Shulman proposed that knowing a subject for teaching requires more than knowing its facts and
concepts. It also requires an understanding of the organizing principles and structures and the rules for establishing
what is legitimate to do and say in a field, stating that: “The teacher need not only understand that something is
so; the teacher must further understand why it is so, on what grounds its warrant can be asserted, and under what
circumstances our belief in its justification can be weakened or denied.” (Shulman, 1986, p. 9)
Among the challenges related to the concept of PCK is whether lines can be drawn to distinguish content
knowledge and pedagogical content knowledge and, if so, how. Goulding, Rowland and Barber (2002) pointed
out that the boundary between knowledge of mathematics content and PCK is not clear-cut. Like Ball (1990) and
Ma (1999), they emphasized that it is the way in which mathematics is known that is crucial (cf. Beswick, 2008).
The MKiT assessment used for the current study consisted of both subject matter and pedagogy questions. The test
itself is proprietary, so actual test items cannot be shown in this study.
Askew et al. (1997) proposed different categories of knowledge:
1. Numeracy subject knowledge: This involves an understanding of mathematics and numeracy appropriate
to what is being taught;
2. Knowledge of teaching approaches: This involves an understanding of teaching styles and different ways
of presenting mathematical and numerical concepts to pupils, including a range of diagrammatic and verbal
representations, so that they gain access to the subject knowledge;
3. Knowledge of students: This involves a knowledge of what particular students currently being taught
understand as well as the level of knowledge of students more generally. For example, it involves a knowledge
of the aspects of the numeracy curriculum that are generally found difficult, common misconceptions and
models of progression.
This study attempts to address the multi-faceted nature of MKiT by viewing it from various perspectives in
order to create a well-rounded picture of each teacher’s MKiT. All teachers in the MOEC-WB study were asked
to complete a multiple choice teacher assessment tool developed by MOEC. The assessment provided a comparable
measure of some aspects of MKiT, containing both subject matter and pedagogy sections. The limitations of
measuring pedagogical knowledge through a multiple-choice assessment were recognized at the time of design of
the study. Thus, other mechanisms were developed to compensate for this limitation. For example, the scenarios
interview was designed to elicit aspects of MKiT through discussion of hypothetical classroom cases. The situations
that arose naturally in the recorded lessons of the case study teachers were also taken into account and investigated
through subsequent stimulated recall interviews. The triangulation of the three sources of pedagogical content
knowledge was used to provide a better-rounded picture of the case study teachers.
Main Report 43
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Table 4: Sources of Data for MKiT Analysis
Data Source Main Strengths Main Shortcomings
MKiT: • Can be given to all teachers; not just case study • Testing theory rather than actual situations
Pedagogical portion of the teachers • it is questionable whether pedagogical MKiT
Teacher assessment • Easily comparable and objective from a scoring can be measured through a multiple-choice
standpoint. assessment
• Does not capture whether teachers can call
upon their MKiT in real life situations
MKiT: • All teachers work on the same scenarios, • Hypothetical situations are not able to fully
Scenarios interview allowing for comparability capture knowledge in action
• Scenarios are designed to target various • The scenarios can only partially bring
pedagogical aspects of knowledge, including pedagogical MKiT to the surface
the handling of student misconceptions.
Practices: • Captures the teachers’ handling of actual • Each teacher had unique situations, not
situations and their knowledge in action. allowing for comparability
Situations from actual
• The stimulated recall allows for a better • The teacher’s MKiT is almost impossible to
lessons and subsequent
understanding of the practices from the identify and assess since the researcher cannot
stimulated recall interviews
teacher’s perspective get into the mind of the teacher and, to a
certain extent, can only conjecture
4.1 TEACHER SUBJECT AND PEDAGOGY COMPETENCY ASSESSMENT
The results of the MKiT assessment, as shown in Figure 16 below, indicate that there was a wide range of scores
among the teachers.
Figure 16: Scatter Plot of Teacher Subject and Pedagogy Scores on the Teacher Assessment
100%
90%
80%
70%
CS9
Pedagogy Score
60% CS5
CS7
CS6
50% CS4 CS3
CS2 CS10
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mathematics Score
44 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Mathematical Knowledge in Teaching
There is a clear correlation between the scores teachers received for teacher subject knowledge and pedagogical
knowledge, with teachers who scored higher on the subject matter portion also tending to score higher on the
pedagogical section. However, there are also many cases of teachers scoring relatively highly in one area, but low
in the other. This provides a good variation when relating the scores to practices and beliefs. It is also important to
note that the subject items were related to the junior secondary school curriculum topics and were not considered
to be particularly advanced. The fact that the majority of teachers obtained scores below 60% and 16 teachers
obtained scores below 40% appears to indicate that in the case of some teachers, the level of subject knowledge
is not much more advanced than that of the students whom they are teaching. This is an important contextual
aspect of Indonesia’s situation compared to relatively more developed countries, where teachers tend to have much
higher levels of subject knowledge. It also ties into the results of MOEC’s national teacher competency testing (Uji
Kompetensi Guru, or UKG), which showed similarly low competency levels. This will be discussed in detail in
section 8.1.
Interestingly, for the case study teachers, all but two are placed well above the trend line shown in Figure 16,
indicating that their pedagogy scores are high relative to their subject scores. Case Study 9 (CS9) obtained the
second highest score in the pedagogy portion and was among the top 10 in the subject portion. CS3 had the highest
subject score of all 104 teachers. Although his pedagogy score is slightly above average, it is very low relative to his
subject score, placing him well below the trend line and indicating a subject-heavy slant. CS5 and CS4 obtained
relatively high scores in both sections.
CS6 and CS7 obtained very strong pedagogy scores, but scores that were near the average in the subject portion.
CS2 and CS10, on the other hand, obtained scores that were well below average in the subject portion, while their
pedagogy scores were the lowest among the case study teachers (although they were only slightly below average
compared to the full sample). These case study results will be of importance in section 4.3 when comparing the
competency scores to how the teachers were able to address hypothetical classroom scenarios. It will also be of
importance when examining their videotaped classroom practices when exploring the knowledge-practices link in
section 6.3.
4.2 TEACHER MATHEMATICAL SCENARIOS INTERVIEW
This section discusses the approach and findings of the mathematical scenarios interviews conducted with
the case study teachers. It examines both individual case responses at the Micro case level and providing a general
summary at the Mezzo case level. The scenarios interview consisted of five hypothetical mathematics lesson situations
where teachers were asked to identify issues, make conjectures on students’ thoughts (e.g. misconceptions held by
students) and strategies for how they would address the situations in their lessons.
Main Report 45
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The following is one example of the types of scenarios used in the interview:
Mrs. Dian’s class was working on an assignment where they had to find the measures of unknown angles in triangles. One
student consistently found the measures of the unknown angles in right triangles by subtracting the known angle from 90. For
example:
90 - 62 = 28
X = 28
62°
X
Mrs. Dian was concerned that this student might run into difficulty when trying to find the measures of unknown angles in more
general triangles.
Intended Explorations
• Recognizing that the student has the correct answer, but that by the student’s solution method, there may be a misconception.
• Probing student to see if there is, in fact, a misconception or if the student understands and just came up with what might be
considered a shortcut.
• Developing a method for ensuring the concept is understood.
Guideline of questions to ask in the interview
• Why do you think Mrs. Dian is concerned?
• (Teacher should identify misconception on his/her own, but if prompting is needed or if the teacher mentions a misconception exists, but
does not specify what it is, then prompt.)
What is the possible misconception that the student has?
• Why might the student have come up with this method of solving the problem?
• If you were the teacher, how would you approach this situation?
All mathematical scenarios and more detailed analysis can be found in the Technical Report.
4.2.1 SUMMARY OF CASE STUDIES ON THE MATHEMATICAL SCENARIOS
The table in Appendix 12.4 provides an overview of all case study teachers, based on distinguishing categories that
emerged amongst teacher responses. The following are summaries of the sections analyzed. Of importance in this
scenario analysis was to understand each case study teacher’s mathematical knowledge in hypothetical situations so
that this could then be compared to their actual assessment tests.
46 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Mathematical Knowledge in Teaching
Mathematics
In general, the level of complexity of the mathematical concepts contained within the scenarios did not create
major difficulties for the teachers. Often teachers would appear to understand the mathematical concepts raised in
the scenario, but then chose an incorrect answer. In the case of Scenario 2 and Scenario 3, there is no disputing which
of the responses was correct. However, Scenario 4 in particular was vague and open to interpretation. Sometimes a
teacher could give a good justification for why an “incorrect” option would work. The vague nature of this scenario
was useful in that it led to interesting discussions. However, it also left the level of MKiT demonstrated through
the scenario open to interpretation. Scenario 5 generated many errors from the teachers, but this might have been
due to the question itself being slightly unclear. Still, the teacher with the strongest subject matter knowledge (as
determined through the teacher assessment) did a particularly admirable job of discussing inequality and identifying
the correct answer, indicating that the scenario still serves an important function in the evaluation of teachers’
MKiT. The teachers who stood out as being particularly strong in their discussions of mathematical concepts were
CS3, CS5 and CS9.
Identify Issues
Most teachers tended to spot the issues quickly, although the speed of recognizing the issues and the ability to
describe it varied. One particularly distinguishing aspect related to the identification of issues was the speed with
which teachers noted the issues in questions and the extent to which they elaborated on the issue itself. In particular,
CS4 and CS7 would elaborate upon and discuss the issue, together with its root causes. It cannot be determined
whether they truly grasped the issues better the other subjects or whether talking through the issue might have been
their method for coming to terms with the situation in their own minds.
Conjectures of Students’ Thoughts
Another interesting distinction was that some teachers placed a greater emphasis on the students’ thinking
processes, while others placed greater emphasis on the mathematical issues. CS3, who placed greater emphasis
on the mathematical issues, was particularly interesting in this regard. The fact that he obtained the highest score
on the subject portion of the MKiT assessment but a relatively lower score in the pedagogy portion probably
helps to explain his focus on the mathematical aspects. By contrast, CS7 was much more focused on the student’s
thinking process, continually placing herself in the mind of the student, although at times she struggled with the
mathematical concepts. Her relatively high pedagogy score and her average subject score may help explain this.
Develop a Strategy
The development of strategies for the scenarios was revealing in terms of the level of knowledge of teachers,
but it actually provided even greater insights into aspects of their beliefs. CS4 and CS5 came up with strategies
that would enable the students to come up with the solution themselves, while CS2 would use phrases such as: “I
would explain…” and “I would repeat…” In their formulation of a strategy, CS3 and CS6 emphasized mathematical
aspects, while CS7 and CS9 came up with creative alternatives.
Main Report 47
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
4.3 LINKING MKIT ASSESSMENT AND SCENARIOS RESULTS
As might be expected, the case study teachers with higher MKiT assessment scores were generally able to
handle the scenarios with greater ease. They tended to identify the issues quickly and provided more complete and
more elaborate responses when discussing the mathematics involved and the strategies they would use. The variance
between subject and pedagogy scores was also reflected in the manner in which teachers managed the scenarios, with
teachers gravitating towards and utilizing their respective strengths. This is an important finding, in that it indicates
how teachers will use their knowledge in formulating and implementing their teaching practices. If one aspect is
greater than the other, then the teacher is likely to utilize their relative strengths.
The MKiT assessment has the benefit of producing a standardized score that can be compared across a large
number of teachers. However, certain aspects of knowledge were demonstrated in the scenarios that could not
have been detected through the assessment. For example, two teachers with low pedagogy scores came up with
creative pedagogical techniques for how they would approach the particularly pedagogical content knowledge. On
the other hand, one teacher with a high assessment score came up with very basic methods for dealing with student
misconceptions, possibly suggesting that knowledge that can be demonstrated through the medium of a written test
is not necessarily easily elicited and drawn upon in classroom situations.
The three micro case studies provided insights into how the MKiT (as captured through the teacher assessment)
and MKiT (as captured through the scenario interviews) appear to be correlated, but also how they diverge
in certain aspects. CS3 provided the best example of how the assessment and scenario results are correlated. CS3
had the highest score of all teachers on the subject matter portion of the assessment and had a slightly higher
than average score in the pedagogy portion relative to the entire sample, but only average compared to the other
case study teachers. In the scenarios, CS3 demonstrated a strong knowledge of the mathematical concepts. This
was particularly true in the case of Scenario 5, which requires a relatively high level of advanced mathematical
thinking and which created problems for many other teachers, but which CS3 managed to address with ease.
However, in terms of the pedagogical aspects of the scenarios, CS3 tended to struggle more than other teachers.
This was particularly true in regards to “knowledge of students”. In Scenario 2, CS3 was prompted multiple times
to conjecture as to why the student had made an incorrect statement. He persisted on saying that he did not know
because he was not the student. This came through in other scenarios as well. Most of the other teachers would
often discuss the issue from the perspective of the student, but CS3 tended not address the scenarios in this manner.
4.4 RELATIONSHIP OF TEACHER KNOWLEDGE WITH STUDENT LEARNING
OUTCOMES
Of all school system factors explored, teacher knowledge had the strongest relationship to student learning
outcomes. The results indicate that both subject matter knowledge and pedagogical knowledge, in almost all cases,
were well above the generally accepted 5 percent statistical significance threshold and typically above the 1 percent
significance threshold (see Figure 17). The significance levels tended to drop when including teacher background
variables such as educational level and experience, likely indicating that these other teacher characteristics also play
an important role in student learning.
48 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Mathematical Knowledge in Teaching
Teacher subject knowledge tended to have a stronger relationship with student learning than did pedagogical
knowledge. This may in part be due to the measurements themselves, with subject matter knowledge lending
itself well to a test whereas pedagogical knowledge is much more difficult to measure through a paper test. Still,
the fact that pedagogical knowledge also tended to have strong results indicates its likely important role in learning
outcomes.
Figure 17: Relationship between Teacher Knowledge and Student Learning17
Overall Subject Pedagogical
assessment score assessment score assessment score
Direction and level of statistical signi cance
4
Thresholds of statistical signi cance
4 3.74 3.75
3 3.22
3.05
3 1%
2 2.19 5%
2.02
2 10%
1
1
0
Regression without teacher variables Regression wth teacher variables
Coe cients 0.20 0.17 0.16 0.14 0.13 0.13
Std. Coe cients 0.24 0.23 0.24 0.23 0.15 0.13
Note: Full regression results can be found in the technical report.
In interpreting the regression coefficient for the degree and its relation to student posttest scores, a percentage point
increase on a teacher’s assessment score is related to a 0.17 percentage point increase in a student’s posttest score
after controlling for multiple background factors and the student’s pretest score. Teacher assessment scores ranged
from 7 percent to 75 percent, with an average score of 48 percent. For example, a 10-percentage-point higher
teacher assessment score resulted in a student’s posttest score being 1.7 percentage points higher. Again, because
the average score on the posttest was only 34 percent, the 1.7 percentage point difference represents a difference of
5 percent relative to the average score. (Similar but lower coefficients were found when looking at the subject and
pedagogy score in relation to students’ posttest score, with a 0.14 and 0.10 percentage point increase, respectively.)
The standardized coefficients indicate that a 1.0 standard deviation increase in teacher assessment scores is related to
a 0.23 standard deviation increase in student test scores.
17
A great deal of caution should be used in assessing a teacher through a pedagogy examination because a teacher’s pedagogical effectiveness
is not about knowing facts about teaching methods that can be reduced to a question in a multiple-choice examination. The video study is
specifically intended to explore how teachers use their pedagogical knowledge “in the moment.” Still, the fact that such a strong relationship
exists between the pedagogy score and student learning indicates that the test is, in fact, capturing some aspect of the teachers’ knowledge and
appears to provide a proxy for what they do in the classroom.
Main Report 49
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
4.5 KEY POINTS ON TEACHER KNOWLEDGE
There are clear distinctions in how teachers approached the mathematical scenarios, with these distinctions
based on their level of subject and pedagogical knowledge. Furthermore, it appeared that teachers would shape
the scenarios according to their relative strengths, seeing the issues in either more mathematical or more pedagogical
terms according to their strengths and developing strategies accordingly. This point is particularly relevant when
exploring relationships and is carried over into the analysis in Section 6.3.
While generally there was a clear relationship between the MKiT Assessment scores and the knowledge
demonstrated in the mathematical scenarios, there were cases where teachers with high scores appeared to
struggle to put that knowledge into play when discussing the scenarios. On the other hand, there were teachers
who demonstrated a high level of knowledge, particularly pedagogical knowledge, when discussing the scenarios,
despite the fact that they obtained relatively lower pedagogy scores.
In relating teacher knowledge to student learning outcomes, there is a clear positive relationship, but this
is only an initial step in understanding why knowledge might be an important factor in student learning.
Knowledge is explored in much greater detail through relations with teaching practices, both with quantitative
analysis and more in-depth qualitative analysis. These results are presented in section 6.3 and later presented as a
thematic exploration in section 8.1.
50 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 5
TEACHER BELIEFS ON
TEACHING AND LEARNING
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The systematic study of teachers’ conceptions of mathematics has been of continuing interest to researchers,
with these conceptions being considered by many researchers to play a significant role in shaping teachers’
characteristic patterns of instructional behavior. Wilson and Cooney (2002, page 128) stated what appears to be
a basic premise of mainstream research into teacher beliefs: “Over the last 15 years, there has been a considerable
amount of research on teachers’ beliefs based on the assumption that what teachers believe is a significant determiner
of what gets taught, how it gets taught, and what gets learned in the classroom.” Teachers do not just rely on their
pedagogical knowledge or on curriculum guidelines. Rather, their beliefs act as a filter through which they make
decisions related to teaching practices (Clark and Peterson, 1986). While teaching practices shape what happens in
the classroom, it is teacher beliefs that can explain why these practices are utilized.
From the point of view of teachers’ professional development, it is important to understand the nature of
teacher beliefs. Cooney et al. (1998, page 331) state that, “An analysis of belief structures, attention to the intensity
with which beliefs are held, and the nature of the evidence that supports beliefs can provide a forum by which our
teacher education programs will be better able to address issues of reform.” Thompson (1984, page 106) focused on
the negative consequence of not taking teacher mathematical beliefs into account when he stated that: “A failure to
recognize the role that the teachers’ conceptions might play in shaping their behavior is likely to result in misguided
efforts to improve the quality of instruction in the schools.” Kagan (1992) argues that teacher beliefs may be the
most significant measure determining teachers’ professional growth.
Table 5: Beliefs Lenses and Description of Categories Used in the Analysis
Lens Categories Main Stance/Emphasis*
Transmissionist Typically more teacher-centered and traditional instructional approach with
verbal explanations
Orientation Connectionist More student-centered, involving students in conceptually-focused discussion;
(Askew et al., 1997) learning about and making connections amongst concepts
Discovery Teacher acts as a guide in student-led practical activities where students
discover methods themselves.
Instructor Skill mastery
Role of Teacher
Explainer Conceptual understanding
(Ernest, 1989)
Facilitator Confident problem solving
Learner Focus Stresses the learner’s construction of mathematical knowledge through social
interaction
Teacher’s Focus in
Approach Content - Conceptual Focus Driven by the content itself, but with an emphasis on understanding concepts
(Kuhs and Ball, Content - Performance Focus Values performance as the key goal whose attainment depends on the
1986) mastery of rules and procedures
Classroom Focus Mathematics teaching based on research knowledge of effective classrooms
School - knowledge Believing that teaching is an act of passing information on to others while
Purpose of learning involves the process of reproducing that information
Schooling
(Renne, 1992) Child-Development More likely to consider children’s needs and characteristics as the primary
factors in instructional decision making
Teaching Approach Progressive Gives recognition and value to new instructional strategies in which students
(Handal, 2003; (socio-constructive) are able to learn mathematics by personally and socially constructing
Dewey, 1938) mathematical knowledge
Traditional Emphasize transmission of knowledge and stress the pedagogical value of
(behaviouralist) formulas, procedures and drill, and products rather than processes
52 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teacher Beliefs on Teaching and Learning
What are the mathematical beliefs and orientations of Indonesia’s eighth grade mathematics teachers? The
analysis of these beliefs focuses first on an analysis of the mathematical belief interviews and then of the mathematical
beliefs survey. The two analyses are then compared and contrasted with the aim of determining the nature of the
mathematical beliefs held by the case study teachers and how these beliefs relate to those of the larger set of 104
teachers surveyed.
5.1 TEACHER MATHEMATICAL BELIEFS SURVEY
The mathematical beliefs survey consisted of items that the participants ranked from 1 to 5 on the Likert
scale. The survey also includes sections in which teachers ranked statements in relation to their beliefs, allocating
proportions to three separate statements based on the extent to which they agreed with each statement relative to
the other two. Thus, a teacher might give 70% to the statement with which they most agreed, followed by 20% for
a statement with which they agreed to a lesser extent, and 10% for the statement with which they agree the least.
The distribution of responses for each question is summarized in graphs that can be found in Appendix 12.2. The
full Technical Report provides a step-by-step analysis of the survey results. The results presented in Figure 18 below
include items to reach a large majority of teachers provided either positive or negative responses.
Figure 18: Likert Scale Statements with Large Majority of Responses being Either Positive or Negative
Strongly Disagree Disagree No opinion Agree Strongly Agree
Study of mathematics is suited mostly to males
56% 37% 3%2%2
Telling the children the answer is an e cient way of facilitating their
mathematics learning 18% 56% 0% 23% 2%
For many citizens it is not necessary to know mathematics beyond
simple calculations (primary school level) 15% 60% 3% 15% 8%
Students learn mathematics best in classes with students of similar abilities
6% 61% 2% 27% 4%
Mathematics is learnt well by working individually on mathematical problems
5% 48% 4% 37% 7%
Students can solve most mathematical problems on the basis of
memorized formula 5% 44% 2% 47% 3%
Listening carefully to the teacher explain a mathematics lesson is the most
e ective way to learn mathematics 4% 37% 5% 43% 12%
The applications of mathematical result are more important than its proof
3% 27% 6% 50% 14%
The most e ective way to teach mathematics is to rst explain the topic,
give example(s) and then ask students to solve mathematical tasks 18% 1% 52% 28%
It is the teacher’s responsibility to provide children with clear and concise
solution methods for mathematical problems 1 11% 0% 65% 23%
Persistent questioning has signi cant e ect on children’s
mathematical learning 9% 6% 62% 23%
Mathematical material is best presented in an expository style: demonstrating,
explaining and describing concepts and skills 8% 2% 55% 35%
I function in my classroom as a learner and partner in learning
with my students 6% 1% 72% 21%
Students learn most e ectively when learning tasks are broken down
into small sequential steps 1 4% 4% 76% 15%
It is important that mathematics content be presented to children
in the correct sequence 4% 2% 51% 43%
Teachers can create, for all children, a non-threatening environment
for learning mathematics 3%2% 50% 45%
Children always bene t by discussing their solutions to
mathematical problems with each other 3%1% 57% 39%
Learning mathematics in school is essentially an active process
2%2% 59% 37%
Main Report 53
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Teachers tended to express strong agreement with statements of a social nature that might indicate disadvantage
or distinction for some group. The highest level of disagreement was expressed for the statement: “The study of
mathematics is suited mostly to males.” Teachers also tended to express disagreement with statements that students
learned best in classes with students of similar abilities and that mathematics at the primary level was sufficient for
some citizens. For questions related to teaching approaches and how student learn, teachers tended not to express
strong disagreement. However, teachers did tend to express moderate disagreement with some statements regarding
teaching practices, with most teachers expressing at least moderate disagreement to the proposition that telling
students the answer was efficient for student learning. In fact, many teachers expressed strong disagreement with
this statement.
Teachers also tended to express agreement with statements related to certain social aspects of teaching,
with nearly half of all teachers expressing strong agreement with the statement that teachers can create a
non-threatening learning environment for students. Ninety-six percent of teachers expressed at least moderate
agreement with the statement that learning mathematics in school is essentially an active process. This is probably
due to an Indonesian policy espousing “active and joyful learning,” making the term ‘active’ resonate with teachers.
In the area of statements related to teaching technique, 90% of teachers expressed at least moderate agreement
with the statement that mathematics material is best presented in an expository manner, involving demonstrating,
explaining and describing concepts and skills. A significant proportion of teachers also expressed at least moderate
agreement with statements related to the correct sequencing of teaching and breaking down learning tasks into small
sequential steps.
In the case of some statements, the proportion of teachers who expressed either agreement or disagreement
was fairly evenly split. For example, the proportion of teachers who expressed agreement or disagreement with
the statement that listening carefully to the teacher explain a mathematics lesson is the most effective method for
learning mathematics was almost equal. Such statements were most useful in analyzing the relationship between
beliefs and practices, since there was a wider range of responses.
Teaching Practice Frequency Items
In comparison to the teachers’ responses through the Likert scale, the teachers’ responses to questions
regarding the frequency with which they utilized certain teaching practices in their own lessons were more
evenly spread. The least frequent response was almost never and the most frequent response was occasionally, followed
by almost always.
54 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teacher Beliefs on Teaching and Learning
Figure 19: Distribution of Frequency Scale Responses in the Mathematical Beliefs Survey
Most of the time
19% Almost always
25%
Half the time
14% Almost never
10%
Occasionally
32%
Of course, the responses must be viewed in light of the fact that teachers may have been responding in terms
of what they felt they were supposed to do, rather than what they actually did. Thus, they may have considered
that a response of almost never should be avoided. Still, the responses can be taken at face value or “stated frequency”
and later compared to the videos to check the actual frequency with which a practice was utilized by a teacher and
thereby to determine the extent to which the two coincide.18
Figure 20: Frequency Scale Results on a Sample of Teaching Practices
Almost never Occasionally Half the time Most of the time Almost always
I only go through one method for doing each question 24% 51% 11% 8% 7%
I jump between topics as the need arises 24% 48% 9% 12% 8%
I teach each student di erently according to individual needs 23% 37% 8% 13% 20%
I encourage students to make & discuss mistakes 16% 30% 19% 13% 23%
I nd out which parts students already understand and don’t teach those parts 17% 25% 11% 25% 23%
I tend to teach each topic separately 13% 30% 16% 19% 23%
I draw links between topics and move back and forth between topics 8% 42% 13% 19% 19%
Students learn through discussing their ideas 7% 52% 18% 16% 8%
Students compare di erent methods for doing questions 5% 38% 14% 16% 28%
I tend to follow the textbook or worksheets closely 3% 46% 18% 18% 16%
Students work on their own, consulting a neighbour from time to time 1% 36% 17% 31% 16%
I tell students which questions to tackle 4% 26% 19% 24% 27%
I avoid students making mistakes by explaining things carefully rst 6% 18% 11% 32% 33%
Students use only the methods I teach them 8% 23% 14% 17% 38%
I teach the whole class at once 9% 11% 5% 23% 53%
I know exactly what maths the lesson will contain 5% 14% 5% 24% 53%
18
An absence of practices in the 3-5 videotaped lessons would, of course, not mean the teacher never uses the practices, but at least general
triangulation can be done.
Main Report 55
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A surprisingly high 75% of respondents stated that they almost never or only occasionally utilize only one
method for addressing a particular question, implying that they often used multiple methods. Similarly, 72%
of respondents stated that they almost never or only occasionally jumped between topics as the need arises. On
the other hand, a large proportion of teachers stated that they almost always or most of the time used practices that
involved: (i) always knowing what mathematics the lesson will contain; (ii) teaching the whole class at once; and
(iii) the students only using methods given by the teacher. Certain practices received an even spread of responses,
from almost never to almost always. Statements in this category included those relating to encouraging students to
make and discuss mistakes; tending to teach each topic separately; and finding out what students already know and
not teaching those parts.
Potentially conflicting results also emerged. For example, only 20% of respondents stated that they almost always
or most of the time “jumped between topics as the need arises.” By contrast, 38% of respondents stated that they
almost always or most of the time “drew links between topics and moved back and forth between them,” a statement
somewhat similar to the first statement. Possibly the teachers perceived a difference between the two statements, but
the variance in the responses may also be a warning regarding their accuracy.
As with the Likert scale statement, only a portion of the frequency statements (16 of the 27) are shown above.
However, the full results can be found in Appendix 12.2.
5.2 CASE STUDY INTERVIEWS ON TEACHER MATHEMATICAL BELIEFS
The full analysis of the case study interviews on beliefs can be found in the Technical Report, including individual
responses to the Beliefs Orientation game, as well as the results of general interviews into how the teachers beliefs
were formed; how the perceived students learn best; and what teaching practices they believed are most effective in
the teaching-learning process. The following subsection summarizes the key findings from the interviews:
No Teacher Fell Completely into an Orientation Category
From the outset of the exploration into teachers’ mathematical beliefs, it was not expected that all the data
for a particular teacher would point to a single categorization. The results confirmed this. Teachers may express
beliefs that would tend to align with certain categorizations. However, even the teachers who could most easily be
placed in a particular category would also tend to express beliefs that would not fully align with, or that would even
contradict, the categorization. Building on these results, the next stages of analysis must keep this in mind and avoid
oversimplifying or typecasting the case study teachers. Instead, their categorizations should be viewed as tendencies
or as classifications of their dominant expressions. Put simply, teachers, like any other people, are capable of holding
different beliefs that may appear to contradict each other.
Beliefs are Contextually Fluid
It became very clear through the interviews that there is a distinction between beliefs in an abstract, general
sense and beliefs in a practical, situationally specific sense; there is a dance that takes place between the two.
At times the abstract and specific embrace as though they are a single entity, at other times they are apart and only
lightly touching one another. This dance took place as teachers spoke of their beliefs, with some teachers appearing
to be overtly aware of the distinction, while others blended belief and action as if there were no contradiction.
56 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teacher Beliefs on Teaching and Learning
For example, it appears that teachers may believe a certain teaching technique is ideal in a general sense, but due
to policy, student achievement level, community and family environment or a host of other possible influencing
factors, the teacher’s belief in how to actually teach in his or her current circumstances may not fully align with those
general beliefs. One teacher was highly aware of this, frequently making statements such as: “I would prefer to teach
using a more advanced method, but due to the level of my students, I must use a different technique… I must adapt
my teaching to the level of the students.” This teacher even stated that within a single lesson, his techniques would
vary based on the circumstances, with these variations affected by the purpose of the lesson segment and the nature
and quality of the students with which he was working.
How, then, should this be taken into account when examining the teacher’s videotaped lessons or analyzing comments
made in interviews relating to teaching practices? As mentioned earlier, a distinction can be drawn between espoused
and enacted beliefs (e.g. Ernest, 1989; Thomson, 1992; Butt and Raymond, 1989; Anderson, 1997), but this can
be misleading. The distinction between espoused and enacted beliefs seems to imply that if there are differences
between the two, then the teacher is either not aware of the contradiction or is prevented from following through
on his or her espoused beliefs due to existing constraints.
How should it be interpreted when a teacher is aware of the contradiction and states that in different circumstances
he or she would teach differently? Possible reasons for this occurring include:
1. The teacher holds a belief or orientation (as espoused in the interview), but consciously decides it is not
applicable to the specific situation and uses a different approach (enacted in the videotaped lessons). This
could be interpreted to mean that in fact, the teacher holds multiple beliefs. While the teacher tends to favor
a certain approach, he or she is also willing to use alternative approaches based on the context.
2. The teacher holds a belief or orientation (espoused), but is unaware that his/her actions (enacted) are do not
align with that belief.19
3. Possibly due to pressures (policy, peers, what was learned in training, etc.), the teacher expresses what he or
she thinks is the “right” answer or response, but he or she does not truly believe it.
4. The teacher is not actually sure or aware of what belief he or she holds, but provides a response anyway.
5. The teacher does not hold a strong belief or tendency towards a certain orientation, so when it comes to
actual classroom situations, the espoused beliefs are easily discarded.
There Appeared to be Limited Direct Reflection on Beliefs
The beliefs interview contained some sections that addressed beliefs in an abstract, general sense, while other
sections addressed beliefs in a more direct, concrete and practical sense. While all teachers were able to discuss
the more concrete aspects of teaching and learning, the sections that tended to address beliefs in a more abstract
sense presented challenges. It appeared that some teachers did not tend to think about the nature of mathematics,
how students learn, and how to teach in an abstract sense. The case study teachers were asked what they thought
about the beliefs survey in which they had participated and were also asked to what extent they had thought about
and reflected upon their beliefs. All but two teachers indicated that they had never directly thought about their
beliefs in a direct or philosophical sense and found the process a bit strange. On the other hand, four teachers
indicated that while they may or may not have directly considered these issues previously, doing so was an enjoyable
learning experience.
19
Alternatively, it is possible that it is only a perceived contradiction by me, but that in the teacher’s mind there is no contradiction. It is
therefore important that I not unreasonably infer differences and contradictions.
Main Report 57
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Adherence of Beliefs versus Pragmatists
Some teachers spoke passionately and eloquently about their views, while others struggled to form and
express their ideas or did not feel certain about taking a stance. Some teachers had clearly thought about issues
and consciously chosen not to take a stance or expressed their response in a nuanced manner, often prefacing their
statement with reference to contextual factors. The degree to which teachers may hold their beliefs is extremely
important, as the degree to which they hold their beliefs is likely to play a role in how they incorporate these beliefs
into their practices. This relates back to the literature that discusses how some teachers doggedly cling to their beliefs
(e.g. Kagan, 1992; Nespor, 1987; Swan, 2006; Calderhead, 1996). However, some research shows that teachers can
and do change their beliefs, with teachers holding more relativistic orientations to teaching mathematics being more
likely to consider and adopt new ideas (Arvold and Albright, 1995). This concept of adherents versus pragmatists
is an important concept for analysis. In later sections of this report, the concept of adaptability and flexibility are
explored in greater detail.
5.3 LINKAGES BETWEEN BELIEFS SURVEY AND BELIEFS INTERVIEW
Through the survey, teachers provided responses to specific items, with it possible to group the items in
relation to specific categories. The interview provided insights into teachers’ thoughts regarding mathematical
beliefs, affording a much more elaborate and detailed view of the teachers’ mathematical beliefs, uncovering
important aspects of these beliefs that could not be captured through a survey. The survey responses were compared
to the interview responses to determine whether the responses appeared to be consistent and whether the responses
from one could enrich the understanding of the responses to the other.
The data from the surveys and interviews were linked in two ways. First, individual questions within the survey
were related to what teachers say in the interviews. For example, the survey statement: “Allowing a child to struggle
with a mathematical problem, even a little tension, can be necessary for learning to occur” was specifically discussed
as one of the topics in the mathematical beliefs interview. For this item in the survey, there was significant variation
in the case study teachers’ responses, with four disagreeing, three agreeing and one strongly agreeing. Relative to
the responses in the interview, six of the teachers’ responses were consistent, one was mixed and one was clearly
inconsistent. In the case of the inconsistent response (CS2), it is likely that the teacher either did not understand the
survey statement or interpreted it differently when participating in the survey. It is also possible that when discussing
the topic in the form of an interview, she had more time to consider the issue and so gave a more nuanced response
that contradicted her response through the survey.
The comparison between the multiple survey items and interview results showed that there was a high level of
consistency between the two sources. This was particularly true when grouping items. For example, survey items
related to how the teacher pictured his or her role were generally very consistent with the interview results. However,
at the individual item level, for the statement: “It is more effective to provide students with the information they
need to know, rather than encouraging them to experiment,” the teachers tended to make statements in the interview
that contradicted their responses through the survey as they attempted to verbalize the degree to which they would
let students work on their own and how much guidance they would need to provide.
58 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teacher Beliefs on Teaching and Learning
In general, the interview results highlighted the complexity of beliefs. In discussing a topic, a respondent might
make multiple statements that seem to be contradictory if they are viewed in isolation. However, when viewed
in the context of an overall narrative, it was possible for the apparent contradictions to be resolved. Seen in this
light, the topic of beliefs is perhaps much better investigated through an interview, in which teachers are able to
elaborate on their ideas, than through a survey, which tends to lead to oversimplification. With survey statements,
teachers sometimes clearly misunderstood their meaning or interpreted them differently when the same subject
was discussed in the interview. Due to the complex nature of teachers’ beliefs, the profile given created through the
interview could be considered considerably more trustworthy than the profile created through the survey, although,
as stated earlier, there was a considerable degree of alignment between the two.
5.4 BELIEFS RECONCEPTUALISED
Various studies have conceptualized teacher beliefs by applying the use of categorizations. Many of these
categorizations have been used as lenses for analysis. A key finding of this beliefs analysis was that while some teachers
exhibited tendencies that align them with particular categorizations, others defied categorization. Sometimes, the
defiance was implicit, with the survey data and beliefs interview not enabling a clear categorization. In the beliefs
interview in particular, some teachers often used the expression “it depends” when discussing the application of
teaching approaches to a particular situation, indicating that for them teaching practices are dynamic and context-
specific. For case study teachers CS3 and CS9, this defiance was explicit, with both specifically stating that they did
not regard their teaching methods as being susceptible to categorization. To further emphasize the point, CS9 gave
examples of how she would switch from one type of approach to another within a single lesson.
This point features prominently in subsequent steps of the analysis that relate beliefs to knowledge and
practices. It also has a significant impact on the development of the new conceptual model on the interaction
between beliefs-knowledge-practice presented in Chapter 7. As will be discussed later, a key finding is that teachers
with a higher level of knowledge also tended to have a greater degree of flexibility in their beliefs. It appears that this
flexibility allows for more adaptation in practices and, in turn, more effective teaching.
Main Report 59
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
60 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 6
EXPLORING THE BELIEFS-
KNOWLEDGE-PRACTICES
RELATIONSHIP
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Previous chapters have defined and analyzed the components of beliefs, knowledge and practices in isolation.
Building on these previous chapters, this chapter explores the relationships between these components. With
reference to the Analytical Steps Framework (see section 2.10.2), this chapter explores dual relationships between
Knowledge-Practices (Point D); Beliefs-Practices (Point E); and Beliefs-Knowledge (Point F). The intersections
between these components have been marked with a circle in the illustration at the beginning of each subsection.
This section addresses a fundamental research question on the beliefs-knowledge-practices relationship:
What is the relationship between teaching practices, teacher mathematical beliefs and mathematical knowledge in
teaching?
Through the analysis and results that are outlined in this section, it becomes clear that a model is necessary to explain
conceptually what has been learned through the findings regarding the relationship between the components. This
model will be described in greater detail in Section 7.
6.1 THE BELIEFS-KNOWLEDGE LINK
Most of the existing research into Beliefs and MKiT tends to
PRACTICES examine either the beliefs-practices link or knowledge-practices
Video
Teaching practices used in the lessons
link. While some recent studies (such as COACTIV, MT21,
A
Askew et al, 1997) have examined beliefs, knowledge and
Stimulated recall
D How teacher describes events within the
lessons and explains decision/actions
E
practices together, the focus has tended to be primarily on the
relationship between beliefs and practices on the one hand and
G
MKiT
Scenarios
MATHEMATICAL BELIEFS
Beliefs Interview
then on the relationship between knowledge and practices on
Demonstrated knowledge of students,
knowledge of practices and subject knowledge
Espoused beliefs through general
discussion and beliefs situations the other. Their studies tend not to touch upon the relationship
B
F
C
between beliefs-knowledge or, if this relationship is mentioned
Teacher Assessment
Subject and pedagogical knowledge test scores
Beliefs Survey
Questions covering nature of mathematics,
how students learn, how to teach and
at all, it is not analyzed in a rigorous manner. In this study, the
beliefs-knowledge relationship is viewed as an important link
teacher’s own practices
in considering why teachers teach the way they do.
6.1.1 MACRO VIEW: LINKING THE BELIEFS SURVEY AND TEACHER ASSESSMENT RESULTS
Analytical Approach
Regression Analysis
The regressions were conducted following the same general multi-level model regression methodology specified in
Section 2.12. In the case of comparing beliefs and knowledge, each of the responses to the statements in the Beliefs
Survey was individually regressed against the teacher’s (i) total assessment score; (ii) subject matter score; and (iii)
pedagogy score. In the case of this analysis, the goal was only to identify relationship patterns, so the variable of
interest was the t-statistic. Through this variable, it could be determined which beliefs statements were positively or
negatively correlated to a statistically significant level with the above three knowledge measures. As with the model
regression methodology utilized throughout this study, matrices are used to describe statistically significant positive
relationships (which are highlighted in green) and negative relationships (which are highlighted in red).
62 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
Figure 21: Proportions of the Relationship between Beliefs Questions and Knowledge
Breakdown of Response Relationships Number Percent Percent within Relationship
No Relationship 37 42
Both Subject and Pedagogy 24 27 46
Subject only 19 21 37
Pedagogy only 8 9 15
Opposite directions for subject and pedagogy 1 1 2
TOTAL 89
HOW TO INTERPRET THE BELIEFS KNOWLEDGE FIGURES IN THIS SECTION
In order to present the relationships found between beliefs and knowledge in a clear and simple manner, matrices were created
that show the beliefs statement along with columns representing MKiT dimensions as well as dimensions on student achievement
level. Below is an example matrix, followed by explanations of the columns:
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
For many citizens it is not necessary to know mathematics beyond simple
calculations (primary school level)
Students can solve most mathematical problems on the basis of
memorized formula
It is important that mathematics content be presented to children in the
correct sequence
• The “Distribution” column shows the Likert scale distribution of responses from all 104 teachers to the beliefs statement.
This is included as a reference for interpretation of the MKiT scores. Dark orange=strongly disagree; light orange=agree;
brown=neutral; light blue=agree; and dark blue=strongly agree
The remaining columns are relationship indicators that are color coded based on the t-statistic result in the regression analysis.
Dark red indicates a negative relationship and statistical significance at the 1% level; medium red at the 5% level; and light red at
the 10% level. Dark green indicates a positive relationship and statistical significance at the 1% level; medium green at the 5%; and
light green at the 10%. The threshold of 5% is used as the threshold for defining statistical significance, but the 10% is still shown
to indicate that in the case of that item, this threshold was approached.
• ”Entry Level” column: This column indicates the relation between the beliefs statement with the entry achievement level
of students in the teachers’ classes. A positive relationship would indicate that teachers having higher entry achievement level
students tended to agree more with the statement.
• ”Learning” column: This column indicates the relation between the beliefs statement with the learning outcome of students
in the teachers’ classes. A positive relationship would indicate that teachers whose students learning outcomes increased to a
greater extent tended to agree to a greater degree with the statement.
• “MKiT” column: This column indicates the relation between the teachers’ overall teacher assessment scores and the beliefs
statements. A positive relationship would indicate that as teachers’ MKiT scores increase, there is a greater tendency to
agree with the statement. The “Subject” column indicates the relation with the mathematics (subject) section of the teacher
assessment, while the “Pedagogy” column indicates the relation to the pedagogy section of the teacher assessment
• ”ME Degree” column: This column indicates the relationship between the teachers’ relative level of agreement with the
statement with whether they hold a mathematics education degree. A positive relationship would indicate that the teachers
holding a mathematics education degree tend to agree with the statement to a relatively greater degree than those with a
pure mathematics degree or a non-mathematics degree.
Main Report 63
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Item Results
The results indicate that in the case of 42% of the Likert scale beliefs statements, no statistically significant
relationship emerge between the statements and MKiT scores. When a relationship could be determined, nearly
half the time (46%), it emerged for both subject and pedagogy knowledge. In 37% of cases, it only emerged for
the subject portion, while in 16% of cases, it only emerged for pedagogy. One curious case involved an item had
in which both subject and pedagogy relationships emerged, but in a contradictory direction. A breakdown of these
relationships will be presented and discussed in this chapter.
Comparison of Subject and Pedagogy Results
In the case of some statements, the results for both subject and pedagogy were related to the beliefs statement
to a statistically significant degree. In the case of other statements, a relationship only emerged for either subject
or pedagogy. There was also one statement with which both subject and pedagogy relationships emerged, but in a
contradictory direction. The following analysis examines each of these cases to see if patterns might explain why
certain differences emerged.
Beliefs statements that had a statistically significant relationship with both subject and pedagogy scores are shown in
Figure 22 below. One interesting finding that emerged from an examination of these statements is that teachers with
relatively higher MKiT scores tended to express a belief in the importance of being thorough; of covering everything
in a topic; and of presenting in the correct sequence (positive relationship), while disagreeing to a relatively greater
extent with statements suggesting that it was only important to cover important ideas or to only to use one method
for each question. They also tended to disagree to a greater extent with statements suggesting that students were able
to solve most mathematical problems on the basis of memorized formula or that learning with understanding can
be achieved through practicing the mathematical exercises provided in the textbooks. From the example presented
in the graphical analysis section, teachers with higher levels of knowledge tended to disagree with the statement
that it was not necessary for many citizens to have a knowledge of mathematics beyond that required to do simple
calculations.
One surprising result is that teachers with higher levels of knowledge tended to disagree to a relatively greater
extent with the statement that drawing links between topics and moving back and forth between topics was
an effective approach. If “making connections” is taken to indicate a connectionist approach, then the higher
knowledge teachers tended to disagree with this connectionist-oriented statement.
64 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
Figure 22: Beliefs Statements Having a Relationship with Both Subject and Pedagogical Knowledge
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
For many citizens it is not necessary to know mathematics beyond simple calculations
(primary school level)
The more students practice mathematical exercises given in the textbooks, they learn
mathematics with understanding
Students can solve most mathematical problem on the basis of memorized formula
It is important that mathematics content be presented to children in the correct sequence
I try to cover everything in a topic
I draw links between topics and move back and forth between topics
I only cover important ideas in a topic
I only go though one method for doing each question
As stated previously, there were cases in which there was a statistically significant relationship between
certain beliefs statements with either subject MKiT or pedagogy MKiT, but not both. These statements provide
interesting insights into possible distinctions between types of MKiT. The statements shown in Figure 23 are those in
which there was a statistically significant relationship with subject knowledge but not with pedagogical knowledge.
The statements tended to involve aspects related to working on mathematical problems. On their statements related
to what takes place in their own classroom, teachers with relatively higher levels of subject MKiT stated that they
used the approach of having students invent their own methods; choosing which questions they tackle themselves;
and learning through discussing their ideas, to a relatively lesser extent than other teachers. This result was somewhat
surprising, in that the researchers had hypothesized that teachers with progressively higher levels of subject MKiT
would tend to be more comfortable letting their students have more freedom to experiment. A possible explanation
is that these approaches would imply higher levels of knowledge in terms of pedagogical theory, but that the teachers
with higher levels of knowledge of subject MKiT tend to be more concerned with providing structure and therefore
remained focused on the problems they had selected. For example, CS3 had the highest score in terms of subject
MKiT, but only an average score in the area of pedagogy. He wanted students to follow his methods and therefore
he tended to present problems in small, sequential steps, with clear instructions on how to solve them. However,
this would contradict the result for another statement that tended to attract a higher level of disagreement from
teachers with higher subject MKiT levels: “It is the teacher’s responsibility to provide children with clear and concise
solution methods for mathematical problems.”
Main Report 65
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Figure 23: Beliefs Statements Having a Relationship with Subject But Not Pedagogical Knowledge
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
Students learn most e ectively when learning tasks are broken down into small
sequential step
It is the teacher’s responsibility to provide children with clear and concise solution
methods for mathematical problems
Paper pencil test is the best way to examine students’ mathematical learning
Mathematics is learnt well by working individually on mathematical problems
Students choose which questions they tackle
Students learn through discussing their ideas
Students invent their own methods
Students learn through doing exercises
In the case of other beliefs statements, there was a statistically significant relationship with the level of
pedagogical knowledge, but not with the level of subject knowledge, as shown in Figure 24. Teachers with higher
pedagogy scores tended to agree with statements that mathematics is a creative field of endeavor. Thus, a greater
proportion of these teachers agreed with statements such as: “Mathematics is a creative subject like arts and music”,
while disagreeing with statements such as: “Expecting students to be creative is unreasonable and does not warrant
the use of precious lesson time.” They also disagreed to a relatively greater extent with a statement that telling
students the answer is an effective way of facilitating the learning process. Generally, statements in this category tend
to imply a more student-centered approach to the learning process, with these teachers more prepared to take the
student’s perspective into account and to develop and adjust the lesson to meet the students’ needs.
Figure 24: Examples of Beliefs Statements with a Relationship with Pedagogical But Not Subject Knowledge
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
Mathematics is a creative subject like arts/music
Expecting pupils to be creative in mathematics is unreasonable and doesn’t warrant the use
of precious lesson time
I adjust my lesson plan based on results of homework assignments
Telling the children the answer is an e ecient way of facilitating their mathematics learning
I adjust my lesson plan based on result of homework assignments
Students work on their own, consulting a neighbour from time to time
66 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
There was only one statement, the statement with which there was a statistically significant and opposite
result between the teachers with relatively higher subject knowledge scores and those with relatively higher
pedagogy scores. This statement was: “I avoid students making mistakes by explaining things carefully first” shown
in Figure 25, The result was somewhat surprising, in that it was conjectured that teachers with relatively high
pedagogy MKiT scores might see value in using mistakes as opportunities for learning, while teachers with relatively
high subject MKiT scores might be more inclined to attempt to explain things carefully in order to avoid mistakes.
One possible is that those with relatively higher subject MKiT scores tend to be more comfortable with mathematical
mistakes and misconceptions that would require their knowledge to explain. Teachers with relatively low subject
MKiT scores, on the other hand, might have wanted to avoid these situations. The term “mistake” is also one that
is open for interpretation and possibly the teachers with higher pedagogical MKiT scores tended to consider that the
term “mistakes” in this statement referred to errors arising from poor initial instruction on the part of the teacher.
Figure 24: Example of Beliefs Statement with Subject and Pedagogy MKiT Scores in Opposite Directions
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
I avoid students making mistakes by explaining things carefully rst
Responses with Knowledge and Student Learning in Opposite Directions
The results from the knowledge analysis described in Section 4 highlighted the fact that there was a strong positive
relationship between levels of both teacher subject knowledge and teacher pedagogical knowledge with student
learning outcomes. When comparing the responses to the beliefs statements with both levels of teacher knowledge
and student learning outcomes, it is generally the case that in the cases of responses to beliefs statements in which
there was a statistically significant relationships with both student learning outcomes and levels of teacher knowledge,
the relationships would both tend to go in the same direction. However, there were some cases in which the levels
of teacher knowledge and student learning outcomes would go in opposite directions. These statements and their
associated relationships are presented in below.
Main Report 67
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Figure 26: Beliefs Statements where MKiT and Student Learning have Opposite Relationships
Entry Level
ME Degree
Pedagogy
Learning
Beliefs Statements Distribution
Subject
MKiT
For many citizens it is not necessary to know mathematics beyond simple calculations
(primary school level)
The more students practice mathematical exercises given in the textbooks, they learn
mathematics with understanding
Students can solve mathematical problems on the basis of memorized formula
It is important that mathematics content be presented to children in the correct sequence
I try to cover everything in a topic
I draw links between topics and move back and forth between topics
I only cover important ideas in a topic
I only go through one method for doing each question
The second statement indicates that teachers with higher levels of MKiT, both in subject knowledge and pedagogical
knowledge, tended to agree to a lesser extent with the statement that textbooks and other published materials are
the best source for curriculum development, with the negative relationship indicated in red for Overall, Subject and
Pedagogical components of MKiT. However, students of teachers who tended to agree with the statement tendered
to have relatively better learning outcomes, with the positive relationship indicated in green under (L)earning.
How might this be interpreted?20 The case studies provide a likely explanation. CS3 and CS9 both scored very high
on the MKiT assessment and both spoke of their enjoyment in creating their own examples for classroom work. It
is likely that a teacher with higher levels of MKiT is much more likely to be able to define and tailor examples than
a teacher with lower levels, who is likely to feel much more comfortable relying on textbooks and other pre-defined
examples. CS2, on the other hand, had a very low MKiT score. In the case of two of her observed lessons, she
spent the instruction time reading step-by-step instructions for calculating an inner and outer tangent to her class
and having her students write down the steps. Further, when asked in the stimulated recall interviews to explain
why the steps were important she would repeatedly say: “They are listed in the steps,” rather than demonstrating a
mathematical understanding of why the steps were important.
But why might there be a positive relationship with learning outcomes? There tends to be a wide variation amongst
Indonesian teachers’ levels of MKiT. For those teachers with lower levels, the textbook and other published materials
may provide useful and possibly necessary support for those teachers. It is possible that teachers with higher levels of
MKiT can be effective in creating their own materials, without having to rely on a textbook. However, by contrast,
teachers with lower levels of MKiT may benefit from the use of such tools, but their students benefit from this.
20
It is important to note that the regressions take into account (control for) multiple variables, including the entry achievement level of
students. Second, it is important to note that higher MKiT teachers could conduct their class using certain approaches that would lead to
higher learning outcomes for them, but would lead to lower learning outcomes for teachers in general.
68 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
For statements such as: “Students only use methods I teach them” and “I only go through one method for doing
each question”, there is also a negative relationship with levels of MKiT, but a positive relationship with student
learning outcomes. Similar to the example regarding the use of textbooks, a possible explanation is that teachers
with higher levels of MKiT may be comfortable with allowing their students to explore methods and engaging
in discussions with students on alternative methods. However, teachers with lower levels of MKiT teachers may
prefer to have students use a single method, but this could in fact reduce confusion for the students. The pattern
of comfort and ability of the teacher could also apply to the statement: “I adjust my lesson plan based on results of
homework assignments.”
The statement: “A highly effective way to teach mathematics is to first explain the topic, give example(s) and then
ask students to solve mathematical tasks” is one with which teachers with higher levels of MKiT tended to agree
to a lesser extent, but with which there was a positive relationship with student learning outcomes. In this case, it
may be that higher MKiT teachers tend to utilize alternative methods. In terms of actual practices, the results in
Section 6.3.1 indicate that teachers with higher levels of MKiT tended to use a wider variety of practices. The more
expository approach implied in this statement was used less often by teachers with higher levels of MKiT.
The general theme in these “opposite direction” statement relationships is that teachers with higher levels of MKiT
tended to agree to a relatively greater extent with statements that could be seen to imply more advanced techniques
and approaches, but that there was not always a positive correlation with higher student learning outcomes. My
interpretation of this is that while certain approaches might be seen to be more advanced or more effective, given
that there is a very wide variation in the levels of MKiT amongst Indonesian teachers, teachers tending to agree
with beliefs statements related to more traditional or basic techniques may be more effective in using the associated
approaches or techniques.
This has important implications. It may be desirable to have teachers believe in and use practices that encourage
students to come up with their own solution methods; to have teachers who adjust their lesson on the basis of
homework assignment results; and not to have teachers who think that telling the students the answer is an efficient
way of teaching. It may also be the case that teachers with higher levels of MKiT are able to use the associated
practices effectively. However, it may be that teachers with relatively lower levels of MKiT could in fact be more
effective using what would be considered less advanced approaches. This implication is one of the key drivers for the
model that is described in Chapter 7.
Flexibility of Beliefs Explored with Standard Deviation on Responses
One result derived from the analysis of the case studies earlier in this section is that teachers with lower levels
of knowledge, as measured through the assessment, tended to hold a more focused or narrower set of beliefs.
On the other hand, teachers with higher levels of knowledge were much more nuanced and “uncategorizable” in
their responses to questions related to beliefs. The teachers with higher levels of knowledge often expressed a high
degree of flexibility regarding which practices to use, with their choice of practices depending on the classroom
and subject context. Could this ‘flexibility’ be detected through the survey responses? One way to determine this
was to separate the teachers into higher and lower assessment score groupings and to compare the average standard
deviation of responses to each Likert scale item. A higher standard deviation would indicate a greater variation in
responses, while a lower standard deviation would indicate a tendency for teachers within the grouping to respond
in a similar manner.
Main Report 69
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
On average, the standard deviations of the teachers in the high-scoring group (1.22) were higher than was the case
for teachers in the low scoring group (1.01). By using the independent group’s t-test, it can be determined that
this difference is statistically significant. This result can be seen as providing at least some evidence to support the
hypothesis that the beliefs of the teachers with relatively high levels of knowledge do tend to vary more, while those
of the teachers with lower levels of knowledge tend to more easily fall into given categories. The regression results
presented in the next section take this idea further.
6.1.2 CASE STUDY INSIGHTS ON THE BELIEFS-KNOWLEDGE RELATIONSHIP
A key finding of the beliefs analysis was that while three teachers expressed belief statements that enabled
the teachers to be easily categorized, all other case study teachers expressed a range of beliefs. This gave rise
to the hypothesis that, rather than attempt to categorize these teachers or declare their beliefs contradictory, it was
important to recognize that this expressed range is significant. It suggests that some teachers are more flexible in
their views, adapting these views according to context. Of particular interest in examining the relationship between
beliefs with levels of knowledge is determining whether teachers who tended to express a more focused or oriented
set of beliefs differed from those tending to express a range of beliefs and who specifically discussed flexibility and
adaptability in terms of their level of knowledge.
Case studies CS2 and CS10 expressed relatively focused viewpoints that, when seen through the analytical beliefs
lenses, were transmissionist (orientation lens), traditional (approach lens) and instructor-centric (teacher role lens)
in nature. This Transmissionist-Traditional-Instructor grouping will be specified as TTI for brevity throughout
the remainder of the study. The comments of CS5 also tended to be clearly orientated towards the other end of
the spectrum. She tended to make statements and show preferences that were clearly Connectionist, Progressive
and Facilitator-oriented (which for brevity’s sake will be referred to as CPF). However, the remaining case study
teachers expressed a range of beliefs, with three teachers in particular (CS4, CS4 and CS9) specifically emphasizing
flexibility. While appearing to lean towards what could be a CPF orientation, at the same time, they saw value in
practices could be described as more transmissionist, traditional and instructor in nature. The contextual aspect
was critical. These teachers would often emphasize the importance of context in determining how to teach and
what methods to use. CS3, CS6 and CS9 openly defied the notion of a beliefs orientation, arguing that different
situations call for different approaches. These teachers specifically discussed the need for flexibility in teaching.
However, an important distinction in this concept of concept is that CS3 tended to consider the use of a particular
approach as being more appropriate for particular sets of students, with more emphasis on TTI for students of lower
achievement levels, but a greater emphasis on CPF for more advanced students. On the other hand, CS6 and CS9
advocated the use of multiple approaches for the same set of students, even jumping from approach to approach
within a single lesson, depending on the topic and context.
How, then, might a teacher’s orientation relate to their levels of knowledge and how does this notion of flexibility
play out when comparing beliefs and knowledge?
70 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
All Case Studies
In Figure 27, the categorized beliefs expressed by the teachers are overlaid with the teacher subject and pedagogy
assessment scores. One thing that immediately stands out is that those teachers whose responses tended to be TTI-
oriented also tended to record scores at the lower end of the teacher assessment, particularly in terms of subject
matter knowledge. On the other hand, teachers who tended to be more CPF-oriented also tended to have relatively
high subject and pedagogy scores, but the results were mixed. The teachers who directly discussed the importance
of flexibility tended to have scores at the upper end of the range.
Figure 27: Scatter Plot of Teacher Subject and Pedagogy Scores on the Teacher Assessment
100%
90%
Flexibility emphasis
80%
70%
CS9
TTI Orientation
Pedagogy Score
60% CS5
CS7
CS6
50% CS4 CS3
CS2 CS10
40%
30%
CPF Orientation
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mathematics Score
Note: TTI=Transmissionist-Traditional-Instructor lens grouping and CPF=Connectionist-Progressive-Facilitator lens grouping.
Case Study Conclusions
However, it must be stated that the very limited number of case studies involved does not provide sufficient proof
to conclude that it is teachers with lower levels of knowledge that tend to hold this beliefs perspective. Indeed, the
fact that the flexible teachers appreciated and used transmissionist/traditional/instructor approaches and that some
of these teachers scored among the highest levels in the assessment suggests that it may be inappropriate to draw
such a clear-cut conclusion. Still, it does raise an interesting area for further exploration to determine whether there
is a reason that teachers with lower levels of knowledge tend to hold a TTI viewpoint.
Main Report 71
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
6.2 THE BELIEFS-PRACTICES LINK
The research community is particularly interested in
PRACTICES
teachers’ mathematical beliefs because of the likely
influence of these beliefs on teaching practices. In
Video
Teaching practices used in the lessons
particular, the research has tended to focus on the extent to
A
D which espoused beliefs are consistent with enacted beliefs.
Stimulated recall
How teacher describes events within the E
lessons and explains decision/actions
This branch of research has its roots in the work of Ernest
MKiT MATHEMATICAL BELIEFS (1989), who proposed a model of espoused and enacted
G
Scenarios
Demonstrated knowledge of students,
knowledge of practices and subject knowledge
beliefs, in which the two types of belief are separated by a layer
Beliefs Interview
Espoused beliefs through general
discussion and beliefs situations
B of constraints and opportunities defined by the social context
F
C
Teacher Assessment
Subject and pedagogical knowledge test scores
of teaching. Results of some studies indicate consistency (such
Beliefs Survey
Questions covering nature of mathematics,
as Thompson, 1984; Philipp, 1995) while others suggest
how students learn, how to teach and
teacher’s own practices
a higher level of inconsistency, with some of these studies
positing conjectures as to why these inconsistencies may exist
(such as Butt and Raymond, 1989; Anderson, 1997; Anderson and Piazza, 1996). Lerman (2001, 2002) and Skott
(2001, 2010) and a number of others have challenged the highly individual perspective underlying beliefs research,
suggesting instead that researchers should take a more social viewpoint. Lerman (2001, page 36) argues that there
may be a ‘family resemblance’ between beliefs across contexts, but that beliefs are qualitatively different because of
these contexts. According to this approach, it makes little sense to talk about beliefs in themselves, irrespective of
context, and to expect a one-to-one correspondence between espoused and enacted beliefs. Skott (2010) argues that
the individualistic approach leaves little room for a more social understanding of teaching practices. He states that a
more participatory view of the role of teachers in classroom practice should be adopted, recommending a shift from
the emphasis on ‘beliefs’ to one on ‘patterns of participation’.
This study was designed to allow for a comparison of beliefs across the full set of 104 teachers, with espoused
beliefs being captured the Beliefs Survey and enacted beliefs being captured through coded video data. At
this level of analysis, the many small cases in which a teacher’s espoused beliefs are not consistent with his or her
actual practices would not be captured to the extent that they could be through the case study analysis. However,
this overall view does provide an indicator as to whether the espoused beliefs of the teachers, as represented by their
responses to belief statements, are related to the practices they used, as captured using low inference coding of the
video footage.
This method has a number of recognized deficiencies that will be discussed together with the analysis of the
case study component. However, it still provides advantages in terms of the identification of general patterns in the
relationship between beliefs-practices. As such, this analysis was approached with the intention of identifying patterns.
The results presented in this subsection indicate that when examining beliefs and practices to identify patterns
across the whole sample of teachers, there is a clear relationship between teacher beliefs and practices. This
lends validity to the idea that understanding a teacher’s belief system can provide important insights into how the
teacher may structure the lesson; what practices the teacher may use; and how the teacher may approach certain
situations. However, it is certainly not the case that there is a clear, one-to-one relationship between teacher’s belief
and the practices that he or she always uses.
72 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
The case studies provide many insights into the dynamics and complexities involved in the selection of
practices by teachers on an event-by-event basis within their classroom. As mentioned previously, some teachers
demonstrated a high level of flexibility in their beliefs regarding approaches and practices, which indicates that
context is an important factor in determining the selection of teaching practices. Teachers do not operate in isolation
from their environment when they choose which practice to utilize. Rather, they take into account a wide range of
influencing factors.
6.2.1 EXPLORING PATTERNS IN BELIEFS-PRACTICES
Analytical Approach
With the full sample of quantitative data, a pattern analysis approach was adopted to identify the relationships
that emerged. Prior to running these regressions, the beliefs items were categorized according to the beliefs
lenses (see Section 5.2). In addition, the practices were conceptualized on the basis of what would be expected to
occur in relation to those beliefs. For example, using the lens of traditional-progressive beliefs, the practice category
of public (whole class) interaction would be hypothesized to involve teachers making statements more towards
traditional beliefs to have a larger proportion of teacher-only interaction than teachers making more progressive
statement who would be hypothesized to have more teacher-student or student-only interaction.
It is important to understand how these results should be interpreted. The identification of relationships
involves comparing teachers’ rating of beliefs statement relative to other teachers. Similarly, it involves comparing
the proportion of time teachers use various practices relative to other teachers. For example, a high proportion
of teachers may strongly disagree with a particular beliefs statement. A teacher who expresses only moderate
disagreement with the statement would still have a higher relative level of agreement with the statement compared
to other teachers. Thus, a positive relationship in the analysis below between a beliefs statement and a specific
practice does not necessarily mean that the teachers in question strongly agreed with the statement. Rather, it
means that they agree with it more strongly than the average respondent. The same is true with practices. A positive
relationship for a given practice simply means that teachers tend to use that practice for a greater proportion of time
than do teachers. It does not necessarily mean that they use it for a greater proportion of time in an absolute sense.
Emergent Links between Beliefs and Practices
Two different approaches were adopted in the analysis of patterns of beliefs-practices. First, certain themes were
examined to determine which teaching practices could be matched with beliefs-practices in terms of statistically
significant positive or negative correlations. Second, the axis was flipped and certain practice themes were examined
to see which beliefs statements emerged as having a positive or negative relationship.
The regressions were run taking into account and controlling for multiple student, classroom, teacher, school,
and community variables. The t-statistics were then collected to determine whether a statistically significant positive
or negative relationship existed between the beliefs statement and the associated teaching practice. A green box
indicates a statistically significant positive relationship, while a red box indicates a statistically significant negative
relationship.
Main Report 73
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The following results are presented by themes. The first theme is discussed in detail, with each belief statement
being analyzed in order to demonstrate the analytical thought process that took place for all statements. For the
sake of brevity, subsequent sections do not provide the same level of detail, but rather summarize key observations.
HOW TO INTERPRET THE BELIEFS-PRACTICES FIGURES IN THIS SECTION
In order to present the relationships found between beliefs and practices in a summarized form, matrices were created with rows
to show the beliefs statements, along with columns representing various practices. Below is shrunken view of a matrix.
Interaction Types Approaches Problem and Question Types
Private (group and individual)
Open problem and question
Public: teacher and student
Private: teacher monitoring
Mathematical language
Qeustion and answers
Non-routine problem
Public: student only
Public: teacher only
Public (whole class)
Private: group work
Real world context
Private: no teacher
Private: seat work
Routine problem
Problem solving
Practical work
Non-problem
Investigation
Discussion
Exposition
Problem
ROLE OF TEACHER
The teacher is the main source of mathematical knowledge
I guide students in finding their own answers to academic
problems
It is more effective to provide students with the information the
need to know, rather than encouraging them to experiment
It is the teacher’s responsibility to provide children with clear an
concise solution methods for mathematical problems
I function in my classroom as a learner and partner in learning
with my students
Teachers can create, for all children, a non-threathening
environment for learning mathematics
I wait for students to approach me before offering extra help
The cells for each belief-practice combination are color-coded on the basis of the result of the relationship revealed through the
regression analysis. Dark red indicates a negative correlation with a statistical significance at the 1% level; medium red indicates
statistical significance at the 5% level; while light red indicates statistical significance at the 10% level. Similarly, dark green indicates
a positive correlation with a statistical significance at the 1% level; medium green indicates statistical significance at the 5% level;
and light green indicates statistical significance at the 10% level. The level of 5% is used as a threshold for meeting the criteria
of statistical significance, but the level of 10% is still shown to indicate that an item was approaching statistical significance. The
columns of practices are the same as were used for the analysis of the relationship between knowledge-practices in Section 6.3.
6.2.2 BELIEFS THEME: THE ROLE OF THE TEACHER
Figure 28 shows the relationship between statements under the theme “role of the teacher” with the associated use
of practices. Of the different themes explored, the results for the “role of the teacher” statements conformed most
closely with predicted responses, possibly indicating that the association between beliefs statements regarding the
role of the teacher and practices is clearer than in the case of belief statements related to how students learn or
how to teach. From even a brief glance at the figure, it appears that the area with the most positive and negative
relationships is in the interaction type found in the classroom, which include the first nine columns on the left.
74 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
Figure 28: Matrix with Beliefs Role of the Teacher related to Actual Practices
Interaction Types Approaches Problem and Question Types
Exposition
Investigation
Practical work
Problem solving
Problem
Non-problem
Mathematical language
Real world context
Routine problem
Non-routine problem
Qeustion and answers
Open problem and question
Private (group and individual)
Public: teacher and student
Private: teacher monitoring
Legend of Statistical Significance and direction
Positive 1% 5% 10%
Public: student only
Public: teacher only
Public (whole class)
Private: group work
Private: no teacher
Negative 1% 5% 10%
Private: seat work
Discussion
ROLE OF TEACHER
The teacher is the main source of mathematical knowledge
I guide students in finding their own answers to academic problems
It is more effective to provide students with the information the need to
know, rather than encouraging them to experiment
It is the teacher’s responsibility to provide children with clear an concise
solution methods for mathematical problems
I function in my classroom as a learner and partner in learning with my
students
Teachers can create, for all children, a non-threathening environment for
learning mathematics
I wait for students to approach me before offering extra help
For this theme, the relationships seem to generally conform to the predicted results:
• “The teacher is the main source of mathematical knowledge”: Teachers who agreed to a relatively greater
extent21 with this statement also tended to utilize practices involving a higher proportion of whole class time.
Furthermore, their whole class time included a relatively higher proportion of teacher-only public interaction.
The teachers also tended to use exposition to a relatively greater extent. The use of these practices is in line
with the predicted/hypothesized direction of the relationship, as they all are associated with more traditional
teaching methods, with the statement being categorized as orientated towards traditional, transmissionist
teaching. Under the teacher role lens, the statement itself would fall into the instructor or explainer category,
rather than the guide category. This matches the use of exposition (lecturing). The negative relationship
with practical work also conforms to the predicted results, because the practice involves students learning by
exploring, rather than learning derived from the knowledge of the teacher.
• “I guide students in finding their own answers to academic problems”: This statement contrasts with
the “main source” statement above, with the relationships tending to go in the opposite direction. There is a
positive relationship between this statement and teacher-student public interaction, indicating that student
interaction in whole-class situations is guided by the teacher. For individual interaction, it is also the case that
the teacher tends to monitor or be involved in these situations, again possibly indicating the teacher’s guiding
role. Finally, there is a positive relationship with the use of questioning, again possibly indicating a guide role.
21
In this case a teacher’s level of agreement is based on his or her Likert scale ranging from 1 to 5 for the statement and compared to other
teachers. If a teacher marks 4 (agree) and the majority of teachers tend to disagree with the statement (marking 1, 2) then this teacher would
have a relatively higher agreement of the statement. If, on the other hand, the majority of teachers strongly agree (marking 5) then that same
response would be relatively low relative to other teachers.
Main Report 75
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
• “It is more effective to provide students with the information they need to know, rather than encouraging
them to experiment”: There is a positive relationship between the level of agreement with this statement and
the extent to which teachers relied upon teacher-only public interaction, with a negative relationship with
student-only interaction. In private interaction, the teachers also tended to monitor the group and seatwork
to a relatively greater extent. Again, this is possibly related to the perception that a higher level of teacher
involvement is a more effective approach.
• “It is the teacher’s responsibility to provide children with clear and concise solution methods for
mathematical problems”: In the case of this statement, a somewhat surprising result was recorded, in
that teachers who expressed agreement to a relatively higher extent also tended to use a relatively lower
level of public interaction and a higher level of private (group and seatwork) interaction. There was also
a negative relationship with the level of teacher involvement in private interaction. However, there is a
positive relationship with the extent to which teachers utilized the Discussion and Exposition approaches,
and a negative relationship with the use of Practical Work is negative. These results conform to the initial
hypothesis. The teachers also tended to use Mathematical Language to a relatively greater extent.
• “I function in my classroom as a learner and partner in learning with my students”: In the case of this
statement, there was a positive relationship between the level of belief with the extent to which teachers
utilized student-only public interaction, and a negative relationship with the extent to which they utilized
teacher-only interaction, as was predicted. The approach of Discussion may also be considered to fit the
learner and partner in learning approach. In this case, one unexpected result was that such teachers were
expected to use questioning to a relatively greater degree, but this result did not occur to a statistically
significant level.
6.2.3 BELIEFS THEME: TOOLS AND RESOURCES, MANAGEMENT AND EVALUATION
The analysis revealed multiple relationships in terms of Tools and Resources, particularly in terms of
interaction practices. Teachers who tended to agree to a relatively greater extent with the statement that textbooks
are the main resource for the teaching of mathematics also utilized whole class teaching practices to a relatively
greater extent. Teachers who agreed that it is important to cover all the topics in the mathematics curriculum and
in the sequence prescribed by the textbook also used whole class teaching practices to a relatively greater extent,
and the nature of the public interaction tending to be more teacher-oriented. There was also a tendency towards a
greater use of mathematical language.
In the area of management and evaluation, teachers who tended to agree to a relatively greater extent with the
statement that: “Paper-pencil testing is the best way to examine students’ mathematical learning” also tended to
dedicate more time to problem-solving and to private time involving seatwork rather than group work. Together,
these statements appear to fit the belief system of teachers who would tend to place an emphasis on testing.
An analysis of the responses to the statement: “It is important that I establish classroom control before I become
too friendly with students” also reveals a number of significant relationships. Focusing on the teacher-student
interaction portion, in the case of public interaction, teachers tended to utilize a higher proportion of student-only
time and for private interaction, the teacher tended to be less involved in monitoring activities. The relationship
between teacher-student interaction practices with the level of both public and private interaction is negative, which
could relate to these teachers tending to not want the teacher-student relationship to remain distant.
76 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
The grouping of Curriculum Assessment and Policy statements stands out for revealing relatively few
statistically significant relationships.
Figure 29: Matrix with Beliefs on Teaching Approach, Assessment, and Resources Related to Actual Practices
Interaction Types Approaches Problem and Question Types
Exposition
Investigation
Practical work
Problem solving
Problem
Non-problem
Mathematical language
Real world context
Routine problem
Non-routine problem
Qeustion and answers
Open problem and question
Private (group and individual)
Public: teacher and student
Private: teacher monitoring
Legend of Statistical Significance and direction
Positive 1% 5% 10%
Public: student only
Public: teacher only
Public (whole class)
Private: group work
Private: no teacher
Private: seat work
Negative 1% 5% 10%
Discussion
MANAGEMENT AND EVALUATION
I base student grades primaliy on homework, quizes, and tests
I adjust my lesson plan based on results of homework assignments
I prefer to assess students informally through observation and conferences
It is important that I establish classroom control before I become too
friendly with students
Paper pencil test is the best way to examine students’ mathematical
learning
I prefer to cluster students’ desks or use tables so they can work together
TOOLS AND RESOURCES
Textbooks are the main resource to teach mathematics
It is important to cover all the topics in the mathematics curriculum in
the textbook sequence
To be sure that I teach students all necessary content and skills, I follow a
textbook or workbook
I generally use the teacher’s guide to lead class dicussions of a story or text
I find that textbooks and other published materials are the best sources
for creating my curriculum
I encourage parents to follow up on classroom activities with students
at home
CURRICULUM, ASSESSMENT AND POLICY
The examination syllabus should not be the main factor in determining
the school syllabus
Teachers should be able to influence public examination boards in the
style of examination, syllabus, content, etc
Public examination syllabus, style, etc. should be decided by experts
Commerce and industry’s needs should determine the mathematics
curriculum
The testing program in my state’district dictates what mathematics
content I teach
I believe that expanding on students’s ideas is an effective way to build
my curriculum
Main Report 77
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
6.2.4 BELIEFS THEME: PRACTICES FOR AND PROCESSES OF LEARNING
In general, in the case of themes of Practices and Processes of Learning, the analysis revealed fewer statistically
significant relationships than was the case in the previous sections. In a sense, this is surprising, as this area
would seem to be more directly related to practices than the previous themes.
Figure 30: Matrix with Beliefs on Learning Related to Actual Practices
Interaction Types Approaches Problem and Question Types
Private (group and individual)
Open problem and question
Public: teacher and student
Private: teacher monitoring
Legend of Statistical Significance and direction
Mathematical language
Qeustion and answers
Positive 1% 5% 10%
Non-routine problem
Public: student only
Public: teacher only
Public (whole class)
Private: group work
Real world context
Private: no teacher
Negative 1% 5% 10%
Private: seat work
Routine problem
Problem solving
Practical work
Non-problem
Investigation
Discussion
Exposition
Problem
PRACTICES FOR LEARNING
Students learn mathematics best in classes with students of similar
abilities
Listening carefully to the teacher explain a mathematics lesson is the
most effective way to learn mathematics
Expecting pupils to be creative in mathematics is unreasonable and
doesn’t warrant the use of precious lesson time
Mathematics is learnt well by working individually on mathematical
problems
The more students practice mathematical exercises given in the
textbooks, they learn mathematics with understanding
PROCESS OF LEARNING
Allowing a child to struggle with a mathematical problem, even a little
tensions, can be necessary for learning to occur
It is important that mathematics content be presented to children in the
correct sequence
Mathematical material is best presented in an expository style:
demonstrating, explaining and describing concepts and skills
Learning mathematics in school is essentially an active process
Students learn mathematics by finding the correct answer to
mathematical problems
Student can solve most mathematical problems on the basis of
memorized formula
Use of calculator hinders students’ understanding of mathematics
Children always benefit by discussing their solutions to mathematical
problems with each other
Persistent questioning has a significant effect on children’s mathematical
learning
The statement: “Learning mathematics in school is essentially an active process” fits well in terms of interaction
results, with the proportion of public time tending to have more teacher and student interaction and private
interaction tending to involve more group work than seat work. Questioning, though, is less and this seems to go
against what would be predicted for a teacher with an active process perspective.
78 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
The results of the analysis for the statement: “Listening carefully to the teacher explain a mathematics lesson is the
most effective way to learn mathematics” is also in line with the expected interaction, with teachers expressing a
relatively high level of agreement with the statement tending to engage in public interaction to a relatively greater
extent, with the interaction tending to have a higher proportion of teacher-only time and less student-only time.
However, while it was hypothesized that there would be a positive relationship between this statement and an
expository approach, while the t-statistic is positive, the relationship did not reach the level of statistical significance.
In the case of the statement: “Expecting pupils to be creative in mathematics is unreasonable and does not warrant
the use of precious lesson time”, the relationship went in the opposite direction to that expected, with teachers
expressing agreement with the statement tending to dedicate a relatively higher proportion of time to student-
teacher pubic interaction, with less teacher-only interaction. It also is unexpected that these teachers’ practice
involved a higher proportion of practical work, which could possibly include exploratory or creative activities.
General Comments on the Pattern Analysis
The pattern analysis, which revealed both expected and unexpected relationships, can be somewhat
overwhelming when examined item by item. However, when considering the picture from a general perspective,
the results mostly conformed to the initial hypotheses, showing that many aspects of teachers’ beliefs play a role in
defining their choice of teaching practices.
As was concluded after the initial beliefs analysis, it can be seen that beliefs are complex and a Likert scale
survey is unable to capture the complex manner in which teachers’ beliefs interact with their practices.
Nonetheless, certain aspects appeared to provide insights into the interaction. In particular, responses to the items
under the theme “Role of the Teacher” tended to be particularly closely linked with interaction types. It may be that
this aspect of teachers’ beliefs clearly translates into how the teacher chooses to interact with students. Looking at the
Approaches grouping of practices, it was frequently found that an expositional approach was associated with belief
statements that indicated a belief in teacher-centered, rather than student-centered, approaches to learning. The
responses to belief statements related to Problem and Question Types tended to have a lower level of association with
practices, although the teachers’ use of questioning tended to be associated with beliefs involving a more student-
centered approach.
6.2.5 CASE STUDY TEACHER INSIGHTS ON BELIEFS-PRACTICES
Case Study Teachers
With the case study teachers, the relationship between beliefs and practices was first analyzed in broad, general
terms across all of teachers’ videotaped lessons. This was undertaken to determine whether any teacher’s actual
overall approach to teaching might be considered to be noticeably different from the results of the beliefs analysis
presented in Section 5. This was followed by an examination of specific events within the lessons. The inputs for
this included not only observation of the video but also, critically, stimulated recall interviews in which the teachers
explained their choice of actions and methods.
Main Report 79
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The Broad Sweep
A broad sweep was conducted across all lessons of each case study teacher, with consideration given to the
practices used in relation to the statements they made during the beliefs interview and their responses to
the beliefs survey. There were no cases when a teacher’s overall approach was significantly different from his or
her stated beliefs. Thus, there were no cases in which a teacher’s statements indicated a certain orientation, but
they generally22 taught in a completely different way. The practices are utilized and the environment created in the
classroom typically conformed to the results and conclusions of the beliefs analysis.
Teachers CS2 and CS10, who tended to make statements indicating an orientation towards transmissionist or
traditional teaching methods, generally used a relatively more teacher-centered approach than those whose
statements indicated an orientation towards a more connectionist or progressive approach. These teachers tended
to use an expositional approach, with the proportion of ‘student and teacher’ and ‘student only’ interaction for public
(whole class) time being relatively lower than with the other case study teachers. These teachers took on the role of
instructor relatively more frequently than they took on the role of guide or explainer. When these teachers utilized
questioning in the classroom, the questions tended to be control and testing type questions rather than genuine or
provoking type questions.
Teachers CS4, CS5, CS6 and CS9, who tended to make statements indicating that they saw their role as being a
guide to the students, also tended to utilize genuine and provoking questions to a relatively greater extent than the
more instructor-oriented teachers, who tended to use control and testing-oriented questions of a greater extent.
The resulting classroom environment was more student-led, with less teacher-led exposition and with a greater
proportion of student-led investigative and problem solving activities. This involved a higher proportion of ‘student
and teacher’ and ‘student only’ public interaction time.
One general difference between beliefs and practices was that there was a tendency for teachers to utilize
traditional teaching approaches to a greater extent than might be implied by their beliefs statements. Some
teachers tended to stress beliefs regarding levels of student involvement and the use of non-routine or open
approaches, but in practice these aspects were less evident. This point is explored in greater detail after discussing
individual instances.
Beliefs-Practices “Differences” in Individual Teaching Instances and Teacher Explanations
When examining specific instances within the lessons, many interesting interactions and incidents occurred
that helped generate insights into the relationship between beliefs-practices and the complexity of this
relationship. The stimulated recall interviews played a particularly important role in the comparison of beliefs-
practices, because they provided insights into teachers’ reasons for the adoption of the approach taken in specific
classroom events. Often in research literature, differences between espoused and enacted beliefs are seen as a conflict.
In a sense, this word captures the fact that teachers do not choose their practices in isolation. Rather, at times,
the environment and context influences their choice of practices and these practices may not match the teacher’s
idealized general conception of the teaching-learning process. However, in discussing the findings of this research,
the term ‘conflict’ has a misleading connotation.
22
This stage is taking the teaching practices as a whole rather than on an instance-by-instance basis.
80 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
The differences that emerged can be grouped according to five themes, as specified below:
(a) Alternating amongst Approaches
In examining approaches on an instance-by-instance basis, the validity of the initial hypothesis when defining
the theoretical framework became clear: All teachers adopted various approaches and no teacher’s actions necessarily
conformed to a single category of beliefs 100% of the time. In a sense, this is somewhat obvious. For example, it would
have been unrealistic to expect teachers who made statements indicating that they saw their role as being a ‘guide
for their students’ to never undertake activities that were instructional or explanatory in nature (the other two
categories under the teacher role lens). Many of the case study teachers stated that in planning the learning process
for a topic, which could extend over a number of lessons, the steps within the learning process required different
levels of student involvement and different teaching approaches.
CS9 made an interesting statement, saying that she felt it was important to alternate amongst approaches and that
this was more beneficial than always following a single method or pattern of teaching. By varying practices by, for
example, sometimes group work, sometimes individual seatwork, students could be exposed to different ways of
thinking and approaching mathematics and mathematical problems.
(b) Matching Approaches with Topic or Situation
Some teachers spoke of tailoring their approaches to suit the topic. For example, CS1 explained that her game of
creating various forms of nets of a cube was not necessarily something that matched her usual style of teaching, but
that she had got the idea from another teacher and felt that the topic required the use of a visual stimulus, which lent
itself well to a game using group work. However, she regarded this as an exception to a general rule. On the other
hand, some other case study teachers were very conscious of adjusting their practices according to the situation. The
manner in which CS9 adopted alternating approaches has already been discussed. CS4 also spoke of how he would
reflect upon the topic to be taught and consider various ways to approach the topic.
While the majority of cases when teachers attempted to match a topic with situation emerged through the stimulated
recall sessions of their own lessons, many instances also emerged through the Mathematical Scenarios interview.
Some teachers would speak specifically about how the given topic would be best approached through a certain type
of practice. In other cases, they wanted to know the context to the situation before selecting an approach.
(c) Choice Due to Perceived Environmental Factors
All case study teachers pointed to various factors that played a role in their determination of how and what to
teach. The curriculum dictated or influenced not only which topics to cover, but also the amount of time teachers
could spend on the topic, which in turn influenced the approach they adopted to cover that topic. When directly
asked about the national assessment and whether it played a role in the way they teach, all teachers agreed that
it was a factor, but to varying extents. CS5 noted that her class was advanced and that they were covering topics
beyond the level prescribed by the curriculum and the national assessment process. CS3 was at the opposite end
of the scale, stating that his class had a low entry achievement level and that he needed to focus on fundamental
issues that were not theoretically part of the curriculum and national assessment process. Some teachers noted that
Main Report 81
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
the national assessment has become more influential in recent years, following the practice of publishing schools’
results. Ragatz (2013) noted an emerging trend between two points in time, 2007 and 2011, with practices possibly
changing as a result of the influence of the national assessment. Over this period, the proportion of teachers who
said they were influenced by the national assessment increased from 70% to 75%. There was also a trend towards
changes in the degree to which various practices were utilized. There was a decrease in the degree to which teachers
used non-routine approaches to problems and to which they used open problems and questioning. In terms of
teaching approaches, there was a decrease in investigative and practical work and an increase in expositional and
problem solving. There are many factors that go towards explaining teachers’ choice of practices, but the changes
could possibly be interpreted as methods oriented toward “teaching to the test” and that the trends may in part be
due to teachers adjusting their practices based on the national assessment.
In the stimulated recall interviews, teachers frequently explained their choices of practices in the context of
their students’ achievement levels and needs. CS3 was particularly intriguing in terms of how he expressed his
beliefs in relation to teaching practices and justifying the practices used in the observed classroom. He specifically
pointed out that he would tailor his teaching to his students’ ability levels. When teaching more advanced students,
as he did in previous assignments at other schools, and when tutoring, he would use approaches considered more
progressive, including student-led activities, the use of open problems and non-routine approaches. He stated that
in these contexts, he would play the role more of a guide than of an instructor. For the class observed in this
research, the teacher repeatedly mentioned his students’ low levels of achievement. In this situation, he felt it was
more appropriate to use more traditional approaches. He felt he needed to direct and control the lesson to a greater
extent and could not conduct activities in which the students took on creative tasks or were asked to complete
unstructured tasks. He spoke of his preference for individual seat work rather than group work with this class,
because he often felt that the group work would be unproductive.
The comments of CS3 bring up multiple interesting points regarding the relationship between beliefs-practices. It is
certainly debatable whether this ‘tailoring’ is beneficial or whether the fact that the teacher chose not to use what he
generally believed to be better practices might limit the students’ opportunities to learn (Clarke, 2011). However, for
the purpose of comparing beliefs-practices, what is of interest is the teacher’s perception that his generally preferred
method of teaching was not applicable for his students. Does this mean he did not “believe” in the methods that he
actually used? However, while it might seem that he did not believe that these were ideal methods, he did appear
to believe that these alternative methods were appropriate when applied in the context of his classroom situation.
One way to interpret this is that CS3 has a range of beliefs that defied categorization within specific orientation.
Rather, his practices have an accompanying range. Rather than thinking of beliefs as a singular point within a beliefs
map, his beliefs imply a range across the map, with the context determining which practices he chose to apply.
(d) Differing conceptions and interpretations of beliefs
It became clear through the interviews that the abstract and complex nature of beliefs makes them particularly
difficult to pin down to a specific approach. The interpretation of certain beliefs can differ from teacher to teacher.
Indeed, while the definition and interpretation used in this study may be founded through a review of the literature,
it is simply another interpretation.
82 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
Atweh and Abadi (2012) noted that multiple curriculum reforms in Indonesia have played an important role in
influencing teachers’ beliefs, often in ways that were uneven and unintended. They attributed differing conceptions
of pedagogical practices and teaching approaches to the inconsistent dissemination of the curricula. In the case
of the Student Active Learning Curriculum of the mid-eighties, which placed a heavy focus on student-centered
learning, there was only a limited provision of professional development and teaching guides. They noted that:
“While teaching materials in terms of activities and posters crowded the classrooms, teachers’ lack of clarity about
their expected roles lead to the overuse of homework and activities leading to concerns by parents and educators
about the decline of actual teaching and learning in schools” (page 2).
Teachers such as CS2 and CS10 indicated that they believed the teachers should act as a guide or facilitator,
yet they tended to use more traditional, teacher-centered practices that fit with a more transmissionist approach.
However, in discussions with CS2, it was clear that she interpreted the terms “guide” and “facilitator” in line with
her transmissionist teaching approach. There were also instances within her lesson where students conducted group
work and she did show aspects more related to a facilitating role, although it still had elements of a transmissionist
approach. The concepts of student-centered learning, active learning and student participation are also good examples
of how definitions can vary. For example, some teachers pointed to choral responses of students as examples of
student-centered learning, active learning and student participation.
(e) Belief in Practices, but Struggling to Implement
The approach of CS7 in the beliefs-practices context contrasts somewhat with the example of CS3. It was her first
year teaching at her school, and she noted that she was still in the process of determining the best way to teach her
students. She strongly believed that her students should be active participants in the learning process. However,
the school was in a poor, rural area and the teacher described the community culture for education as one in which
the students were not accustomed to speaking in the classroom. The video lessons provided multiple examples in
which the teacher attempted to prompt the students to answer questions, but struggled to get students to respond.
She would change the question wording, ask the class as a whole, the thing specific students and using other tactics.
However, she often failed to obtain satisfactory responses and at times became visibly frustrated.23 In the stimulated
recall interviews, she explained that she got frustrated because she felt she had not figured out “how to inspire the
students”.
This example brings up many interesting points. First, it emphasizes the fact that beliefs can be held, but, when
evoked through practices, those practices operate within the classroom environment. Beliefs and practices are not
static, but rather dynamically evolve over time and are likely to play off one another, each shaping the other to some
extent. The teacher was new at the school and it may be that over time, the students could adjust to the practice and
become more participatory, as the teacher was hoping. Alternatively, it could be that over time, the teacher adjusts
her practice, by refraining from attempting to get students to participate to the extent she initially hoped. This is
conjecture only, but it does raise an interesting area for exploration into how the beliefs-practices interaction may
evolve over time, based on the classroom environment.
23
The teacher was asked in the Stimulated Recall interviews if she thought the cameras had an effect on the students. She said no, this was
the way they typically responded to her attempts to get them to participate.
Main Report 83
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
6.2.6 KEY CONCLUDING POINTS ON BELIEFS-PRACTICES
In this analysis, there has been a simultaneous tearing down and building up of the concept of the relationship
between beliefs-practices. The general patterns from both the case studies and the full-sample analysis indicate that
there is a relationship between beliefs and practices, with a general consistency in terms of the link between certain
practices with mathematical beliefs conforming to expectations.
At the same time, certain hypothesized relationships in the full-sample analysis did not always emerge as being
statistically significant. Although rare, in some instances, the relationship flowed in a direction opposite from that
which had been expected. In part, this reflects the complexity of beliefs, indicating that no simple relationship exists:
a teacher’s level of agreement with a statement does not automatically indicate what practices they would use in a
particular situation.
The case studies provided a deeper understanding by enabling teachers to explain how they selected their practices
and how their beliefs may have directly or indirectly played a role in the process. As has been discussed in other
research on beliefs, teachers must take into account a wide range of environmental and situation-specific contextual
factors when selecting their teaching practices, both in preparation of the lesson and in on-the-fly decision making.
Most teachers indicated that they operated in an environment that generally meshed well with their beliefs (CS2,
CS4, CS5, CS6, CS9), but all teachers pointed to factors that influenced their teaching in ways that they might
not otherwise have chosen. Other teachers (CS3, CS7) operated in environments that generated elements of greater
tension (real or simply perceived) with their beliefs. These teachers noted that these factors created particular
challenges for them in the use their preferred practices and approaches. This concept of operating outside of their
preferred set of beliefs or even having at time to use practices that fall outside of their beliefs range is an important
finding that will play a key role in the model proposed in Section 7.
Another important concept that emerged was that some teachers had “flexible” beliefs, in the sense that they openly
recognized the value of practices that implied a range of sometimes contradictory beliefs. In this study, the higher
performing teachers were generally those without a strong orientation, in the sense that they saw value in various
techniques and would switch as they deemed appropriate.
So, what are beliefs when viewed in relation to practices? The definition may be different from simply viewing beliefs
at a theoretical level or in isolation from the classroom. The results in this section indicate that teachers might have
an orientation or preference that can be captured to a certain extent through their relative level of agreement with
statements and through an analysis of their explanations regarding their philosophy of teaching, learning and the
nature of mathematics and of their justifications and explanations for the choices they made when teaching. Their
expressions of beliefs did generally reflect their teaching practices, but their beliefs may shift or adapt depending
on the environmental context. In fact, the results indicate that, far from it being a negative attribute, this flexibility
and adaptability may prove to be a strength for many of the teachers, playing a role in their teaching effectiveness.
84 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
6.3 THE KNOWLEDGE-PRACTICES LINK
Ideally, the link between knowledge-practices would
PRACTICES
involve identifying instances within lessons where the
Video
Teaching practices used in the lessons
teachers invoke different aspects of their knowledge,
A
D because regardless of the extent of a teacher’s knowledge in
Stimulated recall
How teacher describes events within the E
lessons and explains decision/actions
subject and theory, it is the actual invocation of knowledge
MKiT MATHEMATICAL BELIEFS in the classroom that actually makes a difference in the
G
Scenarios
Demonstrated knowledge of students,
knowledge of practices and subject knowledge
teaching-learning process. A number of ambitious research
Beliefs Interview
Espoused beliefs through general
discussion and beliefs situations
B
studies have attempted to identify this link, with varying levels
F
C
Teacher Assessment
Subject and pedagogical knowledge test scores
of success. Some of the most prominent initiatives have come
Beliefs Survey
Questions covering nature of mathematics,
from the work of Hill, Rowan and Ball (2005), who examined
how students learn, how to teach and
teacher’s own practices
the predictive validity of their level of knowledge competence
by using the item response theory score to predict elementary
students’ learning outcomes. They drew on a sample of schools participating in three comprehensive school reform
programs and a matched group of control schools. Multilevel analyses showed that elementary teachers’ level of
mathematical knowledge for teaching indeed correlated with students’ learning outcomes in two different grades. In
fact, the effect was practically linear. This study produced the first conclusive evidence for the practical importance
of teachers’ mathematical knowledge in terms of both the mathematical knowledge that adults use in everyday life
and the specialized knowledge that teachers use in classrooms.
The work of Hill, Rowan and Ball (2005) was considered a ground-breaking quantitative study in its unique approach
to finding that teachers’ mathematical knowledge was significantly related to levels of student achievement. This
result, while consistent with findings from the educational production function literature, was obtained through a
measure focusing on the specialized mathematical knowledge and skills used in teaching mathematics. However, it
had many limitations and could not distinguish between their theorized domains of knowledge.
A grounded approach to data analysis by Rowland, Thwaites and Huckstep (2005) involving the use teacher
videos led to the identification of a ‘knowledge quartet’, with four broad dimensions, or ‘units’, through which
mathematics-related knowledge of the teachers could be observed in practice. These four dimensions were defined
as: foundation, transformation, connection and contingency.
While a methodical coding of the videos in the context of MKiT was not undertaken, the stimulated recall
interviews brought out many insights in the teachers’ levels of MKiT when discussing specific instances in the
lesson. The scenarios interview, while not involving actual interviews, did provide simulated situations that could
enable a comparison of teachers. The detailed aspects of the scenarios analysis can be found in the Technical Report,
but specific aspects of the knowledge-practices relationship are summarized here.
Main Report 85
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
6.3.1 LINKING PRACTICES WITH MKIT ASSESSMENT SCORES
The results of this study and of those of a large body of similar studies indicate that there is a strong positive
relationship between levels of teacher knowledge as measured through the teacher MKiT assessment and
the learning outcomes of students.24 Teachers do not directly pass along their knowledge to students: rather, it
is passed along through actions, through their teaching practices in the classroom.25 Therefore, it is important to
understand how teacher knowledge manifests itself in the classroom and how teaching practices differ according to
teachers’ various levels of MKiT. The influence of knowledge on practices can manifest itself in three possible ways.
First, it is possible that teachers with higher levels of MKiT could tend to use different practices than those with
lower levels. Under this scenario, the teachers with higher levels of MKiT might possibly have a larger set of teaching
practice “tools” and that these additional tools are beneficial for student learning. Second, teachers with higher levels
of MKiT might tend to use certain teaching practices more or less frequently, based on their knowledge. With this
scenario, it may indicate that the exposure of students to certain practices could result in better or worse learning
outcomes. Third, it is possible that teachers use the same approach, but that their execution of that approach varies
according to their level of knowledge. Under this scenario, students may achieve better outcomes because a teacher
with a higher level of knowledge may tend to use a particular practice or practices more effectively. The analysis
explores these different possibilities as a means to explain how learning outcomes tend to be better for students who
have teachers with higher levels of knowledge.
As this section discusses, the results at both the case study level and from the full study level indicate that
to some extent, differences exist in terms of the types and frequency of practices used, with teachers with
higher levels of knowledge tending to use a wider range of practices and tending to use certain practices more
frequently. Still, these differences are by no means dramatic. In fact, differences in the use of teaching practices
are much more distinct when comparing teachers with a mathematics education degree to teachers with a pure
mathematics degrees, or when comparing teachers with varying levels of experience. This seems to indicate that
other factors are at play in the teaching practices of teachers with higher levels of knowledge. When examining
specific practices in detail, including teachers’ use of questioning, the results indicate that knowledge also manifests
itself in the approach to practices, with higher student learning outcomes likely to result in part from teachers with
higher levels of knowledge tending to use certain practices more effectively.
6.3.2 USE OF PRACTICES IN RELATION TO KNOWLEDGE INDICATORS
Do teachers with higher levels of knowledge, as measured through the teacher MKiT assessment, teach any
differently?
The strong positive relationship that was found between levels of both subject matter knowledge and
pedagogical teacher knowledge, as measured through the MKiT assessment, and student learning outcomes, as
measured through tests at the beginning and end of the eighth grade year, indicate that teachers with higher levels
24
Multiple factors play an important role in student learning beyond the teacher. This result is after controlling for various student, home,
classroom, school, teacher background and community characteristics in order to isolate and capture the relationship of teacher knowledge
and student learning as accurately as possible with the given data.
The teacher may work with students outside of the classroom, but teacher tutoring is a variable that is controlled for and it is assumed that
25
most of the teacher influence in the learning process takes place in the classroom.
86 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
of MKiT tended to teach more effectively.26 This would imply that somehow, teachers with higher levels of MKiT
are doing something different in the classroom. In terms of teaching practices, the difference could come about
through: (i) using different practices; or (ii) using the same practices, but using them more effectively. The use of
certain practices and the frequency of these practices coded through the Video Study enable an exploration as to
whether teachers with different levels of MKiT use different practices than do teachers with lower levels.
As Figure 31 shows, a few differences do emerge when comparing teacher knowledge scores to the measured teaching
practices. Teachers with higher MKiT scores tended to use a higher proportion of Mathematical language and symbols
relative to of Real world contexts. They also tended to use questioning techniques more often. In terms of teaching
approaches, they tended to use Investigation to a greater extent. In the case of all other teaching practices, there were
no statistically significant correspondences. While the teaching practices identified as having a statistically significant
correlation are of interest, it may be even more interesting that relatively few such practices were identified as being
statistically significant when compared to other teacher characteristics, such as teacher degree type. Since teacher
knowledge has a strong positive relationship with a large effect size, this lack of differences would imply the impact
on student learning might in large part be due to teachers with higher levels of knowledge being able to using the
same practices in a more effective manner. This proposition will be explored later in the chapter.
Figure 31: Teaching Practices of Teachers in Relation to their Overall MKiT Score
Interaction Types Approaches Problem and Question Types
Exposition
Investigation
Practical work
Problem solving
Problem
Non-problem
Mathematical language
Real world context
Routine problem
Non-routine problem
Qeustion and answers
Open problem and question
Private (group and individual)
Public: teacher and student
Private: teacher monitoring
Legend of Statistical Significance and direction
Positive 1% 5% 10%
Public: student only
Public: teacher only
Public (whole class)
Private: group work
Private: no teacher
Negative 1% 5% 10%
Private: seat work
Discussion
MKiT Assessment Score
Subject Matter Section
Pedagogy Section
Mathematics Education Degree
Subject Score
Do teachers with higher subject knowledge use the measured practices more or less frequently?
While a comparison of teachers’ overall MKiT scores with the teaching practices they adopted did not reveal
many correlations, it is possible that examining the subject and pedagogy scores separately could reveal
additional insights. Subject matter knowledge is also shown in Figure 31. While little changed, it does appear that
teachers with higher subject scores tended to use more Non-routine and Open approaches to problem solving. The
fact that these relationships emerge specifically in relation to the teachers’ level of subject knowledge may indicate
that as teachers have more knowledge, which may include aspects such as ability, tools and confidence, they may
know how to use more complex or less standard approaches, such as Non-routine and Open problems.
26
This result cannot be conclusively said to be causal, but there are strong theoretical reasons to believe that the relationship is a result of
teachers with higher MKiT teaching in a way that led to better learning outcomes.
Main Report 87
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Pedagogy Score
Do teachers with higher levels of pedagogical knowledge use the measured practices more or less frequently?
When looking at the relationship between practices with teachers’ scores on the pedagogy portion of the
assessment, two new findings emerge. First, teachers with higher pedagogical scores tended to use Exposition
to a relatively lesser extent than did teachers with lower scores. Indonesia’s Ministry of Education and its teacher
training colleges generally tend to discourage the use of teacher-oriented approaches in favor of more student-
oriented, at least in recent years. Teachers with higher pedagogy scores may have been exposed to a greater level of
training, which may explain the result. Second, the teachers with higher pedagogy scores also tended to be involved
to a greater extent when private interaction (group or seatwork) took place. This activity would typically involve
the monitoring of students during an activity and could involve asking questions and guiding students. It seems
reasonable that teachers with higher pedagogy scores would see value in these practices.
Degrees in Mathematics Education versus degrees in Pure Mathematics
Do teachers with a mathematics education degree use the measured practices more or less frequently?
When comparing teachers with mathematics education degrees to those with degrees in pure mathematics, the
differences in the practices adopted by the two groups are more striking than when comparing teachers with
varying levels of MKiT. While the results for some aspects are similar (Questioning, Open and Investigation), there
are differences in approach, with a relatively much lower use of Exposition, and a relatively higher use of Discussion
and Problem Solving by teachers with mathematics education degrees. Interestingly, even more differences were noted
in terms of interaction. Mathematics education degree teachers tended to use Private interaction more frequently
and, as with the teachers with higher pedagogy scores, these teachers tended to monitor private interaction more
often. Surprisingly, for public interaction there was a tendency for teachers with mathematics education degrees to
apply proportionally more Teacher-only time and less Teacher and student time. This is somewhat surprising given
Indonesia’s pedagogical push for teachers to conduct more student-centered activities.
Summary of Practices Use
In examining the relationship between practices and MKiT assessment scores, a few relationships did emerge,
but it is the lack of a large number of relationships that is of most interest. This finding suggests that the
positive relationship between levels of teacher knowledge and student learning outcomes may not be due so much
to teachers using different practices in terms of type or proportion, but rather to the fact that when they use similar
practices, they tend to use them more effectively.
88 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Exploring the Beliefs-Knowledge-Practices Relationship
6.3.3 CASE STUDY RELATIONSHIPS
Teachers with higher levels of knowledge tended to use questioning more effectively, both in terms of the types
of questions they asked and in terms of integrating questioning into their lesson approach. They used genuine
and provoking questions to a greater extent, while teachers with lower levels of knowledge tended to use questions
to a lesser extent, with these questions tending to be testing and closed type questions. This suggests that varying
levels of knowledge are a factor in teachers using similar practices more or less effectively.
Knowledge was also generally demonstrated through discussions regarding teaching practices during the
stimulated recall interviews. Teachers with higher levels of knowledge tended to be more comfortable discussing
practices, with their explanations demonstrating a more focused and intentioned uses of the practices.
6.4 KEY CONCLUDING POINTS ON RELATIONSHIPS
An important contextual basis for the beliefs-practices analysis was that the students of teachers who scored
higher on the subject matter and pedagogical assessment tended to have higher learning outcomes. Although
the relationship cannot be declared causal, there are certainly theories as to how a teacher’s level of knowledge
would likely play a role in their effectiveness. In turn, this would have a causal relationship with student learning
outcomes. This could in part be due to: (i) teachers utilizing certain practices with greater frequency; (ii) teachers
using different practices; or (iii) teachers using the same practices, but utilizing them more effectively.
Investigating these three possibilities, it became clear that while there were signs of (i) and (ii), these factors did
not tell full story. Rather, it became clear that (iii) was also an important contributing factor. The practices used
by teachers with higher levels of subject knowledge did differ from those with lower knowledge, but the differences
tended to be less significant than other aspects, such as teacher degree type, which is taken to be another indicator
of contrasting pedagogical vs. subject knowledge. In addition, other contextual factors, such as the teacher’s years
of experience, also played a role. This relative lack of difference in the teaching practices adopted by teachers with
varying levels of knowledge despite the relatively strong relationship with student learning outcomes suggests that
teacher knowledge manifests itself not only through differences in the practices used, but in terms of how effectively
these practices are used. Two teachers with varying levels of knowledge might use a similar set of practices, in terms
both of the selection of practices and the proportional time devoted to these practices, but the teacher with a higher
level of knowledge is able to use those practices more effectively.
In the case of the use of questioning, teachers with higher levels of knowledge used many more genuine,
prompting and directing questions than did the teachers with lower levels of knowledge. They appeared to use
questioning as an effective tool for stimulating student engagement and thought. Lower knowledge teachers tended
to use questions more for the purposes of controlling and testing only.
Main Report 89
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The distinction between teachers with varying levels of knowledge also arose in relation to practices in the
mathematical scenarios interview. Teachers with higher levels of knowledge tended to more quickly and accurately
assess the situation, also developing a coherent plan regarding what practices they would utilize to address the
situation.
Finally, as became clear through the stimulated recall and mathematical scenarios interviews, the teachers
with higher levels of knowledge also tended to have a larger toolbox of practices upon which they could draw
to address different situations. Thus, there were able to select from a wide range of tools to choose what they
thought was the most appropriate tool for the given situation.
When examining beliefs in relation to levels of knowledge, it became clear that teachers with lower assessment
scores tended to exhibit an orientation towards a set of beliefs more readily than teachers with higher assessment
scores. Based on interviews with the teachers, it appeared that this expression of a range of beliefs indicates a more
flexible view of teaching, with a teacher with higher levels of subject and pedagogical knowledge being more likely
to not see teaching approaches as black and white, but rather in shades of grey. If knowledge is seen as a set of tools
that the teacher can draw upon, then a higher level of knowledge may allow the teacher to draw upon a larger set of
tools and therefore to determine the appropriate approach for any given situation.
This finding regarding the flexibility of beliefs does not require a rejection of the use of categories. Categories
can be helpful in conceptualizing the various ways in which teachers might perceive important aspects of the
teaching-learning process. Further, the fact that in this research, many statistically significant relationships were
identified between teaching practices and mathematical beliefs (Section 6.2) indicates that there is a relationship
between teachers’ perceptions regarding teaching and learning and their practices. According to the hypothesis
proposed in the analytical framework, these perceptions play a key role in determining the practices used.27 However,
these findings do call into question the degree to which a teacher’s mathematical beliefs should be seen in “either-or”
terms. The flexible view often expressed by the higher knowledge teachers indicates that teachers can hold a range
of viewpoints that cut across categories and that, rather than being detrimental, this flexibility may in fact assist the
teacher in being more effective by: (i) enabling the teacher to adapt to a particular situation depending on what the
teacher sees as the educational needs of the students; and/or (b) enabling the teacher to mix practices within and
across lessons, which could provide the students a more diverse set of experiences and perspectives in thinking about
mathematics.
27
Correlations between beliefs and practices do not prove causation, but the theoretical model posits that a teacher’s beliefs play a role in
determining the practices the teacher chooses and it is not logical for the relationship to go in the opposite direction where a teacher’s practices
determine her beliefs.
90 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 7
A NEW CONCEPTUAL
MODEL OF BELIEFS-
KNOWLEDGE-PRACTICES
FOR INDONESIA’S TEACHERS
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The MOEC has attempted to mandate the adoption of new teaching practices at various points with the
goal of generate improvements to student learning outcomes. However, there is often little support provided
to teachers to help them to understand the practices and to know how to use them or to understand why they
are important. The result is typically that teachers simply do not adopt the mandated practices or that they do
so only to a very partial extent, based on their own interpretation or in the context of the interpretation of their
direct managers. This study has explored the why of teaching practices to establish clear links between teaching
practices and a teacher’s knowledge and beliefs. In order to convey the complex findings in a simple and clear
manner, a model was developed for use as a tool for to facilitate discussions on policies on teaching practices and
on determining the measures required to ensure that they are actually implemented in the classroom. The need for
this discussion is particularly apparent in the context of the implementation of Indonesia’s 2013 curriculum. This
curriculum was hurriedly implemented on a nationwide scale, before implementation was halted to conduct an
evaluation in December 2014. This model can be used as a tool to consider the measures necessary to ensure that
the desired practices are actually implemented in the classroom.
7.1 REPRESENTING BELIEFS, KNOWLEDGE AND PRACTICES AS DISCS
The main purpose of the model28 is to depict the interaction between beliefs, knowledge and practices. These
components must therefore be represented in a way that allows for a representation of their interaction. All three
components are highly complex and have a slightly different nature and composition, with practices being specific,
concrete actions; while knowledge is measurable but difficult to fully capture in action; and beliefs are even more
abstract. Despite the disparity in their make-up, these three components be represented as discs and ascribed with
the following meanings:
• Practices (actions): This disc represents the set of a teacher’s actions in the classroom in their entirety;
• Knowledge (tools): This disc represents all MKiT held by the teacher, including subject knowledge, knowledge
of students, knowledge of teaching practices and reflective competencies. As a whole, this knowledge could be
considered as the teacher’s set of tools, from which he or she draws when conducting the practices. Knowledge
could conceivably be separated into separate discs of subject and pedagogical knowledge (this option is discussed
later in this chapter). However, in order to maintain the clarity of the model, they are kept together.
• Beliefs (perceptions): This disc represents the full array of mathematical beliefs that a teacher holds, including
his or her full set of perceptions regarding the nature of mathematics, how students learn mathematics, and
effective methods of teaching mathematics.
7.2 DISC SIZE SIGNIFICANCE
The size of the each disc holds important meaning:
• Practices: As stated above, this disc represents the set of a teacher’s actions in the classroom in their entirety.29
A teacher who uses a narrow range of practices in the classroom would be represented by a small practices
disc, while a teacher who utilizes a wide range of practices would be represented by a large disc. The timeframe
and context play an important role in defining this disc, with the model having dynamic aspects.
28
The conceptual model of Beliefs-Knowledge-Practices, are also part of a doctoral thesis by Ragatz (2013a).
29
For the purposes of this study the range of practices would be only those used in the teacher’s 8th mathematics classroom.
92 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
A New Conceptual Model of Beliefs-knowledge-Practices for Indonesia’s Teachers
• Knowledge: The size of the disc represents the extent of the teacher’s mathematics subject knowledge,
knowledge of teaching approaches, knowledge of how students learn mathematics, and reflective competencies.
A teacher with a lower level of knowledge would be represented by a small disc, while teachers with a higher
level of knowledge would be represented by larger discs. The disc size can be thought of as representing both
the depth and the range of the teacher’s knowledge.
• Beliefs: As stated above, this disc represents the full array of mathematical beliefs that a teacher holds.
Thus, the size represents the extent of that array. A teacher with a very focused set of beliefs (for example, a
teacher who strongly believes that students learn mathematics best through the discovery approach and that
discovery methods should be used for all mathematical topics and in as many learning situations as possible)
would have a small disc. On the other hand, a teacher with a wide array of beliefs (for example, a teacher
that sees value in a wide range of approaches and believes that different contexts call for different approaches)
would have a large disc. This is primarily intended to indicate the extent of the array of beliefs and, by
extension, the degree of flexibility that might exist in the teacher’s belief system. The concept of orientation
is in line with Schoenfeld’s (2010) conception, which is also brought into the model through the dynamics
to be discussed later.
It is important to note that the size of the discs is only conceptual in nature. In representing cases of teachers
from this research, the relative sizes of the discs were determined on the basis of evidence of quantitative and
qualitative data gathered, but there are no clearly specified measures. For example, the knowledge disc represents an
array of evidence gathered, including the teacher’s score on the teacher assessment as well as aspects of knowledge
captured in the Mathematical Scenarios interview and in knowledge-related comments from the stimulated recall
interviews. Although the teacher assessment involves an actual score and is quantifiable, there was no attempt to
quantify the other aspects.
7.3 DISC INTERACTION
As explained in the initial conceptual framework (Section 2.6), a teacher’s knowledge and beliefs are hypothesized
to be critical inputs in the determination of what practices the teacher decides to use in the classroom. In the
conceptual framework, the flow of interactions was represented through arrows. This model attempts to reframe
the interaction in subtle but important ways to allow for the representation of more dynamic relationships amongst
the three components. The overlapping of the discs in a Venn diagram represents the interaction of the components.
The extent of the overlap is critical. Following the typical interpretation of a Venn diagram, the interior, overlapping
portion symbolically represents the elements of a given set, while the exterior represents elements that are not
members of the set.
Each zone plays an important role in explaining the interaction amongst the components. Of particular
importance is one of the areas where there is no interaction, where teachers use practices that are not in line with
either their knowledge or beliefs. The model is illustrated in Figure 32 below and will be referred to as the BKP
(Beliefs-Knowledge-Practices) Congruency Model. This is followed by descriptions of the different zones of the
model, followed by their definitions.
Main Report 93
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Figure 32: Conceptual Model of Beliefs-Knowledge-Practices Interaction
PRACTICES
(actions)
Dissonant Zone
Kn ence
Ab
ne
ow Zo
s
sen lief
Zo
led ne
Ab Be
ce
ge
Congruent Zone
(practices-knowledge-beliefs
all aligned in congruence)
Kn
s
rac o
ow d no
tice
in p ed t
an
led t u
sed t ti
ge sed
r u ut no
sep in
dg ld, b
ara pra
te
wle he
eo
fro tices
kno eliefs
m
c
be
Practice
B
KNOWLEDGE BELIEFS
lie
fs
(tools) Absence Zone (perceptions)
The practices disc is central to the model because it ultimately contains all of the actions that actually take
place. Therefore, this aspect is most directly related to student learning. Teachers can operate within four possible
zones when using teaching practices. These zones are defined by whether there is overlap between the utilized
practice with the teacher’s knowledge and beliefs.
• Congruent Zone: The practices are congruent with the teacher’s beliefs and the teacher has the associated
knowledge tools to effectively implement the practices. In this case there is a harmony between all components,
with actions being backed by both conviction and ability.
• Dissonant Zone: The practices are not in harmony with the teacher’s beliefs and the teacher also does not
have the necessary tools to effectively implement the practices.
• Knowledge Absence Zone: The practices are in harmony with the teacher’s beliefs, but the teacher has a
deficit in the area of the associated knowledge tools. This could be refined further with the statement that
this represents a situation in which a practice is within the teacher’s content knowledge, but that the teacher
does not yet have the pedagogical tools necessary to effectively implement the practice. Alternatively, the
teacher may possess the pedagogical skills necessary to implement a practice, but a lack of content knowledge
impedes the effective implementation of the teaching practices to help student learning.
94 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
A New Conceptual Model of Beliefs-knowledge-Practices for Indonesia’s Teachers
• Belief Absence Zone: The teacher possesses the knowledge (tools) necessary to implement the practices, but
does not perceive the practices to be the most correct or effective ones to use. This could be either because of
a lack of belief in the practice generally or possibly a belief that it is not the appropriate practice in a given
situation.
A fifth zone that falls outside of the practices disc is also of interest:
• Practice Absence Zone: The teacher possesses both the knowledge (tools) and beliefs (perceptions) that
would potentially lend themselves to application of related practices in the classroom, but the teacher does
not implement these practices. This could be due to a variety of factors, as discussed later in this chapter.
There are also two zones that have little bearing on how teachers actually teach. One zone represents a situation
in which a teacher has accumulated knowledge, but this knowledge is not connected to any of his or her beliefs and
is not incorporated into his or her practices. The other zone is made up of beliefs that are not tied to the teacher’s
knowledge or used in his or her practices.30
The term “absence” was chosen to label areas in which one of the three components is missing. In this case, the
term is meant to simply convey “the non-existence or lack of ” (Dictionary, O. E., 1989). Various other terms
were considered (e.g. “gap” or “aperture” to represent an opening or hole), but the term “absence” provides a
relatively straight-forward and accurate meaning. One concern is that the term absence may be interpreted as
having a negative connotation. However, it should not automatically be interpreted as negative, as while absence
of knowledge would in almost all cases be considered undesirable, the absence of certain practices or beliefs may in
fact be a positive attribute.
Each of these zones was derived from evidence gathered through the various steps of analysis and will be
discussed as the dynamics of the model are described below, first in describing the discs and then relating these
dynamics to actual situations as found in the analysis.
7.4 GRAVITATIONAL AND REPELLING FORCES WITHIN THE MODEL
The model is not static and there are potentially strong internal and external forces at work. The clearest
gravitational pull would be toward the center of the Congruent Zone, which would involve the teacher choosing
practices that are most in line with his or her beliefs and where he or she possesses the tools that allow him or her
to effectively implement the practice. If sub-layers were to be imagined within the Congruent Zone, the practices at
the heart would be those that the teacher believes in and is most capable of utilizing. This heart of the Congruent
Zone would represent the teacher’s orientation. Toward the edges of the Congruent Zone would be practices that
the teacher may believe in, but with less conviction (toward the left-hand side of the zone) or where the teacher
has a strong conviction, but where his or her knowledge may not be particularly strong (the right-hand side of the
zone). Amongst the case studies, the teachers who best demonstrated the gravitational and repelling forces were CS3
and CS7, who identified policy and environmental forces that pushed them to use practices that they would not
choose if those forces were not exerted. These gravitational forces also fit Skott’s (2009, page 31) conception that
30 It could be argued that a teacher cannot hold a belief on something on which he or she does not have knowledge. A counter to this
would be that the actual knowledge of the concept overlaps with the knowledge disc, but that the beliefs-only dimension refers to the lack of
knowledge of how to put it into practice. Alternatively, some beliefs can be held at a gut-instinct level without a deep knowledge of it.
Main Report 95
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
beliefs should not be seen in isolation but rather as: “... shifting, intersubjectively established, and continually (re-)
generated settings. This means that contexts are construed by individuals as they participate in practices that evolve
in interaction.” He argues that it makes little sense to talk about beliefs in themselves, irrespective of context, nor
to expect a one-to-one correspondence between beliefs as espoused in interviews and practice. This view conforms
to the model used in this analysis.
Although many teachers would prefer to operate deep within the Congruent Zone, the gravitational pull would
be less strong for those teachers who are eager to develop their skills and expand their array of practices.
These teachers might utilize some practices that fall within the Belief Absence Zone or the Knowledge Absence Zone.
An example of a teacher operating in the Knowledge Absence Zone would be that of a teacher who believes that a
new practice would be applicable in his or her classroom and is willing to try it, but that she still lacks the tools
necessary to effectively implement the practice. This may be most applicable to newer teachers, who are still open to
exploration, but who have yet to develop the necessary tools. Specific examples will be discussed later.
The Belief Absence Zone may be entered by choice if the teacher has the appropriate tools to implement a
practice, but the teacher is still somewhat skeptical regarding its effectiveness. Most likely a teacher would not
enter this zone completely voluntarily if he or she teacher truly does not believe in the practice.
Figure 33: Comparison of Teachers Reacting to Influences
External in uences External in uences
on which practices to use on which practices to use
PRACTICES PRACTICES
Dissonant Zone Dissonant Zone
Internal in uences Internal in uences
pushing outward pulling inward
Congruent Zone Congruent Zone
KN
KN
OW
OW
S
S
EF
EF
LE
LE
LI
LI
BE
BE
DG
DG
E
E
Teacher open to learning and experimenting Teacher wanting to stay in congruent zone
with new practices and use same practices
96 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
A New Conceptual Model of Beliefs-knowledge-Practices for Indonesia’s Teachers
For teachers operating in the Knowledge Absence Zone and Belief Absence Zone, there is a possibility that the
practices in this area will eventually become part of the Congruent Zone. Working in the absence zones may be
less effective than the Congruent Zone initially, but it may be beneficial in the longer-term. A teacher might, through
use and repetition of a practice, develop the knowledge necessary to implement it effectively or become convinced
that the practice is in fact effective and thus incorporate it into his or her belief system.
In contrast to the Congruent Zone, the teacher would not choose the Dissonant Zone of his or her own volition,
because it does not fit either the teacher’s personal knowledge set or beliefs set. This state would only arise if
the cultural, system or environmental forces are so strong that the teacher feels obligated to use practices other than
those they would choose without such influences. Potentially large outside influences include the requirements
of a curriculum, evaluation requirements established by the principal or supervisor, or a teacher working in a
school located in a cultural environment that differs significantly to the one with which the teacher is accustomed.
Operating in this zone is likely to be detrimental because the teacher would lack the tools to properly implement the
practices and would also lack conviction or enthusiasm. Most teachers would have a particularly strong incentive to
resist this situation. Even if they attempt to use the prescribed practices initially, they would tend to slip back into
the Congruent Zone after their probable failure to use the given practices effectively.
A teacher may be fortunate enough to be working in a system or school environment that is in alignment with
her/his beliefs and utilizes her/his set of tools. In this case, the Congruent Zone would be large and the Dissonant
Zone would be almost non-existent. The teacher may also operate in an environment that would provide support
for ventures into the Knowledge Absence Zone and Belief Absence Zone through training, supportive peers, or other
elements that would help the teacher to expand his or her Congruent Zone. On the other hand, if there are cultural,
system or environmental forces that push the teacher outside of this zone, the teacher may resist and attempt to stay
within this zone. If a large proportion of a teacher’s practices fall within the Dissonant Zone, such as in a case where
an educational reform process requires a teacher to use a new and completely foreign teaching approach, then it
would almost certainly have a severe negative effect on the teacher, both through deterioration in his or her level of
confidence (due to lack of tools) and to a deterioration in his or her morale (due to teaching in ways that are not in
line with the teacher’s beliefs).
7.5 THE CULTURAL DIMENSION AS A KEY DRIVER
One of the key underlying factors that may determine whether teachers are able to adopt specific teaching
practices is the extent to which the practices fit within teachers’ cultural sphere or context. For example,
a student-centered approach may go against certain norms that are deeply culturally ingrained. If teachers are
traditionally viewed as the source knowledge and sayings like, “Quiet students, happy teacher”31 shape what is
considered acceptable student-teacher interaction, then it would be very difficult to change their practices through
training course that simply provides knowledge of the approach. Even teachers who are open to the idea of a
student-centered learning approach would likely internalize the training within their own (at times deeply ingrained)
conceptions and possibly interpret and reshape the approach to fit within their own pre-existing beliefs. Indeed,
Indonesian teachers do tend to state beliefs that are in line with active learning and student-centered approaches,
but what is seen in the classroom that may tends to be involvement of students, but through a teacher-led approach.
Practices are more likely to be adopted if they align with cultural norms or if the training takes into account the
cultural aspect and is designed to specifically reshape conceptions and beliefs.
31
This saying was found carved into one of the ceiling beams of a rural school in Indonesia.
Main Report 97
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
7.6 APPLICATION OF THE MODEL TO SPECIFIC INSTANCES
The model can be transformed in a variety of ways, particularly in terms of the degree of overlap and size of
the discs. The following are important examples of how the model can change, but is by no means an exhaustive list.
7.6.1 LARGE DISSONANT VERSUS LARGE CONGRUENT ZONE
The congruent and dissonant zones represent the biggest potential competing tensions faced by a teacher
in terms of teaching practices. As mentioned above, all teachers would attempt to minimize operating in the
Dissonant Zone. A teacher operating in an environment that is either in line with his or her knowledge and beliefs or
in an environment where the teacher has control over the practices he or she chooses will have a minimal Dissonant
Zone. On the other hand, a teacher facing pressure to use practices outside his or her Congruent Zone might feel
obligated or forced to use practices that place him or her in the Dissonant Zone.
The extreme example below could only come about through a major shift, such as in the case of a reform
through which teachers are required to completely change their practices. Even if such a state could exist in the
short term, a teacher is very unlikely to remain in this unbalanced state in the long term, as he or she would likely
gravitate back to the Congruent Zone and stop using the dissonant practices. It may also be possible that in the long
term, the teacher would come to accept and incorporate the practices into his or her Congruent Zone, although it
would be unlikely to occur unless the teacher were to receive intensive support that would build both a belief in the
practices and the associated knowledge tools required to effectively utilize the practice.
Figure 34: Large Dissonant vs. Large Congruent Zone
PRACTICES
Dissonant Zone
PRACTICES
Congruent Congruent
Zone Zone
KN
KN
FS
IE
OW
OW
L
BE
LE
LE
S
DG
DG
EF
E
LI
E
BE
98 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
A New Conceptual Model of Beliefs-knowledge-Practices for Indonesia’s Teachers
7.6.2 COMPARISON OF HIGHER AND LOWER KNOWLEDGE TEACHERS
As has been mentioned, three of the key findings from the analysis are that:
1. Teachers with a lower level of lower knowledge appear to have a clearer or narrower set of beliefs than teachers
with higher levels of knowledge, who express a range of beliefs, recognize context and note that practices vary
depending on the context;
2. Teachers with a higher level of knowledge tend to utilize a larger variety of teaching practice tools, such as a
larger range of questioning techniques; and
3. Teachers with a higher level of knowledge and a larger array of beliefs vary their practices to a greater extent.
These findings were drivers for the design of this model in terms of meeting the need to establish a means to show
overlap amongst beliefs-knowledge-practices and also to demonstrate the extent and size of these three components.
The example below is based on the analysis of the data on CS9 and CS232 and illustrates the above points. When
the teacher with a relatively high level of knowledge (CS9) also has a larger array of beliefs (consistent with Point 1
above), this is related to greater flexibility in his or her teaching practices (Point 2) and ability to vary the practices
(Point 3). On the other hand, the teacher with a relatively low level of knowledge (CS2) has a lower knowledge disc,
which limits the extent to which practices can fall within the Congruent Zone. The teacher’s beliefs disc is also shown
to be smaller than the higher knowledge teacher’s, but it is sufficiently large that a large portion of the beliefs disc
falls outside the knowledge disc, creating a large Knowledge Absence Zone.33
Figure 35: Illustrative Contrasting Example between Higher and Lower Knowledge Teachers
PRACTICES
PRACTICES Large
Knowledge Absence
Zone
Congruent
Zone Congruent
Zone
KN
S
OW
EF
KN
LI
LE
BE
DG
OW
S
EF
E
LE
LI
DG
BE
E
32
The sizes and overlap of the discs are based on interpretation of the teacher assessment and mathematical scenarios interview, beliefs survey
and beliefs interviews, and lesson observation of practices along with the associated stimulated recall interviews. The disc sizes and overlaps
are not exact measurements and therefore should be understood to be interpreted only.
33
The knowledge aperture zone is reflective of the teacher’s attempt to teach inner and outer tangents, but had to rely completely on the
textbook and only read the steps to the students without being able to explain the purpose of the steps.
Main Report 99
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A key observation from the figure is the highly contrasting sizes of the Congruent Zone as a result of the large
differences in the size and degree of overlap of the knowledge and beliefs discs. The higher knowledge teacher
has a much larger set of tools, a larger array of accompanying beliefs and can therefore utilize a larger set of teaching
practices. The size of the Dissonant Zone is also minimal in the case of the higher knowledge teacher, while the
lower knowledge teacher has a larger Dissonant Zone due to the need to teach all aspects of the curriculum without
necessarily having all the tools required to properly teach all the topics required by that curriculum.
The observed case of CS2 provides a good example of the Knowledge Absence Zone in practice. She received a lesson
plan from a different teacher on how to draw inner and outer tangents. It was a numbered list of steps involved.
She did not fully understand the steps involved, so she spent the lesson requiring the students to copy down the
steps. She struggled to draw the tangent on the board herself and made some minor errors. In the stimulated
recall interview, she was asked to explain the purpose behind the specific steps, but only responded that they were
the instructions, so the students should do them. These factors indicated that she was lacking certain aspects of
knowledge (both subject and pedagogical) to effectively teach that particular topic. This can be contrasted with the
case of CS3, who taught the same topic. This teacher obtained the highest subject score on the MKiT Assessment
and was able to conduct the lesson without notes, being easily able to describe the steps involved and their purpose.
In his case, the practices he used were within his Congruent Zone.
Another example of the Knowledge Absence Zone from the case studies is that of CS7. She was a relatively young
teacher and although she had three years of teaching experience, it was her first year in her current school. She
strongly believed that students should be actively engaged, but mainly due to the context in which she operated (a
poor rural environment in which the cultural norm appeared to be for students to refrain from proactively speaking
out in the classroom), she was struggling to get the students to participate. In the stimulated recall interviews, she
expressed her frustration, claiming that she could not figure out how to get them to become more engaged with
the process. She clearly had a belief in a practice and was attempting to implement it, but she did not possess
the knowledge required to engage them. She may persist with the use of this practice and, over time, develop the
knowledge required to motivate students to participate more actively. In this case, the knowledge disc would expand
and the Knowledge Absence Zone would close. Another possibility is that, over time, her beliefs will change and she
will no longer see active engagement as an effective way to teach. In this case, she would reduce her Practices and
Beliefs discs, and the Knowledge Absence Zone would also be reduced.
The Belief Absence Zone is an area in which a teacher would rarely operate. It would require that a teacher use
practices that are within the knowledge disc and that the teacher has in fact chosen to use the practice, but that
the teacher does not have an underlying belief in the effectiveness or usefulness of the practice. One case study
example that at least approaches that zone is CS3, who indicated that he was not using his ideal set of practices
(which would involve a student-centered approach in which he would act as a guide and facilitator) because the
achievement level of his students was low and the practices would therefore not be effective. Instead, he chose to
take on a more instructor-oriented role. It appeared that he had both the subject and pedagogical knowledge to
utilize this instructor-oriented approach, so its use would not place him in the Dissonant Zone. While this case
might be considered one where the teacher does not believe in the practice, it could perhaps be better described as
a situation in which the teacher believes in the practice within the context of the given situation, in which case it
would not fall in the Belief Absence Zone.
100 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
A New Conceptual Model of Beliefs-knowledge-Practices for Indonesia’s Teachers
A more likely scenario is one in which teachers feel forced to use practices due to curriculum requirements, school
requirements or other authoritative directives that require a specific practice to be used. The case study teachers
did discuss how the curriculum influences their practices and CS3 expressed a level of frustration that his system
influences limited his freedom to teach in the manner that he felt appropriate. However, the case studies did not
produce a clear example of a teacher operating in the Belief Absence Zone. In a sense, it is a zone that remains
theoretical.
7.6.3 KEY CONJECTURES FROM THE MODEL
The model reflects many important points derived from the analysis, enabling a range of conjectures related to
the interaction of teaching practices, teacher knowledge and teacher beliefs. A few major implications are briefly
discussed below.
There has been an immense amount of research in relation to effective teaching, both in terms of what teacher
background factors characterize effective teachers and of what practices are most effective for generating
higher student learning outcomes. This study has also attempted to explore the relationships between teaching
practices and student learning outcomes and to gauge how knowledge and beliefs relate to student learning outcomes,
although it is important to reiterate that it is the practices that are the conduit to student learning. By stepping back
and viewing effective teaching within this model, what conclusions can be drawn regarding effective teaching?
One key implication of the model is that effective teaching practices are relative rather than absolute, emerging
through their relationship with knowledge and beliefs. What might be described as an effective practice for a
teacher with a high level of knowledge might be ineffective for a teacher with a lower level of knowledge, if using
that practice places the teacher outside his or her Congruent Zone.
For example, many would argue that the practice of provoking and directing questions is more effective
than simply using testing questions. However, would that practice necessarily be effective for all teachers in all
situations? Most likely not, if utilizing the practice places the teacher outside his or her Congruent Zone. If a teacher
does not have an understanding of why provoking/directing questions is an effective method, then the practice
may not fall within the teacher’s belief disc as an effective method. If the teacher lacks the pedagogical knowledge
of how and when to use provoking/directing questions and also lacks the subject matter knowledge to effectively
direct student thinking and then to respond appropriately to students questions, then it falls outside the teacher’s
knowledge disc.
Pushing the teacher to use a practice for which he or she is not equipped with adequate knowledge or in which
he or she does not believe in its effectiveness could in fact result in a deterioration of the learning environment.
The teacher might guide the students in the wrong direction; it could create confusion for both the students and
teacher; the teacher might lack enthusiasm, and; when students provide responses, the teacher may not have the
tools necessary to provide effective responses. The model would imply, therefore, that effective teaching practices
are relative rather than absolute and are dependent on individual teacher characteristics, including knowledge and
beliefs. Teachers who are forced to use practices or approaches that place them outside their Congruent Zone are less
likely to use them effectively. This is particularly true if the practice falls outside both their knowledge and beliefs.
Main Report 101
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A related conclusion is that in order for teachers to adopt a new teaching method it is important to address
not only the “how to” aspect of the practice (knowledge), but also the “why” aspect (beliefs). Often training
gives a basic level of “how to” and provides some theory on “why”, but in order for it to be truly understood and
adopted, the practice must be readily applicable and practical. It also would tend to require a process over a period
of time rather than a one-shot intervention.
Often it is said that teachers who attend a course in a training center will tend to not apply it once they return
to the classroom. Even if they apply it then they often quickly fall back into their previous pattern. An extended
process which involves teachers learning, practicing in their own classroom, reflecting and re-applying is more
likely to build deeper understanding and a belief in its applicability and effectiveness. The process must be extensive
enough that over time the new practice is enveloped within the teachers’ knowledge and beliefs. This is explored in
section 10, which describes a pilot professional development module designed to change teaching practices through
building of knowledge and reshaping of beliefs.
102 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 8
THEMES ON TEACHING
PRACTICES AND POLICIES
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The Technical Report provides a step-by-step analysis for each individual teaching practice. While this is useful
on its own, it becomes more meaningful when considering the practices in a larger context. The following section
presents the most relevant findings in a thematic format. This is followed by a thematic discussion from a policy
perspective. As will be seen, there are overlaps and connections amongst the themes, highlighting the nature of the
interconnections between the different perspectives on what takes place in the classroom; why it takes place; and
how to provide support to teachers to enable them to improve their practices.
8.1 HOW TEACHER KNOWLEDGE IS ENACTED IN THE CLASSROOM
Indonesia’s teachers have low levels of both subject and pedagogical knowledge, as shown by the results of the Ministry
of Education and Culture (MOEC) teacher competency exam, which includes both subject and pedagogical items.
This study and a recent study on certification in Indonesia (de Ree, 2014) make it clear that the level of teacher
subject and pedagogical knowledge is the strongest educational factor in student learning outcomes. Therefore, only
by addressing the deficits in teacher knowledge will student learning outcomes start to improve.
Figure 36: Distribution of Scores of Mathematics Teachers in UKG
All Secondary 57% 23% 20%
Junior Secondary 43% 28% 29%
Senior Secondary 77% 17% 6%
Vocational Secondary 87% 11% 2%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
1 to 50 51-60 Over 60
Indonesia teachers have very low levels of subject and pedagogical knowledge
The MOEC recently developed a teacher competency assessment (Ujian Kompetensi Guru, or UKG) that includes
items related to both subject matter and pedagogy. To this point, more than one million certified teachers have
taken the assessment. The results for 50,000 mathematics teachers indicate that there is a wide range of competency
levels, but that a very large proportion of teachers have low levels of subject and pedagogical knowledge, with more
than half of all teachers scoring below 50%. The scores at the senior secondary level were particularly low, with 77%
of general school teachers (SMA) and 87% of vocational school teachers (SMK) falling below the 50% mark. If 60%
is considered as a threshold for a base level of competency, very few teachers reached this level. Only 20% obtained
a score over 60%, with just 6% of SMA teachers and 2% of SMK reaching this mark.
104 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Teacher knowledge is crucial in improving student learning outcomes
While there is broad agreement that it is good to have teachers with strong competencies, some believe most teachers
already have sufficient levels. However, two recent studies34 by the MOEC and the World Bank indicate that this
is far from the case. In both studies, teacher subject and pedagogical knowledge emerged as the education system factor
with strongest relationship with student learning outcomes. While the positive relationship is not surprising, the fact
that the relationship was so strong highlights just how essential teacher subject matter and pedagogical knowledge
are in the Indonesian context, particularly since a large proportion of teachers have very low competency scores. The
results indicate that teachers with particularly low levels of subject and pedagogical knowledge produce only small
benefits in terms of student learning outcomes. Improvements in the knowledge of teachers with the lowest levels
could produce the greatest impact in student learning outcomes. However, even for the teachers with relatively high
scores, increases in competency still produce increased learning gains. This result indicates that there is no point
at which teachers reach a sufficient level. Therefore, continuously improving teacher knowledge for teachers of all
levels is beneficial.
Why teacher knowledge plays such an important role in student learning
International research into teacher knowledge has explored its complex nature and multiple dimensions.
Mathematical Knowledge in Teaching (MKiT) has emerged as a commonly used term in research to describe the
specialized nature of the knowledge that mathematics teachers should possess. A typical distinction is to separate
subject matter from pedagogical content knowledge, but further distinctions can also be made. Askew et al. (2003)
break the knowledge down into three specific categories:
• Subject matter knowledge: What teachers know about mathematical concepts and their ability to solve
mathematical problems.
• Knowledge of students: What teachers know about students’ knowledge of mathematics, including typical
misconceptions students have, how to assess student knowledge and an understanding of their own students’
ability levels and the varied ways they best learn mathematics.
• Knowledge of teaching: What teachers know about how to teach the subject of mathematics, including the
knowledge of planning and enacting mathematics effectively in the classroom.
How higher teacher knowledge plays a role in Indonesia’s classrooms
The Indonesia TIMSS35 Video Study explored teaching practices in Indonesia’s eighth grade mathematics classrooms
through extensive quantitative and qualitative analysis. The study goes beyond just identifying the relationship
between teacher competency and student learning outcomes. It also explores how different teachers use their
knowledge in the classroom and how this makes a difference to student learning outcomes. The study found that
there are clear differences in the practices of teachers with higher levels of knowledge. These differences include not
only the range of practices used, but, more importantly, the level of effectiveness with which the same practices were
executed.
34
DeRee (2014) Teacher certification and beyond: An empirical evaluation of the teacher certification program and education quality
improvements in Indonesia, World Bank
35 Trends in Mathematics and Science Study.
Main Report 105
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A larger toolbox…
Teachers with higher levels of knowledge tended to utilize a greater range of practices. Techniques such as
investigation, open-ended questioning, the use of mathematical language and symbols, and the use of non-routine
problems and applications were found more often in classrooms in which teachers had higher levels of knowledge.
In part, this appears to be because the techniques themselves require a greater amount of subject mastery to be
conducted effectively. For example, open-ended questioning requires the teacher to develop questions that can
be answered in a variety of ways, with these questions tending to be relatively complex. When in the classroom,
teachers must be prepared for the different possible responses of students and they must have contingencies for the
different responses, which possibly involve higher-order thinking.
…containing better tools
While higher-knowledge teachers tended to have a larger toolbox, they also tended to use the same tools
more effectively. While higher- and lower-knowledge teachers may utilize the same methods and practices, such
as group work, higher-knowledge teachers were more effective. They were able to interact with students more, spot
misconceptions that the group might have, and also come up with better ways to help correct the misconceptions.
Further in-depth qualitative analysis uncovered important differences in the ways higher-knowledge teachers taught
more effectively while using the same practices. Teachers with higher levels of knowledge tended to:
11. Make fewer mathematical errors;
12. Use more accurate language and be more concise in their explanations;
13. Come up with new problems that were not originally in the lesson plan;
14. Make greater attempts to probe student thinking;
15. Be more able to spot student misconceptions and effectively correct them;
16. Involve a greater range of students, including those who might not provide correct answers;
17. Demonstrate more confidence and command in the classroom;
18. Stimulate more teacher-student interaction, with the interaction involving a greater degree of higher order
thinking;
19. Use questioning to a greater extent and encourage students to think and share ideas rather than just to follow
instructions; and
20. Make connections between procedures and concepts.
The above points could play a role in student learning from a variety of angles. The use of more concise language and
the making of fewer errors would likely lead to less confusion and a greater level of understanding among students.
Misconceptions that students hold may be: (i) identified; (ii) probed to determine the cause; and (iii) addressed using
appropriate techniques so that they do not become embedded in the student’s mind and cause future problems and
misunderstanding at later stages of learning. A mastery of both the subject and pedagogical techniques can allow
teachers to run their lessons with greater confidence, possibly leading to a higher level of involvement by students.
Higher levels of pedagogical knowledge can enable a higher level of involvement of students because of the teachers’
better understanding of the thought processes of students. A teacher with higher levels of pedagogical knowledge
will also be better able to address situations that do arise in the classroom. Teachers with higher levels of knowledge
are also able to better utilize problems; set up activities; and provoke higher-order thinking.
106 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
A contrasting approach was found in many Indonesian classrooms
While certainly no two classrooms are the same and no two teachers teach in exactly the same way, Indonesia’s
eighth grade mathematics classrooms exhibit many common features in their teaching practices and follow
similar patterns. In the majority of classrooms, whole-class interaction (as opposed to group work or seat work)
predominantly involves one-way teaching practices, with little meaningful student involvement. Often, the cues
provided by teachers for student interaction are simple testing questions, with almost 90% of student responses
consisting of single word answers. Indeed, often these questions are pseudo-questions that involve filling in the
blank (teacher: “centi…” students: “…meter”) or questions where students repeat an answer to a question in chorus
as the teacher points to on the board. Such interaction may serve the purpose of keeping the attention of students,
but it does not provoke deep thinking. By contrast, teachers with higher levels of knowledge tend to play the role of
guide; to use more effective questioning techniques; and to conduct activities conducive to higher-order thinking.
Contrasting approaches in the handling of a mathematical scenario
How might teachers with varying levels of knowledge, given the same situation, respond differently? To determine
an answer to this question, ten teachers were asked to take part in in-depth case studies. They participated in
interviews to probe their level of mathematical knowledge in teaching by discussing a number of hypothetical
mathematical situations that might be encountered in the classroom. Below is an example scenario where a student
had a misconception related to fractions:
Arif was asked by his teacher to make drawing to compare 3/4 and 5/6. Arif made the following drawings:
and claimed that 3/4 and 5/6 are the same amount.
Main Report 107
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The following chart summarizes the general tendencies of teachers with higher levels of knowledge compared to
those with lower levels, although it should not be taken to represent the cases of all teachers.
LOWER-KNOWLEDGE TEACHERS HIGHER-KNOWLEDGE TEACHERS
Identification of the mathematical error
All were able to identify the error, but some had difficulty Tended to not only identify, but also concisely explain the error.
verbalizing it.
Understanding the student’s reasons for the misconception
Some struggled to identify likely reasons. Some connected it to similar difficulties faced by their own
One teacher felt it was not the teacher’s role to “guess or assume” students.
what the student was thinking. All came up with the most likely reason, but some also
attempted to come up with other possible explanations.
If they were in the teacher’s position, how would they approach the situation?
None mentioned attempting to work with Arif to identify why he Some teachers wanted to first ask Arif questions in order
thinks they are the same. to understand why he made the error and the cause of the
misconception.
Tended to jumped into definitions of fractions and mathematical Many used creative tasks involving cognitive conflict to help Arif
explanations. see the error and then construct a new, correct understanding.
This and other scenarios used in the interview revealed important insights into how the multiple dimensions of
teachers’ knowledge of subject, of students, and of how to teach, could be employed in classroom situations. For the
scenarios that were more challenging mathematically, the teachers with lower levels of knowledge tended to struggle
to identify and explain the issues. The teachers’ knowledge of students came out mostly in terms of not only whether
they could attempt to “get into the mind of the student” and thereby to determine possible reasons for the error,
but whether they took steps to come up with questions to probe students’ reasoning and to understand why they
came to the wrong conclusion. Finally, teachers with higher levels of knowledge tended to utilize creative tasks to
help correct errors and misconceptions. They also tended to include the use of cognitive conflict or new examples,
rather than just relying on the existing example.
Policy Implications Related to Teacher Knowledge
Teacher quality is an extremely complex, multi-dimensional concept. Teacher knowledge is certainly only part of
the equation. Still, the strong correlation between teacher competency test scores and student learning outcomes
indicates that it is a relevant indication of teacher quality. Further qualitative analysis indicates that this relationship
is causal, in that the knowledge could be seen to be employed in the classroom practices in ways that would be
likely to produce better learning outcomes. It also indicates that policies supporting and encouraging teachers to
strengthen their subject and pedagogical knowledge are likely to lead to improved student learning outcomes.
Indonesia’s Teacher Professional Management System (TPMS)
The MOEC has undertaken the massive task of developing a Teacher Professional Management System (TPMS)
in which the key elements form an “integrated framework” of requirements. The system makes the teacher directly
accountable for both work performance, through an annual performance-based teacher appraisal (Pemantapan
Kerja Guru, or PKG), and professional development, through continuous professional development (CPD). The
108 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
appraisal procedure requires open discussion and agreement between the teacher and an assessor, and approval by
the principal, if the principal is not acting as the assessor. Both the appraisal and CPD processes are monitored and
supported by supervisors at the school level. Teachers must also pass a minimum standards test in professional and
pedagogic competency (Uji Kompetensi Guru, or UKG) in order to progress up through the professional levels, with
the rate of progression depending on the teacher’s total number of credit points, which is established on the basis of
the results in the PKG and CPD. The UKG is used as a contextual input for the PKG, and the two of them together
should then be used to define the CPD plan of each individual teacher.
Figure 37: Integrated Framework for the Teacher Professional Management System
What a teacher does
Feedback based on observation
Performance
Appraisal
Continuous
Competency Professional
Test Development
What a teacher knows Tools for improvement
Identi cation of knowledge areas for improvement Tailored plan based on identi ed needs
The possible uses of the teacher competency assessment (UKG)
The UKG can serve important purposes in teacher management, including the hiring of teachers, identifying areas of
improvement for incumbent teachers, providing the appropriate support to improve competencies, and rewarding
teachers who demonstrate high levels of competency and a strong rate of improvement.
• Hiring of teachers: The UKG can be used in the hiring process. Only teachers who can demonstrate a threshold
mastery or strong subject and pedagogical knowledge should be hired;
• Professional development support of teachers: The UKG is already incorporated into MOEC’s TPMS. The
results can be used to identify teacher’s strengths and areas where improvements are required. The areas for
improvement can then be addressed through the teacher’s annual professional development and results plan;
Main Report 109
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
• Progression and promotion: The TPMS also uses the UKG results to determine decisions related to a teacher’s
progression and promotion. The thresholds have not yet been established but, if the results are to encourage
teachers to improve their knowledge, the thresholds must be meaningful and teachers must meet the
competency requirements in order to be promoted.
But the effective use of competency testing requires strong political will…
If utilized properly, the UKG can serve as a tool to help teachers identify areas for improvement; to set goals
and targets; and to seek out ways to improve their level of competency. However, to encourage improvements in
levels of competency, the UKG must also serve the purpose of setting thresholds and ensuring that teachers reach
those thresholds. This requires strong political will. However, if implemented, it could play an important role in
improving levels of teacher competencies and generating better student learning outcomes over the long term.
…as well as a reliable and effective instrument
If the UKG is to fulfill these roles, the instrument and the way it is administered must be generally regarded as
a valid, reliable and credible measure of teacher competency. An invalid tool could lead to poor decisions, poor
outcomes and a frustrated teaching workforce. The assessment items must be validated thoroughly and must be
based on subject and pedagogical knowledge that is truly relevant for teachers for each subject and at each school
level. Investing in a valid, reliable and credible instrument will ensure its acceptance by all stakeholders. By contrast,
an instrument that lacks these attributes will create serious credibility issues that could have the opposite of the
intended effect.
The importance of continuous professional development (CPD)
One of the key roles of the UKG and the performance appraisal is to serve as a feedback mechanism. By identifying
areas in which teachers need to improve, they can engage in professional development activities that will address
their deficiencies. This requires teachers to be aware of and to have access to a wide range of materials that are
necessary to address these deficiencies. This would involve the cataloging and mapping of materials available from
teacher training centers, district/city governments, third-party vendors and online resources. Teachers should be
provided with clear support to engage in CPD. They should also receive credit for their efforts, perhaps by being
accredited on the basis of their passing tests or producing specific outputs through the CPD process. Support should
be provided for avenues to enable teachers to participate in CPD, with these avenues including teacher working
groups (KKG/MGMP36); online courses, such as the MOEC’s mathematics education teacher training center
(P4TK Matematika); and courses offered by the Open University (Universitas Terbuka, or UT) and by teacher
training colleges (LPTK). A clear strategy involving the provision of a number of options will be critical for the
TPMS to function optimally and to achieve real improvements in levels of teacher competency and performance.
Teacher working groups at the primary school level are Kelompok Kerja Guru (KKG) and at the secondary school level are Musyawarah
36
Guru Mata Pelajaran (MGMP).
110 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
8.2 TEACHER-STUDENT INTERACTION
One of the key focal areas of the Video Study was on teacher-student interaction, with the findings providing a
new perspective on teaching-learning culture and norms in Indonesia, as well as showing how student-teacher
interaction relates to student learning outcomes. The study shows that the approaches encouraged through the
2013 curriculum tended to have positive relationships with student learning outcomes. However, it also shows that
the type of teacher-student interaction envisioned by the new curriculum was rare. The new curriculum encourages
the increased use of a student-centered learning approach in which the teacher’s role is to serve more as a guide and
facilitator in the process of student learning. Therefore, the challenge will be to enable teachers to facilitate changes
in classroom culture and to assist students to better understand their role within this new student-centered learning
culture. The transition towards higher levels of student interaction and the adoption of measures to encourage
higher-order thinking could shape the next generation, building their confidence to express ideas and ensuring
that they are more able to undertake tasks requiring higher-order thinking. However, in order to achieve this,
teachers will require a great deal of support if they are to be able to effectively utilize the new classroom practices.
Somewhat disturbingly, the trend in the period from 2007 to 2011 indicates that the extent to which teacher-
student interactions take place in whole-class situations is not only low, but actually decreasing. By contrast, the
extent to which lecturing and teacher-centered learning takes place is increasing. In other words, the quality of
teacher-student interaction is actually getting worse.
8.2.1 WHAT IS A STUDENT-CENTERED APPROACH?
When student-centered instruction predominates in the classroom context, students and teachers share the focus,
with much higher levels of student participation. Instead of just listening to the teacher, students and teachers
interact to a much higher degree. Group work is encouraged, discussion and questioning is more prevalent, and
students learn to collaborate and communicate with one another. The perceived benefits of student-centered learning
include the following:
• Students learn important communication skills through more intense interactions with the teacher and other
students;
• Through deeper discussions and the sharing of ideas, students may be able to develop a deeper understanding
of concepts;
• Students learn to direct their own learning process by asking questions and completing tasks independently;
and
• The construction of knowledge for oneself can lead to greater genuine learning and mastery of subject.
Different teaching practices promote the development of different cognitive skills. More traditional, teacher-
centered approaches tend to promote factual knowledge and routine problem solving skills, while student-centered
practices promote reasoning skills. There are benefits and tradeoffs to each of these approaches.
Main Report 111
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
8.2.2 INTERACTION AND RECENT POLICIES ON TEACHING PRACTICES
In 2013, Indonesia implemented a substantial reform to its national curriculum, with this reform involving not
only a change in content, but also a change in emphasis on the specific approaches adopted in the teaching-learning
process, with a far greater emphasis on student-based learning.37 As will be discussed in Section 8.3, the models of
learning emphasized in policies over the past decade are encouraging students to proactively seek out knowledge and
to make independent observations. Students are encouraged to formulate problems (ask), rather than just solving
problems (answer). Students are also taught to work together and collaborate to solve problems to a far greater
degree.
In turn, teachers are encouraged to use classroom practices that foster independent learning; observation; questioning;
processing; making judgments; communicating and creating a more open and student-centered teaching/learning
paradigm (MOEC, 2013). The teacher’s role is to serve more as a guide and facilitator, rather than being the source
of all information. According to this model, interaction and engagement with students becomes central to the
teaching-learning process.
Various studies have indicated that, even with the major reforms to the Indonesian educational system that took place
more than two decades ago, the student-centered approach is relatively uncommon. Somerset (1997) described the
teaching-learning process as almost always involving the use of traditional, teacher-centered approaches. Hendayana
et al. (2003) agree, stating that teachers tend to dominate mathematics lessons, preventing students from participating
to a meaningful degree and in meaningful manner. Through their observation of indignation classrooms, they
found that activities tended to predominantly involve students copying notes and listening to teachers, with very
little interaction between students and with a large number of students appearing to be disengaged.
Fauzan et al. state that the classroom climate generally has the following characteristics: pupils are passive throughout
the lesson; ‘chalk and talk’ is the preferred teaching style; there is an emphasis on factual knowledge; questions tend
to require only single word responses, often provided in chorus; there is a lack of learning questioning; only correct
answers are accepted and acted upon; and whole-class activities often involve writing, with no hands-on work
carried out (2002).
This study provides a more recent picture of what takes place in the classroom, with its two phases in 2007 and
2011 providing the opportunity to spot emerging trends and changes. Following the same analytical and coding
approach to that adopted in the ground-breaking 1999 TIMSS Video Study (Hiebert et al, 2003) also enabled a
cross-country comparison of results.
8.2.3 INTERACTION OF TEACHERS AND STUDENTS
When examining the proportion of time allocated to whole-class activities compared to that allocated to group
and seatwork, Indonesia’s classrooms tend to have a slightly higher than average time allocated to whole-class
interaction, at 66%, with this proportion being in the middle compared to other countries that participated in the
1999 TIMSS Video Study.
37
The 2013 curriculum was initially rolled out to 6,221 schools in 2013, then rolled out nationally for the 2014/2015 school year, but the
rollout was put on hold in order for a more formal evaluation to be conducted. It is expected that refinements will be made to the curriculum
and that it will be rolled out again at a slower pace between 2015 and 2019.
112 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Figure 38: Proportions of Whole-class vs. Group and Seatwork
Hong Kong SAR 80 20
Czech Republic 79 22
United States 68 32
Japan 66 34
Indonesia 64 36
Switzerland 55 45
Australia 52 48
Netherlands 44 55
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Public (whole-class) interaction Private (group work and seatwork) interaction
While Indonesia tends not vary too greatly from other countries when it comes the time allocated to whole-class and
private interaction, a deeper examination reveals that not only is the student-centered learning approach relatively
uncommon, but the use of this approach is actually declining. The average proportion of time allocated to whole-
class activities has increased from 59% in 2007 to 64% in 2011. When private interaction is broken down into
group work and seatwork, it is clear that the increased whole-class time has come at the expense of group work, with
the proportion of time dedicated to this work declining from 21% to 15%, despite the fact that the group work is
considered a cornerstone of student-centered learning.
Figure 39: Whole-class Work is on the Increase, while Group Work is Declining
2007 59 21 20
2011 64 15 21
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Whole class Group work Individual seatwork
Whole Class Group Work Seatwork
Whole-class interaction by type
Whole-class interaction does not necessarily exclude a student-centered approach. Interaction between teachers
and students was broken down into teacher-only, teacher-student, and student-only interactions. Teacher-only
interaction is considered to imply a traditional approach, while teacher-student and student-only interactions
are often associated with progressive and student-centered learning approaches. These divisions also relate to the
teacher’s conception of his or her role as instructor (for skill mastery); explainer (for conceptual understanding); or
facilitator (for confident problem solving) (Kuhs and Ball, 1986).
Main Report 113
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Whole-class interaction time by teachers and students in 2007 and 2011
2007 61 21 18
2011 74 11 15
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Teacher only Teacher and student Student only
Whole Class
Note: The changes between 2007 and 2011 are statistically significant for “teacher only” and “teacher and student.” Whole-class time refers to time when the interaction,
regardless of the participants, is intended for the whole class.
Again, the trend indicates that student-centered approaches are increasingly less commonly utilized. The analysis
shows that a much greater proportion of classroom time was teacher-only time in 2011 than in 2007, with this
proportion increasing from 61% to 74%. This time typically involved expositional or lecture activities. On the other
hand, student involvement in the form of teacher-student interaction, declined from 21% to 11%. A troubling
aspect of this decrease is that the students of teachers who used more teacher-only time tended to have lower
learning outcomes, while those with more teacher and student interaction had higher learning outcomes.38 The
relationship with student learning indicates that the 2013 curriculum’s student-centered learning emphasis may lead
to better learning outcomes, but the proportion and trends indicate that implementation of the new curriculum
will face challenges.
Teacher role in group work and seatwork
In looking at time used for group work and seatwork, another important finding emerged. While there were no
statistically significant differences in student learning outcomes relative to the proportion of time allocated for
group work relative to seatwork, there was a positive relationship between classrooms in which the teacher was
actively monitoring and providing feedback with student learning for both group and seatwork activities. Even
when a student centered approach is adopted, the teacher still has a significant role to play as a guide and facilitator.
38
Student-only time did not have a statistically significant relationship with student learning.
114 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Private (group and seatwork) interaction time in 2011
Group Work Individual Seat Work
Teacher
Students Teacher involvement
only involvement Students 37%
48% 52% only
63%
Group Work Seatwork
Student-teacher interaction is important
The fact that a positive relationship was identified between levels of teacher-student interaction and student learning
outcomes in whole-class, group work and seatwork indicates that active engagement with students, regardless of the
type of activity in which they are involved, is likely to be an effective means of improving these outcomes. It also
indicates that student-centered learning should not be treated as student-only learning. In the Indonesian context,
the involvement of the teachers as guides and facilitators appears to play a crucial role in this regard.
Interaction in terms of words spoken
Average teacher and student words per 50 minutes of public interaction
7,000
6,000 5,798 5,902
5,360 5,452 5,536
5,148
5,000
4,000
3,243
3,000
2,000
1,016 1,018
1,000 766 824 810
509 640
0
Indonesia Netherlands Switzerland Czech Australia Hong Kong United
Republic States
Average Teacher Words Average Student Words
Main Report 115
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Another way to gain an understanding of teacher-student interaction is through the amount of actual speaking that
occurs. Counts of words spoken focused only on full-class interaction, rather than group or individual work, and
were standardized to indicate the number of words spoken over 50 minutes. Indonesian teachers generally spoke
far fewer words than teachers in other countries, with only 3,243 words in an average lesson, compared to a range
from between 5,198 to 5,902 in other countries. The number of words spoken by students was also lower than in
other countries, but the difference was less dramatic, at an average of 509 in Indonesia compared to a range from
between 640 to 1,108 in other countries.
Figure 40: Ratios of Words from the Teacher for Every Word from a Student
Indonesia 11
Hong Kong 16
Netherlands 13
Switzerland 10
Czech Republic 9
Australia 9
United States 8
0 2 4 6 8 10 12 14 16 18
The lower rate of verbal communication by both teachers and students may signal a slower pace, or possibly less
active and engaged participation. Although the number of words spoken by both teachers and students is lower than
in other countries, the ratio of teacher talking time to student talking time shows that in Indonesia, teachers uttered
11 words for every word spoken by students, compared to a range from between 8 to 16 words in other countries.
8.2.4 WHAT TYPE OF INTERACTION REALLY MATTERS?
While the proportion of time spent on different types of interaction can provide a starting point for understanding
what sort of practices may lead to higher learning outcomes, it is what the teacher does in the available time that
really matters. A particularly important practice is the use of questioning in the classroom. Students whose teachers
utilized questioning to a greater extent tended to have better learning outcomes. However, the reality was that in
many classrooms, questioning tended to be superficial. The majority of questions that teachers asked were testing
questions that would typically require only one-word responses and little or no higher order thinking. The teachers
who were most effective tended to use more advanced forms of questioning that would encourage higher order
thinking. The study shows that students of teachers who employed this technique tended to achieve better learning
outcomes. Unfortunately, however, the use of open-ended questions by teachers was quite rare.
116 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Box: The Use of Questioning from the Perspective of the Teacher’s Goals
Questioning can be a powerful instrument in a mathematics teacher’s repertoire of teaching practices, serving various purposes
in defining the dynamics of the classroom and developing student learning. Through interviews and the observation of the case-
study teachers in the Video Study, an interesting picture emerged regarding how teachers conceive of and use questioning.
Teachers said that they used questioning to check understanding (the majority); keep the students’ attention; direct the students’
attention; and understand what students are thinking. They also explained that they called on individual students as a way of
building their self-confidence, although some teachers also sometimes used it as a form of punishment involving the ritual
humiliation of a student for not paying attention. Ainley (1988) developed a framework of questioning based on the teacher’s
goals by which questions were categorized in the following way:
• Pseudo questions: Questions intended to establish an acceptable behavior or a social contract with students
• Testing questions: Questions intended to find out whether students respond correctly
• Genuine questions: Questions intended to seek information (thus, teachers do not know the answers to these types of
questions)
• Direct or provoking questions: Questions intended to provoke students’ thinking by making new connections or clarifying
existing ones and by exploring new areas of mathematical knowledge
MAJORITY VERY RARE
Pseudo Testing Genuine Directing
Questions Questions Questions Questions
Establish an The teacher knows The teacher really does The teacher wants to
acceptable behavior. the answer and not know what the make the student
Questions only wants to check to see student is thinking and think more about
requiring or allowing if the student can wants to know or something or to think
agreement provide it better understand of it in a di erent way
That’s a really interesting Good answer. Now, can
Arewe
Are ready
weready answer, but i don’t know you explain the
2 + 3 = ... ?
tobegin?
to begin? how you got it. Could you implications for solving
explain your logic? with a di erent triangle?
In Indonesia’s classrooms, the use of pseudo and testing questions predominated. However, relatively effective teachers would use
genuine questions and direct or provoking questions to a far greater extent. The latter are considered to be particularly effective
at encouraging higher-order thinking.
That’s a really interesting
answer, but i don’t know
Main Report 117
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Teachers’ explanations of their use of questioning
Teachers were interviewed regarding their practices as they watched their own videos. One teacher had a particularly
profound response to how she saw the role of questioning. She explained that: “Questions provide a window that
let me see into the minds of my students. Their responses give me clues about how to adjust my teaching.” Teachers
also saw questioning as a tool to support specific students. One teacher explained, “I called on Rivan to answer. He
is behind the other students and needs to build his self-confidence. The question was easy and I was pretty sure he
could answer it.” The study also highlighted the difficulty of engaging students in interaction, particularly in whole-
class situations. Although there are exceptions, the Indonesian cultural norm in a whole-class setting tends to be
one in which students are not accustomed to verbalizing responses that require more than one word. Cases where
students spoke up without any prompting from the teacher were extremely rare.
Despite the challenges, there are ultimately benefits of actively engaging students
In order for teachers to transform their teaching practices from ones that involve a teacher-centered approach to
ones that involve student-centered learning, a great deal of continuous training, support and materials will need
to be provided. Teachers will need to change the culture of the classroom, particularly with regards to students’
understanding of their new role within a student-centered learning culture. The Video Study results indicate that
the focus on greater student interaction and the encouragement of higher-order thinking are a move in the right
direction. Such a move could shape the next generation, enabling its members to have greater confidence to express
their ideas; a stronger ability to undertake tasks requiring higher-order thinking; and, in turn, to more effectively
shape Indonesia’s future.
8.3 TEACHING PRACTICES ENCOURAGED OVER THE PAST DECADE
The policies on teaching practices over the past decade have tended to encourage progressive practices that focus on
active learning (as explored in detail in Section 8.2) with higher levels of student interaction and collaboration and
a greater emphasis on reasoning, innovation, creativity and higher order thinking. This approach was emphasized to
an even greater extent in the 2013 curriculum.39 The findings of the Video Study show that many of these practices
have a positive relationship with student learning outcomes. However, the problem is that the Video Study also
finds that there is a very wide gap between current practices and those expected of both students and teachers, with
many of the desired practices rarely found in mathematics classrooms. A holistic approach to supporting teachers
to improve their knowledge and skills will be essential in addressing this gap and could be driven by the MOEC’s
new Teacher Professional Management System (TPMS), which involves a component of continuous professional
development (CPD).
39
The 2013 curriculum was first rolled out to 6,221 schools in August of 2013 and then rolled out nationwide in August 2014 to all schools
across Indonesia. In December 2014 the newly elected administration made the decision to halt the rollout for schools that had implemented
the curriculum less than one year, citing an insufficient evaluation and rushed rollout. The 2013 curriculum continues to be used in the
schools for the initial rollout while the evaluation and plans for re-rolling out to all remaining schools is in process.
118 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
The active learning approach
The active learning approach encourages students be more interactive and collaborative as well as to seek out and
make observations. Students are directed to formulate the problem (ask), not just to solve the problem (answer).
They are expected to think analytically (make a decision), rather than just in a mechanistic manner (routine).
Students are also expected to work together and collaborate in solving problems.
The role of the teacher
This more open and student-centered teaching/learning paradigm requires that the teacher become more of a guide
and facilitator than simply the provider of information through one-way teaching. Classroom practice must foster
independent learning, observation, questioning, processing, making judgments, communicating and creating.
Practices in the context of mathematics
For mathematics education, there has been an emphasis on the use of investigative and practical (hands-on) teaching
approaches with more of a connection to real-world contexts. The lesson activities should encourage higher order
thinking through techniques such as the use of open-ended questioning, non-routine problems, and the application
of various enquiry methods.
A study of practices used by mathematics teachers
The Indonesia TIMSS40 Video Study examined in extensive detail the practices of 200 eighth grade mathematics
teachers. The findings of the study provide useful insights into what currently takes place in Indonesia’s classrooms,
with particular relevance to the 2013 curriculum. The findings indicate that many of the practices encouraged in
the 2013 curriculum have a positive relationship with student learning outcomes, providing evidence that the new
approaches could lead to better learning outcomes in the Indonesian context. However, the study also found that
there is an extensive gap between current practice and what is expected in the 2013 curriculum, with many of the
desired practices rarely found in mathematics classrooms.41 The study also goes beyond what practices teachers
use and explores in detail why teachers use the teaching practices. This gives critical insights into what would be
required to support teachers in actually changing what happens in the classroom.
What happens in Indonesia’s mathematics classrooms?
Classrooms are complex environments in which teachers must make many decisions and deal with students who
learn at different paces and in different ways. Every classroom is different and each teacher faces a unique set of
circumstances. While a summarized snapshot of Indonesia’s classrooms tends to hide this complexity, it is still
extremely useful in understanding what teaching practices are normally used and their relative frequency.
40
Trends in International Mathematics and Science Study (TIMSS).
41
The filming of classrooms took place two years before the 2013 curriculum was introduced, so the differences are unconnected with whether
teachers are now adopting the new curriculum. Rather, it provides a comparison of what practices were being used before the curriculum was
introduced and can also serve as a benchmark for, at some point in the future, determining the extent of changes in practices resulting from
the new curriculum.
Main Report 119
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Interaction: As was highlighted in section 8.2, in Indonesia’s mathematics classrooms, a teacher-centered approach
tends to be the norm. A large proportion of the lessons are dedicated to whole-class time (where the whole class
participates together), while only 15% is made up of group work, which is considered a cornerstone of a student-
centered approach. The 2013 curriculum’s emphasis on collaborative learning would tend to require a greater
amount of group activities.
In whole-class situations (where all the students and the teacher are working together), 74% of lesson time was teacher-
only, generally indicating a strong teacher-centered approach. Only 11% involved teacher-student interaction and
the remaining 15% was student-only. The small amount of teacher-student interaction is an indicator that teachers
tend to avoid taking on the role of guide and facilitator, as required by a more student-centered approach.
Methods: In terms of teaching methods, 61% of mathematics time was devoted to exposition and another 22% to
problem-solving, with discussion (9%) practical work (7%) and investigation (1%) being much less common. The
2013 curriculum for mathematics emphasizes activities such as investigation and practical (real-life) methods, which
are currently very rare.
Figure 41: Proportion of Lesson Time by Methods Used
9 61 1 7 22
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Discussion Exposition Investigation Practical work Problem solving
Mathematical problems and questions: The approach to questions and problems provides an extremely stark
contrast, with the dominant approaches being closed questioning (97% compared with 3% open-ended); routine
problems (96% compared with 4% non-routine); and mathematical language and symbols (93% compared with
7% real-world contexts).
Figure 42: Problem and Question Contexts
Real-world
contexts Open Non-routine
7% 3% 4%
Math
Closed Routine
language
97% 96%
93%
120 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
A gap to fill
These findings from the Video Study demonstrate that teachers will need to teach in a very different style than they
do currently and that there is a large gap to fill between the ideal approach and that to be adopted. In nearly every
case, the practices encouraged through policies promulgated over the past decade are rarely used. The encouraged
student-centered learning approach clearly contrasts to existing practices. While problem-solving was used 22% of
the time, the use of practical work and investigation, which align with the espoused practices, were rare. The use
of open-ended questioning, non-routine problems and real-world contexts were particularly rare. As a whole, the
results indicate that there is a significant gap between the practices actually used and those that are expected to be
used. Since these policies have been in place over the past decade, the fact that they are so rarely found highlights
the extent of the challenge in translating policy into actual classroom practice.
Relationship of practices to student learning
A critical question is whether the approaches encouraged by policy are applicable and effective in the Indonesian
context. The Video Study utilized student pre- and post-testing in order to measure student learning outcomes over
the course of the eighth grade year. The changes in learning outcomes were then correlated with the practices of
the teachers (while controlling for a variety of student, classroom, school and community characteristics) in order
to identify relationships.42 While caution should be taken in interpreting the relationships as being causal, they do
provide a useful entry point for considering how practices relate to learning in the Indonesian context. The results
were then further probed in a qualitative manner to better understand how certain practices are being used by
different teachers and how these practices may benefit student learning outcomes.
Interaction: Practices that involved increased student-teacher interaction and student-centered learning tended to
have a positive relationship with student learning outcomes. Conversely, there was a negative relationship between
teacher-centered activities, such as teacher-only lecturing, with student learning outcomes. There was also a positive
relationship between a higher level of involvement of the teacher in monitoring group work and seatwork activities,
indicating that even in activities in which students are engaging in activities on their own, they benefit from teacher
support. In the Indonesian case, it appears that the teacher still has a critical role to play in monitoring and support,
even in student-centered activities.
• Task time: As would be expected, teachers who devoted a higher proportion of lesson time to the study of
mathematics (rather than on organizational or non-mathematical activities) tended to achieve better student
learning outcomes. Students with teachers dedicating a higher proportion of time to solving problems also
tended to perform better.
• Method: Classrooms in which the investigation method was used were rare, but when these practices were
implemented, there was a positive relationship with student learning outcomes. By contrast, there was a
negative relationship with the extent to which Teacher lecturing and exposition predominated. In the case of
methods, such as problem-solving, practical work and discussion, no clear relationship with student learning
outcomes could be determined.
42
In the analysis multiple student, classroom, teacher, school and community variables were controlled for in order to understand how
practices and learning outcomes are related across contexts.
Main Report 121
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Problem and question context: Students of teachers who utilized open-ended questioning and non-routine problems
to a relatively greater extent tended to achieve better learning outcomes. Interestingly, however, there was a negative
relationship between the extent to which teachers used of real-world contexts (rather than the use of mathematical
language and symbols) with these outcomes.
Teacher knowledge and practices
There was also a positive correlation between the results of the teacher competency tests in mathematics and
pedagogy with student learning outcomes, as well as with the quality of teaching practices. The role of teacher
knowledge is explored as a theme in Section 8.1. Of all the education system factors measured, teacher competency
scores had the strongest positive correlation with student learning outcomes. This indicates the critical importance
of having teachers who have a mastery both of their subject and the associated pedagogical methods. Importantly,
the teachers with higher levels of knowledge also tended to use the practices encouraged over the past decade to
a relatively greater extent. On examination of the videos, it is clear that the practices of these teachers are more
advanced, requiring a higher level of competencies in order to use them, and to use them effectively.
The greater use of student-teacher interaction, open-ended questions, non-routine problems and approaches such
as investigation also require that teachers have a high level of mathematical and pedagogical competencies. In order
to conduct activities involving higher order thinking, the teacher must not only be able to engage in higher order
thinking him/herself, but to lead discussions and guide students to enable them to do the same. While setting routine
problems that require simple answers is an easy process to control, the use of open-ended questions and non-routine
problems creates a greater potential for the class to go in different directions to that planned by the teacher. Teachers
with lower levels of competency tend to be more reluctant to allow themselves to get caught in these situations. A
higher level of interaction with students and activities to probe student thinking and understanding require not only
strong mathematical skills, but also a high level of pedagogical skills.
The Video Study results indicate that those teachers with lower levels of knowledge will have particularly difficulty
utilizing the more advanced practices. Even if they do adopt the practices, they are more likely to struggle to use
them effectively. Not only will teachers require support in learning the new practices and the necessary associated
pedagogical knowledge, but they will need to increase their knowledge of their subject as well.
The findings from the Video Study indicate that there is a positive relationship between many of the practices
encouraged over the past decade and through the 2013 curriculum with student learning outcomes. However, there
is a large gap between the practices used and what the new curriculum expects. The results also indicate there are
barriers that cannot be resolved through quick training.
Change is notoriously difficult
Changing teaching practices is notoriously challenging and this can be seen in previous Indonesia curriculum
reforms. The 1984 curriculum reform was specifically designed for student-centered learning, but this study as well
as others (e.g. Somerset 1997, Hendayana et al., 2003; Karnasih and Soeparno, 1999; Soedjadi, 2000; Fauzan et
al., 2010) indicates that the teacher-centered approach held strong. Teachers take the information and interpret it
in the context of their existing beliefs and environmental situation. With weak or limited training, teachers will be
even less likely to change their practices. With limited support, teachers will often not implement in the intended
manner, possibly with detrimental results.
122 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Effective change requires alignment with teacher knowledge and beliefs
A useful model for understanding teachers’ use of practices is to also consider two major influences on their practices:
their subject and pedagogical knowledge, and their beliefs of how students learn and how to best teach. Teachers will
attempt to align their practices to their knowledge and beliefs. The Video Study results indicate that it is when the
practices are aligned with knowledge and beliefs that teachers are most effective. When they have to utilize practices
for which they do not have sufficient mathematical or pedagogical knowledge, or that do not align with their beliefs
on the teaching-learning process, they will not be effective. For example, teachers forced to use a student-centered
approach where they are a guide and facilitator rather than an instructor will struggle if they do not know how to
use the approach and/or they do not believe this is the way that students learn.
Figure 43: Theoretical Model Showing Alignment of Teaching Practices with Knowledge and Beliefs
PRACTICES
Dissonant Zone
PRACTICES
Congruent Congruent
Zone Zone
KN
OW
KN
S
EF
OW
LE
LI
BE
DG
LE
E
S
DG
EF
E
LI
BE
Teacher using many practices not aligned Teacher whose practices are mostly
with her knowledge set or beliefs aligned with her knowledge set or beliefs
The implications of this model for the 2013 curriculum are crucial. If teachers do not have sufficient knowledge in
the new approaches and they do not believe they are effective approaches, then teachers will do one of two things: (i)
they will either revert back to the practices that do align with their knowledge and beliefs; or (ii) they will continue
to use the practices, but use them ineffectively. This second scenario could be even more detrimental in the short
term than if teachers use their normal practices.
After the initial hurried national rollout of the 2013 curriculum, it has been put on hold while a proper evaluation
can be conducted. The 2013 curriculum has the potential to significantly change what takes place in Indonesia’s
classrooms and to improve student learning outcomes, but the extreme gap between teaching practices found
before the 2013 curriculum was implemented and what is expected of both students and teachers following its
implementation must be bridged. A holistic approach to supporting teachers in improving their knowledge and
skills is essential and can be driven by MOEC’s new Teacher Professional Management System, which involves a
Main Report 123
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
component on Continuous Professional Development. It involves identification of competencies both through
subject competency testing and through classroom observation. These tools can help in identifying where teachers
can improve, including with 2013 requirements, and then support teachers in their improvement.
8.4 THE NATIONAL EXAM AND OTHER INFLUENCES ON TEACHING
PRACTICES
Teaching practices are influenced by a large number of factors. System and policy influences could come in the form
of curriculum changes, textbook changes, new training offered in pre-service or in-service programs and activities
related to what is emphasized in teacher appraisal. Changes may also come through less direct interventions, such as
modifications to the content and level of emphasis of the national exam, influences of teaching peers, or a teaching
methodology that happens to be hot or trending.
However, while educational ministries and other stakeholders often attempt to change what happens in the
classroom, actually achieving this is notoriously difficult. There have been many failed attempts to implement
in-service training programs or policies that emphasize the use of certain approaches or practices. Simply learning
about a new practice or approach is typically not enough for a teacher to change his or her behavior in the classroom.
Education system trends or major reforms may shift the system through multiple influences. Indonesia’s teacher
reform effort and the passage of the 2005 Teacher Law was intended to improve teacher quality, with a cornerstone
feature of the reform being teacher certification.
8.4.1 WHAT DO THE TRENDS TELL US ABOUT TEACHER REFORM AND OTHER INFLUENCING
FACTORS?
Indonesia’s Teacher Reform and Teaching Practices
Unlike an intervention whose specific goal is to encourage the use of specific practices, Indonesia’s reform did not
provide any explicit training that emphasized specific methods. Rather, the reform encouraged teachers to upgrade
their qualifications. Since the reform began, a much larger proportion of the teacher workforce now has a 4-year
degree (an increase from 29% in 2006 to more than 65% in 2012) and has been certified (with nearly half of all
teachers having obtained certification by 2012). Such shifts might possibly reflect a change in what happens in the
classroom. The payment of a bonus that effectively doubles the teacher’s salary following certification could also
have effects on teaching practices by eliminating the need for a second job, increasing motivation, and so on.
A Study to Measure Trends in Teaching Practices
The Indonesia Video Study of TIMSS Eighth Grade mathematics classrooms measured multiple types of classroom
practices, both in terms of frequency and as a proportion of lesson time. While many practices showed little change,
some trends did emerge. Surprisingly, many of the trends went in the opposite direction of what might have been
expected. The following brief presents the trends in the period from 2007 to 2011 and considers why they might
have occurred.
124 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Trends in Interaction
The Video Study explored the interaction between teachers and students in depth in order to understand levels and
types of participation. The first, top layer of interaction considered was whether mathematics time took place in
whole class (all in the classroom participating together), group work (with multiple groups, typically of three to six
students) or individual seatwork (with students conducting tasks on their own). In both phases, the time allocated
to whole class activities took up most of the class time, at 59% in 2007, increasing to 64% in 2011. In comparing
this proportion to the seven countries that participated in the 1999 TIMSS Video Study, Indonesia’s whole class
time was in the middle, with the country average being 63%. Group work made up 21% in 2007, but decreased
to 15% in 2011. This result was somewhat surprising, since the general emphasis in pre-service and in-service
programs is on encouraging encourage group work and collaboration.
Figure 44: Proportion of Mathematics Time by Structure of Interaction
2007 59 21 20
2011 64 15 21
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Whole Class Group Work Individual Seatwork
When breaking down the above three components further into the specific types of teacher-student interactions that
take place, further interesting patterns emerge.
When looking only at the whole-class time (which was 59% in 2007 and 64% in 2011), the trend clearly indicates
an increase of teacher only time from 61% to 74%. The increase came mainly at the expense of a large decrease in
teacher and student interaction, which fell from 21% to 11%.
Figure 45: Proportion of Whole Class Time by Interaction Type
2007 62 38
2011 52 48
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Teacher monitoring/facilitating Students only
When looking at the proportion of group work in terms of whether the teacher either monitors and facilitates
(going from group to group to check progress and possibly provide feedback and guide the groups) rather than just
letting the groups function on their own, the trend is for teachers to do less monitoring and facilitating. While 62%
of time in 2007 involved monitoring and facilitating, this dropped to 52% in 2011.
Main Report 125
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
A less pronounced but similar trend can be found for individual seatwork, where a slight drop from 40% to 37%
was registered.
Are these trends good or bad?
An important point in examining teaching practice proportions is that there is no right or wrong practice. In fact,
teachers will often mix their practices, using different approaches within and across lessons. The “correct” method
is context-specific and will depend on the stage of the lesson, the students’ abilities and other factors. Still, relating
practices to student learning outcomes provides general clues as to what more effective teachers may utilize or what
may generally work well in the Indonesian context.
Proportions were analyzed lesson-by-lesson and compared with pre- and post-test scores given at the beginning
and end of the eighth grade year to measure student learning outcomes. For proportions of whole class, group and
individual seatwork, no statistically significant relationships emerged. Interestingly, however, a clear pattern emerged
among all three sub-layers. In whole-class interaction, there was a statistically significant negative relationship with
teacher-only interaction and a positive relationship with teacher and student interaction. For group work and
individual seatwork, there was a statistically significant relationship with teacher monitoring and facilitating, while
there was a negative relationship with student-only time.
This general result across all sub-layers appears to indicate that students in classrooms where there is more teacher-
student interaction tend to perform better. A teacher-centered approach in which students are less active appears to
be relatively less effective than our more student-centered approaches. This finding has important implications for
the 2013 curriculum, under which student-centered approaches and more active learning are emphasized.
So, in terms of trends, it appears that the extent to which those practices associated with positive learning outcomes
decreased, while those associated with negative outcomes increased. Again, the use of particular practices is contextual
and cannot be easily labeled as good or bad. Still, the general trends combined with their relationships to student
learning outcomes are not promising.
Trend in Teaching Method
Teaching methods were broken down into various types based on the teaching method or activity involved. The
major shift took place in terms of the proportion of time involving exposition or lecturing, which increased from
52% to 51%. The proportion of time involving the practices of discussion, practical work and investigation, on the
other hand, all decreased.
Figure 46: Proportion of Teaching Methods and Approaches Used
2007 13 53 3 10 21
2011 9 61 1 7 22
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Discussion Exposition Investigation Practical work Problem solving
126 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Again, the relationships of these practices present an unappealing picture. There was a negative relationship between
the use of Exposition with student learning outcomes, while there was a positive relationship with the use of
investigation. The proportion of time involving the use of Problem solving, which has a positive relationship with
student learning outcomes, remained unchanged between the two points in time.
Trends in Problem Time and Approach
The proportion of time spent on studying mathematics that was allocated to solving problems relative to non-
problem time (explaining mathematical concepts and other mathematics-related work not specifically involving
problems) decreased significantly over the two points in time, from 76% to 64%. In 2007, this proportion was
already lower than the other countries, but by 2011 proportion, the proportion was 18% less than the average for
other countries.
Practices are interrelated and the increase in time allocated to Exposition tended to involve more lecture time by the
teacher. This in part explains the decline in problem time. Teachers tended to spend more time explaining concepts
in whole-class time, with less time spent on group and individual seatwork also leading to less problem time.
Figure 47: Proportion of Mathematics time as Problem vs. Non-problem time
Indonesia 2007 76 24
Indonesia 2011 64 36
Japan 82 18
Hong Kong 85 15
Czech Republic 85 15
Australia 85 15
Switzerland 87 13
United States 89 11
Netherlands 96 4
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Problem Non-problem
As with the previously presented results, the trend appears to be going in the wrong direction, given that there is a
positive relationship between problem time and student learning outcomes.
Mathematical Problems by Type
When examining the types of problems and the language used, there is little variation between the two points in
time, with problem types being largely routine, closed and using mathematical language rather than non-routine,
open and involving real world contexts (applications) at both points. However, a couple of important trends did
emerge.
Main Report 127
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Routine versus Non-routine problems
While problems could predominantly be classified as routine in 2007, with 88% of problems falling into this
category, by 2011, this had increased to 96%. Time spent on non-routine problems, which would tend to involve a
greater degree of complexity and higher order thinking, only made up 4% of the total.
Figure 48: Proportion of Problem Work as Involving Routine versus Non-Routine Problems
2007 88 12
2011 96 4
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Routine Non-routine
Mathematical Language versus real World Contexts
When classifying problem time in terms of the problem context, the large majority involved mathematical language
and symbols. In 2007, the average time spent on these problems was 89%, increasing to 93% by 2011. These
problems tend to be procedural problems. Problems using real world contexts, on the other hand, were rare, making
up only 7% of the total by 2011.
Figure 49: Proportion of Problem Work as Involving Mathematical Language vs. Real World Problems
2007 89 11
2011 93 7
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mathematical language and symbols Real world
When considering the relationship between these approaches to mathematical problem with student learning
outcomes, mixed results emerge. There was a positive relationship between the use of non-routine problems and
student learning outcomes. This, combined with the fact that problems of this type would tend to be more complex
and involve a greater degree of higher order thinking, indicate that the trend is going in an undesirable direction.
On the other hand, there was a negative relationship between the use of real world contexts and student learning
outcomes. In this case, the increase in the use of mathematical language and symbols would be a positive trend.
However, this goes counter to MOEC policies and training activities, which tend to encourage the use of real-world
contexts (Contextual Teaching and Learning).
128 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Themes on Teaching Practices and Policies
Interpretation of this category is particularly challenging and raises fundamental issues when consideration is given
to what is tested. One important consideration relates to the test used to measure student learning outcomes. The
items were drawn from the item bank for the national examination and also included TIMSS, which both tend
to emphasize procedural problems. While a few of the problems used in the pre and post tests involved real world
contexts, the majority were procedural and could therefore favor students who were exposed to such problems to a
greater extent.
When examining learning outcomes involving only the real world problems in the test, the students whose teachers
tended to use real world problems in their lessons performed better, although the result did not reach the level of
statistical significance. At the very least, however, they did not perform less well. Thus, it is important to consider
what is emphasized on the test in terms of what sort of outcomes might be obtained.
Would Certification Explain the Trends?
The general hypothesis posed at the beginning of the study was that the teacher reform could impact what happens
in the classroom and that this could be seen through trends in teaching practices. The teaching practices of certified
and uncertified teachers may differ because of either varying levels of teacher motivation or fundamental underlying
differences between the two groups. Certified teachers may alter their teaching practices because of the higher
motivation provided through the incentive than increased salary, an increased in status, or an increased sense of
obligation. Such teachers might also dedicate more time to teaching because they do not feel the need for a second
job, which could conceivably lead to changes in the way teachers plan and execute their lessons.
Figure 50: Differences in Teaching Practices of Certified Teachers vs. Uncertified Teachers in Indonesia TIMSS Video
Study of Mathematics for the 2011 Phase
Public: teacher and student
Private (group or seatwork)
Mathematical organization
Question and answer
Public: student only
Public: teacher only
Public (whole class)
Nonmathematical
Problem solving
Math language
Practical work
Mathematical
New concept
Investigation
Nonproblem
Assessment
Nonroutine
Discussion
Real world
Exposition
Rhetorical
True-false
Problem
Practice
Routine
Review
Closed
Open
4
Direction and level of statistical signi cance
Thresholds of statistical signi cance
3
1%
2 5%
10%
1
0
(1)
10%
(2) 5%
1%
(3)
(4)
Note: Bars that pass the 5% threshold are considered to be statistically significant.
Main Report 129
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
In the 2011 sample, 53% of teachers were certified, providing a nice balance for comparative purposes. However, in
looking at teaching practices, the striking feature is the lack of statistically significant differences between certified
and uncertified teachers. Figure 51 shows results in which the teaching practices were placed one at a time into a
regression model that controlled for multiple contextual variables. Only one difference emerged: certified teachers
tended to use whole-class interaction to a relatively greater extent and private (group and seatwork) interaction to
a relatively lesser extent. It appears that certification could in part explain the trend towards increased whole-class
time, but would not explain any other trends.
These results appear to indicate that certification itself did not have an impact on teaching practices and behavior.
Considering that the reform did not emphasize a specific change in teaching approach, this lack of difference should
not be considered particularly surprising. Almost every teacher who went through the certification process actually
obtained their certificate, so there was no separation of teachers during the process in terms of quality.43 Although
approximately two-thirds of the teachers took an additional 90-hour course, such training would be highly unlikely
to alter teaching practices in any measurable way. The certification study found that certified teachers were less likely
to hold a second job, but it did not identify any other differences in teacher behavior such as number of teaching
hours, levels of absenteeism, or student outcomes. The Video Study accords with those results by further indicating
that there are no discernible differences in the practices of the two groups.
What are the Implications for the 2013 Curriculum Reform?
These problem context and approach categories bring up interesting aspects related to testing, raising questions
regarding what is truly being measured and how teachers might change their teaching practices to better assist their
students to succeed in high-stake assessments, such as Indonesia’s national examination. Teachers seem to have
moved towards more routine, formulaic, and possibly rote learning approaches. These actions could be considered
“teaching to the test.” The increase in mathematical language and the use of routine approaches and closed problems
and questions may have to do with an increased emphasis on performing well in the national examination, where
questions typically require a routine approach in their solution.
There was a positive relationship between teachers’ self-reported influence of the national examination and their
use of routine problems, procedures and mathematical language contexts. Teachers were asked whether the national
examination influenced their teaching. Those who said they were more influenced by the national examination
also tended to use routine problems, closed problems, and mathematical language contexts, which is evidence that
teachers see these practices as being an effective means of preparing students for this examination. In addition, the
percentage of teachers who said they were highly influenced by the national examination increased from 70% to
75% in the period from 2007 to 2011, which indicates the national examination is increasingly influencing how
teachers prepare and implement their lessons.
This finding raises an important contradiction. Many educationalists argue that the teaching approach encapsulated
in these routine, closed, and formulaic techniques does not promote true understanding of mathematics. Still, if
teachers are “teaching to the test” and these approaches tend to help students score better, is it not a wise strategy?
Could it, in fact, lead to better learning, albeit more through a mechanistic, rote-learning approach? The results
regarding non-routine and open approaches are an indication that the less mechanistic approach could, in fact, lead
to better learning, but it is used infrequently.
43
The queuing criteria such as experience and educational level did mean that differences do exist between certified and uncertified teachers,
but these factors were controlled for in the analysis.
130 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 9
OVERARCHING
CONCLUSIONS FROM
THE STUDY FINDINGS
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The Video Study provides many insights into the intricacies of the teaching-learning process, generating significant
insights what takes place in Indonesia’s mathematics classrooms, why it takes place, and how it relates to student
learning outcomes. The following are the overarching messages and conclusions from this study:
1. Many of the practices encouraged through policies promulgated by the MOEC over the past decade, and
particularly through the 2013 curriculum, have a positive relationship with student learning outcomes,
indicating that proper implementation of these approaches should improve student learning.
2. However, teachers rarely use these practices, indicating that simply promulgating policies does not lead to
teachers utilizing the practices implied by those policies. Rather, it takes a major effort to achieve change in
terms of methods and practices. Teachers will need strong support if they are to understand and utilize the
practices described by the policies properly.
3. The strong, positive correlation between teacher subject and pedagogical knowledge on the one hand
and student learning outcomes on the other, indicates that this knowledge is a critical element in teacher
effectiveness. However, the national teacher competency test indicates that in general, levels of teacher subject
and pedagogical knowledge are very low. Support to in-service teachers to improve their knowledge and
targeted hiring to ensure that only teachers with a certain level of competency are hired would improve the
quality of the teacher workforce.
4. Teacher-student interaction is an important element of the student learning process. Students in classrooms
with a greater level of teacher-student interaction achieve better learning outcomes. This holds true whether
the students are involved in whole-class, group work or individual seatwork activities. There tends to be a
negative correlation between the extent to which teacher centered practices such as teacher-only talking time
are used and student learning outcomes.
5. The most effective teachers in Indonesia tended to have a flexible and adaptable beliefs structure rather
than falling into a single orientation. Generally, teachers who had a more transmissionist belief system,
where the role of the teacher is to be the source of knowledge and traditional practices of lecturing and rote
learning, also tended to have lower learning outcomes of their students. Teachers who tended to have a more
connectionist belief system, where the teacher’s role as facilitator and student-centered learning approaches
are dominant, tended to have higher learning outcomes of their students. But the most striking finding on
beliefs was that teachers who had a flexible belief system and saw the need to adapt teaching practices to
each unique context were the teachers whose students tended to have the highest learning outcomes. These
teachers saw the need for more traditional methods in some instances and more progressive methods in other
instances. These teachers tended to use a wider range of practices in their lessons. They were highly aware of
their own students’ needs and attempted to tailor their lessons in accordance with those needs.
6. In the period from 2007 to 2011, there has been a decline in the extent to which many practices that have
a positive relationship with student learning outcomes are utilized. The National Exam and the curriculum
have had an increased influence on the choice of teachers in classroom practices and appear at least to some
extent to be driving the negative trends.
132 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Overarching Conclusions from the Study Findings
7. According to the interactive model of beliefs-knowledge-practices produced through this study, teachers are
most effective when their teaching practices align with their levels of mathematical knowledge in teaching
(MKiT) and mathematical beliefs, so that they are operating in the “congruence zone”. They are least effective
when they are pushed to use practices that are not aligned with their knowledge or beliefs, or when they are
operating in the “dissonance zone”. This has two important implications. First, the concept of “best practices”
is relative rather than absolute. What would be considered the best teaching approach may not be the same
for all teachers. Second, if policies encourage and require teachers to use specific practices, such as active
learning, these policies must be accompanied with the provision of professional development support that
helps teachers to build the knowledge required to effectively implement these practices and to understand
their merit so that it develops and engrains a belief in teachers that it is an appropriate practice.
8. On the basis of this study, it can be seen that teachers need support to develop skills in the following priority
areas:
• Engaging students actively in the learning process;
• Paying attention to the language used;
• Paying attention to the accuracy of the mathematical content (e.g. What is volume? What is 3D space?);
• Enriching teachers’ roles. Teachers need to move from direct teaching to posing “strategic questions”; to
listening to students carefully; and to eliciting students’ understanding;
• shifting the emphasis of their teaching from a very procedural pattern, to a more investigative and
meaningful pattern;
• Incorporating non-routine tasks into lessons to make these tasks more challenging and more productive
of higher-order thinking.
9. Teachers require support if they are to change their practices. Results from a pilot geometry module
(presented in the next section) implemented through KKG/MGMP indicate that continual support through
the provision of “in-on-in” activities and practical applications appear to be an effective way to change teacher
beliefs; to improve levels of teacher knowledge; and to change practices in the classroom.
Main Report 133
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
134 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
CHAPTER 10
TURNING STUDY FINDINGS
INTO PRACTICE: VIDEO STUDY
RESULTS CONVERTED INTO A
CPD MODULE
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The insights gained through the Video Study highlighted the general importance of strengthening both teacher
subject knowledge and pedagogical knowledge. It also identified specific areas where teaching practices should be
enhanced. The study has also become particularly relevant in the context of the 2013 curriculum, which emphasizes
teacher practices such as discovery learning, problem-based teaching and higher order thinking.
A natural extension of the study was to incorporate the lessons learned into modules that teachers could use in their
Continuous Professional Development (CPD) activities. More than 600 hours of video footage captured activities
conducted in natural classroom time, providing useful real-life examples of what teachers do in their classroom. This
unique and invaluable data provided a perfect platform upon which to develop teacher professional development
activities.
10.1 USING KEY STUDY FINDINGS TO IDENTIFY AREAS FOR
IMPROVEMENT
The impetus for the creation of a module came from the findings of the Video Study and the desire to utilize
these findings to better practice. Through an understanding of relationships with student learning outcomes
and by witnessing how teachers use the practices through in-depth analysis of the videos, the key challenges and
opportunities were identified.
The challenge(s): Indonesian teachers typically possess low levels of mathematics subject knowledge which impacts on what
they teach and the way they teach. Teachers spend considerable time introducing new concepts; less time on practicing the
concepts; and very little time on assessment and reflection. They are very teacher-centered in their approach to teaching. They
rarely situate mathematics within real-life situations and they tend to use routine problems to demonstrate concepts, and hence
provoke minimal higher order thinking, which is usually associated with solving non-routine problems. Teachers often use
language that does not accurately convey mathematical concepts. Indonesian teachers rely on question/answer type questions,
where the intent of the questioning is to test if a student knows the answer, rather than initiating discussion or seeking information
related to student thinking.
The opportunities: Video is a powerful medium for promoting discussion and discourse. It has the distinctive ability to capture the
power and intricacy of classroom practice, since it encourages teachers to engage in comparative reflection; descriptive reflection;
and critical reflection. It also affords opportunities to analyze teaching from different perspectives, including from the teacher, the
student, the content (discipline) and the interactions among participants. Teachers are able to gain new insights into their own
teaching practice by both experiencing the challenges of colleagues and identifying new ideas for their own practice, such as
questioning techniques and new activities (Zhang et al., 2011). In order to overcome the challenges faced by Indonesian teachers,
new levels of professional awareness and theorizing need to be established to encourage critical thinking and personal theorizing.
The findings of the video study and the desire to put these findings into practice served as the impetus for creation
of the module, which focused on developing the skills described in the introduction to this section.
10.2 AN INNOVATIVE APPROACH TO CPD THROUGH THE USE OF VIDEO
The central goal of the study was to determine the level of success of a professional development initiative aimed
at enhancing teaching and learning opportunities for teacher educators, classroom teachers and their students in
Indonesian schools. The professional development initiative was designed around the use of classroom video data,
136 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module
which was used as a catalyst to promote new pedagogical practices. The video data, and related learning framework
and tools, was designed to encourage classroom teachers to reflect more deeply on their practices. This involved the
establishment of teacher working groups to develop new levels of professional engagement. The main objective was
to conduct a comprehensive analysis of a pilot that will determine the effectiveness of a new approach to promote
new and contextually innovative teaching practices.
10.2.1 IN-ON-IN APPROACH
With the stated goal of achieving behavioral change, the module was designed with an in-on-in approach, in
which teachers first engage in learning activities in a working group session. Following this they are assigned with
KEY DESIGN PRINCIPLES FOR BEHAVIORAL CHANGE IN TEACHING PRACTICES
Behavioral change is most likely to occur when:
1. Activities to facilitate this change are conducted over an extended period of time;
2. There is an iterative cycle of learning, practice, reflection and modification;
3. There are opportunities to engage in discussions with peers;
4. The content and subject of the activities relate directly and are relevant to the teacher’s classroom context;
5. The content is practical and has practical applications, rather than being purely theoretical;
6. The approach strengthens both the teacher’s knowledge and beliefs in relation to the practices;
7. Activities relate to both subject matter and pedagogical content, so that the teacher strengthens both simultaneously.
“homework” tasks to complete in their own classrooms. In the subsequent working group session, teachers share
their experiences and reflect on what worked, what didn’t work, and how they could do better in the future.
The in-on-in approach dictates that what is learned has to be practical and to have practical applications, rather than
being purely theoretical. Through the learning process, both subject matter and pedagogical aspects are addressed
simultaneously through the activities in order to build teacher knowledge and to enable them to apply this
knowledge in the pedagogical context of the classroom. Teachers must be provided with reasons and a rationale for
the activities they engage in, with measures to develop a foundational understanding that helps convince teachers
that the practices are useful and beneficial.
10.2.2 ELPSA FRAMEWORK
The ELPSA (Experiences, Language, Pictures, Symbols and Application) learning framework is embedded in the
design of the module. The ELPSA framework is built on the basis of constructivist and social learning theories.
The ELPSA framework views learning as an active process in which students construct their own ways of knowing
and developing understanding through both individual reflection and through social interactions with others. It
should be emphasized that form of learning defined by the ELPSA is not a linear process. Learning is a complex
and unpredictable process that does not take place in a linear sequence. Thus, the various elements of the model
should be considered as interrelated and overlapping. While the components of the ELPSA framework can be
discussed individually, these components cannot be implemented in isolation. Rather, the various components are
incorporated and intertwined with each other throughout the lesson.
Main Report 137
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Figure 51: Example of ELPSA for the Topic of Right Triangles
Experience Students construct
right-angled triangles
to scale and label
Language
Terminology including:
right angle, hypotenuse, 900 Pictures
5
reinforced 3
4
c
a
Students encounter triangles that
b
do not have a right angle as an
experience to move toward
trigonometry
Formula c2 = b2 + a2
reinforced
Application
Students are given task to
determine the area of the oor Symbols
in their classroom Introduction of formula for area
of right-angled triangle
A = 1/2 bh
10.2.3 INCORPORATION OF VIDEOS INTO THE MODULE
One of the advantages of developing the module on the basis of the video study is that those designing the module
could draw upon more than 600 hours of footage containing real-life examples of classroom practices. Example clips
were incorporated into the session activities for tools to stimulate discussion. These examples were not necessarily
intended to portray best practices, but rather to serve as stimulants to provoke discussion on specific issues and
subjects.
10.2.4 FACILITATORS AS AN INTEGRAL PART OF SESSION ACTIVITIES
The least costly method of delivering CPD through teacher working groups would be to design the module so that it
could be utilized without any facilitation. If this approach were adopted, the materials would need to be completely
self-explanatory. In addition, teachers would need to have a sufficient level of motivation to conduct the activities
in a fairly self-directed manner. While this option was considered, it would have severely limited the complexity of
the activities included in the module. Also, it would have few if any regulators to control the quality and consistency
of delivery. It was decided that the module could be developed much more effectively if it involved the selection
of two members within the working group to serve facilitators and leaders, together with measures to develop the
capacities of the selected two facilitators.
138 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module
10.3 IMPLEMENTATION IN TEACHER WORKING GROUPS
Teacher working groups were selected as the venue for the delivery of the module. These groups were selected for
this purpose primarily on the basis of the benefits of their use from the point of view of both effectiveness and
sustainability. This section briefly examines the theory and core concepts of teacher professional development in
order to provide a context for the implementation of the module.
Teacher working groups, known in Indonesian as Kelompok Kerja Guru (KKG) when they involve primary school
teachers, and Musyawarah Guru Mata Pelajaran (MGMP) when they involve secondary school teachers, are clusters
of teachers within a defined geographical area. For secondary school working groups, the number of schools within a
cluster tend to range from three to six, and involve an average of approximately 20 teachers. In the secondary school
groups, the groups are subject specific, so, for example, a single group may consist of 20 mathematics teachers. The
frequency of meetings varies, but they tend to be held every second week.
Teacher working groups are intended to provide a forum for teachers to discuss issues and share ideas. They are also
intended to be vehicles for professional development activities. Teacher working groups provide advantages over
more traditional professional development in-service training. Working groups can provide a very different type of
learning and professional development experience. They have an advantage over traditional in-service training in
that:
• Training activities can be performed on a regular basis (every two weeks rather than once a year);
• Regular meetings allow for the reinforcement of concepts and multiple iterations of training and professional
development activities;
• The structure of training involves a “for teachers, by teachers” approach, which has been shown to be the
most successful method for professional development;
• Training activities can be more easily modified to meet the needs of a local context, particularly since all
teachers come from the same general location;
• Teachers form a support system upon which they can rely on and to which they have easy access;
• The teacher working group methodology conforms to principles of adult learning, in that:
–– Teachers are able to structure learning around their interests and needs;
–– Teachers are able to apply life experience and prior knowledge to the learning situation;
–– Teachers are able to be actively involved in the learning process;
–– The learning style can be structured to adapt to the needs of working group members;
–– Teachers are able to be self-directed;
–– The working group provides an affiliation for its members;
• Learning activities tend to be participatory and experiential.
Main Report 139
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
10.4 IMPACT ON TEACHER BELIEFS, KNOWLEDGE AND PRACTICES
The quantitative analysis examined three areas: (i) teacher beliefs; (ii) teacher knowledge; and (iii) teaching practices.
For each of these areas, data was gathered before the module began and after it ended.
• Beliefs: A survey was used that asked teachers to rate statements on a scale of 1 to 5 in their level of agreement.
The items covered: (i) teaching mathematics; (ii) the process of learning mathematics; and (iii) learning
activities.
• Knowledge: Teachers were asked to participate in pre and post-tests. The tests were different, but contained
the same general question types and contained anchor items (which appeared on both the pre and post
tests) and parallel items (which were similar, but were changed so that the answer would not be the same).
It also contained groupings of questions, as were either directly related, indirectly related or unrelated to
what was learned in the course module. For example, open-ended questioning was one of the main topics of
the module. Thus, the tests contained items that would test teachers’ ability to create open-ended questions
and answer open-ended questions. This would be considered directly related topics of the module. Items
that are indirectly related touched slightly upon what was learned in the module, while indirect items had
no relationship to the module. The mix of these questions enables an analysis to determine the direct and
indirect impacts of the module, while also containing control questions, which should not change as a result
of the intervention.
• Practices: Students were asked to participate in a practices survey to rate on a scale of 1 to 5 the extent to
which certain practices took place or to the extent to which their teacher uses techniques or be described in
a certain manner.
For all of these instruments, the pre result was compared to the post-result to determine whether the behavior of the
treatment group had changed in relation to the control group and, if so, to what extent. Another end goal is to gain
an understanding of the relationship between beliefs-knowledge-practices.
• Teacher Beliefs: Treatment teachers showed a statistically significant increase in their positive responses
to statements regarding hands-on activities; inquiry-oriented activities; performance-based assessment; and
the use of informal questioning to assess student understanding. All areas except the performance-based
assessment were part of the pilot module. A statistically significant increase was recorded amongst teachers in
the treatment group for statements related to students participating in student-led discussions and working
in cooperative learning groups, as well as for statements related to students working on extended mathematics
investigations or projects. Again, these items relate to what was emphasized in the module.
• Teacher Knowledge: The treatment group had a clear and significant increase in the teacher competency
assessment, indicating that the module accomplished the goal of simultaneously developing both teacher subject
knowledge and teacher pedagogical practices. Of particular interest was that teachers were much more able to
address items related to open-ended questions. This improvement in terms of teachers being more comfortable
with, and able to correctly answer, open-ended items is an important achievement of the module.
• Teaching Practices: The results appear to indicate that teachers in the treatment group changed their
practices in ways that students interpreted to be related to: (i) asking students to explain their answers
and explain what they think; (ii) giving students time to explain their ideas and respecting students’ ideas
and suggestions; (iii) expecting the students to work through the hard problems; and (iv) providing more
feedback and using multiple ways to explain the topics. These suggest that the module was effective in
increasing student involvement and in encouraging teachers to use questioning and rich problems.
140 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module
The quantitative analysis indicates that the module was successful in changing beliefs; improving teacher subject
knowledge; and, most importantly, changing behavior in the classroom.
The qualitative data gathered through observations, interviews and surveys also provided many insights into the
effectiveness of the module, both in terms of strengths and deficiencies and areas where improvements could be
made. There was clearly an appreciation on the part of participants and facilitators for the content of the module.
The ELPSA framework was very well received, with teachers expressing the belief that it was very relevant and helpful
for them in transforming their classroom practices. The sessions on the use of questioning in the classroom and of
hands-on experience were also particularly well received. Challenges also arose and many lessons were learned.
10.5 KEY LESSONS LEARNED AND CONSIDERATIONS
Many lessons were learned through the implementation of the module in terms of its content, implementation
and effectiveness. These lessons learned also raise questions regarding the provision of continuous professional
development in general. The following is a summary of the key lessons and considerations, as well as a discussion of
sustainability in terms of scaling up and expanding the practices covered in the module in the area of mathematics
and for other subjects.
CONTENT
The content of the modules was generally very well received. There were almost no comments expressing disagreement
with the module’s approach or topics and there was near universal agreement that the content was beneficial. In
general, participants expressed the belief that the ELPSA framework and the topics of the module were highly
relevant to their own practice.
There were mixed responses regarding the level of difficulty of the content. Some participants indicated that the
concepts and approaches were not entirely new, but that they still found it valuable to develop their ability to
implement them in the classroom. Other teachers indicated that the content was challenging and felt that they
did not acquire a sufficient level of knowledge to be confident about implementing the new practices in their
own classroom. These teachers tended to regard the approaches as valuable, but indicated that they required more
support if they were to implement them effectively.
Some teachers also indicated that they would like to go into more depth on some of the topics. In particular, some
teachers felt that it would be beneficial to spend more time on the topic of questioning and ways in which a culture
of questioning could be created in the classroom. On the basis of this feedback, it might be desirable to consider
expanding the module to include a second session on questioning. Another approach would be to develop a separate
module consisting of three sessions that could go into much greater depth on this topic.
The issue of the time required both to conduct the working group sessions and to implement the sample lesson plan
also arose. It appears that in some of the sessions, the content was too dense and could not be completed effectively
within the 3-4 hour duration of the working group sessions. This led to some activities not being conducted or
being rushed. Revisions to the module should reduce the content to ensure that all activities can be implemented
appropriately.
Main Report 141
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
IMPLEMENTATION
Using Working Groups for the Delivery of CPD
Teacher working groups have both advantages and disadvantages as a system of delivery for CPD. Some specific
issues that the pilot revealed that included the following:
• The level of attendance at working groups can vary significantly: While in some working groups, teachers
from all schools in the cluster attend, others rotate teachers within the school. In many districts, there is a
policy of having a day dedicated to working groups for particular subjects. For example, working groups
for mathematics teachers may be conducted on Thursdays, so no mathematics classes are scheduled for
that day. However, in Jakarta, this is not the case, so teachers need to find time in their schedules to attend.
This negatively affected the level of attendance. Attendance was also affected by conflicting obligations, as
discussed in the next point.
• A module implemented through a working group may be affected by teachers’ conflicting obligations:
Teachers have many conflicting obligations, being required to engage in training on the new curriculum, to
prepare for national examinations, and to conduct many other activities. This creates significant challenges in
terms of maintaining the attendance of participants in working groups throughout the semester.
• Teachers have different interests and needs, which may not all be addressed through a single module:
While some topics may be of interest to most or all teachers, different teachers will have different professional
development interests and needs. It may be worth considering adopting a certain degree of flexibility to
enable teachers to engage only in the specific activities that meet their needs or to enable multiple sub-groups
to conduct separate activities/modules.
• The quality of the facilitator played a critical role in ensuring the effectiveness of working group
activities: This point is discussed in greater detail below.
• The support provided to working groups is not consistent across different schools and districts: School
and educational agency authorities have varying perceptions of the degree to which working groups are the
responsibility either of the district, the school, or the teachers themselves. One district commented that
working groups are out of their jurisdiction and that the district could not oblige teachers to attend working
groups.
Importance of Facilitators
At the design stage, consideration was given to the idea of developing stand-alone modules that would not require
the training and use of facilitators, but could be used in a self-directed manner by teachers. While this approach
would be less expensive and could be more easily and widely distributed to working groups, the content itself would
have had to be simplified and include even more detailed step-by-step instructions. There would have also been a
greater reliance on the motivation of the members of the working groups to implement the module on their own
and to engage in all the associated activities. Through the piloting of the module, it became clear that the decision
to train and deploy facilitators was correct, with facilitators playing a vital role in ensuring the effectiveness of
the module. Training and deploying facilitators provided the necessary leadership to conduct the activities and to
facilitate an understanding of the content. It also allowed for the establishment of a support network through which
facilitators were able to share their experiences and pose questions with facilitators in other working groups.
142 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module
In addition, the level of commitment and understanding of the facilitators were important in determining how
effectively the sessions were conducted. In some cases, facilitators did not have a sufficient level of understanding of
the materials. In these cases, it was found that the sessions were much less effective. This indicates the importance of
facilitator training, underscoring the need to allow sufficient time to go through all key activities. The most effective
sessions tended to be those in which simulations were conducted during the training process. This indicates that
additional practice and a greater use of simulations in the training process would be beneficial.
Training for facilitators should be more practical. Facilitators should engage in simulation sessions so they clearly understand
their role in facilitating MGMP meetings, especially in providing input for the development and implementation of lesson
plans. Facilitators have to prepare themselves beforehand so they understand the topic and are able to manage the meetings
well. It would be good if facilitators could provide some other examples to enrich the modules.
Responsibility for the Provision of CPD
One challenge created by the decentralization of the responsibility for the provision of educational services to
district governments is that continuous professional development is now the responsibility of these governments,
through district-level educational agencies. The capacity of districts to provide high quality CPD varies, but is
generally very low. How local governments perceive their role in the provision of CPD also varies. In the case of
the pilot, the study team was able to meet with district-level officials and propose that teachers receive credit for
successful completion of the module. But how could this process be scaled up? Under current arrangements, if a
working group or set of teachers were to want to conduct the module, they would need to reach an agreement with
the district to receive credit. Alternatively, should CPD be a supply-driven process in which districts offer the CPD
activity to the working groups? Should it be voluntary or obligatory for all teachers to participate?
The answer to these questions is beyond the scope of this study. However, challenges in the implementation of
the module did underscore the need to clarify the role of the MOEC, provincial and district level agencies, and
of schools and teachers in the provision of CPD, particularly in terms of how teachers will receive credit for their
participation. For the delivery of CPD through online courses, such as courses offered by P4TK Mathematics, it
appears that there is already a mechanism for teachers to receive credit. However, it is important for the rules and
processes to be clearly defined.
Educational agencies, school authorities and other stakeholders should support and opportunities for teachers to attend the
MGMP activities and to create opportunities for them to do so. At present, teachers need a letter of permission from their
principal to attend MGMP meetings. Teachers’ PKB activities should be mandatory and scheduled.
EFFECTIVENESS
The results of the impact evaluation indicate that the module led to statistically significant changes in teacher
beliefs, improvements in teacher knowledge, and changes in the classroom. The qualitative results also indicate that
teachers valued the professional development activities and felt that they benefited from these activities. Input from
the facilitators, teachers and observers have factors to identify factors that lead to effective implementation, but also
indicate that there were aspects that hindered the effectiveness of the module.
Main Report 143
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Working group meetings tend to be most effective when these conditions are present: (i) the topics meet with the
needs of the participants; (ii) the facilitators prepare and perform the activities well; (iii) learning tools, stationeries,
laptops, projector, and other facilities are available; (iv) all participants are actively involved in discussions, questioning,
engaging in practice, and exploring through trial and error; (v) the activities can be effectively implemented within
the allocated time (180-240 minutes).
A key area for revision relates to be content load, given the time constraints that apply to working group sessions.
In particular, the content of the module has to be adjusted to achieve a better fit with the time available so that
the teacher learning process is more effective. Most RPP analysis, which was intended to be conducted during the
sessions, had to be assigned as additional homework because of the lack of time. Each module required 3-4 hours
of intensive activities, with 15 minutes break each session. Regular MGMP meetings usually run for four hours,
but with a slower pace of activities. Specifically, Session 2, on questioning, could be conducted over two meetings,
since the activities required more time than most others. Session 5, on enquiry learning, could also be conducted
over two meetings.
SCALING UP AND SUSTAINABILITY
There is significant interest in CPD modules of this type. Teachers who participated in the pilot were generally very
enthusiastic and requested additional support. Teachers in the control group were also very interested in utilizing
the module in their working group in the future. Many of the educational agencies in the participating districts are
developing plans to support the rollout of the module to the rest of the MGMP in their district. In dissemination
events with district officials and teachers, there were many requests for support. However, despite this enthusiasm,
scaling up the implementation of the module requires careful consideration.
10.6 WIDER POLICY IMPLICATIONS EMERGING FROM THE PILOT
Policies related to teaching practices over the past decade have tended to emphasize more modern, student-centered
teaching approaches, with these approaches involving a higher level of student interaction and a more intense
focus on reasoning and higher order thinking. For example, Regulation 19/2005 states: “The process of learning
in the educational unit should be organized to be interactive, inspiring, fun, challenging, motivating learners to
actively participate and provide enough space for innovation, creativity, and independence in accordance with their
talents, interests, and physical and psychological development of learners.” A similar statement was also included in
Regulation 32/2013. The video study results indicate that eighth grade mathematics teachers tend to use traditional
methods and that the approaches tend to be procedural, rather than encouraging reasoning and higher order
thinking.
Importantly, the results of the video study indicate that when teachers used more interactive methods that encourage
reasoning, students tend to have better learning outcomes. This shows that there are benefits to the teaching methods
that have begun to be encouraged over the past decade. However, it is also clear that in mathematics classrooms (and
likely with other subjects), there is a gap between the methods that are encouraged and methods that are actually
put into practice. How can teachers incorporate more progressive methods into their arsenal of practices? Teachers
clearly require support, and there are no quick fix solutions. Changing practices is notoriously difficult and requires
long-term, continuous, and intensive support.
144 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Turning Study Findings into Practice: Video Study Results Converted into a CPD Module
This pilot was specifically designed to strengthen teacher knowledge and to improve teaching practices in the areas
that encourage greater student interaction and that utilize inquiry-based methods that promote student reasoning.
The results indicate that the approach used in this model changed teacher beliefs: improved teacher knowledge; and
led to changes in practices used in the classroom. Although the results were positive, a module of this type must be
seen merely as a seed for the development of a much larger support system. Teachers will need a much higher level of
support provided on an ongoing basis in order to fully build and strengthen their subject and pedagogical knowledge.
It is clear that the system for the provision of CPD at present is not working effectively in Indonesia. Since
decentralization, teachers have received less CPD and the quality and relevance of the offerings has declined. While
district governments are responsible for the provision of CPD, the majority do not have sufficient capacity or the
willingness to provide opportunities for effective CPD to teachers. In some cases, local governments do not even
necessarily consider it their obligation to provide CPD. Even for those that do see it as their obligation, in many
cases, the offerings tend to be haphazard, of low quality and of limited relevance. Very often, they consist of short
workshops in which a speaker is brought in to give a presentation. While teachers receive credit for participation in
these workshops, they often fail to meet the needs of teachers and have little impact on classroom practices. At least
in the short-term, it is not realistic to expect local governments to be able to develop high quality CPD activities.
In addition, a larger framework is required to give structure to CPD as a whole, rather than having patchy, poorly
integrated offerings.
The P4TK Mathematics centers do tend to offer good courses and programs, but these benefit only a small number
of teachers each year. With a country as large and diverse as Indonesia, in-house P4TK training at their centers for
the majority of teachers is simply not a feasible large-scale solution. Online methods may provide a way to reach a
larger number of teachers, with the pilot geometry module serving as an example of how online methods may be
effectively utilized. This methodology certainly has good prospects, but it cannot be relied upon as the only method
for the provision of CPD.
Teacher working groups offer many advantages, as specified in detail in this report. Among other advantages, they
can be a very effective means of building skills through the in-on-in process, which ensures that what is learned can
be applied in the classroom and then reflected upon with fellow teachers. However, there are many challenges to the
use of teacher working groups as a large-scale method of providing CPD. Working groups function very differently
from district to district, or even from school to school. This makes it difficult to develop and implement a program
consistently. It may also necessitate the development of high quality modules that can be run by teachers themselves.
The development of the geometry module used in this pilot is required extensive time and resources. However, the
payoff is that it can have a significant positive impact in terms of improving the level of teachers’ subject knowledge
and pedagogical knowledge. Most importantly, it may result in real positive changes to what occurs in the classroom.
There can be no single method for the provision of CPD. A holistic approach, with many different offerings that
utilize a range of different methodologies, is necessary. Within Indonesia’s teacher professional management system,
individual teachers are required to develop an annual CPD plan. To achieve this effectively, they should have a
catalog of offerings that can be used to address their specific needs. This requires a system that provides many
options under a holistic dynamic pedagogies framework. This CPD module has planted a seed for the professional
development of mathematics teachers. The approach adopted through the module will be continued through a
project in NTB under a DFAT Global Partnership for Development (GPFD) grant. However, such approaches
need to be applied for teachers of other subjects and for every level of schooling, from early childhood through
senior secondary.
Main Report 145
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
The central government is best positioned to develop the necessary framework and to produce high quality activities
that can be implemented across Indonesia. However, ultimately, it is districts that are responsible for the provision
of CPD. In this capacity, they must provide support for their teachers by establishing mechanisms that will give
access to and credit for CPD activities. None of this is easy, requiring inputs and support from many stakeholders.
However, it is critically important if Indonesian students are to benefit from improved learning outcomes. For more
than a decade, Indonesian teachers have been provided with very little support and only limited opportunities
to develop the skills necessary to improve Indonesia’s human capital. The level and quality of teachers’ subject
knowledge and pedagogical knowledge is still generally very low, which ultimately impacts the ability of Indonesian
students to grow up and become productive citizens. It is vitally important for the nation’s future to address this.
146 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
REFERENCES
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Abelson, R.P. (1979). Differences between belief and knowledge systems. Cognitive Science, 3 (4), pp. 355-366.
Ainley, J. (1988). Perceptions of teachers’ questioning styles. Proc. 12th Conf. of the Int. Group for the Psychology of
Mathematics Education, pp. 92.
Ainley, J. (1987). Telling questions. Mathematics Teaching, 118, pp. 24-26.
Ainley, M. (2006). Connecting with learning: Motivation, affect and cognition in interest processes. Educational
Psychology Review, 18 (4), pp. 391-405.
Ali, R.M. (2007). Teacher talk in mathematics classrooms: Questioning to establish procedural competence.
Proceedings of the Second International Conference on Science and Mathematics Education, pp. 342.
Anderson, D.S. and Piazza, J.A. (1996). Changing Beliefs: Teaching and Learning Mathematics in Constructivist
Preservice Classrooms. Action in Teacher Education, 18 (2), pp. 51-62.
Andrews, P. (2010). Comparing Hungarian and English mathematics teachers’ professional motivations. In V.
Durand-Guerrier, S. Soury-Lavergne & F. Arzarello (Eds.), Proceedings of the Sixth European Conference on
Research in Mathematics Education (CERME 6, January 2009), (pp. 2452-2462). Lyon, France: Institut
National de Recherche Pedagogique.
Andrews, P. (2009). Comparative studies of mathematics teachers’ observable learning objectives: validating low
inference codes. Educational Studies in Mathematics, 71 (2), pp. 97-122.
Andrews, P. (2007a). Mathematics teacher typologies or nationally located patterns of behaviour. International
Journal of Educational Research 46, no. 5: 306-18.
Andrews, P. (2007b). The curricular importance of mathematics: A comparison of English and Hungarian teachers’
espoused beliefs. Journal of Curriculum Studies 39, no. 3: 317-38.
Andrews, P. and Hatch, G. (2000). A comparison of Hungarian and English teachers’ conceptions of mathematics
and its teaching. Educational Studies in Mathematics, 43 (1), pp. 31-64.
Arvold, B. and Albright, M. (1995). Tensions and Struggles: Prospective Secondary Mathematics Teachers
Confronting the Unfamiliar.
Askew, M., Brown, M., Rhodes, V., Johnson, D. and Wiliam, D. (1997). Effective teachers of numeracy. London,
King’s College London.
Atweh, B.D. and Abadi (2012). Investigating Teachers’ Pedagogical Beliefs in Indonesia and Australia, The Asia-
Pacific Education Researcher, Vol 21, No 2
Aubrey, C. (1993). An investigation of the mathematical knowledge and competencies which young children bring
into school. British Educational Research Journal, 19 (1), pp. 27-41.
148 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Ajzen, I. and Fishbein, M. (1980). Understanding attitudes and predicting social behaviour. Prentice-Hall Englewood
Cliffs, NJ.
Ball, D.L. (1991). Research on teaching mathematics: Making subject matter knowledge part of the equation.
Advances in research on teaching, 2, pp. 1-48.
Ball, D.L. (1991). Research on teaching mathematics: Making subject matter knowledge part of the equation.
Advances in research on teaching, 2, pp. 1-48.
Ball, D.L. (1990). Prospective elementary and secondary teachers’ understanding of division. Journal for Research in
Mathematics Education, , pp. 132-144.
Ball, D.L. (1988). Unlearning to teach mathematics. For the Learning of Mathematics, , pp. 40-48.
Ball, D.L., Lubienski, S.T. and Mewborn, D.S. (2001). Research on teaching mathematics: The unsolved problem
of teachers’ mathematical knowledge. Handbook of research on teaching, 4, pp. 433-456.
Ball, L., Thames, M. and Phelps, G. (2008). Content Knowledge for Teaching: What Makes It Special? Journal of
Teacher Education, 59 (5), pp. 389.
Barkatsas, A. and Malone, J. (2005). A Typology of mathematics teachers’ beliefs about teaching and learning
mathematics and instructional practices. Mathematics Education Research Journal, 17 (2), 69-90.
Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., Klusmann, U., Krauss, S., Neubrand, M.
and Tsai, Y.M. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student
progress. American Educational Research Journal, 47 (1), pp. 133.
Bennett, N., Summers, M. and Askew, M. (1994). Knowledge for teaching and teaching performance. Look before
you leap, pp. 23-36.
Bereiter, C., Scardamalia, M., De Corte, E., Verschaffel, L., Entwistle, N. and van Merriënboer, J. (2003). Powerful
Learning Environments: Unravelling Basic Components and Dimensions.
Berk, R.A. (1988). Fifty reasons why student achievement gain does not mean teacher effectiveness. Journal of
Personnel Evaluation in Education, 1 (4), pp. 345-363.
Berkowitz, S. (1997). Analyzing qualitative data. In J. Frechtling and L. Sharp (Eds.), User-friendly handbook for
mixed method evaluations. Arlington, VA: Division of Research, Evaluation and Communication, National
Science Foundation. Retrieved February 10, 2013, from http://www.nsf.gov/pubs/1997/nsf97153/start.htm
Beswick, K. (2012). Teachers’ beliefs about school mathematics and mathematicians’ mathematics and their
relationship to practice. Educational Studies in Mathematics, 79(1), 127-147.
Beswick, K. (2008). Middle School Mathematics Teachers’ Knowledge for Teaching Percent. 11th Annual Congress on
Mathematical Education, Mexico City, Mexico.
Main Report 149
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Beswick, K. (2007). Teachers’ beliefs that matter in secondary mathematics classrooms. Educational Studies in
Mathematics, 65 (1), 95-120.
Beswick, K. (2005). The beliefs/practice connection in broadly defined contexts. Mathematics Education Research
Journal, 17 (2), 39-68.
Bjork, C. (2005). Indonesian education: Teachers, schools, and central bureaucracy. Routledge.
Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33
(8), pp. 3-15.
Blum, W., Baumert, J., Neubrand, M., Krauss, S., Brunner, M., Jordan, A. and Kunter, M. (2005). COAKTIV–A
Project for Measuring and Improving the Professional Expertise of Mathematics Teachers. 15th ICMI
Conference on “The Professional Education and Development of Teachers of Mathematics”(2005, 15-20 May,
Brazil).
Boaler, J. (1998). Open and closed mathematics: Student experiences and understandings. Journal for research in
mathematics education, pp. 41-62.
Borich, G.D. (1996). Educational testing and measurement: Classroom application and practice. Harper Collins.
Brophy, J. and Good, T.L. (1986). Teacher behaviour and student achievement. Handbook of research on teaching,
3, pp. 328-375.
Brown, D.F. and Rose, T.D. (1995). Self-Reported Classroom Impact of Teachers. Action in Teacher Education, 17
(1), pp. 20-29.
Bruns, B. and Luque, J. (2014), Great Teachers: How to Raise Student Learning in Latin America and the Caribbean.
World Bank, Washington, D. C.
Butt, R.L. and Raymond, D. (1989). Studying the nature and development of teachers’ knowledge using collaborative
autobiography. International Journal of Educational Research, 13 (4), pp. 403-419.
Calderhead, J. (1996). Teachers: Beliefs and knowledge.
Carroll, J.B. (1963). A Model of School Learning. Teachers College Record 64 (8): 723–33.
Chan, C. C. and Wong, A. K. C. (1991). Knowledge Discovery in Data Bases: 107-123.
Clark, C. and Peterson, P. (1986). Teachers’ thought processes’ in MC Wittrock (ed.): Handbook of Research on
Teaching.
Clarke, D., Keitel, C. and Shimizu, Y. (2006). The learners’ perspective study. D.Clarke, C.Keitel and Y.Shimizu
(eds.) Mathematics Classrooms in Twelve Countries: The Insider’s Perspective. Rotterdam: Sense Publishers
150 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Clarke, N. (2011). With good reason?: students with low prior attainment reasoning in classroom mathematical activity,
DPhil Thesis, University of Oxford
Cobb, P., Wood, T. and Yackel, E. (1990). Classrooms as learning environments for teachers and researchers. In R.
B. Davis, C. A. Maher & N. Noddings (Eds.), Constructivist views on the teaching and learning of mathematics
(pp. 125–146). Reston, VA: National Council of Teachers of Mathematics.
Cohen, D. (1990). A revolution in one classroom: The case of Mrs. Oublier. Educational Evaluation and Policy
Analysis, 12(3), 311—329.
Cooney, T.J., Shealy, B.E. and Arvold, B. (1998). Conceptualizing belief structures of preservice secondary
mathematics teachers. Journal for Research in Mathematics Education, 29 (3), pp. 306-333.
Correa, C. A., Perry, M., Sims, L. M., Miller, K. F. and Fang, G. (2008). Connected and culturally embedded
beliefs: Chinese and US teachers talk about how their students best learn mathematics. Teaching and Teacher
Education, 24(1), pp. 140-153.
Davis, B. and Simmt, E. (2006). Mathematics-for-teaching: An ongoing investigation of the mathematics that
teachers (need to) know. Educational Studies in Mathematics, 61 (3), pp. 293-319.
Davis, B. and Simmt, E. (2003). Understanding learning systems: Mathematics education and complexity science.
Journal for Research in Mathematics Education, 34 (2), pp. 137-167.
Denscombe, M. (2008). Communities of Practice A Research Paradigm for the Mixed Methods Approach. Journal
of mixed methods research, 2(3), 270-283.
Dewey, J. (1938). Education and experience. Touchstone, New York, NY.
Dictionary, O. E. (1989). OED online. Oxford University Press. http://www. oed. com, Accessed Dec, 1, 2012.
Doyle, W. (1990). Classroom Management Techniques.” In Student Discipline Strategies: Research and Practice,
edited by O. C. Moles, 113–27. New York: New York University Press.
Dunkin, M.J. (1976). Problems in the accumulation of process-product evidence in classroom research. Journal of
Education for Teaching, 2 (2), pp. 175-187.
Eisenhart, M., Borko, H., Underhill, R., Brown, C., Jones, D. and Agard, P. (1993). Conceptual knowledge falls
through the cracks: Complexities of learning to teach mathematics for understanding. Journal for Research in
Mathematics Education, 24 (1), pp. 8-40.
da Ponte, J. P. (1994). Mathematics teachers’ professional knowledge. In J. P. da Ponte and J. F. Matos (eds),
Proceedings of the 18th International Conference for the Psychology of Mathematics Education, Vol. 1 (Lisbon:
University of Lisbon, Program Committee of the 18th PME Conference), 195–210. ERIC ED 383 537.
Main Report 151
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Dickinson, P. (2000). Teachers ‘questions. Mathematics Teaching, 171, pp. 4-7.
Ernest, P. (1991). The philosophy of mathematics education. Falmer.
Ernest, P. (1989). The impact of beliefs on the teaching of mathematics. Mathematics teaching: The state of the art,
pp. 249-254.
Ernest, P. (1989). The knowledge, beliefs and attitudes of the mathematics teacher: A model. Journal of Education
for Teaching, 15 (1), pp. 13-33.
Fang, Z. (1996). A review of research on teacher beliefs and practices. Educational Research, 38 (1), pp. 47-65.
Fauzan, A. (2002). Applying Realistic Mathematics Education (RME) in teaching geometry in Indonesian primary
schools. Thesis, University of Twente.
Fennema, E., Carpenter, T. P., Franke, M. L., Levi, L., Jacobs, V. R. and Empson, S. B. (1996). A longitudinal
study of learning to use children’s thinking in mathematics instruction. Journal for Research in Mathematics
Education, 27, 403-434.
Fennema, E. and Franke, M. L. (1992). Teachers’ knowledge and its impact. Chapter in Handbook of research on
mathematics teaching and learning: A project of the National Council of Teachers of Mathematics. , (pp. 147-
164). New York, NY, England: Macmillan Publishing Co, Inc, xi, 771 pp.
Filmer, D. (2006). A Millennium Learning Goal: Measuring Real Progress in Education. CGD Working Paper 97.
Fisher, C. W., D. C. Berliner, N. N. Fully, R. S. Marliave, L. S. Cahen, and M. M. Dishaw (1980). “Teaching
Behaviors, Academic Learning Time, and Student Achievement: An Overview.” In Time to Learn, edited by
C. Denham and A. Lieberman, 7–32. Washington, DC: National Institute of Education.
Fleischman, Howard L., et al. (2010). Highlights from PISA 2009: Performance of US 15-Year-Old Students in
Reading, Mathematics, and Science Literacy in an International Context. NCES 2011-004. National Center
for Education Statistics
Goldin, G. (2003). Affect, meta-affect, and mathematical belief structures. Beliefs: a hidden variable in mathematics
education?, pp. 59-72.
Gondoseputro, T. T. (1999). The cross-cultural perspective of teachers’ beliefs and their influence on teaching
practices: A case study of two teachers teaching secondary mathematics in Australia and Indonesia. In Making
the difference: Proceedings of the Twenty-second Annual Conference of The Mathematics Education Research Group
of Australasia Incorporated (pp. 494-501).
Greeno, J.G., Collins, A.M. and Resnick, L.B. (1996). Cognition and learning. Handbook of educational psychology,
pp. 15-46.
152 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Grossman, P.L. and Stodolsky, S.S. (1995). Content as context: The role of school subjects in secondary school
teaching. Educational Researcher, 24 (8), pp. 5.
Guba, E.G. and Lincoln, Y.S. (1989). Fourth generation evaluation. Sage Publications, Inc.
Guskey, T.R. (1986). Staff development and the process of teacher change. Educational Researcher, 15(5), pp. 5-12.
Handal, B. (2003). Teachers’ mathematical beliefs: A review. The Mathematics Educator, 13 (2), pp. 47-57.
Handal, B. (2002). Teachers’ beliefs and gender, faculty position, teaching socio-Economic area, teaching experience
and academic qualifications. Proceedings of the 2002 International Biannual Conference of the UWS Self
Research Centre, pp. 1–6.
Hanushek, E. A., & Rivkin, S. G. (2010). Generalizations about using value-added measures of teacher quality. The
American Economic Review, 267-271.
Hanushek, E. A., & Rivkin, S. G. (2010). Using Value-Added Measures of Teacher Quality. Brief 9. National Center
for Analysis of Longitudinal Data in Education Research.
Harbison, R.W. and Hanushek, E.A. (1992). Educational performance of the poor: lessons from rural Northeast Brazil.
World Bank, Office of the Publisher, 1818 H Street, NW, Washington, DC 20433
Hattie, J. (2003). Teachers make a difference: What is the research evidence? Research conference of the Australian
Council for Educational Research, Melbourne, VIC, Australia. Retrieved May, pp. 2007.
Hattie, J. (1999). Influences on student learning. Inaugural lecture given on August, 2, pp. 1999
Hiebert, J. (1997). Making Sense: Teaching and Learning Mathematics with Understanding. Heinemann, 361 Hanover
Street, Portsmouth, NH 03801-3912.
Hiebert, J. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 Video Study. DIANE
Publishing.
Hiebert, J., Gallimore, R., Garnier, H., Givvin, K.B., Hollingsworth, H., Jacobs, J., Chiu, A., Wearne, D., Smith,
M. and Kersting, N. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video
study (NCES 2003-013). US Department of Education. Washington, DC: National Center for Education
Statistics.
Hiebert, J. and Stigler, J.W. (2000). A proposal for improving classroom teaching: Lessons from the TIMSS video
study. The Elementary School Journal, pp. 3-20.
Hill, H.C., Schilling, S.G. and Ball, D.L. (2007). Developing measures of teachers’ mathematics knowledge for
teaching. The Elementary School Journal, 105 (1).
Main Report 153
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Hill, H.C., Rowan, B. and Ball, D.L. (2005). Effects of teachers’ mathematical knowledge for teaching on student
achievement. American Educational Research Journal, 42 (2), pp. 371.
Hill, H.C., Schilling, S.G. and Ball, D.L. (2007). Developing measures of teachers’ mathematics knowledge for
teaching. The Elementary School Journal, 105 (1).
Howe, K. R. (1988). Against the quantitative-qualitative incompatibility thesis, or, Dogmas die hard. Educational
Researcher, 17, 10–16.
Hoyles, C. (1992). Mathematics teaching and mathematics teachers: A meta-case study. For the Learning of
Mathematics, 12(3), 32-44.
Jalil, A. (2011). Comments made in an informal discussion during a case study visit for this research.
Kagan, D.M. (1992). Implication of research on teacher belief. Educational psychologist, 27 (1), pp. 65-90.
Karnasih, I and Soeparno. (1999). Teaching mathematics has to focus on logic. Kompas, Indonesia, May 17, 2000.
Knipping, C. (2003). Learning from comparing: A review and reflection on qualitative oriented comparisons of
teaching and learning mathematics in different countries. ZDM, 35(6), 282-293.
Kounin, J. S. (1970). Observing and Delineating Technique of Managing Behavior in Classrooms. Journal of
Research and Development in Education 4 (1): 62–72.
Krauss, S., Baumert, J. and Blum, W. (2008). Secondary mathematics teachers’ pedagogical content knowledge and
content knowledge: validation of the COACTIV constructs. ZDM, 40 (5), pp. 873-892.
Kuhs, T.M. and Ball, D.L. (1986). Approaches to teaching mathematics: Mapping the domains of knowledge,
skills, and dispositions. East Lansing: Michigan State University, Center on Teacher Education.
Lerman, S. (2002). Cultural, discursive psychology: A sociocultural approach to studying the teaching and learning
of mathematics. Learning discourse, pp. 87-113.
Lerman, S. (2001). A review of research perspectives on mathematics teacher education. Making sense of mathematics
teacher education, pp. 33-52.
Lerman, S. (2000). The social turn in mathematics education research. Multiple perspectives on mathematics teaching
and learning, pp. 19-44.
Lerman, S. (1990). Alternative perspectives of the nature of mathematics and their influence on the teaching of
mathematics. British Educational Research Journal, 16 (1), pp. 53-61.
Lester, F. (2003). Implications of research on students’ beliefs for classroom practice. Beliefs: a hidden variable in
mathematics education?, pp. 345-353.
154 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Leung Frederick K.S. and Ragatz, A. (2010). Inside Indonesia’s Mathematics Classrooms: A TIMSS video study of
teaching practices and student achievement, World Bank, Washington D.C.
Lindmeier, A. and Ufer, S. (2010). Modelling and Measuring Components of Mathematics Teacher Knowledge
and Competencies. Proceedings of the 34th Annual Conference of the International Group for the Psychology of
Mathematics Education. (Vol. 3, pp. 209-216). Belo Horizonte, Brazil: PME.
Johnson, R. B. and Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has
come. Educational researcher, 33(7), 14-26.
Ma, L. (1999). Knowing and teaching elementary mathematics: Teachers’ understanding of fundamental mathematics in
China and the United States. Lawrence Erlbaum Associates.
Mullis, I. V., Martin, M. O., Foy, P. and Arora, A. (2012). TIMSS 2011 international results in mathematics.
Martino, A.M. and Maher, C.A. (1999). Teacher questioning to promote justification and generalization in
mathematics: What research practice has taught us. The Journal of Mathematical Behaviour, 18 (1), pp. 53-78.
Mason, J. (2002). Minding your Qs and Rs: effective questioning and responding in the mathematics classroom.
Moyer, P.S. and Milewicz, E. (2002). Learning to question: Categories of questioning used by preservice teachers
during diagnostic mathematics interviews. Journal of Mathematics Teacher Education, 5 (4), pp. 293-315.
Muijs, D. and Reynolds, D. (2002). Classroom Factors and Student Achievement in Primary Math. American
Education Research Association.
Mullis, I.V.S. (2007). PIRLS 2006 international report: IEA’s progress in international reading literacy study in primary
schools in 40 countries. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston
College.
Mullis, I., Martin, M. and Foy, P. (2008). TIMSS 2007 international mathematics report. Achievement. Journal of
educational psychology, 96 (1), pp. 174.
Pajares, M.F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of educational
research, 62 (3), pp. 307.
Nespor, J. (1987). The role of beliefs in the practice of teaching. Journal of curriculum studies, 19 (4), pp. 317-328.
OECD (2014), Measuring Innovation in Education: A New Perspective, Educational Research and Innovation,
OECD Publishing. http://dx.doi.org/10.1787/9789264215696-en
OECD (2007). PISA 2006 science competencies for tomorrow’s world, Volume 1: Analysis.
Main Report 155
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
O’Neill, G.P. (1988). Teaching Effectiveness: A Review of the Research. Canadian Journal of Education/Revue
canadienne de l’éducation, 13 (1), pp. 162-185.
Op‘t Eynde, P., Corte, E. and Verschaffel, L. (1999). A Socio-constructivist Perspective on the role of Emotions in
Mathematical Problem Solving. 8th EARLI-conference, Göteborg.
Ornstein, A.C. (1976). Can We Define a Good Teacher? Peabody Journal of Education, 53 (3), pp. 201-207.
Pajares, M.F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of educational
research, 62 (3), pp. 307.
Philipp, R.A., Sowder, J.T., Flores, A. and Schappelle, B.P. (1995). A responsible mathematics teacher and the
choices she makes: A case study. Providing a foundation for teaching mathematics in the middle grades, pp.
301-325.
Porter, A.C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31
(7), pp. 3-14.
Pratt, M. L. (1991). Arts of the contact zone, Profession, 91, 3–20.
Ragatz, A. (2013a), Teaching Practices of Indonesia’s Eighth Grade Mathematics Teachers: Formation, Execution and
Effect, Dissertation Thesis, University of Oxford, Department of Education
Ragatz, A. (2013b). Chapter 5 In: Chang, M.C. of The Role of Politics and Evidence-based Policy Making: The Case of
Teacher Reform in Indonesia, World Bank, Washington DC
Ragatz, A. and Chen, D. (2009). Transforming Indonesia’s Teaching Force, World Bank, Washington DC
Raudenbush, S.W. (2004). What are value-added models estimating and what does this imply for statistical practice?
Journal of Educational and Behavioral Statistics, 29 (1), pp. 121-129.
Renne, C. (1992). Elementary school teachers’ views of knowledge pertaining to mathematics. Annual Meeting of
the American Research Association, San Francisco, CA.
Ragatz, A. (2014). Teaching Practices in Indonesia’s 8th Grade Mathematics Classrooms: Formation, Execution and
Effect. Doctoral thesis, University of Oxford.
Rowan, B., Correnti, R. and Miller, R. (2002). What Large-Scale Survey Research Tells Us About Teacher Effects on
Student Achievement: Insights from the Prospects Study of Elementary Schools. The Teachers College Record,
104 (8), pp. 1525-1567.
Rowland, T., Thwaites, A. and Huckstep, P. (2005). The knowledge quartet: A framework for reflection, discussion,
and professional development. 15th ICMI study: The professional education and development of teachers of
mathematics. Retrieved October, 31, pp. 2006.
156 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Sanders, W.L. and Rivers, J.C. (1996). Cumulative and residual effects of teachers on future student academic
achievement. Knoxville: University of Tennessee Value-Added Research and Assessment Center.
Scheerens, J. (1993). Effective schooling: Research, theory and practice. School Effectiveness and School Improvement,
4 (3), pp. 230-235.
Schoenfeld, A.H. (2011). Toward professional development for teachers grounded in a theory of decision making,
ZDM Mathematics Education (2011) 43:457–469
Schoenfeld, A.H. (2010). How We Think: A Theory of Goal-Oriented Decision Making and Its Educational Applications.
Schoenfeld, A.H. (1992). Learning to think mathematically: problem solving, metacognition and sense-making
in mathematics. In D. A. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning (pp.
334-370). New York: MacMillan.
Schoenfeld, A.H. (1989). Explorations of students’ mathematical beliefs and behavior. Journal for Research in
Mathematics Education, 20(4), 338-355.
Schmidt, W.H., Houang, R.T., Cogan, L., Blömeke, S., Tatto, M.T., Hsieh, F.J., Santillan, M., Bankov, K., Han,
S.I. and Cedillo, T. (2008). Opportunity to learn in the preparation of mathematics teachers: its structure
and how it varies across six countries. ZDM, 40 (5), pp. 735-747.
Schwab, J.J. (1978). Science, curriculum and liberal education: Selected essays, Joseph J. J.Schwab. Chicago: University
of Chicago Press.
Schmidt, W.H., Houang, R.T., Cogan, L., Blömeke, S., Tatto, M.T., Hsieh, F.J., Santillan, M., Bankov, K., Han,
S.I. and Cedillo, T. (2008). Opportunity to learn in the preparation of mathematics teachers: its structure
and how it varies across six countries. ZDM, 40 (5), pp. 735-747.
Shulman, L.S. (1999). Knowledge and teaching: Foundations of the new reform. Learners and pedagogy, pp. 61-77.
Shulman, L.S. (1987). Knowledge and teaching. Harvard Educational Review, 57 (1), pp. 1-22.
Shulman, L.S. (1986). Those who understand: Knowledge growth in teaching. Educational researcher, 15 (2), pp. 4.
Skemp, R. (1976). Instrumental understanding and relational understanding. Mathematics Teaching, 77, pp. 20–26.
Skott, J. (2011). Beliefs vs. Patterns of participation–towards coherence in understanding the role the teacher.
Skott, J. (2010). Shifting the direction of belief research: From beliefs to patterns of participation.
Skott, J. (2009). Contextualising the notion of ‘belief enactment’. Journal of Mathematics Teacher Education, 12 (1),
pp. 27-46.
Main Report 157
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Skott, J. (2004). The forced autonomy of mathematics teachers. Educational studies in Mathematics, 55 (1), pp.
227-257.
Skott, J. (2001). The emerging practices of a novice teacher: The roles of his school mathematics images. Journal of
Mathematics Teacher Education, 4(3), 3-28.
Snow, R.E. and Lohman, D.F. (1984). Toward a theory of cognitive aptitude for learning from instruction. Journal
of educational psychology, 76 (3), pp. 347-376.
Soedjadi. (2000). Teaching mathematics has to focus on thinking process. Kompas, Indonesia April 17, 2000.
Somerset, A. (1997). Strengthening quality in Indonesian junior secondary school: An overview of issues and
initiatives. Ministry of Education and Culture, Jakarta.
Srivastava, P. and Hopwood, N. (2009). A practical iterative framework for qualitative data analysis. International
journal of qualitative methods, 8 (1), pp. 76-84.
Steadman, S. (2005). Practically perfect? Excellence in practice-based educational research. A discussion paper for
seminar 3 in the ESRC TLRP seminar series: Quality in Educational Research. Sussex: University of Sussex.
Stillman, G. (2000). Impact of prior knowledge of task context on approaches to applications tasks. The Journal of
Mathematical Behaviour, 19 (3), pp. 333-361.
Swan, M. (2000). GCSE mathematics in further education: challenging beliefs and practices. Curriculum Journal,
11 (2), pp. 199-223.
Swan, M., (2006). National research and development center for adult literacy and numeracy and National institute
of adult continuing education (England/Wales), Collaborative learning in mathematics: a challenge to our
beliefs and practices. National Institute of Adult Continuing Education.
Tatto, M.T., Schwille, J.S., Senk, S., Ingvarson, L.C., Peck, R. and Rowley, G.L. (2009). Teacher education and
development study in mathematics (TEDS-M): Conceptual framework. Dr. Lawrence Ingvarson, pp. 150.
Thompson, A.G. (1992). Teachers’ beliefs and conceptions: A synthesis of the research. Handbook of research on
mathematics teaching and learning, 127, pp. 146.
Thompson, A.G. (1985). Teachers’ conceptions of mathematics and the teaching of problem solving. In E. A.
Silver (Ed.), Teaching and learning mathematical problem solving: Multiple research perspectives (pp. 281-294).
Hillsdale, NJ: Lawrence Erlbaum.
Thompson, A.G. (1984). The relationship of teachers’ conceptions of mathematics and mathematics teaching to
instructional practice. Educational studies in mathematics, 15 (2), pp. 105-127.
158 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
References
Udin, S. and Johnston, M. (2006). Cross-Cultural Influences on Teacher Education Reform: Reflections on
Implementing the Integrated Curriculum in Indonesia, Journal of Education for Teaching: International
Research and Pedagogy, Vol.32 (1), p.3-20
Vegas, E., and I. Umansky. (2005). “Improving Teaching and Learning through Effective Incentives.” In Incentives to
Improve Teaching: Lessons from Latin America, edited by E. Vegas, 21–62. Washington, DC: World Bank.
Warfield, J., Wood, T. and Lehman, J. D. (2005). Autonomy, beliefs and the learning of elementary mathematics
teachers. Teaching and Teacher Education, 21, 439-456.
Watson, A. (2008a). Developing and deepening mathematical knowledge in teaching: Being and knowing. In
MKiT 6, Nuffield Seminar Series, 18th March, at University of Loughborough.
Watson, A. (2008b). How secondary teachers structure the subject matter of mathematics. Proceedings of the British
Society for Research into Learning Mathematics, 28, pp. 126-131.
Watson, A. (2007). Developing and deepening mathematical knowledge in teaching: being and knowing.
Mathematical Knowledge in Teaching Seminar.
Watson, A. (2002). Working with students on questioning to promote mathematical thinking. Mathematics
Education Review, 15, pp. 32-42.
Watson, A. and Geest, E.D. (2005). Principled teaching for deep progress: Improving mathematical learning beyond
methods and materials. Educational Studies in Mathematics, 58 (2), pp. 209-234.
Watson, A. and Mason, J. (1998). Questions and Prompts for Mathematical Thinking, ATM, Derby.
Weber, R. P. (Ed.) (1990). Basic content analysis (No. 49). Sage.
Weidle, R. and Wagner, A. (1994). Die Methode des Lauten Denkens. - In: Huber, G.; Mandl, H. (Eds.), Verbale
Daten. 2. Aufl. Weinheim: Psychologie Verlags Union, p. 81-103
Wilson, M., Cooney, T., Leder, G., Pehkonen, E. and Toerner, G. (2002). Beliefs: A hidden variable in mathematics
education?
Winne, P.H. and Marx, R.W. (1980). Matching students’ cognitive responses to teaching skills. Journal of educational
psychology, 72 (2), pp. 257-264.
Wragg, E. C. (1993). Multimedia in Education: Bane or Boon?” Journal of Educational Television 19 (2): 73–79.
Main Report 159
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
160 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
APPENDICES
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
While a few key appendices are included in this report, many important pieces of information are captured in
a separate technical report. For more extensive details on the instruments, processes and analysis, please see the
technical report.
CODING LAYERS AND DEFINITIONS
Description Definition
STRUCTURE OF TIME
Mathematical time Time spent on mathematical content presented either through a mathematical problem or outside the
context of a problem, e.g., talking or reading about mathematical ideas, solving mathematical problems,
practicing mathematical procedures, or memorizing mathematical definitions and rules.
Mathematical At least 30 continuous seconds devoted to preparing materials or discussing information related to
organization time mathematics, but not qualifying as mathematical work (e.g., distributing materials used to solve problems,
discussing the grading scheme to be used on a test, distributing a homework assignment).
Non-mathematical time At least 30 continuous seconds devoted to non-mathematical content, e.g., talking about a social function,
disciplining a student while other students wait, or listening to school announcements on a public-
address system.
LESSON SEGMENT PURPOSE
Review This category, more technically called “addressing content introduced in previous lessons,” focused on
the review or reinforcement of content presented. These segments typically involved the practice or
application of a topic learned in a prior lesson, or the review of an idea or procedure learned previously.
Examples include:
• Warm-up problems and games, often presented at the beginning of a lesson;
• Review problems intended to prepare students for the new content;
• Teacher lectures to remind students of previously learned content;
• Checking the answers for previously completed homework problems; and
• Quizzes and grading exercises.
Introducing New This category focused on introducing content that students had not worked on in an earlier lesson.
Content Examples of segments of this type included:
(New content)
• Teacher expositions, demonstrations, and illustrations;
• Teacher and student explorations through solving problems that were different, at least in part, from
problems students had worked previously;
• Class discussions of new content; and
• Reading textbooks and working through new problems privately.
Practice This category focused on practicing or applying content introduced in the current. These segments only
occurred in lessons where new content was introduced. They typically took one of two forms: the practice
or application of a topic already introduced in the lesson, or the follow-up discussion of an idea or formula
after the class engaged in some practice or application. Examples of segments include:
orking on problems to practice or apply ideas or procedures introduced in an earlier lesson;
• W
Class discussions of problem methods and solutions previously presented; and
•
Teacher lectures summarizing or drawing conclusions about the new content presented earlier.
•
Evaluation/Assessment Tasks or questions that are given to students evaluate the knowledge of students (through exam or quiz).
PUBLIC AND PRIVATE INTERACTION
(1) Full class/Public Public presentation by the teacher or one or more students intended for all students
interaction
• Teacher interaction The teacher lectures to all students.
• Teacher and student Presentation made by both teachers and students (in intervals), for all students.
interaction
• Student interaction Presentation made by students, aimed at the teacher and all students
162 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Appendices
Description Definition
(2) Individual/Private All students work at their seats, either individually, in pairs, or in small groups, while the teacher often
interaction circulates around the room and interacts privately with individual students
a. Teacher and group Students work in groups or have discussions with the teacher going from group to group to provide
interaction guidance.
b. Group interaction Students work in groups or have discussions without teacher’s guidance.
c. Teacher and Teachers provide individual counseling to students.
individual
interaction
d. Student interaction Each student works alone with no interaction with the teacher.
(3) Student presents A student presents information publicly in written form, sometimes accompanied by verbal interaction
information between the student and the teacher or other students about the written work; other students may
attend to this information or work on an assignment privately
PROBLEM SOLVING STRATEGY
Exposition The teacher lectures while students listen and answer closed questions (with no discussion).
Discussion The teacher and student or students discuss their own ideas about mathematics.
Problem solving The teacher provides a problem / situation as a basis to discuss ideas in mathematics.
Practical Equipment or situations in the real world are used to get ideas in mathematics.
Investigation Students explore the issues (problems) in various mathematical situations.
PROBLEM VS. NOT PROBLEM
Problem Events that contained a statement asking for some unknown information that could be determined by
applying a mathematical operation.
Simple questions asking for immediately accessible information was not counted as problems. Examples
of mathematical problems included:
• A dding, subtracting, multiplying, and dividing whole numbers, decimals, fractions, percents, and
algebraic expressions;
• Solving equations;
• Measuring lines, areas, volumes, angles;
• Plotting or reading graphs; and
• Applying formulas to solve real-life problems.
Non-problem A non-problem segment is defined to be mathematical work outside the context of a problem. Without
presenting a problem statement, teachers (or students) sometimes engaged in:
• resenting mathematical definitions or concepts and describing their mathematical origins;
P
•
Giving an historical account of a mathematical idea or object;
•
Relating mathematics to situations in the real world;
•
Pointing out relationships among ideas in this lesson and previous lessons;
•
Providing an overview or a summary of the major points of the lesson; and
• Playing mathematical games that did not involve solving mathematical problems (e.g., a word search for
mathematical terms).
METHOD OF PROBLEM SOLVING
Using Procedures Problem statements that suggested the problem was typically solved by applying a procedure or set of
procedures. These include arithmetic with whole numbers, fractions, and decimals, manipulating algebraic
symbols to simplify expressions and solve equations, finding areas and perimeters of simple plane figures,
and so on.
Example: “Solve for x in the equation 2x + 5 = 6 - x” were classified as using procedures.
Stating Concepts Problem statements that called for a mathematical convention or an example of a mathematical concept.
Problem statements.
Examples: “Plot the point (3, 2) on a coordinate plane” or “Draw an isosceles right triangle” were classified
as stating concepts.
Main Report 163
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Description Definition
Make a connection Problem statements that implied the problem would focus on constructing relationships among
mathematical ideas, facts, or procedures. Often, the problem statement suggested that students would
engage in special forms of mathematical reasoning such as conjecturing, generalizing, and verifying.
Example: “Graph the equations y = 2x + 3, 2y = x - 2, and y = -4x, and examine the role played by
the numbers in determining the position and slope of the associated lines” were classified as making
connections.
Show A statement concerning the word "buktikan" or "show".
PROBLEM CONTEXT
Real world Math problems presented within a real-life context.
Examples: “Estimate the surface area of the frame in the picture below,” and “Samantha is collecting data
on the time it takes her to walk to school. A table shows her travel times over a two-week period; find the
mean.”
Math language Problems presented only with mathematical language,
Examples: “Graph the equation: y = 3x + 7,” and “Find the volume of a cube whose side measures 3.5 cm.”
PROBLEM CONTEXT
Open The correct answers from more than one problem.
Closed The correct answer from a matter of just one.
PROBLEM CONTEXT
Routine Problem can be solved directly using the formula, definition or proposition
Non-routine Problem that cannot be solved with a usual procedure (directly using the formula, definition or
proposition), but instead must be solved using a non-routine strategy (options specified below)
164 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Appendices
GRAPHICAL MAPPING OF FINDINGS FOR EACH STAGE OF ANALYSIS
The following map highlights key points that were found in each of the steps of the analysis. The blue boxes contain
key findings at the component level of Beliefs, Knowledge and Practices. The red boxes contain findings of the
dual relationship analysis of beliefs-knowledge, beliefs-practices and knowledge-practices. The green box contains
analysis of the beliefs-knowledge-practice interaction.
PRACTICES
D E
HIgher MKiT teachers varied practices Use of Questioning in the classroom General relationship clearly exists
more (greater variety) Wider range for some teachers Clearest in “Role” concept
Less variation on MKiT assessment than Handling of “pivot moment” is a Teachers tended to see their teaching as
teacher degree di erentiating aspect of teachers more connectionist than in actual practice
Striking lack of di erence in frequency of Strategies Tensions exist in practices-beliefs
practices likely about how done Interaction/Approach/Problem (environment; real vs. perceived)
Lower knowledge > fewer practices Practices related to student learning Narrower beliefs > fewer practices
G
Teachers whose beliefs-knowledge and practices appear to be most e ective
Teachers with higher knowledge and a wider range of beliefs are also able to
utilize a greater range of practices
Concept of relative e ectiveness
Context plays a role in choice of practices based in part on beliefs and knowledge
at teacher’s disposal
MATHEMATICAL
MKiT F
BELIEFS
Wide range in Indonesian teachers Higher MKiT > wider range Flexibility concept emerges
Many teachers with minimal level Higher MKiT > more exibility Teachers have range based on context
Sub/ped correlation Cases of Beliefs-Knowledge opposite Perception on factors
Some sub-ped di erences Beliefs Learning Beliefs as a range rather than a point
Teacher’s degree is important distinction Concept of relative e ectiveness Link to learning outcomes
Strong link to Learning Outcomes Beliefs expressed in discussing scenarios
Scenarios approaches di erent
Main Report 165
DISTRIBUTION OF RESPONSES FOR EACH STATEMENT IN THE MATHEMATICAL BELIEFS SURVEY
166
LIKERT SCALE RESPONSES (SCALE OF 1-5)
How Students Learn Mathematics
Strongly Disagree Disagree Neutral Agree Strongly Agree
PRACTICES FOR LEARNING
Students learn mathematics best in classes with students of similar abilities 6% 61% 2% 27% 4%
Telling the children the answer is an e cient way of facilitating their mathematics learning 18% 56% 0% 23% 2%
Listening carefully to the teacher explain a mathematics lesson is the most e ective way to learn mathematics 4% 37% 5% 43% 12%
Discovery methods of learning mathematics are relevant for the earliest concepts only, e.g. addition, volume, etc 16% 3% 54% 27%
Discovery methods would be useful for older pupils if time and syllabus permitted 3%5% 74% 18%
Expecting pupils to be creative in mathematics is unreasonable and doesn’t warrant the use of precious lesson time 4% 57% 8% 31% 0%
Mathematics is learnt well by working individually on mathematical problems 5% 48% 4% 37% 7%
The more students practice mathematical exercise given in the textbooks, they learn mathematics with understanding 9% 0% 50% 42%
PROCESS OF LEARNING
Allowing a child to struggle with a mathematical problem, even a little tension, can be necessary for learning to occur 1% 36% 6% 47% 10%
It is important that mathematics content be presented to children in the correct sequence 4% 2% 51% 43%
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
Mathematical materials is best presented in an expository style: demonstrating, explaining and describing concepts and skills 8% 2% 55% 35%
Learning mathematics in school is essentially an active process 2% 2% 59% 37%
Students learn mathematics by nding the correct answer to mathematical problems 12% 1% 69% 19%
Students can solve most mathematical problems on the basis of memorized formula 5% 44% 2% 47% 3%
Use of a calculator hinders student’s understanding of mathematics 5% 33% 8% 43% 12%
Students learn most e ectively when learning tasks are broken down into small sequential steps 5%4% 76% 15%
Children always bene t by discussing their solutions to mathematical problems with each other 3%1% 57% 39%
Persistent questioning has a signi cant e ect on children’s mathematical learning 9% 6% 62% 23%
Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Teaching Mathematics
Strongly Disagree Disagree Neutral Agree Strongly Agree
Main Report
ROLE OF TEACHER
The teacher is the main source of mathematical knowledge 5% 53% 3% 30% 9%
I guide students in nding their own answers to academic problems 3%2% 76% 19%
It is more e ective to provide students with the information they need to know, rather than encouraging them to experiment 2% 51% 6% 37% 4%
It is the teacher’s responsibility to provide children with clear and concise solution methods for mathematical problems 1%11% 65% 23%
I function in my classroom as a learner and partner in learning with my students 6%1% 72% 21%
Teachers should help pupils to develop mental strategies 64% 35%
Teachers can create, for all children, a non-threatening environment for learning mathematics 3%2% 50% 45%
I wait for students to approach me before o ering extra help 10% 62% 3% 25% 1%
APPROACH AND SEQUENCING
I teach subjects separately, although I am aware of the overlap of content and skills 5% 44% 15% 36% 0%
The most e ective way to teach mathematics is to rst explain the topic, give example(s) and then ask students to solve
mathematical tasks 18% 1% 52% 28%
ASSESSMENT AND EVALUATION
I base student grades primarily on homework, quizzes, and tests 1% 30% 0% 60% 9%
I adjust my lesson plan based on results of homework assignments 5% 57% 8% 29% 1%
I prefer to assess students informally through observations and conferences 43% 7% 44% 6%
It is important that I establish classroom control before I become too friendly with students 11% 5% 64% 20%
Paper pencil test is the best way to examine student’s mathematical learning 2% 33% 9% 50% 6%
I prefer to cluster student’s desks or use tables so they can work together 1% 14% 6% 59% 20%
TOOLS AND RESOURCES
Textbooks are the main resource to teach mathematics 3% 28% 4% 53% 12%
It is important to cover all the topics in the mathematics curriculum in the textbook sequence 1% 42% 5% 47% 5%
To be sure that I teach students all necessary content and skills, I follow a textbook or workbook 3% 25% 3% 65% 4%
I generally use the teacher’s guide to lead class discussions of a story or text 15% 15% 69% 1%
I nd that textbook and other published materials are the best sources for creating my curriculum 2% 22% 11% 59% 6%
I encourage parents to follow up on classroom activities with students at home 7% 8% 54% 31%
CURRICULUM, ASSESSMENT AND POLICY
The examination syllabus should not be the main factor in determining the school syllabus 1% 30% 8% 57% 4%
Teachers should be able to in uence public examination boards in the style of examination, syllabus, content, etc. 19% 23% 50% 8%
Public examination syllabus, style, etc. should be decided by experts 3% 21% 20% 39% 17%
Commerce and industry’s needs should determine the mathematics curriculum 6% 32% 18% 33% 11%
The testing program in my state/district dictates what mathematics content I teach 18% 11% 62% 9%
I believe that expanding on students ideas is an e ective way to build my curriculum 1%7% 15% 65% 12%
167
Appendices
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
FREQUENCY OF TEACHING PRACTICES IN THE TEACHER’S LESSONS
Frequency of Teacher’s Own Teaching Practices
Almost never Occasionally Half the time Most of the time Almost always
Students learn through doing exercises 15% 18% 20% 47%
Students work on their own, consulting a neighbour from time to time 1% 36% 17% 31% 16%
Students use only method I teach them 8% 23% 14% 17% 38%
Students start with easy questions and work up to harder questions 3% 17% 5% 24% 50%
Students choose which questions they tackle 16% 36% 10% 16% 23%
I encourage students to work more slowly 7% 16% 11% 23% 44%
Students compare di erent methods for doing questions 5% 38% 14% 16% 28%
I teach topic from the beginning, assuming they know nothing 19% 35% 10% 17% 20%
I teach the whole class at once 9% 11% 5% 23% 53%
I try to cover everything in a topic 21% 23% 13% 24% 21%
I draw links between topics and move back and forth between topics 8% 42% 13% 19% 19%
I avoid students making mistakes by explaining things carefully rst 6% 18% 11% 32% 33%
I tend to follow the textbook or worksheet closely 3% 46% 18% 18% 16%
Students learn through discussing their ideas 7% 52% 18% 16% 8%
Students work collaboratively in pairs or small groups 4% 38% 25% 16% 18%
Students invent their own methods 18% 56% 10% 11% 6%
Students work on substantial tasks that can be worked on at di erent levels 3% 29% 20% 24% 25%
I tell students which questions to tackle 4% 26% 19% 24% 27%
I nd myself encouraging students to work more quickly 9% 30% 22% 16% 24%
I only go through one method for doing each question 24% 51% 11% 8% 7%
I nd out which parts students already understand and don’t teach those parts 17% 25% 11% 25% 23%
I teach each student di erently according to individual needs 23% 37% 8% 13% 20%
I only cover important ideas in a topic 15% 32% 20% 20% 14%
I tend to teach each topic separately 13% 30% 16% 19% 23%
I know exactly what maths the lesson will contain 5% 14% 5% 24% 53%
I encourage students to make and discuss mistakes 16% 30% 19% 13% 23%
I jump between topics as the need arises 24% 48% 9% 12% 8%
168 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Appendices
RANKING RESPONSES
Ranking of Preferred Teaching Style from 1 (most) to 7 (least)
1 2 3 4 5 6 7
Giving clear explanations of de nitions and procedurs to follow in
di erent exercise and problems of application, in the topics studied 42% 15% 14% 11% 6% 7% 7%
Organizing problem-based activities for the pupils to work in small groups
where they can present their ideas to the whole class for discussion 15% 24% 23% 17% 11% 3% 7%
Testing pupils at the end of each activity or topic, in order to have 15% 18% 10% 11% 17% 20% 9%
su cient marks for assessment in each Attainment Target
Designing activities that provide space for pupils’ self-paced learning 12% 19% 17% 17% 11% 9% 16%
Providing opportunities for pupils to develop their communication skills
so that they can express their mathematical ideas with con dence 11% 21% 17% 18% 17% 11% 5%
Giving pupils lots of exercises for application as homework 9% 12% 16% 9% 13% 12% 29%
Designing classroom work that promotes connections between
di erent mathematical topics studied 7% 8% 10% 11% 22% 24% 20%
Ranking of Priorities in Teaching from 1 (most) to 5 (least)
1 2 3 4 5
Strategies for investigation and problem solving 23% 42% 21% 7% 8%
Interpretation for concepts and representations 23% 20% 29% 15% 14%
Fluency in recalling facts and performing routine skills 21% 21% 17% 26% 16%
Awareness of the nature and values of the educational
system (eg ‘exam technique’) 13% 12% 15% 27% 34%
Main Report 169
A Video Study of Teaching Practices in TIMSS Eighth Grade Mathematics Classrooms
SUMMARY OF TEACHER RESPONSES TO MATHEMATICAL SCENARIOS
Identify issues (error, Conjectures of student
Mathematics Develop a strategy
misconception) thoughts
CS2 Multiple errors Missed some issues More focused on the maths, “I would explain…” “I would
less on student thinking repeat…”
Confidently discussed maths,
but made mistakes
CS3 No mathematical errors, Quick to spot most issues, Struggled to make Would try to connect to daily
but chose some answers but got confused with issues conjectures about students’ life “Step by step”
that would be considered regarding student thinking thoughts, but was fine with
Would tap into his personal
incorrect according to the mathematical errors.
item bank for new problems
answer key
Constant theme of trying and approaches
Ease in discussing maths to help students connect to
real world and how students
have difficulty with abstract
concepts
CS4 Minor errors Took time to spot issues, Sometimes discussed Work in groups
but generally did and then students’ thoughts
Some difficulty in discussing Students work out for
elaborated more than other
maths themselves
teachers.
CS5 No errors Quick to spot errors Sometimes discussed Strategies focused on
students’ thoughts students discovering for
Ease in discussing maths
themselves
CS6 Minor errors Very quick to spot issues Would often discuss students’ Went to mathematical
thoughts explanations
Ease in discussing maths
Would connect to own
experiences
CS7 Generally very good, but Quick to spot errors, Would often discuss students’ Game for integer problem
could not answer the elaborated on roots thoughts
Come up with new
inequality scenario
Would connect to own perspective
Ease in discussing most experiences
maths (except inequality)
CS9 No mathematical errors, Very quick to spot issues Very clear about student Creative ways of explaining
but did not identify correct thought processes. topics from different
answer in scenario 5 perspectives
Multiple avenues
Extreme ease in discussing
maths
CS10 Multiple errors Took time to spot issues, but Not too focused on student Struggled to come up with
generally did thoughts strategies
Difficulty in discussing maths
170 Understanding What Teaching Practices are Used, Why They are Used and How They Relate to Student Learning
Appendices
TECHNICAL REPORT AS A SEPARATE APPENDIX
A great deal of data and information was generated for this study and has been documented in detail in a separate
technical report. This was done in order to keep the main report at a manageable length and to not bog down
the reader in procedural details. The technical report can be considered a large appendix to this main report. The
technical report will be of interest for those who are interested in the following:
• Full literature review of international research related to the components of the study
• In-Depth description of design, methods and analytical approach
• Full results of all analysis
• Detailed presentation of results including step-by-step
• Full description of new conceptual model for the dynamic beliefs-knowledge-practices relationship
• Very little policy discussion
• Full appendices of instruments, case study profiles, results used for pattern analysis
Main Report 171
÷
+ 5
1