Improving Teachers and
School Leadership in Indonesia
Impact Evaluation of Guru Penggerak Program at the Primary Level
Noviandri N. Khairina, Noah Yarrow,
Jacobus Cilliers, and Indah S.Z. Dini.
02 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Improving Teachers and School
Leadership in Indonesia: Impact
Evaluation of Guru Penggerak
Program at the Primary Level
This report was prepared by Noviandri N. Khairina,
Noah Yarrow, Jacobus Cilliers, and Indah S.Z. Dini.
Acknowledgement
This work is a product of the staff of The World Bank. Financial support for
this work was generously provided by the Australian Government through the
Australia-World Bank Indonesia Partnership (ABIP). This research product also
benefited from additional financing that comes from the China-World Bank Trust
Fund. The findings, interpretations, and conclusions expressed in this work do not
necessarily reflect the views of the Executive Directors of The World Bank or the
governments they represent. The World Bank does not guarantee the accuracy
of the data included in this work. The boundaries, colors, denominations, and
other information shown on any map in this work do not imply any judgment
on the part of The World Bank concerning the legal status of any territory or the
endorsement or acceptance of such boundaries.
Rights and Permissions
© 2023 The World Bank
1818 H Street NW, Washington DC 20433
Telephone: 202-473-1000; Internet: www.worldbank.org
Some rights reserved
The material in this work is subject to copyright. Because The World Bank
encourages dissemination of its knowledge, this work may be reproduced, in
whole or in part, for non-commercial purposes as long as full attribution to this
work is given.
All queries on rights and licenses, including subsidiary rights, should be addressed
to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington,
DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org.
Attribution
Please cite the work as follows: Khairina, Noviandri; Yarrow, Noah; Cilliers,
Jacobus; Dini, Indah. 2023. Improving Teachers and School Leadership in
Indonesia: Impact Evaluation of Guru Penggerak Program at the Primary Level.
World Bank, Jakarta. © World Bank.
Contact Information
The authors can be contacted at nkhairina@worldbank.org, ejc93@georgetown.
edu, idini@worldbank.org.
Photo Credit:
Cover and Chapter Opening: Freepik, Midjourney
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 03
Table of Contents
Table of Contents 4
Tables and Figures 4
Abbreviations and acronyms 5
Acknowledgement 5
Executive Summary 6
1. Understanding the Context: Indonesia’s Education System 10
2. New Wave of Reforms: Emancipated Learning 12
3. The Program – Pendidikan Guru Penggerak (PGP) 14
Program Design, Delivery, and Curriculum 14
Selection process 16
Theory of Change 17
4. Methodology 18
Empirical Approach for the Impact Evaluation—Regression Discontinuity Design 18
Sample Selection - Examining the Cutoff for Selection 19
Limitations 21
5. Data and Instruments 22
Implementation Assessment (phone survey) 22
School Visit for Impact Evaluation 23
Instruments 23
Construction of Indicators and Indices 28
Timing of Data Collection 28
Descriptive Statistics 28
6. Results 30
Phone Survey for Implementation Assessment Results 30
Impact Evaluation Results 33
Quality of Implementation and Other Training Programs 34
Self-reflecting and Self-improving Teachers 36
Student-centered Teaching 37
Leadership: Communities of Learning, Coaching, and School vision 42
Additional Findings from This Evaluation 44
7. Conclusions and Recommendations 48
References 52
Appendix 1 – Pendidikan Guru Penggerak 53
Appendix 2 – Teach Classroom Observation Instrument: 60
Appendix 3 – Empirical Strategy 64
Appendix 4 – Additional Tables and Results 65
04 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
List of Tables
Table 1 Comparison of 'Penggerak’ programs in the Merdeka Belajar reform 12
Table 2 Sample for Phone Survey and School Visit Data Collection 22
Table 3 Summary of Phone Survey Data Collection 23
Table 4 Mapping between MoECRT’s groupings of final outcomes and our evaluation instruments 24
Table 5 Description of the Teach classroom observation instrument 26
Table 6 Summary of the MoECRT classroom observation instrument 27
Table 7 Teacher Characteristics - by PGP 29
Table 8 Summary of main results 33
Table 9 Quality of Implementation by District 34
Table 10 Exposure to training (by selection into Batch 5) 35
Table 11 TOC I. Self-reflection and self-development 36
Table 12 Leadership confidence and aspirations 46
Table 13 Leadership confidence and aspirations (by gender) 47
Table 14 School Principal’s perception of the teacher’s leadership ability 47
Table 15 TEACH Results 62
Table 16 Overview of TEACH Instrument 63
List of Figures
Figure 1 Pendidikan Guru Penggerak – Program Design 15
Figure 2 PGP Actors roles and relationships 16
Figure 3 PGP Theory of Change: Final Outcome 17
Figure 4 Location of Sample Districts 20
Figure 5 Distribution of sampled teachers by assignment to treatment 20
Figure 6 The relationship between the running variable and the probability of being selected to participate in PGP 21
Figure 7 Timeline of Data Collection 28
Figure 8 Quality of implementation for different components of the program 30
Figure 9 Distribution of perceived quality of the facilitators, instructors, and mentors 31
Figure 10 District-level variation in perceived quality of the facilitator, instructor and mentor 31
Figure 11 Mentor Training Effectiveness 32
Figure 12 How workshop can be improved 32
Figure 13 Proportion of teachers that said logistics can be improved 32
Figure 14 Teachers' five-year career plans 33
Figure 15 reflection and self-development 36
Figure 16 TOC II.1 Student-centered teaching – positive discipline 38
Figure 17 TOC II.2 Student-centered teaching – differentiated learning 39
Figure 18 TOC II.3 Student-centered teaching – socioemotional competencies 40
Figure 19 TEACH Scores 41
Figure 20 TOC III.1 Create communities of learning (restricted sample) 42
Figure 21 TOC III.2 Coaching (Restricted sample) 43
Figure 22 TOC IV.2 Vision of student-centered school development (restricted sample) 44
Figure 23 Enjoyment of teaching 45
Figure 24 TEACH Result - Time on learning 61
Figure 25 Teach Primary Score of All Areas 61
Figure 26 Distribution of Average Teach Primary Scores by Area and Element 63
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 05
Abbreviations and Acronyms
BGP/BBGP MoECRT’s Implementation Unit on Teacher and Education Personnel Development
CGP Guru Penggerak Training Participants
EMIS Education Management Information System
GTK Directorate General of Teachers and Education Personnel
KM Merdeka Curriculum
KSPSTK School Principals, School Inspectors, and Education Personnel
LMS Learning Management System
MoECRT Ministry of Education, Culture, Research, and Technology
MoRA Ministry of Religious Affairs
OECD Organisation for Economic Co-operation and Development
PGP Guru Penggerak Program
PISA Programme for International Student Assessment
PNS Civil Servant
PSPK Center for Education and Policy Studies
RDD Regression Discontinuity Design
SD Standard Deviations
TALIS Teaching and Learning International Survey
ToC Theory of Changes
WDR World Development Report
ACKNOWLEDGEMENT
This report was prepared by a World Bank team led by Noviandri Khairina, and comprising (in
alphabetical order) Jacobus Cilliers, Indah Dini, and Noah Yarrow. Alexander Michael Tjahjadi and Ross
Babineau provided support on data cleaning and analysis. Sylvia Njotomihardjo provided administrative
assistance to the task team over the course of the preparation of this report. Additionally, special
acknowledgment goes to Nuriza Saputra for the design of this report.
This report was developed under guidance provided by Cristian Aedo (Education Practice Manager,
East Asia and Pacific Region), Achim Schmillen (Human Development Practice Leader, East Asia and
Pacific Region), Shinsaku Nomura (Education Cluster Lead, East Asia Pacific Region) and Satu Kahkonen
(Country Director for Indonesia and Timor-Leste).
The team is indebted to Rythia Afkar, Ezequiel Molina, Koen Geven, Lars Sondergaard and other
reviewers for their valuable feedback for preparing and improving this report.
The World Bank Task Team would like to thank the Ministry of Education, Culture, Research, and
Technology (MoECRT), for all the support and input provided during this research. Particularly to
Prof. Dr. Nunuk Suryani, M.Pd. (Director General of Teachers and Education Personnel (GTK), Mr. Iwan
Syahril, Ph.D. (Director General of Early Childhood and Basic Education), Dr. Kasiman (Acting Director of
KSPSTK), Dr. Praptono (Secretary of the Directorate General of Early Childhood and Basic Education),
the Setditjen GTK Team (Mila Novita, Reza Maulana Hamzah), the Pokja Team at the Directorate of
KSPTK (Rita Dewi Suspalupi, Otong Kusnadi, Yudi Herman, and Agus Rohmani), as well as the Technical
Team of Teachers and Education Personnel (Tim Teknis GTK).
06 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Background
EXECUTIVE The most significant recent education reform in Indonesia is
'Merdeka Belajar', officially translated as “emancipated learning,”
SUMMARY which aims to transform the education system into having a more
student-centered approach and promote lifelong learning. The
reform places a great emphasis on teacher competence, independence,
freedom, and ability to teach based on student’s needs, and aims to
delegate more autonomy to teachers and schools for enhanced student-
centered learning. The program includes many components, one of
which is teacher training.
The World Bank is supporting the Ministry of Education, Culture,
Research, and Technology (MoECRT) in its ongoing education reform
effort. As part of this support, and after comprehensive discussions and
careful deliberation, it was agreed that the World Bank would conduct
an evaluation study specifically for the Guru Penggerak program
given its importance for improving education quality in Indonesia.
Program
‘Pendidikan Guru Penggerak’ (PGP) program—roughly translated to
instructional leaders’ program— is a teacher professional development
program launched in 2020, with 2,800 teachers participating in the
Batch 1 training. The program has dual objective of improving teaching
practices and improve leadership by training the new cadre of school
leaders who will replace the outgoing school leadership. The ministry
is planning to train over 400,000 teachers by the end of 2024 and has
started to expand the pace and scale of implementation. Starting this
year (2023), a PGP certificate is a mandatory requirement for becoming a
1) school principal; 2) school supervisor; or 3) other leadership position
in the education sector. The training is rolled out in batches. The number
of teachers participating in each batch has increased from 2,800 in Batch
1 in October 2020 to 55,000 teachers who will be selected in Batch 10 by
August 2023. It is estimated that about 94,000 teachers have participated
in Batch 1-6 and currently ongoing batch 7, 8, and 9.
The PGP program offers an integrated approach to enhance teachers'
instructional leadership and creating student-centered learning
environments. It combines online learning using a learning management
system (LMS), monthly face-to-face workshops, and individual mentoring
in schools over a six-month period. Online learning covers ten modules
focused on student-centered learning and instructional leadership.
Seven in-person workshops cover teacher competencies, community
development, instructional leadership, coaching practice, collaboration
for student-centered programs and sustainable self and school
development. Six one-on-one in-school mentoring sessions focus on
teacher competencies, evaluating classroom practice, and designing and
implementing student-centered programs.
1
Kabupaten Banjarnegara, Kabupaten Blora, Kabupaten Semarang, Kabupaten Sumedang, and Kota Jakarta Timur
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 07
Evaluation design
We evaluate PGP Batch 5 using a Regression Discontinuity Design, taking advantage of the fact that selection relies on
an overall score given to teachers, based on their performance in a series of interviews and simulations. This estimation
strategy compares the performance of teachers who scored just below the cut-off—and who consequently did not participate in
the program—with teachers just above the cutoff—who did participate in the program. We surveyed 350 current teachers and 31
former teachers who had become school principals by the time of the survey, across five districts in Java province.1 The sample
focused on teachers who were closest to the threshold for selection in their respective districts. We also observed their teaching
in the classroom, and surveyed their students, another teacher in the school, and the school principal. The study also aimed
to provide feedback from participating teachers and mentors, conducting phone surveys with 400 teachers and 10 mentors, to
measure implementation quality.
This study reports results based on two different data collection activities (see figures below for the timeline of data collection
activities). First, for the Implementation Assessment, three rounds of phone surveys were conducted during the training to
capture information on the quality of implementation and teachers' perceptions of the program. Second, for the Impact
Evaluation, in-person data collection took place in March and April 2023—roughly 10 months after the start of the training,
and two months after the completion of the training. This include data collection using government’s and TEACH classroom
observation to measure the teaching practices.
Timeline of Data Collection
Start of the training Phone survey Phone survey
program round 2 round 3 End of school visit
Group B: Group A: data collection
June 2022 200 teachers December 200 teachers March 2023
2022
May November January April
2022 2022 2023 2023
Phone survey Mentor School visit data
round 1 Phone survey collection
Group A: 10 mentor 381 sample
200 teachers teachers teachers
End of the training
program
Results
First, the program was well-implemented and well-received by the teachers. Teachers rated all components of the program
highly. Moreover, overall satisfaction with this program is higher than for teachers who participated in other training programs.
Teachers are also more likely to believe that this training program will greatly improve their teaching skills, with 86 percent for
this program versus 53 percent for others.
One area for potential improvement is the support provided to teacher mentors. Overall, the perception of the quality of
the mentors is lower than the perceived quality of the instructors or facilitators. There is also substantial regional variation, with
an average score of 8.3 out of 10 in one district. Mentor teachers also provided a low rating for the quality of training that they
received. Their training is exclusively online, and there is no guidance on how to use the classroom observation instrument that
they are required to administer during coaching sessions.
Second, teaching practices improved substantially, especially in classroom culture and positive discipline. Overall
teaching practices, measured by the Teach classroom observations instrument, increased by 0.8 to 1.1 standard deviations (SD),
which suggests the teaching practices in the classrooms improved on average, and the extent of improvement is substantial.
The largest improvements were in providing a supportive learning environment, followed by positive behavioral expectations,
08 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
lesson facilitation, and promotion of critical thinking skills. There is also consistent evidence, from both the student surveys and
MoECRT classroom observation instrument, of improvements in positive discipline. Students were less likely to report that they
were scolded or punished by a teacher, less afraid to ask questions in the class, and more likely to have a class agreement or
social contract. Teachers are 34 to 50 percentage points more likely to apply the “restitution process” when disciplining children,
which entails reminding children of the class agreement when they misbehave, and asking children to self-reflect on what they
did wrong.
However, there is no evidence of improvements in differentiated instruction, or in teachers’ socio-emotional skills in the
classroom. Along with positive discipline, these were three elements of teaching practices that the program aimed to improve.
This result suggests that it may be harder to change teachers’ socio-emotional skills than to shift specific instructional practices,
such as applying the restitution process.
Third, there is evidence that the PGP teachers provided instructional support to other teachers in the school, but only
if we restrict our sample to schools where there are no other PGP graduates. A requirement of the program is that PGP
teachers establish communities of learning and provide instructional support to other teachers in the school. Schools are now
more likely to have communities of learning as a result of the program, but this might be because almost 90 percent of control
schools already have a community of learning. Almost 40 percent of teachers in our control group teach in schools where there is
at least one PGP graduate, so the control teachers might have already benefited from the program. When restricting the sample
to schools where there are no other PGP teachers, we find that the peer teachers are more likely to indicate that the PGP teacher-
initiated discussions about student-centered learning, helped them map out students’ needs, asked them to self-reflect on their
learning process, and helped them conduct a strength mapping exercise.
Fourth, the program improved teachers’ enjoyment of teaching, their aspirations to become school principals, and their
own self-confidence as a leader. This is very encouraging, since it suggests that the new teaching practices reduced the anxiety
and stress that teachers face in the classroom and increases the likelihood that these improved teaching practices will be
sustained over time. However, at the time of the data collection, we don’t find evidence of improvement in teacher’s leadership
abilities as measured by perception of the school principals.
Limitation
The positive impacts found in this study has the potential lack of generalizability due to limitation that our sample were located
in districts within Java Island, which is home to most well-developed provinces and districts in Indonesia. Our sampling
strategy for the regression discontinuity design (RDD) meant that we only sampled teachers close to the cutoff, and it happened
that the only districts that had a large enough sample of short-listed teachers who scored close to the cutoff were all in Java.
Consequently, while the internal validity of our evaluation method is strong, the findings of this study cannot be generalized to
all provinces and districts across Indonesia. Furthermore, the focus of this study is on CGP at the elementary school level, so that
different outcomes may be found at lower or higher educational levels.
Policy conclusions
We conclude that PGP should continue, given the high quality of implementation and its positive impact on teaching
practices and leadership skills. We do, however, expect significant implementation challenges as the ministry scales up the
rollout of training over the next 18 months. Our results also indicate areas for improvement, both in the quality of implementation
and design of the program
1. Additional support to mentors. While MoECRT has developed a comprehensive handbook for mentors, there is still room
for improvement. In particular, additional guidance is required for the classroom observations, such as how to rate teacher
competencies according to the government instrument, and what kind of feedback to provide to teachers based on the
observations. This can also be achieved by enhancing mentor training to include additional pedagogical strategies and
effective mentoring techniques. By equipping mentors with a broader range of knowledge and skills, they will be better
prepared to support teachers in their professional growth. Lastly, mentors also need to have robust support system and
access to resources to overcome challenges, share experience, and enhance their effectiveness in guiding the teachers.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 09
2. Supporting teachers in differentiated learning and socio-emotional skills. This support can take the form of materials
(guides, tools, practical tips, other resources) related to differentiated learning in the classroom or easily accessible self-
paced training for teachers, while also ensuring a learning community at the school level. Support can also encompass
additional mentoring sessions during the program and follow-up mentoring sessions after the program ends.
3. Standardize best practice and provide clear guidelines and procedures for implementation of district-level
activities. As the program scales up, it is relying more on district-level implementation, which could be variable. Clear
guidelines and related resources, as well as coordination between the Ministry of Education, Culture, Research, and
Technology (MoECRT) and related stakeholders, including local government and Balai Guru Penggerak, is vital for effective
program execution.
4. Reconsider the broad and ambitious objectives of the program. The program might be overly ambitious in the wide
range of teacher attitudes and skills that it hopes to change. Socio-emotional competencies, in particular, are very hard
to change in adults. Moreover, the official standards for reaching competence in the different domains, as defined in the
program’s classroom observations instrument, are highly ambitious. A different strategy is to prioritize a smaller set of
skills that are key to improving student outcomes, and setting standards that are realistically achievable by all teachers.
This will be increasingly important as the program continues to scale up and is implemented in remote and under-
resourced locations where the baseline teacher capacity might be weak.
5. Conduct additional research. The results of this study are not generalizable to Indonesia as a whole. Future research
should:
A. Expand data collection to more districts, including remote and rural locations.
B. Collect data on teachers who participated in both earlier and later batches of training.
C. Measure students’ cognitive and non-cognitive skills.
D. Conduct a prospective randomized evaluation in one of the upcoming batches, using a randomized
phased in approach. This will strengthen the confidence that the program had a causal impact,
increase the generalizability of the findings, and also improve statistical power.
10 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
1.
UNDERSTANDING THE
CONTEXT: INDONESIA’S
EDUCATION SYSTEM
Indonesia has a large and complex education system (MoHA) oversees regional governance, including teacher
involving both centralized and decentralized systems training at the local level and sub-national financing.
under multiple ministries. With over fifty-three million
students2 enrolled in basic and secondary education and Despite significant progress in expanding access,
over three million teachers and education personnel, the education quality in Indonesia remains a significant
education system overseen by two ministries. The Ministry challenge. For the last 30 years, Indonesia has experienced
of Education, Culture, Research, and Technology (MoECRT) rapid progress in expanding access to primary and
makes and coordinates the implementation of education secondary education. However, learning quality remains a
policies for primary, secondary as well as higher education; challenge. A recent World Bank report 3 showed that only
and the Ministry of Religious Affairs (MoRA) manages and 7.8 years-equivalent of learning were acquired among youth
implements policies for the religious education sector or who on average attended 12.4 years of schooling. Results
madrasahs across the country. MoECRT schools operate from international assessments, such as PISA 2018, show
under a decentralized system implemented by provincial that reading literacy for Indonesia students lags other
or local governments. Madrasahs operate in a centralized countries. 4 These findings are also aligne d with re sults
system unde r the MoRA following the same national from the latest national assessment (2021), which showed
curriculum as MoECRT schools, but with the ministry that half students do not meet Indonesia’s minimum
managing the institutions, developing religious curricula, literacy standards and only a third met minimum numeracy
policies, and conducting teacher training. In addition to standards. 5
the roles of MoECRT and MoRA, the Ministry of Home Affairs
2
MoECRT Education Statistics, MoRA’s EMIS 2022/2023
3
Human Capital Index, 2020
4
Organisation for Economic Co-operation and Development (OECD), 2018
5
MoECRT’s minister presentation on the national assessment (ANBK) result, April 2022
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 11
Global evidence indicates that the quality of teachers 2020, the re are 50,135 e leme ntary school principal age
and their te aching are the most important factors in between 51-55 and 56,758 principals age over fifty-five. 9
improving student learning outcomes. 6 In 2018, the This means that a hundred thousand school principals
OECD conducted a Teaching and Learning International are expected to retire within the next decade.10 Therefore,
Survey (TALIS) which collected data on teachers' teaching training for school principal candidates is a government
practices and the impact on student learning outcomes priority. This provides an opportunity for improving school
across countries. The study consistently showed the leadership at scale. Evidence shows that school leadership
impact of teacher quality on student achievement.7 Well and management are crucial to enhancing student learning.
educated and skillful teachers who have deep pedagogical From Brazil to India to Sweden, research reveals a significant
knowledge and are well equipped to meet diverse needs correlation between principal management skills and
in class can deliver effective and high-quality instruction. student performance (WDR, 2018). Effective principal
The enhancement of teacher skills and knowledge affects training focuses on how to improve teacher-student
the education system as teacher training continues to interaction, such as providing feedback to teachers and
evolve (Barber and Mourshed, 2009). Therefore, investing developing action plans to enhance student performance
in teacher education is a strategic approach to improve the and classroom behavior.
overall quality of education.
The COVID-19 pandemic brought an unexpected
The quality of teachers has been identified as one of the opportunity for the education sector, despite the
most significant issues in Indonesia’s education sector. challenges and significant impact it caused. With limited
Data from a nationally representative survey conducted by in-person classes during the period of school closures,
World Bank in 2019 found that teacher absenteeism is high, the government introduced a simplified curriculum, which
even though most of this absence appears to be excused ultimately led to the development of the new Kurikulum
and officially excused (e.g. teacher was absence to attend Merdeka, which aims to provide freedom and a more
training or completing other administrative task). Teacher relevant learning experiences for students based on their
subject knowledge was also found to be low, with only a needs.11 Furthermore, the pandemic engendered a shift
third teachers surveyed able to score at least 80 per cent to online learning and the utilization of digital platforms
in an assessment based on material from the Grade 3 and which in turn increased the digital literacy skills of both
Grade 4 curricula. 8 teachers and students. However, evidence shows that in
spite of these innovations, the pandemic closures caused
There is also an urgent need to de velop high-quality significant learning loss, particularly for students from the
school principals in Indonesia, given that most school most disadvantaged families, or those who had limited
principals (72 percent) are currently aged over 50. In access to distance learning during school closures.12
" The enhancement of teacher skills and knowledge affects the
education system as teacher training continues to evolve (Barber
and Mourshed, 2009). Therefore, investing in teacher education is a
strategic approach to improve the overall quality of education.
"
6
OECD, 2005
7
OECD, 2020
8
Yarrow et. al., 20209 Badan Pusat Statistics, 2020 as cited in Katadata, 2021
10
Mandatory retirement age for school principals is 65 years old (Circulate Letter from Head of Civil Servant Agency No. K26-30/V.119-2/99 on October 3rd, 2017.
11
MoECRT, 2023
12
World Bank,2023
12 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
2.
NEW WAVE OF REFORMS:
EMANCIPATED LEARNING
‘Merdeka Belajar’ reform, officially translated as in terms of the overall objective to develop students with
“emancipated learning”, is the most recent policy reform Pancasila profiles,13 but are different in terms of means of the
in Indonesia’s education system. The reform, initiated intervention and duration of the program (as summarized
in 2019, emphasizes teacher competence, independence, in table below). In February 2022, MoECRT introduced a
freedom, and ability to teach based on student needs, and new curriculum, ’Kurikulum Merdeka’, which had been
aims to delegate more autonomy to teachers and schools implemented as a pilot in 2,500 schools participating in
for enhanced student-centered learning. The overarching ‘Program Sekolah Penggerak‘. Shortly afterwards, the
objective of this reform is to transform Indonesia’s education Ministry enabled implementation of the new curriculum for
system for the 21st century, to be more student-centered, and academic year 2022/2023 by providing access to an online
to promote lifelong learning. By April 2023, the ministry had platform, ‘Platform Merdeka Mengajar ‘, for all teachers to
launched 24 Merdeka Belajar programs for regional education learn about the curriculum. These are important factors
offices, schools, teachers, and students, and covering all in our analysis, since even teachers in the control group for
levels of education from early childhood, primary, secondary, this study might have benefited from other programs and
vocational, and up to higher education. interventions.
The education sector reform involves multiple programs Student-centered learning, the key element of the
targe ting a variety of stake holde rs in the se ctor. The reform, places students at the center of learning process
Ministry started the reform by abolishing the high-stake as opposed to the traditional teacher-centered model.
national exam in 2020 and introducing new national The concept of student-centered learning is not a recent
assessments. This was followed by a series of interventions, trend in global education, as it has been discussed in the
including the ‘Penggerak’ programs: Organisasi Penggerak, international literature for over two decades. In essence,
Guru Penggerak, and Sekolah Penggerak. The three are similar learner-centered education refers to a pedagogical approach
13
The "Profil Pelajar Pancasila" refers to the profile of students who embody the values and principles of Pancasila, which is the philosophical foundation of the
Indonesian state. It encompasses the characteristics, attitudes, and behaviors that reflect the ideals of nationalism, humanism, social justice, democracy, and
belief in one supreme God. The "profil pelajar pancasila" aims to develop students who are morally upright, responsible, respectful, and actively contribute to
the betterment of society, guided by the principles of Pancasila.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 13
that gives learners (and demands from them) a relatively high as Singapore and South Korea. South Korea has promoted
level of ownership over the content and process of learning student-centered learning since the 1990s, initiated by The
in accordance with their needs, capacities and interests 31 May Education Reform Report in 199517 and later through
(Schweisfurth, 2013).14 This places the learner rather than the a series of revisions to the curriculum over the following
teacher, content or system, at the heart of the teaching and decades. An example of a student-centered initiative is
learning processes.15 Learner-centered education involves the ‘Creative Experimental Activity’ introduced in 2009 for
student-activating approaches, problem-based learning, middle and high school students where they participate in
minimal guidance approach, discovery learning, open-ended interesting and experiential activities for 3-4 hours a week
learning environments, collaborative/cooperative learning, (Park 201618). Meanwhile in Singapore, the shift toward a more
project-based learning, and case-based learning (Baeten et student-centric learning began with the “Student-centered,
al. 201016). Values-driven” vision articulated in 201119, which emphasizes
holistic education centered on values, social and emotional
Similar approaches have been implemented in other well-being and character development for every child.
countries with high-performing education sectors such
Table 1. Comparison of 'Penggerak’ programs in the Merdeka Belajar reform
Program Sekolah Program Pendidikan Program Organisasi
Penggerak Guru Penggerak Guru Penggerak
Objective To improve student learning To produce instructional Massive community
outcomes by improving leaders who will encourage empowerment effort
competencies of existing student-centered learning to improve teacher and
school leaders and teachers. and act as agents of principal quality through
transformation in the various training models and
education system design
Actors School Principals, Teachers, Mentor, NGOs or development
Facilitators of Sekolah instructors and Facilitators partners
Penggerak of Guru Penggerak
Program Duration 3 years 9 months (Batch 1-4) 2 years
6 months (Batch 5-13/
present)
Target Beneficiaries School principals, teachers, Teachers School principals, teachers,
supervisors supervisors
Source: MoECRT presentation on Merdeka Belajar, episode 4, 5, and 7 - (https://merdekabelajar.kemdikbud.go.id/)
14
Schweisfurth, 2013
15
Yang and Tan, 2019
16
Beaten et.al., 2010
17
Kim, 2004
18
Park, 2016
14 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
3.
THE PROGRAM: PENDIDIKAN
GURU PENGGERAK (PGP)
As part of the ongoing program of reform, MoECRT and supervisor candidates to complete PGP. In 2022, the
launched a new in-service teacher training program in policy was expanded with Permendikbud No.26 2022 Article
2020.20 The program, called ’Pendidikan Guru Penggerak’, 1323 , whereby PGP certification also provide opportunities to
is a professional development program offering training become school supervisor or other leadership positions in
and mentoring focused on instructional leadership. It aims the education sector.
to empower teachers to facilitate student growth, inspire
their colleagues to implement student-centered learning,
Program Design, Delivery, and Curriculum
and serve as role models and agents of transformation
in the education system to realize the profile Pelajar
Pancasila. The PGP initiative was introduced as a set of other This program aims to develop a new generation of
programs: “transformational organization program” and education leaders. Ultimately, the goal is to prepare teachers
“transformational school program” (Program Organisasi to become transformational leaders in the education sector,
Penggerak and Program Sekolah Penggerak). The ministry is becoming school principals, supervisors, teacher professional
planning to train 407,400 teachers by the end of 2024. As of development instructors24, thus contributing to an increase
June 2023, the ministry has just completed recruiting 62,681 in the number of students with a Pancasila profile; or those
teachers to participate in PGP.21 who think critically, creatively, independently, have faith, fear
God Almighty, are of noble character, work together and are
A new policy and regulation provides a pathway globally competent.
for teachers to become principals or supervisors.
The government has implemented a new regulation, The PGP program combines online learning using a
Permendikbud No. 40 202122 to regulate teacher to be learning management system (LMS), monthly face-to-
assigned as to school principals. It requires school principal face workshops, and individual mentoring in schools
20
MoECRT, 2020
21
MoECRT, 2023
22
Permendikbud No. 40, 2021
23
Permendikbud No. 26, 2022
24
GTK, 2023
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 15
Figure 1. Pendidikan Guru Penggerak – Program Design
5 Months
Online learning Online learning Online learning
Package 1 Package 2 Package 3
1 Month
Facilitator Instructor
Orientation Official Official Official Official Official Official Official
Workshop workshop workshop workshop workshop workshop workshop workshop
and individual and individual and individual and individual and individual and individual and individual
mentoring mentoring mentoring mentoring mentoring mentoring mentoring
Teacher's
Mentor
Source: Master Deck of Guru Penggerak program– Batch 5, MoECRT
over a six-month period. PGP is based on an andragogical knowledge and develop a plan for action, known as "Aksi
approach throughout a six-month learning cycle, providing Nyata" for implementation in the participants' classrooms
teachers with access to a learning community where they or schools as a practical application of their learning.
can collaborate and implement student centered learning Participants go through a total of 10 modules, organized into
projects at school during and after training. In the process, three categories or module packages, as follows:
they regularly reflect on their learning, exchange feedback
among participants, mentors, and facilitators, and create • Package 1 focuses on the vision of Guru Penggerak
action plans to improve their skills. Communities of practice and the importance of creating a student-centered
are developed in a natural and organic manner. The proportion learning environment, highlighting the values, roles,
of activities consists of 70 percent studying in place of work and vision necessary to cultivate a positive learning
(on-the-job training), 20 percent study with peer teachers in culture.
the PGP program, and 10 percent training with facilitators,
• Package 2 centers around student-based learning
instructors and the teacher’s mentor. The learning journey
practices, encompassing inclusive and differentiated
(see figure below) begins with an orientation workshop, where
learning paradigms, mapping student learning needs,
teachers are introduced to the program's policy, culture and
and implementing strategies for differentiation. It
learning targets. Subsequently, teachers explore the modules
emphasizes socio-emotional learning and coaching
on the LMS, enabling them to acquire knowledge and skills
as a supportive system.
at their own pace. In addition, teachers participate in offline
workshops to deepen their understanding of the content and • Package 3 explores the role of instructional leaders
engage in collaborative activities with their peers (fellow Guru in school management, covering decision-making,
Penggerak participants). Throughout the program, mentors asset management within the school ecosystem, and
assist and provide feedback for teachers as they apply their managing programs that promote student leadership
newly acquired skills in their schools. and community engagement. These modules equip
teachers with the mindset and skills to prioritize
The online learning component is conducted through student-centered learning, foster collaboration
MoECRT’s Learning Management System (LMS). The LMS among peers, and implement asset-based school
adopts an appreciative inquiry approach, inspired by the programs collectively.
Teacher Inquiry Program at the British Columbia Teachers
Federation. Each module follows the "MERDEKA" cycle, The workshops are conducted in-person at the district
which consists of seven steps. It begins with self-reflection level, facilitated by mentors. This is an opportunity for
on the material, followed by independent study by the participants to collaborate, deepening their understanding,
participants. This is supplemented by a discussion forum and reflect on their teaching practices through eight six-hour
and facilitated group assignments. Participants then engage workshop sessions. These include orientation followed by
in mid-learning reflection and proceed to work on individual self-development and networking with fellow educators,
tasks. An enrichment session with the instructor follows, the school’s vision and mission based on student-centered
where participants deepen their understanding of the philosophy, and learning practice in the classroom. Next,
topics. Subsequently, participants integrate their acquired participants are expected to be able to carry out academic
16 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
supervision applying a coaching mindset. Later sessions focus Facilitators play the role of assisting participants in the
on developing an asset-based and collaborative program online learning process within the Learning Management
work plan for student-centered school development and System (LMS). This includes self-paced asynchronous learning
good practice sharing with school principals, offices, or and synchronous discussions. They are also responsible
learning communities. for assessing and providing feedback on participants'
assignments. Instructors are tasked with enhancing the
Individual mentoring sessions take place at each understanding of each module studied by the CGP in virtual
participant’s school. Mentors advise 3-8 PGP participants, (synchronous) environments. Lastly, Mentors guide face-to-
each receiving six individual mentoring sessions with 4-training face workshops and offer individual support within school
hours. Mentors observe the application of knowledge from the settings. Each role carries specific responsibilities that are
LMS and workshops in the classroom, provide feedback on interconnected to ensure the effectiveness of the program
teaching practices, support improvement plans and assess the (please see Appendix 1.D.).
learning process in the classroom and peer teacher coaching.
Figure 2. PGP Actors roles and relationships
Instructor
Participants
(CGP)
Mentors facilitate
workshop and
individual mentoring
Facilitators and
instructors facilitate Mentor
online learning Facilitator
Coordination
Source: Master Deck of Guru Penggerak program– Batch 5, MoECRT
Selection process
school principals, along with an application essay. Selection
The program is highly selective and centralized, handled by includes verification and validation of the documents and
the Directorate General of Teachers and Education Personnel essay grading. Short-listed teachers can join the second stage
GTK) of MoECRT. Both civil servant (PNS) and non-civil servant of the process, where they are graded by six pedagogical
(non-PNS) teachers in both public and private schools, who competencies through micro-teaching simulations, and
have at least a bachelor’s degree and five years of teaching participate in eight leadership competency interviews, with a
experience, are eligible to apply if they’re teaching in regular score ranging from one to five. The two selection criteria are:
schools (teachers at MoRA schools/madrasahs are not
eligible). Applicants register online in the Guru Penggerak25 1. The sum of their scores across the 14 different assessments
website, uploading their national ID, education certificate, should be above a certain threshold.
appointment letter, and endorsement letter from their
25
sekolah.penggerak.kemdikbud.go.id
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 17
2. They should receive no score in the “less competent”
Theory of Change
category (below 3) in any of the competencies. If there
are multiple candidates within a district who meet these
The Theory of Change for the PGP training program (see
requirements, they will then be ranked based on their
Appendix 1) aims to reach all elements of education
overall score and selected according to the available quota.
institutions; students, schools and school personnel, creating
Quota for each district in each batch could be varied and
students with a Pancasila profile. To achieve this, it establishes
will be determined by MoECRT’s discretion.
a student-centered system within schools, developing quality
teachers, principals, and school supervisors and focuses on
In Batch 5 in 2022, roughly 20,000 teachers were short-listed
four final outcomes, each with a set of key indicators. These
from all levels of education (kindergarten, primary, junior
final outcomes and indicators serve as a framework for our
secondary, senior secondary, and vocational schools) in 166
impact evaluation study which is aimed at measuring the
districts in Indonesia. Of these teachers, roughly 8,105 were
impact and effectiveness of the PGP program in achieving its
selected based on their performance on the simulation and
intended goals.
interview.
Figure 3. PGP Theory of Change: Final Outcome
Teachers have the Teachers plan, Teachers support The teacher initiates
moral, emotional, and implement, reflect, peers/communities the development of
Final
spiritual maturity to and evaluate student- in their schools to a student-centered
Outcome
behave according to centered learning in implement student- school based on
the code of ethics. schools. centered learning. collaborative resources.
" Guru Penggerak Program aims to develop a new generation of
education leaders. Ultimately, the goal is to prepare teachers to become
transformational leaders in the education sector, becoming school
principals, supervisors, teacher professional development instructors
"
18 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
4.
METHODOLOGY
This study focuses on evaluating the impact of the program This report therefore has two components: an Impact
on teacher-level outcomes, particularly teaching practices Evaluation, evaluating the impact of the program on its
and leadership skills. Our study aims to answer the following intended outcomes, and an Implementation Assessment,
primary research questions: to measure implementation quality, which is to assess it’s
accordance with the guidelines (in terms of frequency and
Primary Research Question modalities) and teacher’s perception on the quality.
1. Can a government-run, in-service teacher
professional development plan implemented
at scale (i) improve teacher leadership and Empirical Approach for the Impact Evaluation—
mentoring skills; and (ii) induce more student- Regression Discontinuity Design
centered teaching practices?
2. Will these improvements in teaching The program is evaluated using a Regression Discontinuity
practices also spill over to other teachers
Design (RDD), taking advantage of the fact that selection
and benefit the school as a whole?
for PGP participation relies on teacher performance in
In addition to these primary research questions, the study interviews and simulations. This is a transparent method of
also aims to provide immediate feedback from participating converting their performance in each component into one
teachers and mentors to further support the ministry in overall score (see the section above), with only teachers
improving the implementation quality of PGP. Hence, a who receive a high score being selected. The cut-off value
separate activity sought to answer the following secondary for selection is different in each district since the available
research questions: slots in each district are different.
Secondary Research Question The estimation strategy compares the performance
of teachers who scored just below the cut-off—and who
1. What are teachers’ perceptions of the program?
consequently did not participate in the program—with
2. What is the quality of implementation of the teachers just above the cutoff, who did participate in
program, and how does this vary by district?
the program. Since these teachers have very similar
That is, are all the elements of the program
implemented correctly? characteristics on average, the teachers who scored just
below the cut-off are a valid counterfactual for how the
3. How do policies on becoming future principals
influence teacher motivation to participate in PGP graduates would have performed if they were not
the program? selected. 26
26
The counterfactual is beneficiaries’ outcomes if they did not participate in the program. It is critical have a valid measure of the counter-factual in order to
estimate the causal impact.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 19
The empirical strategy is slightly more complicated (see In this report the results are therefore reported with the
Appendix 4 for more details on the estimating equations). following specifications:
Regression analysis is used to fit a relationship between the
score—what is often referred to as the running variable—and
1. Very narrow bandwidth: The sample is
the relevant outcome indicator (e.g., teaching practices).
restricted to teachers who scored just two
This allows us to de termine the e xpe cted outcome for a
points below and two points above the
non-PGP graduate, if they were precisely at the cutoff for
cutoff. This model is estimated with a linear
selection. Similarly, we use linear regression to estimate the
specification for the running variable.
expected outcome value for a PGP graduate at the cutoff. The
difference between these two values is our estimate of the
2. Narrow bandwidth. The sample is restricted
treatment effect.
to teachers who scored just three points below
and three points above the cutoff. This model is
When performing analysis using an RDD, there are two
also estimated with a linear specification for the
decisions to make. First, the size of the bandwidth: this is
running variable.
the number of teachers who are included in the sample for
analysis. For example, one could restrict the sample to only
3. Full sample, including all surveyed teachers.
teachers who scored one value below and one value about
This model is estimated with a quadratic
the cutoff; or expand it slightly to teachers who scored two
specification of the running variable.
values below and two values above the cutoff; or expand it
even further, up to all teachers in the sample. But analysts
face a trade-off here: if the sample is restricted to teachers We only consider a result to be robust if the magnitude is
who scored very close to the cutoff (i.e., a narrow bandwidth), consistently and meaningfully large under the three different
then the result is the least biased. However, it is also very estimation choices, and statistically significant for at least one
imprecise because of the very small sample. If the sample of the strategies. Note that it is possible that the estimated
is expanded (for example teachers who scored between 10 coefficients are similar in magnitude, but only the estimation
below and 10 above the cutoff), then the sample is larger, using the full sample is statistically significant, given the
and hence there is far more statistical power to detect a increased statistical power from using the full sample. We
statistically significant impact of the program. But the larger still consider this to be a robust result.
sample introduces a risk of bias if the model is not correctly
specified, because it includes teachers who scored poorly
and therefore may be very different in a variety of ways from Sample Selection - Examining the Cutoff
those that scored very well. for Selection
Following best practice, this report therefore shows the
results for multiple different bandwidths. The sample is drawn from all primary-level teachers who were
short-listed to participate in Batch 5. Out of these teachers,
A second decision is how to model the relationship between 350 teachers who were closest to the threshold for selection
the running variable and the outcome. The easiest way is in their respective districts/cities, were sampled for data
to estimate a linear relationship. This is appropriate (and collection. The sampling process involved identifying five
the best approach) if the bandwidth is very narrow. But it locations that had the most teachers close to the threshold,
becomes problematic with a wide bandwidth. In this case, it and then sampling 350 teachers whose score was closest
is appropriate to allow for a non-linear relationship between to the threshold. The final sample had 262 treated and 138
the running variable and the outcome. control teachers. The sampled locations are: Kabupaten
Banjarnegara, Kabupaten Blora, Kabupaten Semarang,
Kabupaten Sumedang, and Kota Jakarta Timur, all are located
in Java island as shown in Figure 4.
20 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Figure 4. Location of Sample Districts
Kota
Jakarta
Timur
Kab. Kab. Kab.
Sumedang Semarang Blora
Kab.
Banjarnegara
Figure 5 below shows the distribution of the selection score an unbiased causal estimate of the impact of participating in
(which we refer to as the “running variable"), broken down by the program.
whether the teacher in our sample was selected to participate
in PGP or not. The treated teachers (i.e., those selected into Figure 5(f) shows the distribution for all the teachers, after
the program) are shown in green bars, the control teachers we have “centered" the running variable. We do this in two
(who were not selected into the program) are shown the clear steps. First, we determine the cut-off value for selection into
bars. Panels (a) to (e) show the distribution separately for each the program separately for each district and teacher contract
district in our sample. There is some non-compliance in some type. Second, we subtract a teacher's score by this cutoff
of the districts: i.e., there are some teachers who scored above value.27 A value of zero means that they received the minimum
the threshold yet were not selected, and some who scored score required for a teacher of their contract type and in their
below the threshold and who were selected. This has some district to be selected. Figure 5(f) clearly shows how there are
implications for the empirical strategy (see Appendix 4) and far more treated teachers to the right of the cut-off, compared
also reduces the statistical power. But it does not undermine to the left of the cut-off, which supports our evaluation design
the internal validity of the results: we are still able to estimate and empirical strategy.
Figure 5. Distribution of sampled teachers by assignment to treatment
(a) Kabupaten Banjarnegara (b) Kabupaten Blora (c) Kabupaten Semarang
(d) Kabupaten Sumedang (e) Kota Jakarta Timur (f) Overall (running variable centered at zero)
Note: The running variable is the teachers’ overall score in the second round of selection. In Figure (f) it is centered to be equal
to zero at the cutoff for selection for a given teacher contract and district.
27
For example, it is 94 for all teachers in Banjarnegara and 95 for all teachers in Sumedang. See the appendix on how we determine the cutoff for the districts with non-
compliance. In two districts the cutoff value that maximizes the power of the first stage is slightly lower for PNS teachers.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 21
Figure 6 below confirms that there is a discontinuous jump in the probability of being selected at the cutoff. The proportion
of teachers who were selected jumps from just above zero below the cutoff, to about 70 percent above the cutoff. This is
encouraging, since we have more statistical power if there is a larger jump in the probability of being treated at the cutoff value.
Figure 6. The relationship
between the running variable
1
and the probability of being
selected to participate in PGP
.8
.6
.4
Note: Y axis is the proportion of
teachers who were selected to .2
participate in Batch 5. X-axis is
the running variable, centered
0
at zero. Each dot represents the
mean value for teachers with -20 -10 0 10 20
a given value of the running
Sample average within bin Polynomial fit of order 2
variable.
Limitations Fourth, the study only examines the impact of teachers who
participated in Batch 5. Since the program is scaling up over
Although the internal validity of our evaluation strategy is time—from 2800 teachers who participated in Batch 1 to
strong, one limitation is the potential lack of generalizability 55,000 teachers who will beselected in Batch 10—the quality
of the findings to the population of PGP graduates as a whole. of implementation across regions could be more varied in
later batches. The characteristics of the teachers selected
First, the sample is restricted to five districts in Java. Our in the different batches might also be different, since it was
sampling strategy for the RDD meant that we only sampled more selective at the start. Therefore, the program might
teachers close to the cutoff, but the only districts that had a have had a larger impact in the earlier batches, relative to
large enough sample of short-listed teachers who scored close later batches.
to the cutoff were all in Java provinces. Our results cannot tell
us what the impacts are in more remote and rural locations Another limitation is that this study has low statistical power.
outside Java. Plausibly, the impacts would be smaller if the Regression Discontinuities Designs are more data-hungry,
quality of implementation was weaker. Therefore, further since analysis should be restricted to observations close to
research focusing in remote and rural areas is warranted to the cutoff. Moreover, the non-compliance discussed above
understand the impact of the program across the country. also reduced statistical power. The implication is that we can
only detect statistically significant effects that are very large.
Second, results are not generalizable to the types of teachers
who scored well in the selection process, since the RDD only It is also important to note that other activities related to
estimates the impact for teachers very close to the cutoff. the Merdeka programs— Sekolah Penggerak, Organisasi
This is a shortcoming that is inherent in any evaluation using Penggerak—were also taking place in both the treatment and
RDD. In contrast, results of a randomized evaluation can be control schools. This has implications for how we interpret
generalized to all treated teachers in the evaluation sample, the results. We are measuring the treatment effect of the PGP
including the best-performing teachers. program, relative to the status quo of other training programs
that are taking place in some of the schools.
Third, the study is focused on primary school teachers, and
does not include secondary school teachers. Hence, there is a
possibility that the observed impact cannot be generalized to
teachers of different levels.
22 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
5.
DATA AND INSTRUMENTS
This study reports results based on two different data program. We discuss each of these data collection activities
collection activities (see Figure 7 for the timeline of in turn.
data collection activities). First, for the Implementation
Assessment, three rounds of phone surveys were conducted
Implementation Assessment (phone survey)
during the training to capture information on the quality of
implementation and teachers' perceptions of the program.
Second, for the Impact Evaluation, in-person data collection The purpose of the phone survey activity was to gather
took place in March and April 2023—roughly 10 months after teacher’s and mentor’s feedback on the program design and
the start of the training, and two months after the completion the quality of implementation. For the teacher survey, the
of the training. The data from the school visits is used to sample was drawn from all teachers participating in PGP
construct outcome measures to estimate the impact of the Batch 5 from 13 districts across Indonesia (see Table 2).28
Table 2. Sample for Phone Survey and School Visit Data Collection
Phone Survey (13 districts) School Visit (5 districts)
1. Kab. Bengkulu Utara 8. Kab. Semarang 1. Kab. Banjarnegara
2. Kab. Kudus 9. Kota Binjai 2. Kab. Blora
3. Kab. Lamongan 10. Kota Jakarta Timur 3. Kab. Semarang
4. Kab. Maros 11. Kota Salatiga 4. Kab. Sumedang
5. Kab. Pacitan 12. Kota Tanjung Pinang 5. Kota Jakarta Timur
6. Kab. Pasaman 13. Kota Yogyakarta
7. Kab. Pekalongan
28
The sample of districts for the Implementation Assessment and Impact Evaluation is different for two reasons. First, we were able to collect data in more
districts with phone surveys, whereas cost constraints meant that the in-person data collection could only take place in five districts. Second, these districts
were selected for an initial design of a randomized evaluation of PGP. This design was not feasible given changes in the timing of the program. A different set
of districts were more appropriate for the RDD.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 23
Table 3. Summary of Phone Survey Data Collection
Phone Survey Date of Survey Sample Measure
PGP Teacher – Round 1 June 2022 200 Quality of the LMS
PGP Teacher – Round 2 November 2022 200 Quality of workshop and individual coaching
Mentor Teacher (Pengajar Praktik) December 2022 10 Mentor preparedness and implementation of
workshop and coaching
PGP Teacher – Round 3 January 2023 199 Quality of overall program
Over 1,000 teachers participated in PGP Batch 5 from these aspirations. Additionally, detailed questions were included to
13 districts. We selected 400 teachers through a randomized gauge their perception of the quality of PGP implementation.
sampling approach and then assigned these teachers in two To ensure a larger sample size for data analysis, some
cohorts of 200 teachers, conducting three rounds of data questions from Round 1 were retained and incorporated
collection. The first cohort of 200 teachers was surveyed at into Round 2. Furthermore, specific questions from Round 1
the beginning and the end of the training. The second cohort were repeated in Round 3 to facilitate a comparison of the
of teachers was surveyed in the middle of the training. teachers' perceptions before and after participating in the
program.
For the mentor teacher survey, a total of 10 mentors were
selected through random sampling. The sample was drawn
from all mentor teachers in the same 13 districts of the School Visit for Impact Evaluation
PGP Batch 5 teachers. Since these mentors were randomly
selected, we cannot link the teachers sampled for the phone
Instruments
survey and these sampled mentors.
The data collection instruments for the school visits
The teacher phone survey was conducted in multiple rounds were developed in partnership with MoECRT, with the
throughout the implementation period of PGP Batch 5. Each goal of capturing most of the outcomes identified in
round aimed to collect information on different elements of MoECRT’s theory of change (Table 4 below). Four different
the training, as summarized in the table below. The mentor survey instruments were developed: a teacher survey,
survey was conducted once, during Batch 5 implementation. administered to the sampled teacher; a peer-teacher survey,
administered to a teacher who teaches the same grade or
The instruments used for data collection were collaboratively an adjacent grade to the target teacher; a school principal
developed with the team from MoECRT. In each round of survey; and a student survey that was administered to ten
the survey, we collected basic demographic information randomly selected students taught by the target teacher.
such as gender, age, employment status, highest education We also administered two different classroom observation
completed, years of teaching experience, and the school instruments, to measure teaching practices of the target
level where the teachers were currently teaching. teachers. After data collection started, an additional module
for sample teachers who had become a school principal by
The surveys administered to the teachers inquired about
the time of data collection.
their motivation for participating in the PGP and their career
" The data collection instruments for both phone survey and school visits
were developed in partnership with MoECRT, with the goal of capturing
"
most of the outcomes identified in MoECRT’s theory of change.
24 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table 4. Mapping between MoECRT’s groupings of final outcomes and our evaluation instruments
Major grouping Grouping Description Instrument
I Teachers have the I.2 Teacher reflects on their own competencies Teacher survey
moral, emotional, Self-improvement and strengths and weaknesses; develops a
and spiritual self-development plan; and executes this
maturity to behave plan.
according to the
code of ethics I.3 Teacher masters social-emotional Student survey
Socio-emotional competence in teaching practice in schools.
competencies This includes: identifying emotions,
responding positively to emotions, and
showing empathy to students’ point of view,
II Teachers plan, II.1 Teacher (a) engages students in crafting Student survey,
execute, reflect Positive discipline class beliefs; (b) applies, monitors or teacher survey,
on, and evaluate managers positions consistently in the Classroom
student-centered learning process in the classroom and observations
learning in schools. school, and (c) applies the restitution
process to the students when they make a
mistake.
II.2 Teacher frequently assesses students and Student survey,
Differentiated adapts content and teaching practices to teacher survey,
learning the needs of the child. classroom
observations.
II.3 Teacher (a) structures teaching to improve Classroom
Socio-emotional students’ socio-emotional skills, (b) uses observation
competencies various strategies to integrate social and
emotional learning.
III Teacher encourages III.1 Teacher develops the community of learning Peer survey,
peers/communities Create communities with peer-group to enhance competency school principal
in their schools to of learning and student-centered learning survey
implement student-
centered learning III.2 Teacher implements coaching to develop Peer survey,
Coaching their peer-group to enhance student- school principal
centered learning survey
IV Teacher initiates IV. Teacher has a vision to develop student- Peer survey,
student-centered 1 and IV.2 vision of centered learning and initiates development school principal
school development student-centered with relevant stakeholders survey
with collaborative school development
resource (asset)
based
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 25
The teacher survey was administered to the target their views of the teacher (e.g., “I like the way the teacher
treatment and control teachers in each school. It included treats me when I need help), again reported on a Likert
basic demographic characteristics, as well as questions scale ranging from 1 (Strong agree) to 4 (agree), and some
about prior exposure to teacher training programs, and binary questions about specific teacher behaviors. Finally,
the perceived perception of the quality of the PGP, if they it also asked the student to list the number of students who
participated. It also asked about teachers’ self-confidence were afraid of the teacher or were afraid to ask the teacher
as a leader, enjoyment of teaching, and whether the teacher questions. In order to make this salient, the student first
had conducted the self-reflection activities that form part had to list three classmates, and then report how many of
of the program: i.e., whether a teacher had done strengths these classmates were afraid of the teacher.
mapping, had a strength mapping document, and could
mention areas of improvement. Two different classroom observation instruments were
administered to the targeted teacher, one developed
The peer teacher survey was administered to a teacher internally by MoECRT, and the World Bank Teach instrument.
who teaches in the same grade as the target teacher, or an The be ne fit of using the gove rnme nt's own instrume nt
adjacent grade if there was no other teacher in the same is that it is more likely to capture specific teaching
grade. The main purpose was to capture a more objective practices targeted by the program, since the instrument
me asure of the targe t teache r’s engage ment with other was explicitly developed with this program in mind. For
teachers in the schools (outcome groupings III.1, III.2, example, this instrument is used as a monitoring tool by
and IV). It therefore included questions about whether the mentors when they observe teaching, so they include
a community of learning existed, and whether a teacher indicators that are key outcomes of the program. But there
had done the following: mapped out student needs, self- is a risk that teachers are only changing behavior along the
reflected on their strengths and weaknesses, and discussed specific dimensions of this instrument. It is for this reason
student-centered learning with another teacher. It also that we decided to also use the Teach instrument, which
asked the teacher whether they had been observed in would provide a diffe rent and inde pe ndent asse ssme nt
the classroom, and who initiated the above-mentioned of overall change in teaching practices. Moreover, the
activities. The enumerators then indicated whether the Teach instrument has been rigorously tested, piloted,
target teacher initiated the activity or not. and validated for data collection purposes in a variety of
contexts, and has a detailed set of guidelines and specific
The school principal survey included the same demographic training protocols for applying the instrume nt. We can
questions and questions about exposure to training as the therefore have confidence in the reliability of the measure,
teacher and peer teacher surveys. In addition, it included and data is likely to be captured with less noise.
a battery of questions to measure the school principal’s
assessment of the quality of leadership of the sampled The Teach classroom observation tool measures three
teacher; and questions related to outcome IV from the broad categories of teaching practices: classroom culture,
theory of change: whether a community of learning exists in instruction, and developing socio-emotional skills. Each of
the school, whether the school principal had been asked to these broad categories has two or four clusters of behavior
reflect on the school learning process, whether someone had (see table below), nine in total. In turn, each cluster includes
talked to them about the school vision. As in the peer survey, a series of behaviors. Enumerators were trained to provide
the enumerators then indicated whether the school principal a score for each of these behaviors. A single measure for
mentioned the target teacher as initiating the activity or each cluster is calculated by taking the mean of the scores
discussion, or not. of all the behaviors related to the cluster. Then the mean of
all the clusters creates a score for each category. Finally, the
The student survey was developed to measure students’ mean across all categories creates an overall Teach score.
perceptions of teaching quality. It included a series of The overall score is standardized to have a mean of zero and
questions on a Likert scale—ranging from never (1) to always standard deviation of one. The results for the overall score,
(4)—that relate to the teaching practices of the sampled as well as for each category and cluster of behaviors, are
teacher (for example: “the teacher asks how I am doing and reported.
how I am feeling”). It also included some questions about
26 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table 5. Description of the Teach classroom observation instrument.
Category Cluster Behavior
Classroom Supportive • Teacher treats all students respectfully
culture Learning • Teacher uses positive language with students
Environment • Teacher responds to students’ needs
• Teacher does not exhibit bias and challenges stereotypes in the classroom
Positive • Teacher sets clear behavioral expectations for classroom activities
Behavioral • Teacher acknowledges positive student behavior
Expectations • Teacher redirects misbehavior and focuses on the expected behavior,
rather than the undesired behavior
Instruction Lesson • Teacher explicitly articulates the objectives of the lesson and relates
Facilitation classroom activities to the objectives
• Teacher explains content using multiple forms of representation
• Teacher makes connections in the lesson that related to other content
knowledge or students’ daily lives
• Teacher models by enacting or thinking aloud
Checks for • Teacher uses questions, prompts or other strategies to determine students’
Understanding level of understanding
• Teacher monitors most students during independent/group work
• Teacher adjusts teaching to the level of students
Feedback • Teacher provides specific comments or prompts that help clarify students’
misunderstandings
• Teacher provides specific comments or prompts that help identify
students’ successes
Critical thinking • Teacher asks open-ended questions
• Teacher provides thinking tasks
• Students ask open-ended questions or perform thinking tasks
Socio-emotional Autonomy 1. Teacher provides students with choices
skills 2. Teacher provides students with opportunities to take on roles in the
classroom
3. Students volunteer to participate in the classroom
Perseverance 1. Teacher acknowledges students’ efforts
2. Teacher has a positive attitude towards students’ challenges
3. Teacher encourages goal setting
Social and 1. Teacher promotes students’ collaboration through peer interaction
collaborative 2. Teacher promotes students’ interpersonal skills
skills 3. Students collaborate with one another through peer interaction
MoECRT’s classroom observation instrument has 13 We construct binary variables equal to one if a teacher
different teaching behaviors. For each of these behaviors received a top score for each respective teacher behavior.
the enumerator provides a score ranging from one to four. These binary indicators are used in the analysis rather than
Unlike the Teach instrument, this instrument does not have the Likert scales, for two reasons. First, they relate to the
standardized guidelines or training protocols on how to specific teacher competencies as expressed in theory of
score the te achers along the diffe rent dime nsions. The se change. Second, it aids interpretation of results. Moreover,
behaviors can roughly be classified into the three elements of for most behaviors the plurality of teachers received the top
student-centered teaching: positive discipline, differentiated score, so the meaningful variation is captured in the binary
instruction, and socio-emotional skills. indicator. Results do not change in any meaningful way when
using the Likert scales rather than binary variables in the
analysis.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 27
Table 6 below provides a summary of the MoERCT indicators, grouped by domain. The second column describes the teacher
competency required to receive a top score of four. Note that these are extremely high (perhaps unrealistically high) standards
for top performance.
Table 6. Summary of the MoECRT classroom observation instrument
Variable name Teacher competency reached
Panel A. Positive discipline
Teacher communicates Teacher communicates reciprocally with their students (teacher-student) and also gives
reciprocally students the opportunity to interact with each other (students).
Manager position Teacher uses the principle of restitution whereby reasonable mistakes occur, and
they accept the mistakes of students. The teacher asks the students why they made a
mistake and helps them find a way out. This type of teacher sometimes uses the position
of monitor and manager in students in certain situations. Example: "What are you going
to do now? What can we do to correct this mistake?"
Motivation Teacher fosters students' desire to learn by making during learning sessions engaging
and interesting. Example: The teacher designs learning that matches the interests of the
students.
Panel B. Differentiated learning
Adjust learning environment Teacher modifies the learning environment according to learning objectives, student
characteristics, and social and emotional learning objectives.
Support and independent Teacher supports students according to their learning needs. Students can do their
assignments assignments independently as a result of the support
Modify learning strategies Teacher smoothly modifies strategies, materials, and groupings to optimize students'
opportunities to learn and meet their learning needs. Students are actively involved
throughout learning and focused on work that not only develops skills and challenges
them to solve problems
Concrete, clear, constructive Teacher provides constructive feedback (clear and containing development advice) not
feedback only on the student's work but also their learning process (e.g., student thinks, how the
pupil solves problems, communication techniques).
Differentiated assessment As learning progresses, an assessment process occurs that is according with the learning
objectives. There is evidence of teachers' efforts to differentiate to meet students'
learning needs and ensure students understand the assessment process
Panel C. Socio-emotional
Empathetic Teacher finds out what their student is working on and thinking. The teacher listens
actively to the opinions expressed by all students
Positive emotions Teacher displays a consistently positive expression of emotions (displays happy,
enthusiastic, friendly emotions
Socio-Emotional Content in Activities that reinforce students' socio-emotional competence have been integrated
Learning Activities into learning at: a warm opening stage, a core involving-challenging-engaging stage,
and an optimistic closing stage by fusing it with academic content being studied
28 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Construction of Indicators and Indices total of 310 schools.
We structure our results by the groupings of final outcomes, Descriptive Statistics
as developed by MoECRT (see Table 4 above). Within each
grouping, binary indicators are constructed to indicate Table 7 shows basic descriptive statistics of the sample.
whe the r the spe cific de rivative indicator (see Table 2) is The first column shows the mean values for those who were
met. We then take the arithmetic mean of all the indicators selecte d for PGP Batch 5; the se cond column shows the
that relate to a specific grouping of outcomes (e.g., mean values for those who were not, the third column shows
positive discipline), in order to have an index of the overall the mean values for all 350 sampled teachers; and the final
performance. The groupings related to student-centered column shows the diffe rence be tween PGP and non-PGP
learning have two indices, based on the student survey and teachers. 218 out of the 350 sampled teachers were selected
classroom observations data, respectively. We deviated to participate in the program, and 132 were not. Panel A uses
from MoECRT’s structure, by reporting the socio-emotional administrative program data, Panel B relies on data collected
competencies all in one table. during the school visits.
Timing of Data Collection The average age in the sample is 38, the majority (74 percent)
are female, all have a bachelor's degree but only about six
Data collection took place between March and April 2023 percent also have a master’s degree. Sampled teachers have
– roughly 10 months after the start of the training, and two on ave rage 15 ye ars of experie nce , and the majority are
months after the completion of the training. No primary permanent civil servant teachers (62 percent).
data collection took place prior to the start of the training.
During data collection the target sample of 350 teachers in The final column shows the difference between the PGP and
five districts29 were surveyed and observed in the classroom. non-PGP schools. As expected, those who were selected
A random sample of 10 of the se teachers’ stude nts we re to participate received a higher score on average. They are
surveyed, along with their school principals, and the peer also less likely to be female and civil servants. This might
teacher. During the implementation of the survey, 31 be because men and civil servant teachers received a lower
teachers had to be replaced because they had become score in the selection tests, on average: women received
school principals (the survey firm had been provided with an overall score of 97, compared to men who received an
a list of 50 potential replacement teachers). 65 percent of overall score of 98. And civil servants received a score of 98
the school principals are female, compared to 70 percent compared to 96 for contract teachers. If we control for the
of PGP graduates who are female. The school principals final score, then we find that women are 7 percentage points
were still surveyed in order to capture basic demographic more likely to be selected, and there is no difference between
characteristics, and also their perceptions of the quality of civil servant and other teachers in the probability of being
the program and self-confidence as a leader. But, since the selected. Encouragingly, all the teachers who were selected
majority of the outcomes related to teaching practices, the to participate in Batch 5, according to the government
additional 31 replacement teachers were also surveyed. records, also indicated that they did so in the survey. There
Because there were cases where there was more than one is one non-selected teacher who ended up participating in
target teacher in the same school, data were collected in a Batch 7.
Figure 7. Timeline of Data Collection
Start of the
Phone survey round 2 Phone survey round 3 End of school
training program
Group B: Group A: visit data
200 teachers 200 teachers collection
June 2022 December March
2022 2023
May November January April
2022 2022 2023 School visit data
2023
Phone survey Mentor
round 1 Phone survey collection
Group A: 10 mentor 381 sample
End of the training
200 teachers teachers teachers
program
29
Banjarnegara, Kab. Blora, Kab. Semarang, Kab. Sumedang, dan Kota Jakarta Timur
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 29
Table 7. Teacher Characteristics - by PGP
Variable (1) (2) (3) T-test
PGP Control Total Difference
Mean/SE Mean/SE Mean/SE (1)-(2)
Administrative Data
Selection score 100.243 91.356 96.891 8.887***
(0.258) (0.318) (0.305)
Female 0.693 0.818 0.740 -0.126***
(0.031) (0,034) (0.023)
Civil servant teacher 0.683 0.515 0.620 0.168***
(0.032) (0.044) (0.026)
Contract teacher 0.243 0.341 0.280 -0.098**
(0.029) (0.041) (0.024)
Sekolah Penggerak 0.050 0.015 0.037 0.035*
(0.015) (0.011) (0.010)
Survey data
Masters degree 0.064 0.045 0.057 0.019
(0.017) (0.018) (0,012)
Teacher certificate 0.830 0.674 0.771 0.156***
(0.025) (0.041) (0.022)
Age 37.670 37.765 37.706 -0.095
(0.398) (0.502) (0.311)
Years experience 15.601 15.530 15.574 0.071
(0.350) (0.445) (0.274)
Years of experience in school 7.583 8.523 7.937 -0.940
(0.398) (0.533) (0.320)
PGP—batch 5 1.000 0.000 0.623 N/A
(0.000) (0.000) (0.026)
PGP—batch 7 0.000 0.008 0.003 -0.008
(0.000) (0.008) (0.003)
Organisasi Penggerak 0.271 0.235 0.257 0.036
(0.030) (0.037) (0.023)
Sekolah Penggerak 0.096 0.076 0.089 0.021
(0.020) (0.023) (0.015)
Kurikulum Merdeka 0.321 0.439 0.366 -0.118**
(0.032) (0.043) (0.026)
N 218 132 350
Notes. The value displayed for t-tests are the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical
level.
Table A.4.1 in the Appendix shows descriptive statistics for from the perspective of the program, which aims to develop
the larger sample that includes the 31 teachers who were leadership qualities, but provides challenges from an
replaced because they became school principals. Roughly analytical perspective, since the teaching practices of these
eight percent of the teachers who were originally sampled principals cannot be observed. There is risk of selection bias
had become school principals by the time of data collection. if these teachers who became principals have better or worse
This proportion is higher for the PGP graduates (11 percent) teaching practices.
than the non-PGP graduates (4 percent). This is good news
30 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
6.
RESULTS
The report first discusses the Implementation Assessment program, the survey provides valuable insights and regular
results, before moving onto the Impact Evaluation results. updates for the government to improve the program.
Phone Survey for Implementation As shown in Figure 8, teachers’ reported perception of the
Assessment Results quality of the program is extremely high. When asked to
rate, between 0 and 10, how the different components of
The phone survey results are structured to answer the the program help them understand PGP, and how much
secondary research questions related to the quality of they learnt from online Learning Management System
implementation. These include understanding teachers' (LMS), almost all teachers gave an overall score of 10,
perceptions of the program, evaluating program and the mean value ranges between 9.2 to 9.47 for these
implementation quality across different districts, and different components. There is clearly social desirability
assessing participants' perceptions of the policy regarding bias (or politeness) at play in these very high scores.
future prospects as school principals. Since it was taken Nonetheless, these high scores are positive indicators of
three times during the implementation of the training teachers’ perceptions of the program.
Figure 8. Quality of implementation for different components of the program
Workshops Helps Understand PGP 9.33
Coaching Helps Understand PGP 9.20
Learned from LMS assignments 9.47
Learned from accessing LMS 9.38
0 2 4 6 8 10
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 31
Figure 9 shows that teachers’ perceptions of the quality Figure 10. District-level variation in perceived
and knowledge of the instructors and facilitators is also quality of the facilitator, instructor and mentor
uniformly high. The majority of teachers provided a score of
10/10, and 75% of teachers gave a score of 9 or 10. The only 10
exception is teachers’ assessment of the mentors’ mastery
of topics during the workshops, where the median rating
9.5
was 9/10, and a quarter of teachers gave a score below that.
Although a median score of 9/10 is high in absolute terms, it
is an outlier relative to all other indicators of performance. 9
Figure 9. Distribution of perceived quality of the 8.5
facilitators, instructors, and mentors
8
Facilitator Mentor (workshops)
Instructor Mentor (coaching)
Notes. Box plots for the district average responses for different questions
where teachers rated different domains of the quality of instructors/
facilitators/mentors with a score between 0 and 10. Instructor is the
average rating across all three modules.
We also surveyed mentors to understand mentor’s training
and preparation (Figure 11). Although they received
handbook materials and underwent a fully online training,
their rating on how much the training prepared them as a
mentor is much lower than all the other indicators discussed
in this section. Also, they are not uniformly applying these
materials in their schools. This is concerning, since these
mentor teachers should be applying these changed teaching
methods, if they are expected to help other teachers do so.
5 6 7 8 9 10
Figure 11. Mentor Training Effectiveness
Facilitator quality
Instructor quality (module 1) Training prepared
me for role as 8.25
Instructor quality (module 2) mentor
Instructor quality (module 3)
Applied PGP
Instructor preperation materials at school 8.05
Instructor knowledge
Material useful
Mentor master topics (coaching)
for professional 9.85
Mentor master topics (workshop) development
0 2 4 6 8 10
Notes. Box plots for different questions where teachers rated different
domains of the quality of instructors/facilitators/mentors with a score Another area for improvement is related to workshop
between 0 and 10. The median score is 10 for all variables except mentor
logistics. A total of 76 percent of teachers responded that
mastery of topics during the workshop, which is nine.
logistics needed to be improved. When asked open-ended
questions about specific logistical improvements, many
There is also some regional variation in the perceived quality teachers expressed concerns about the workshop locations
of the different components of the programs. For example, due to inadequate basic facilities such as functional toilets,
Figure 11 shows that the average perception of the mentor’s difficulties in transportation access to reach the venues,
mastery of topics during the workshops ranges from 8.3 in and issues regarding meals and lack of mineral water for the
one district and 9.6 in another district. entire day's sessions.
32 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Figure 12. How workshop can be improved gaps and disparities in workshop venues across different
districts, as some areas have well-equipped facilities while
Topic and others face challenges in this regard.
14%
Activities
Teaching Finally, to understand teachers' perception of the new
8%
Materials regulation30 regarding their prospects of becoming school
principals upon completing the PGP program, we conducted
Facilitator 30%
a comparison of teachers' career aspirations at the beginning
Duration and 33%
(phone survey Round 1) and the end (phone survey Round
Timing 3) of the program, using the same set of 200 teachers. The
Modality (Online, 2% findings revealed that a significant number of teachers
offline, hybrid) altered their aspirations during the program. For instance,
Logistics 76% initially, 34 percent of teachers expressed a desire to become
school principals, which increased to 50 percent by the end
Nothing 13% of the program. Among those teachers who did not express
a desire to become school principals, the main reason cited
0 2 4 6 8 was because they felt too young or lacking in experience.
Nevertheless, this shift indicates a notable change in
teachers' career goals associated with their participation in
Figure 13. Proportion of teachers that said logistics can be the PGP program.
improved
Figure 14. Teachers' five-year career plans
95%
90% Teacher
44%
89%
33%
Proportion based on district
87%
83% School
34%
principal
82% 50%
80%
School
70% 14%
supervisor
62% 15%
40%
Other
40% educational 9%
10% leadership 3%
0% 20% 40% 60% 80% 100% 0 .1 .2 .3 .4 .5
Proportion of teachers said that Baseline
Proportion of Teachers
logistics can be improved Endline
There are regional variations in the challenges faced In conclusion, the overall perception of the program among
with workshop logistics. The colors in the above figure teachers is positive. The quality of implementation is generally
represent different districts, indicating varying levels of good, although there is room for improvement such as mentor
agreement among teachers regarding the need for logistical preparation and workshop logistics. Notably, a significant
improvements. For instance, in one district, 95 percent of number of teachers have indicated a shift in their career
teachers agreed that logistical improvements are necessary, aspirations, with more individuals expressing a willingness to
while in another district, only 10 percent expressed the same take on leadership roles, such as becoming school principals
sentiment. This highlights the importance of addressing the and school supervisors.
30
Please note that Permendikbud No. 40/2021 and Permendikbud No.26/2022 were introduced after selection process of Batch 5. Participants of Batch 5 weren’t aware
of the ’benefit’ of participating in the program when they applied for the program.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 33
Impact Evaluation Results
The main results of the Impact Evaluation can be summarized as follows (see Table 8):
1 2
The quality of Teachers’ application of positive discipline, as measured by the
implementation was high, MoECRT classroom observation instrument, improved. However,
and teachers viewed the there is no observed improvement in teachers’ socio-emotional
program highly favorably. competencies, nor in the providing differentiated instruction.
3 4
Teaching practices, as measured using There is evidence that PGP graduates took the
the Teach classroom observation initiative to support other teachers in their schools.
instrument, substantially improved. The For example, other teachers in the same school as
gains are especially large for the following the PGP graduates are more likely to have conducted
indicators: lesson facilitation, providing a strength mapping exercise, been asked to self-
a supportive learning environment, and reflect on their learning process, discussed student-
promoting critical thinking. centered learning, and mapped out student needs.
Table 8. Summary of main results
5
Teachers enjoy teaching more and are Indicator Impact
more self-confident in their leadership
abilities. But head teachers have a lower ToC I. Self-development and reflection
assessment of their leadership skills, Self-development Small positive
relative to the control teachers. ToC II. Teaching practices
Positive discipline Larger positive
Differentiated learning None
Socio-emotional competencies None, or negativea
Teach instrument Large positive
ToC III. Support other teachersb
Learning community None
Self-development plan Large positive
Coaching Large positive
ToC IV. Initiate student-centered school development
Vision, initiated by PGP teacher Small positive
Other outcomes
Enjoyment of teaching Large positive
Self-confidence as a leader Large positive
a
No impact, based on student responses, negative impact based on
government classroom observations. School principal's perception of Small negative
b
Only when the sample is restricted to schools where there are no leadership ability
other PGP graduates in the school
34 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Quality of Implementation and Other relevant. Finally, all the teachers indicated that they had
Training Programs received mentorship in person. The average number of times
that they met with the mentor was 7.6 out of 8 total number
Table 9 provides evidence of a high quality of implementation, of individual mentoring sessions. The length of each session
as reported by the teachers who participated in the PGP was roughly three hours. And they also rated the quality of
program. Results are shown separately for each district in our the mentors highly, at 9.25 (out of ten). There is no strong
sample, as well as the overall sample. The mean score for how difference across districts, with the exception of Banjarnegara,
much teachers believed they learnt from accessing the learner where teachers typically provided slightly lower scores in
management system was 9.3 (out of 10). Almost all teachers the ir subjective asse ssme nt of the diffe rent compone nts
indicated that they interacted with an instructor, and both of the program. We do not know whether this is because of
the facilitator and instructor received high scores: 9.47 and weaker quality of implementation, or different expectations/
9.04, respectively. All participants reported having attended standards from the teachers.
a workshop, and these were always held in person (although
in some districts the workshops had a hybrid modality for a Overall, we can conclude that the teachers rated the quality of
small fraction of the teachers). About 85 percent of teachers the program highly, and that while it is intensive (it requires a
indicated that they believed that the workshops were very lot of time from teachers), it was well-implemented.
Table 9. Quality of Implementation by District
Banjarnegara Blora Semarang Sumedang Jakarta Total
Timur
Device used
Phone 0.95 0.95 0.88 0.95 0.96 0.95
Tablet 0.00 0.00 0.12 0.09 0.16 0.09
Laptop 0.98 0.98 1.00 0.95 1.00 0.99
Computer 0.09 0.09 0.31 0.16 0.33 0.21
Device used most frequently
Phone 0.11 0.04 0.27 0.30 0.12 0.14
Tablet 0.00 0.00 0.00 0.00 0.00 0.00
Laptop 0.86 0.95 0.73 0.63 0.88 0.84
Computer 0.00 0.00 0.00 0.07 0.00 0.01
Learning Management System
Learnt from LMS (0-10) 8.73 9.41 9.04 9.72 9.51 9.32
Interacted with instructor 1.00 0.96 1.00 0.95 0.98 0.98
Quality of facilitator (0-10) 9.05 9.57 9.42 9.63 9.58 9.47
Quality of instructor (0-10) 8.75 9.03 9.12 9.09 9.28 9.06
Workshop
In-person 1.00 1.00 1.00 0.98 0.98 0.99
In-person and online 0.00 0.00 0.00 0.02 0.02 0.01
No. workshops attended 7.30 7.18 7.27 7.14 7.39 7.25
Very relevant 0.73 0.86 0.92 0.88 0.88 0.85
Quite relevant 0.25 0.14 0.08 0.12 0.12 0.14
Mentorship
At teacher's school 1.00 1.00 1.00 1.00 1.00 1.00
At district 0.00 0.00 0.00 0.02 0.00 0.00
At another PGP school 0.00 0.00 0.00 0.00 0.02 0.00
Mentored in person 1.00 1.00 1.00 1.00 1.00 1.00
Mentored in group 0.00 0.03 0.04 0.07 0.09 0.05
Mentored online 0.00 0.19 0.00 0.02 0.11 0.09
No. times met 7.05 7.15 6.77 8.02 8.67 7.60
Length of session (in minutes) 184.09 180.27 195.77 191.06 201.05 189.39
Quality of Pengajar Praktik (0-10) 9.05 9.24 9.27 9.44 9.28 9.25
Observations 60 94 64 48 84
Notes. The first five columns report mean values for each respective district, the final column shows mean values for the whole sample of 350 teachers. All
indicators are binary variables, unless otherwise indicated. A value of 1 for a binary variable means that 100 percent of teachers said yes.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 35
We also aske d te ache rs in our full sample about the ir our estimates might underestimate the true impact of the
exposure to training in general (results in Table 10, split PGP program.
between the teachers who were selected to participated in
Batch 5 of PGP, and those who were not). There are a few As for the characteristics of the training that they attended
points worth noting. First, almost all teachers who were not (Panel B), only about 70 percent of Batch 5 teachers
selected (our control group) also participated in some kind indicated that they received training in PGP. This might
of training over the past 12 months. So, there is a high degree be because the survey allowed them to only select one
of exposure to training for teachers. Second, even though the training, and they had participated in another training more
control teachers are not PGP graduates, about 40 percent of recently. This again highlights the high degree of teachers’
teachers in the control listed that at least one other teacher exposure to training. Moreover, teachers in the treatment
(or the school principal) in the school is a PGP graduate. This group are generally more satisfied with the PGP training
suggests that our estimates of the school-wide impact of the that they participated in, relative to the control, and they
program might be substantially under-stated, because so are substantially more likely to believe that the training will
many control teachers have already been influenced by peer greatly improve their teaching. This is encouraging for the
teachers who graduated from PGP in previous batches. Third, program. Finally, a high proportion of teachers who did not
a higher proportion of the control teachers indicated that participate in the program have learnt about the different
their school participated in pilot Kurikulum Merdeka (KM): components covered in PGP, such as student-centered
43 percent, relative to 33 percent in the treatment group. learning (97 percent), differentiation (89 percent), and
If teachers improve their teaching practices and leadership the main assets/capital (73 percent). Nevertheless, these
skills because of the other programs operating in schools— proportions are larger, and close to 100 percent, for Batch
Organisasi Penggerak, Sekolah Penggerak, and KM—then 5 teachers.
Table 10. Exposure to training (by selection into Batch 5)
Variable (1) (2) (3) T-test
Selected batch (5) Control Total Difference
Mean/SE Mean/SE Mean/SE (1)-(2)
Panel A. All teachers
Participated in training 100.243 0.949 0.979 0.047***
(0.258) (0.019) (0.007)
Teacher or school principal in PGP 0.693 0.394 0.743 0.544***
(0.031) (0.042) (0.022)
School participated in Organisasi Penggerak 0.683 0.226 0.249 0.036
(0.032) (0.036) (0.022)
School participated in Sekolah Penggerak 0.243 0.073 0.081 0.013
(0.029) (0.022) (0.014)
School participated in pilot Kurikulum Merdeka 0.050 0.431 0.365 -0.103**
(0.015) (0.042) (0.025)
137 381
Panel B. Teachers who participated in training
Training in PGP 0.704 0.000 0.458 0.704
(0.029) (0.000) (0.026)
Utilization of Merdeka Mengajar Platform 0.095 0.369 0.190 -0.275***
(0.019) (0.042) (0.020)
Satisfied with training (0-10) 9.070 8.377 8.828 0.693***
(0.063) (0.106) (0.058)
Greatly improve teaching 0.856 0.531 0.743 0.325***
(0.023) (0.044) (0.023)
Learn about student-centered learning 1.000 0.969 0.989 0.031***
(0.000) (0.015) (0.005)
Learn about restitution process 0.996 0.792 0.925 0.204***
(0.004) (0.036) (0.014)
Learn about differentiation 0.996 0.885 0.957 0.111***
(0.004) (0.028) (0.011)
Learn about main assets/capital 1000 0.731 0.906 0.269***
(0.000) (0.039) (0.015)
Know the main assets/capital 1000 0.800 0.930 0.200***
(0.000) (0.035) (0.013)
N 243 130 373
Notes. The value displayed for t-tests are the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical level.
36 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Self-reflecting and Self-improving Teachers below or three points above the cutoff; and column (4) shows
results when using the full sample of teachers but allowing for
The first set of outcomes relates to teachers’ ability to self- a quadratic relationship between the running variable and
reflect and improve based on their own perceived strengths the dependent variable.
and weaknesses, as reported in Table 11 and Figure 15. The
first column in Table 11 shows the mean value of the outcome We only confidently conclude that there is a positive impact
for teachers who were not selected but scored one or two if the results are consistently large in magnitude across three
points below the cutoff value for selection in their district. We different estimation strategies in columns (2) to (4)
can interpret this as the counterfactual outcome: the mean
in the control. Column (2) shows the impact of the program For exposition purposes, Figure 15 shows the results of
when the sample is restricted to teachers who only scored two Table 11 graphically, when the sample is restricted to those
points above or two points below the cutoff for selection in who scored two below or two above the cutoff. The blue bar
their districts. This is our preferred estimate of the treatment indicates the mean in the control group (from column (2)), and
effect. For robustness, column (3) shows the impact when the orange bar shows the estimated treatment effect (column
the sample is restricted to teachers who scored three points (3)).
Table 11. TOC I. Self-reflection and self-development
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x< 0 |x| ≤ 2 |x| ≤ 3 Full sample
Index 0.70 0.14 0.03 -0.04
(0.03) (0.13) (0.12) (0.09)
Maps strength 0.90 -0.12 -0.16 0.01
(0.06) (0.22) (0.21) (0.16)
Shows strength mapping document 0.20 0.56** 0.29 -0.06
(0.07) (0.28) (0.24) (0.18)
Can mention area for improvement 1.00 -0.03 -0.05 -0.07
(.) (0.05) (0.05) (0.05)
Observations 31 89 127 350
Notes. Row headings denote the dependent variable, x refers to running variable, centered at zero. The first column is the mean for non-PGP teachers who
scored one or two points below the cutoff for selection their district. Columns (2) to (4) show the local average treatment effects, estimated using equations (1)
and (2). In column (2) the sample is restricted to teachers who scored between two points below and two points above the cutoff. In column (3) the sample is
expanded to teachers who scored between three below and three above the cutoff. In column (4) the same is expanded to all surveyed teachers. All estimations
control for the teacher's contract type and x, interacted with treatment. Column (4) also controls for 2, interacted with treatment. Each regression includes
additional control variables that are selected using the adpative Lasso procedure. Baseline variables that are either predictive of treatment or predictive of the
dependent variable are included. Data from the teacher survey. All dependent variables, except for the indices, are binary. "Index" is the arithmetic mean of the
all the other variables in the panel. "Map strength"=1 if a teacher mentioned that they have mapped their strengths and weaknesses. "Shows strength mapping
document"=1 if a teacher can show the document. ***, **, and * indicate significance at the 1, 5, and 10 percent critical level.
Figure 15. Reflection and self-development
Non-PGP Impact
Index 0.7 0.14 Notes. The blue bar indicates the mean in for
teachers who scored one or two points below
the cutoff for selection. The orange bar shows
Show strength 0.2 0.56**
the estimated local average treatment effects,
mapping document restricting the sample to teachers who scored
two below or two above the cutoff for selection.
Data from the teacher survey. All dependent
Map strength -0.12 0.9 variables, except for the indices, are binary.
“Index” is the arithmetic mean of the all the
other variables in the panel. “Maps strength”=1
if a teacher mentioned that they have mapped
Can mention area -0.03 1
for improvement their strengths and weaknesses. ***, **, and
* indicate significance at 1, 5, and 10 percent
critical level. See Appendix 4 for more details on
-0.2 0 0.2 0.4 0.6 0.8 1 1.2 the empirical strategy and regression results.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 37
Figure 15 shows that there was no impact on the proportion As shown in Figure 16, there is consistent evidence of
of teachers who indicated that they mapped their strengths improvements in application of positive discipline, based
or weaknesses, mostly because all teachers, including those on both the student-level (Panel A) and teacher-level
in the control, stated that they did so. However, there is 56 (Panel B) data. Students are more likely to state that their
percentage point increase in the proportion of teachers who class has a social contract (or class agreement), and that
can show evidence of a mapping document. This represents the teacher reminds them of the class agreement. This is a
an almost fourfold increase, relative to only 20 percent of key component of positive discipline that was emphasized
teachers in the control who can show such a document. There in the PGP program. Students are also substantially less
is also no change the proportion of teachers who were able likely to state they have been scolded or punished by the
to mention areas that they want to improve on. But, again, teacher in the past, and less likely to mention someone who
all teachers in the control were able to mention areas of is afraid to ask the teacher questions. Although only one
improvement, so there are substantial ceiling effects (i.e., not outcome is statistically significant, the impacts are large
much “room to grow” even in the control group). in magnitude, and consistently large across the different
estimation approaches. Moreover, the impact on the overall
For simplicity all future analysis will only show results visually index is large—suggesting a 40 percent improvement in
but will discuss whether the results are robust to different the proportion of students who indicated positively to the
bandwidths and specifications. different questions— and statistically significant at the one
percent level.
Student-centered Teaching
These results are mirrored in the observed teaching
practices, using the MoECRT instrument. PGP teachers
The second family of indicators, according to MoECRT’s
were 50 percentage points more likely to demonstrate the
theory of change, is student-centered teaching. Student-
“manager” position during the classroom observations,
centered teaching has three sub-components: positive
compared to only 10 percent for the non-PGP teachers.
discipline, differentiated instruction, and socio-emotional
This is a sixfold increase. The manager position is also a key
skills. They are defined as follows:
component of positive discipline and relates to the social
contract: the teacher is required to apply the restitution
1) Positive discipline. Teachers: (a) have clear rules process when children misbehave by reminding children
for appropriate behavior and engage students in of the social contract and asking them to reflect on why
crafting these rules; (b) apply the manager position the y made a mistake . PGP te achers are also more like ly
consistently in school31; and (c) apply the restitution to communicate reciprocally with their students and
process when students misbehave. Restitution means provide opportunities for them to learn and interact from
that teachers do not punish students, but rather help each other, although this impact is smaller in magnitude
students understand what they did wrong.32 and not statistically significant. There is no evidence of
2) Differentiated instruction. Teachers frequently improvements in “motivation provision”, which is defined
assess students and adapt the content and teaching as providing engaging lessons matched to the need of the
practices to the needs of the child. child.33 The impact on the overall index is not statistically
significant, but is large in magnitude: PGP teachers are 40
3) Socio-emotional skills: Teachers structure teaching percent more likely to master the different competencies
to improve students’ socio-emotional skills and use related to positive discipline.
various strategies to integrate social and emotional
learning.
31
‘The manager position was the term used by MoECRT during PGP training to indicate how teacher positioned themselves in implementing positive discipline. This
position expects teachers to apply restitution process such as asking students to be responsible for their behavior, supporting students to find solutions to their
problems, and analyzing their needs.
32
The classroom observation instrument defines the “manager position” as a teacher who applies restitution, so we do not draw a distinction between (b) and (c) in
our analysis.
33
One could argue that this relates more to differentiated instruction than positive discipline.
38 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Figure 16. TOC II.1 Student-centered teaching – positive discipline
Panel A. Student level
Index 0.63 0.25***
No-one afraid ask 0.43 0.2
Not punished 0.82 0.22
Not scolded 0.62 0.42**
Reminder social contract 0.43 0.25
Teacher encourage 0.63 0.2
Social contract 0.85 0.15
0 0.2 0.4 0.6 0.8 1 1.2
Panel B. Teacher level
Index 0.47 0.19
Motivational Provision -0.18 0.6
Manager Position 0.1 0.5**
Teacher Communicated 0.7 0.26
-0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2
Non-PGP Impact
Notes. See Figure 15 notes for description of how data is constructed. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable
“Index” is the arithmetic mean of the constituent binary indicators. See Table 6 for construction of the dependent variables in Panel B.
In contrast, there is no consistent evidence for a positive impact on differentiated instruction (Figure 17). The impact on the
student-level index is very small and not statistically discernable from zero. There is a larger impact on the teacher-level outcomes,
but the impact on the overall index is not statistically significant. There are large increases in the proportion of teachers who
provide good feedback to students and provide differentiated assessment. But a negative (albeit statistically insignificant) impact
on teachers’ modification of learning strategies. Overall, we cannot conclude with a high degree of confidence that PGP teachers
are more likely to enact differentiated instruction.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 39
Figure 17. TOC II.2 Student-centered teaching – differentiated learning
Panel A. Student level
Index 0.53 0.02
Explains Clearly 0.55 0.09
Help Classwork -0.01 0.61
Patient Explaining -0.06 0.46
Assignments Completed -0.13 0.36
Provide Learning 0.54 0
Help student -0.01 0.61
Student Happy 0.56 0.06
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Panel B. Teacher level
Index 0.32 0.1
Differentiated 0.1 0.28
Constructive Feedback 0.13 0.19
Learning Strategies -0.26 0.5
Independent Assignments 0.47 0.14
Learning Environment 0.4 0.15
-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Non-PGP Impact
Notes. See notes in Figure 15. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable “Index” is the arithmetic mean of the
constituent binary indicators. See Table 6 for construction of the dependent variables in Panel B.
Next, we examine teachers’ socio-emotional skills in magnitude across different specifications. Overall, we cannot
the classroom, the third dimension of student-centered conclude with any confidence that the program improved
teaching. Panel A in Figure 18 shows that students are no teachers’ socio-emotional skills.
more likely to provide a positive assessment of how they
are treated by the teacher. In fact, there is a reduction in the This null result is consistent with the international evidence
proportion of students who like the way they are treated by that it is difficult to improve adults’ socio-emotional skills. A
the teacher. Similarly, the classroom observations suggest recent review of evaluations of programs that aim to improve
that PGP teachers might show weaker socio-emotional skills. socio-emotional skills concluded that programs targeting
But these negative estimates are not consistently large in adults had small impacts (if at all), and that programs work
better for younger participants.34
34
Puerta, et al., 2016
40 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Figure 18. TOC II.3 Student-centered teaching – socioemotional competencies
Panel A. Student level
Index 0.54 0.07
Always Kind -0.03 0.84
Greet Student 0.57 0.21
Treated Teacher -0.12 0.55
Teacher Bad Day 0.27 0
Teacher Asked 0.5 0.2
-0.2 0 0.2 0.4 0.6 0.8 1
Panel B. Teacher level
Index -0.74*** 0.5
Socio Emotional -0.29 0.43
Positive Emotions -1.2*** 0.5
Emplathetic -0.74* 0.57
-1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
Non-PGP Impact
Notes. See notes in Figure 15. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable “Index” is the arithmetic mean of the
constituent binary indicators. See Table 6 for construction of the dependent variables in Panel B.
Finally, Figure 19 shows results from the classroom Although there is not a perfect mapping between the
observations conducted using the World Bank Teach two instruments, it is likely that classroom culture relates
instrument. Teachers who participated in the PGP program closely to positive discipline. Classroom culture includes
improved their teaching practices (as measured by Teach) indicators such as treating students with respect, using
substantially, by over one standard deviation. As a general positive language, setting clear behavioral expectations,
rule, an impact of 0.6 or larger is considered large, and an acknowledging positive student behavior, and redirecting
impact of 0.2 or smaller is considered small. The estimated misbehavior. It is important to note that “socio-emotional”
impact on the overall score for the reduced sample—i.e., is a very different meaning to what is in MoECRT’s theory
teachers who score just two points below or two points of change. The Teach classroom observation instrument
above the cutoff—is not statistically significant, but it is very measures whether the teacher is using teaching practices
large, and the estimates are statistically significant for the that will promote a students’ non-cognitive skills; it does
expanded samples, even though these effect sizes are smaller not measure whether teachers themselves exhibit socio-
(see Appendix 4). The largest and most robust improvements emotional competencies, which is arguably much harder to
are lesson facilitation, a supportive learning environment, change.
and encouraging critical thinking.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 41
Figure 19. TEACH Scores
Panel A. Indices
Classroom Culture -0.36 1.33
Instruction -0.05 0.76
Socio-Emotional -0.2 0.58
Overall -0.26 1.12
-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Non-PGP Impact
Panel B. Separate indicators
Supportive Learning 3.48 0.66
Positive Behavioral 3.35 0.44
Lesson Facilitation 3.67 0.67
Check Understanding 3.35 0.34
Feedback -0.17 2.23
Critical Thinking 3.07 0.57
Autonomy 2.83 0.23
Perseverance 2.25 0.44
Social Collaborative 2.65 0.18
-0.5 0 1 1.5 2 2.5 3 3.5 4 4.5 5
Non-PGP Impact
Notes. See notes in Figure 15. Data from Teach classroom observations instrument. The dependent variables The indices in Panel A are arithmetic means of
their constituent indicators standardized to have a mean of zero and standard deviation of one. The dependent variable Overall is the arithmetic mean of these
three indices, also standardized to have a mean of zero and standard deviation of one. Effect sizes in Panel A are reported in standard deviations. The dependent
variables in Panel B range between 1 and 4.
42 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Leadership: Communities of Learning, control group might already have benefited from exposure to
Coaching, and School vision other PGP teachers. Similarly, 34 of the target teachers in the
treatment group indicated that other teachers in the school
or the school principal are PGP graduates. In these cases,
For the final set of outcomes relate to the impact that PGP
our target teacher might not have initiated the conversations
teachers have on other teachers in the school, and school-
with the peer teacher about student-centered learning,
wide vision and leadership. 35 Unless otherwise stated,
because someone else had already done that.
the results come from data collected in the peer teachers.
Overall, we observe no evidence of positive effects on these We test for this by further restricting the sample to schools
school-wide outcomes. But there is a positive impact when where no teacher has participated in PGP (control), or only
we restrict the sample to schools where there are no other one teacher has teacher has participated (treatment group).
PGP teachers in the school. This suggests that the positive With this restricted sample there is much stronger evidence
spillovers onto the rest of the school may have already of instructional leadership initiated by the treated teachers.
taken place, due to previous PGP graduates in our sample of Figure 20 shows that the pe e r teache rs are substantially
schools. more likely to indicate that they mapped their strengths
Table A.4.7 in the appendix shows that there is no evidence and weaknesses. They are also more likely to indicate that
that the peer teachers are more likely to participate in learning they were asked to self-reflect on their learning process,
communities or engage in self-development as a result of the and more likely to mention the target teacher as the person
target teachers’ participation in the program. Similarly, there who initiated this process.36 Similarly, Figure 21 shows
is no evidence on improvement in instructional leadership large improvements in the proportion of peer teachers
(Table A.4.9), or the target teacher’s engagement with the who indicated that they were asked by a fellow teacher to
school community about a school vision for student-centered discuss student-centered teaching, they were asked to map
teaching (Table A.4.11). out student needs, and that this was initiated by the target
teache r. There is also some e vide nce (Table 14) that the
Part of the reason for the muted impact on these outcomes
PGP teachers are more likely to initiate a conversation with
could be that about 40 percent of teachers in the control group
the school principal about the school’s vision for student-
mentioned that there is at least one teacher in their school
centered teaching.
who participated in PGP training. So, the peer teachers in the
Figure 20. TOC III.1 Create communities of learning (restricted sample)
Panel A. Learning Community
Exists -0.04 0.81
Teacher or principal initiated -0.24 0.63
Initiated teacher -0.10 0.06
Peer teacher joined 0.69 0.21
Improve teaching 0.69 0.24
-0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Non-PGP Impact
35
Category III of the theory of change is: “teachers encourage peers/communities in their schools to implement student-centered learning”. This is in turn
divided into two sub-components: (i) establishment of communities of learning; and (ii) applying coaching principles and engaging with their peer teachers
to create student-centered learning. Category IV is: teachers have a vision for student-centered school development.
36
For some specifications and samples these effect sizes are too large, since it implies that more than 100 percent of the peer teachers performed the activity
in the treatment group and deserve further inspection (as done in the appendix), but what is important is that they are robust to different specifications and
samples.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 43
Panel B. Self-development plan (peer survey)
Map strengths 0.63 1.07***
Asked to Self-Reflect as a teacher -0.35 0.63
Asked Self-Reflect on learning process 0.56 0.72*
Mention area of improvement 0.56 1.4***
Maps doc shown 0.19 0.79***
Index 0.49 0.38**
-0.5 0 0.5 1 1.5 2 2.5
Non-PGP Impact
Notes. See notes in Figure 15. All dependent variables are constructed from the peer teacher survey. Index is the arithmetic mean of all the other dependent
variables shown in both the panes. All variables are binary. The sample is restricted to schools where there is no other PGP graduate teaching at the school.
Figure 21. TOC III.2 Coaching (Restricted sample)
(a) Discuss Student-Centered 0.63 0.77**
--- Initiated by PGP teacher 0.5 0.77*
(b) Maps student need 0.44 0.37
--- Initiated by PGP teacher 0.25 0.71**
(c) Observed teaching 0.81 0.22
--- Initiated by PGP teacher 0 0.2
Index 0.44 0.51**
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Non-PGP Impact
Notes. See notes in Figure 15. All dependent variables are constructed from the peer teacher survey. Index is the arithmetic mean of all the other dependent
variables shown in the table. All variables are binary. The sample is restricted to schools where there is no other PGP graduate teaching at the school.
44 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Figure 22. TOC IV.2 Vision of student-centered school development (restricted sample)
Index 0.6 0.27
---School community involved -0.15 0.69
---Initiated PGP teacher (principal survey) 0.31 0.86**
---Initiated PGP teacher 0.25 0.4
---Determined by joint discussion 0.75 0.25
-0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4
Non-PGP Impact
Notes. See notes in Figure 15. All dependent variables are constructed from the peer teacher survey. Index is the arithmetic mean of all the other dependent
variables shown in the table. All variables are binary. The sample is restricted to schools where there is no other PGP graduate teaching at the school.
Note that the program requires participating teachers reasons to be happy while I teach”. They are also less annoyed
to perform some of these activities with other teachers when teaching, and feel less tense, mad, and nervous when
in the school. So, it is very encouraging from a program teaching. One possible interpretation is that teachers feel like
implementation perspective that the teachers are initiating they are more in control of their classroom, in line with the
these conversations about student-centered teaching with improvement in classroom culture.
their fellow teachers, but this does not constitute evidence
of improved leadership skills overall. We also do not see any This strong positive result on enjoyment of teaching is very
impact on whether the target teacher actually performs encouraging from a sustainability point of view, since teachers
classroom observations, so they are not providing targeted who enjoy their job are more likely to persist in applying
feedback to the peer teachers on how to improve their improved teaching practices.
teaching.
Some questions could be asked of all teachers, including
the 31 PGP teachers who had become school principals by
Additional Findings from This Evaluation the time of data collection. We could not conduct classroom
observations for these teachers, because they are no longer
This sub-section shows results on outcomes that are not teaching stude nts. But we could ask about the ir career
explicitly mapped to MoECRT’s theory of change but are aspirations and self-confidence as a leader.
important indicators that could be impacted by the program.
Table 12 report results using the full sample of targeted
Figure 23 shows that the PGP te ache r's se porte
lf-re d teachers, including the 31 PGP who had become school
enjoyment of teaching increased substantially. Teachers were principals by the time of data collection. Recall that we could
asked to answer a battery of questions related to how much not conduct classroom observations with these teachers,
they enjoy teaching. The first row shows results for the index, because they are no longer teaching, but we asked all of them
and the subsequent row reports results for all the respective about their beliefs. Teachers who participated in the program
indicators, ranging from 1 “Highly disagree” to 5 “Highly are 44 pe rce ntage points more like ly to be come school
agree”. To construct the mean index, we first switch the values principals, compared to those who do not. It also improved
of the questions that were asked in a negative way, so that aspirations: teachers who participated in the program are
positive means more enjoyment of teaching. We then take 66 percentage points more likely to state that they want to
the arithmetic mean of these indicators, standardized to a become a school principal or inspector (or they had already
mean of zero and standard deviation of one. Teachers provide become a school principal).
more positive re sponse s to the state me nt “I ofte n have
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 45
Figure 23. Enjoyment of teaching
Generally enjoy -0.12 4.97
Prepare teach -0.27 4.77
Reasons happy 4.57 1.17***
Teach enthusiasm 4.67 0.24
Reason angry -1.13* 2.37
Feel annoyed -1.77** 2.1
Really teach -1.13 2.9
Generally frustates -0.32 1.37
Tense nervous -1.65*** 1.77
Overall -0.38 2.3***
-3 -2 -1 0 1 2 3 4 5 6 7
Non-PGP Impact
Notes. See notes in Figure 15. Data come from the teacher survey. All dependent variables (except for the index) are Liker scale questions ranging from (1)
“Strongly Disagree” to (5) “Strongly Agree”. Overall is the arithmetic mean, standardized to have a mean of zero and standard deviation of one.
46 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table 12. Leadership confidence and aspirations
Control Mean Impact
(1) (2) (3) (4)
|x| < 3 |x| < 4 Full
Panel A. Leadership ambitions
Became a school principal 0.06 0.06 0.14 0.02
(0.04) (0.04) (0.11) (0.09)
Plan to be a school principal or inspector 0.19 0.19 0.84*** 0.50***
(0.07) (0.07) (0.26) (0.19)
Panel B, Self Confidence as a Leader
Index 0.70 2.64** 2.29*** 1.61***
(0.23) (1.07) (0.85) (0.57)
I have necessary ability to become a good leader of this school 3.68 1.26* 1.38** 0.66*
(0.16) (0.67) (0.54) (0.38)
I know what is needed to help teachers perform their duties 4.06 0.89 0.58 0.46
(0.11) (0.56) (0.43) (0.30)
I believe in my ability to influence teachers 3.71 1.54** 1.33** 0.80**
(0.14) (0.67) (0.54) (0.36)
I know how to encourage teachers 3.84 1.52** 1.22** 0.70*
(0.15) (0.68) (0.53) (0.37)
Trainings have equiped me to become a school principal 3.10 1.63** 1.57** 1.55***
(0.19) (0.79) (0.64) (0.49)
Observations 32 95 135 381
Notes. Data come from the teacher survey. The dependent variables in panel A are binary. All dependent variables (except for the index) in Panel B are Likert
scale questions ranging from (1) "Strongly Disagree" to (5) "Strongly Agree". Index is the arithmetic mean of these indicators.
Their self-confidence as a leader improved substantially. These results are encouraging, since a large object of PGP is to train a
new generation of school leaders.
Table 13 shows the same results as Table 12 but broken down by gender. Although statistical power is reduced with the sub-
samples, there is suggestive evidence that program had a much larger impact on teachers’ self-confidence for males, compared
to females. The impact on a teacher’s career aspirations also seems to be smaller for females relative to males.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 47
Table 13. Leadership confidence and aspirations (by gender)
Impact (|x| < 3) Impact (|x| < 4)
(1) (2) (3) (4)
Male Female Male Female
Panel A. Leadership ambitions
Became a school principal 3.23 0.36** -0.08 0.20
(2.47) (0.17) (0.36) (0.12)
Plan to be a school principal or inspector 4.27 0.64** 1.15* 0.71***
(3.53) (0.27) (0.67) (0.24)
Panel B, Self Confidence as a Leader
Index—self-confidence as a leader 24.01 1.37** 6.32** 1.49**
(30.41) (0.68) (2.71) (0.65)
I have necessary ability to become a good leader of this school 6.27*** 0.72 2.90* 0.92*
(1.75) (0.63) (1.56) (0.50)
I know what is needed to help teachers perform their duties 16.23 0.64 1.98 0.48
(50.05) (0.45) (1.56) (0.44)
I believe in my ability to influence teachers 35.12 0.75 5.49* 0.73
(121.02) (0.47) (2.89) (0.47)
I know how to encourage teachers 13.71 0.55 4.57** 0.56
(18.11) (0.40) (1.98) (0.38)
Trainings have equiped me to become a school principal 13.96 0.55 2.51 1.29*
(19.33) (0.80) (1.56) (0.72)
Observations 20 75 33 102
Notes. See table 10. Data come from the teacher survey. The dependent variables in panel A are binary. All dependent variables (except for the index) in Panel
B are Likert scale questions ranging from (1) "Strongly Disagree" to (5) "Strongly Agree". Index is the arithmetic mean of these indicators.
However, this program did not have a positive impact on the current school principals’ perceptions of the quality of the leadership
ability of the PGP graduates who are still teachers. In fact, there is a large negative impact of 0.44 standard deviations on the
leadership index. The largest reduction is in their belief in the ability of a teacher to influence and lead teachers. Note that the
mean is extremely high, so the school principals almost always give everyone a value of five (out of five).
Table 14 School Principal’s perception of the teacher’s leadership ability
Control Mean Impact
(1) (2) (3) (4)
|x| < 3 |x| < 4 Full
Overall 4.77 -0.44 -0.55** -0.48***
(0.05) (0.27) (0.24) (0.18)
Know how to be a good leader 4.80 -0.28 -0.43 -0.25
(0.07) (0.40) (0.38) (0.25)
Helps team accomplish tasks 4.87 -0.43 0.69*** 0.63***
(0.06) (0.28) (0.29) (0.21)
Ability to influence and lead teacher 4.77 -0.70* -0.87*** -1.02***
(0.08) (0.37) (0.32) (0.26)
Encourage good team 4.63 -0.34 -0.19 -0.11
(0.10) (0.45) (0.35) (0.25)
Observations 31 89 127 350
48 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
7.
CONCLUSIONS AND
RECOMMENDATIONS
The summarize, this report finds that the PGP program trend. The fact that the impacts are different depending
was well-implemented and positively received by on the choice of classroom observation instrument merits
teachers, whose teaching practices improved along a further investigation. Both instruments cover very different
range of dimensions, including: application of positive domains of instructional quality and positive culture. The
discipline, classroom culture, and instruction. Teachers key difference might lie in the level of difficulty or complexity
also increased their enjoyment of teaching. However, there measured. MoECRT’s instrument sets high standards for good
is no evidence that the program improved teachers’ socio- teaching. These two instruments therefore measure very
emotional competencies, nor that it increased differentiated different points across the distribution of teaching practices.
instruction, as measured by MoECRT’s own classroom
observation instrument. Finally, there is also evidence that Our findings present compelling evidence that the
participants of the program engaged other teachers in the current program design and approach effectively
school in conversations around student-centered teaching supports teachers in enhancing their teaching practices,
and self-reflection. They are also more likely to become particularly in relation to their enjoyment in teaching and
school principals, aspire to become school principals, and utilization of positive discipline with students. Despite
have more self-confidence in their leadership abilities. ongoing efforts by the government to eradicate violence and
discrimination in schools over the years, this program has
The ve ry large positive impacts on te aching practice s, successfully aided teachers in comprehending the principles
as measured using the Teach classroom observation of positive discipline and enhancing their classroom
instrument, are impressive. For comparison, recent study management skills. Through this program, teachers not only
on teacher professional development in East Asia and Pacific grasp the concept but also acquire the necessary guidance
found that programs implemented by government is often for implementing it in their classrooms. This achievement
ineffective.37 The government should be commended for should be sustained.
designing and implementing a program that bucks this
37
Afkar et al., 2023. The World Bank’s regional flagship report collects information about various teacher professional development programs in East Asia and
Pacific and found that these programs often lack the four element of effective teacher training program: (a) a focus on content knowledge, (b) opportunities
to practice what is learned with colleagues, (c) continued support through follow-up visits focused on training content, and (d) career incentives through
promotion or increased salary.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 49
Moving forward, MoECRT should build on their successes, Undoubtedly, successful implementation will rely on strong
and bear closely in mind implementation challenges as coordination between MoECRT and the local government/
the program gets scaled up up in the next 18 months. Balai Guru Penggerak (BGP) operating in each district.
The results and impacts observed in the selected districts
of Batch 5 (involving 8,000 participants from 166 districts)
cannot be assumed to be replicated automatically if the B. Enhancing Mentor’s Preparedness
government fails to maintain the quality of implementation
in subsequent batches, such as Batch 8 which commenced This improvement is crucial in ensuring that mentors
with 11,730 participants from 284 districts in 34 provinces, are equipped with the necessary skills and knowledge
as of May 2023.38 to effectively support teachers in the short term.
Throughout the program, mentors play a vital role in
Drawing upon the findings presented in this report, we facilitating teachers' comprehension of the training modules
provide the following recommendations to enhance the and providing assistance during workshops. However, their
quality of PGP implementation. significance extends beyond these tasks as mentors serve as
valuable coaches for teachers. Their responsibilities include
conducting classroom observations, engaging in discussions
A. Standardizing best practices with school principals regarding teachers' learning and
providing feedback to enhance performance.
The primary focus of this recommendation is to urge
policymakers to take the necessary actions to uphold quality More ove r, it lays the foundation for a sustainable
of Guru Penggerak program(PGP) implementation across model of educational reform that can drive continuous
all districts. Our findings indicate that there is variation in improvement in the long term. While the current
teacher satisfaction, particularly concerning the logistical recruitment process for mentors is open to the general public,
arrangements for workshop implementation. Despite MoECRT has planned to designate permanent mentors in
the current provision of guidelines39 by MoECRT to local each district from the pool of PGP graduates from previous
governments and Balai Guru Penggerak on organizing PGP batches. To ensure the effectiveness of mentorship, it is
activities, there is still room for further improvement. For crucial to establish regular performance assessments and
example, MoECRT should: provide ongoing professional development opportunities
for mentors. This approach will not only enable mentors to
• Providing more detailed definitions of the better support teachers participating in PGP in subsequent
"comfortable room" in the guidelines batches, but it also holds the potential to influence positive
reform within their respective schools as they can implement
• Implementing a checklist system to effectively and share the learned practices.
monitor compliance with the intended standards.
" This report finds that the PGP program was well-implemented and
positively received by teachers, whose teaching practices improved
along a range of dimensions, including: application of positive discipline,
classroom culture, and instruction. However, there is no evidence that
the program improved teachers’ socio-emotional competencies, nor
that it increased differentiated instruction..
"
38
MoECRT, 2023
39
Keputusan Direktur Jendral on GTK and Guidebook for facilitators, workshop, and individual coaching. Guideline for workshop available here
50 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
There are many ways to better support the PGP mentors,
including but not limited to the following: C. Supporting Teachers in Differentiated
Learning and Socio-Emotional Skills
1. Provide Comprehensive Guidance for Mentors:
To further enhance the program's effectiveness, it is
MoECRT has developed a comprehensive handbook for
recommended to provide teachers with additional support
mentors, specifically focusing on their individual coaching
in differentiated learning and socio-emotional skills. This can
responsibilities. While the handbook serves as a valuable
be achieved through additional development of guidelines
resource, there is room for improvement, particularly in
and tools, such as examples tools for diagnostic assessments,
the utilization of classroom observation tools. These tools
lesson plans with practical tips and suggestions for how to
are vital for mentors to objectively observe teachers in the
introduce differentiated learning in the classroom. Support
classroom and provide constructive feedback to enhance
can also encompass additional mentoring sessions during
their teaching practices. However, the current classroom
the program and follow-up mentoring sessions after the
observation tools used in the program lack detailed
program ends.
guidance on how to use it and the scoring. By incorporating
this detailed guidance, mentors will not only be better
equipped to utilize the tools effectively within the program
D. Reconsider the broad and ambitious
but also extend their application to their respective roles
objectives of the program.
within their schools, offering coaching to other teachers.
This enhancement will foster improved teacher support
and development, contributing to the overall success of The program might be overly ambitious in the wide range of
the program. teacher attitudes and skills that it hopes to change. Socio-
emotional competencies, in particular, are very hard to
2. Enhance Mentor Training: MoECRT should invest in change in adults. In fact, a review of the evidence showed
enhancing the training provided to mentors. This should that programs targeting socio-emotional skills of adults
include incorporating additional pedagogical strategies, have limited to no impact. Moreover, the official standards
leadership skills, and effective mentoring techniques. for reaching competence in the different domains, as defined
Additionally, teachers should also receive assistance in in the classroom observations instrument, are highly
utilizing the tools included in the handbook. By equipping ambitious. For example, it might be counter-productive to
mentors with a broader range of knowledge and skills, expect the following:
they will be better prepared to support teachers in their
professional growth. “Teacher displays a consistently positive expression
of emotions (displays happy, enthusiastic, friendly
3. Ensure Sustainable Mentor Support: MoECRT should emotions)”
establish a robust support system for mentors, including
regular mentoring meetings, peer learning opportunities, “Teachers smoothly modify strategies, materials, and
and access to resources. This support network will help groupings to optimize students' opportunities to learn
mentors overcome challenges, share experiences, and and meet their learning need”
continuously enhance their effectiveness in guiding
teachers. “Students are actively involved throughout learning
and focused on work that not only develops skills and
By implementing these measures, it is expected that MoECRT challenges them to be problem solvers.”
can significantly improve the preparation of PGP mentors,
enabling them to perform their tasks more effectively in the A different strategy is to prioritize a smaller set of skills
short term. Furthermore, this investment in mentorship will that are key to improving student outcomes, and setting
serve as a sustainable model for driving continuous reform standards that are realistically achievable by all teachers,
and improvement within the education system in the long including:
run.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 51
• No longer including a goal of improving teachers’ socio- 1. Conduct a randomized control trial. A randomized
emotional competencies. controlled trial will provide stronger evidence of the
causal impact of the program, strengthen the external
• Reduce the definition of what entails differentiated validity of the study, and also increase statistical power
instruction, focusing on conducting student assessments relative to a RDD. This could be implemented through a
and adapting the level of teaching to students’ levels of randomized phased in approach, where teachers assigned
ability, rather than also adapting the teaching style to the to the control participate in a later batch.
learning needs of the child. The latter is extremely difficult
for teachers to implement. 2. Expand data collection to more districts, including more
remote areas. Since implementation is decentralized,
• Changing the targets for excellent teaching in the it is possible that the impacts of the program will vary
classroom observation instrument. substantially by region.
This will be increasingly important as the program continues 3. Expand data collection to teachers who participated
to scale up and is implemented in remote and under- in earlier and later batches of the program. The
resourced locations where the baseline teacher capacity program is scaling up over time—from 2800 teachers
might be weak. who participated in Batch 1 to 55,000 teachers who were
selected in Batch 10—and there is international evidence
The "Teachers' Learning Journey"40 developed by MoECRT that the effectiveness of programs decreases with scale.
serves as an opportunity to critically assess whether the ideal The characteristics of the teachers selected in the different
teacher envisioned is grounded in rationale and practicality. batches might also be different, since it was more selective
This evaluation will enable the identification of areas that at the start. All of this means that the program might have
require supplemental interventions to support sustained had a larger impact in the earlier batches, relative to later
and lasting changes in teachers' professional development. batches.
4. Measure student-level cognitive and non-cognitive
E. Further research skills. An important next step is to adcertain whether
the improvements in teaching practices translated
into improvements in student outcomes. Given the
As discussed in the limitations section above, the results are improvements in application of positive discipline in the
not generalizable to Indonesia as a whole, since the sample classroom, it is possible that the program improved both
only includes primary school teachers in five districts in Java cognitive skills and other non-cognitive skills such as
that were short-listed for Batch 5 training. This was done social behavior and self-confidence.
for cost considerations. Moreover, despite the relatively
large sample of 350 teachers, statistical power is limited 5. Longitudinal Study: Implement a longitudinal study
due to some non-compliance and the RDD approach which design that captures data before, immediately after
prioritizes results from teachers very close to the threshold program completion, and tracks teachers' evolution over
for selection. We therefore do not have enough statistical time, particularly as they transition into school principal
power to perform sub-group analysis. There are also no roles. Additionally, utilizing national assessment data on
results on student-level outcomes. Additional data collection the learning environment can offer valuable insights.
and further our understanding of the broader impact of the
program. By undertaking future research with these considerations,
a more comprehensive and nuanced understanding of the
program's impact on teachers and the education system as a
whole can be achieved.
40
Official definition is currently no available. MoECRT often refers ‘teacher learning journey’ as part of teacher’s development following the paradigm of
‘Merdeka Curriculum’, which means a comprehensive self-reported assessment of individual’s teacher competence and their improvement plan, for
example here: https://gurudikdas.kemdikbud.go.id/news/modul-pelatihan-peningkatan-kompetensi-numerasi-untuk-guru
52 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
REFERENCES
Afkar, R., Béteille, T., Breeding, M., Linden, T., Mason, A., Mattoo, A., Pfutze, T., Sondergaard, L., and Yarrow, N. 2023. Fixing
the Foundation: Teachers and Basic Education in East Asia and Pacific. World Bank East Asia and Pacific Regional Report.
Washington, DC: World Bank. doi:10.1596/978-1-4648-1904-9.
Baeten, M., Kyndt, E., Struyven, K., Dochy, F. 2010. Using student-centered learning environments to stimulate deep approaches to
learning: Factors encouraging or discouraging their effectiveness. Educational Research Review, 5(3), p.243-260
GTK. 2023. Sapa GTK Episode 12: Guru Penggerak, as cited in Kemendikbudristek Dorong Guru Penggerak Pimpin Transformasi
di Satuan Pendidikan. https://gtk.kemdikbud.go.id/read-news/kemendikbudristek-dorong-guru-penggerak-pimpin-
transformasi-di-satuan-pendidikan
Katadata. 2022. Mayoritas Kepala Sekolah SD Berumur Lebih dari 55 Tahun. https://databoks.katadata.co.id/
Kim, J. 2004. Education reform policies and classroom teaching in South Korea, International Studies in Sociology of Education,
14(2), p.125-146, doi: 10.1080/09620210400200122
Lee, S. Chia, A., Pereira, A., Tay, L. 2022. Singapore's Student-Centred, Values-Driven education system: A case study of Teacher
Professional Learning. Centering Whole-Child Development in Global Education Reform (1), p.21. Routledge
Ministry of Religious Affairs (MoRA). 2023. EMIS Kemenag 2022/2023. https://emis.kemenag.go.id
MoECRT. 2020. Merdeka Belajar Episode 5: Guru Penggerak. https://www.kemdikbud.go.id/main/blog/2020/07/kemendikbud-
luncurkan-merdeka-belajar-episode-5-guru-penggerak
MoECRT. 2023. Kurikulum Merdeka. https://kurikulum.kemdikbud.go.id/kurikulum-merdeka/
MoECRT. 2023. Sebanyak 11.730 Guru Ikuti Pendidikan Guru Penggerak Angkatan 8. https://www.kemdikbud.go.id/main/
blog/2023/05/sebanyak-11730-guru-ikuti-pendidikan-guru-penggerak-angkatan-8
MoECRT. 2023. Unduhan. https://sekolah.penggerak.kemdikbud.go.id/gurupenggerak/unduhan/
MoECRT Education Statistics. 2023. Statistik Pendidikan. https://statistik.data.kemdikbud.go.id
OECD. 2005. Teachers Matter. https://www.oecd.org/education/school/34990905.pdf
OECD. 2018. Programme for international student assessment (PISA) 2018. https://www.oecd.org/pisa/publications/PISA2018_CN_
IDN.pdf
OECD. 2020. Teaching and Learning International Survey (TALIS) 2018 Insights and Interpretations. https://www.oecd.org/education/
talis/TALIS2018_insights_and_interpretations.pdf
Park, R. 2016. Preparing students for South Korea’s creative economy: The success and challenges of educational reform. Vancouver
B.C.: Asia Pacific Foundation of Canada.
Permendikbud No.40. 2021. Peraturan Menteri Nomor 40 tentang Penugasan Guru sebagai Kepala Sekolah. https://jdih.kemdikbud.
go.id/detail_peraturan?main=2940
Permendikbud No.26. 2022. https://peraturan.bpk.go.id/Home/Details/224728/permendikbudriset-no-26-tahun-2022
Puerta, M., Valerio, A., and Bernal, G. 2016. Taking stock of programs to develop socioemotional skills: A systematic review of program
evidence. doi: 10.1596/978-1-4648-0872-2
Schweisfurth, M. 2013. Learner-centred Education in International Perspective Whose pedagogy for whose development?. New York:
Routledge
World Bank. 2020. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. © World Bank, Washington, DC.
World Bank. 2023. The Invisible Toll of COVID-19 on Learning. Indonesia Economic Prospects. © World Bank, Washington, DC.
Yang, J. and Tan, C. 2019. Advancing student-centric education in Korea: Issues and challenges. The Asia-Pacific Education
Researcher., DOI: 10.1007/s40299-019-00449-1, 1-11.
Yarrow, N., Afkar, R., Masood, E., Gauthier, B. 2020.Measuring the Quality of MoRA's Education Services (English). Washington, D.C. :
World Bank Group.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 53
APPENDIX 1 –
PENDIDIKAN GURU PENGGERAK
A. Theory of Change
Pancasila students who are creative, collaborative, globally diverse, critical thinker, independent, as well as
faithful, devout to the Almighty God, and noble in character.
Schools in Indonesia have a student-centred learning ecosystem in a sustainable manner.
General
Purpose Teachers continuously Teachers continuously The principal leads the Supervisors provide
plan, implement, reflect develop themselves and school collaboratively assisstance to principals
and evaluate student- other teachers by self- to realize the school's and tecahers to realize a
centered learningwith reflection, sharing, and vision that favors school vision that favors
parent involvement. collaboration. students and fosters students and fosters
student leadership. student leadership.
Teachers have the moral, Teachers plan, Teacher support peers/ The teacher initiates
emotional, and spiritual implement, reflect, communities in their the development of a
Final maturity to behave and evaluate student- schools to imolement student-centered school
Outcome according to the code of centered learning in student-centered based on collaborative
ethics. schools. learning. resources.
Self and Classroom: Peers: School:
• Teachers have the paradigm of a • Teacher assist other teachers • Teachers have a vision of student-
student-centered learning leader. in planning, implementing, centered school development.
• Teachers have an understanding reflecting, and evaluating • Teacherts have the skills to map
and begin to plan, implement, student-centered learning in assets collaboratively for student-
Intermediate reflect and evaluate student- schools. centered learning.
Outcome centered learning in schools. • Teachers have the ability and • Teachers have the ability to
• Teachers have the ability and carry out reflective practice for develop programs to create
carry out reflective practice for peer development in creating a student-centered learning
self-development in creating student-centered learning. environment.
student-centered learning. • Teachers have the ability and carry
out reflective practice for student-
centered school development.
Regular PGP: Special Region PGP: Recognition PGP: Intensive PGP:
• Facilitators and • Facilitators guide • Practical Teachers • Facilitators guide
instructors guide face-to-face learning. learn while guiding face-to-face learning.
online learning • Facilitators guide CGP (Candidate of Guru • Facilitators
• Practical Lecturer individual facilitation. Penggerak) in online accompany school
guides workshops & learning practices and guide
• Facilitators guide
Follow-up individual mentoring reflection, provides • Practical Teachers take individual facilitation.
Activities • Facilitators and feedback and concrete actions in • Facilitators lead
Practical Teachers assessment. schools. reflections, provide
guide reflection, • Facilitators from feedback and
provide feedback and the Working Unit assessments.
assessment. conducts assessment
and feedback on real
action
Effective public Development of PGP Data and Recruitment & Education
Foundational outreach and teaching materials. competency based Debriefing. Management by
Activities communication. CGP recruitment. Working Unit and
MoECRT
54 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
B. Guru Penggerak program: Curriculum
Curriculum overview
Guru Penggerak Program Currirulum
1 Paradigm and Vision
of Guru Penggerak 2 Student-centered
Learning Practices 3 Leadership in School
Development
1. Reflection on the National
Philosophy of Education 1. Meeting Student’s Learning Needs 1. Value-based Decision Making
2. Values and Roles of Guru Penggerak 2. Social and Emotional Learning 2. Leadership in Resource Management
3. Vision of Guru Penggerak 3. Coaching for Academic Supervision 3. Management of Impactful Programs
4. Positive Culture for students
Workshop: Workshop: Workshop:
0. Orientation 3. Leader’s Role in Learning 5. Collaboration in Student-
1. Community Practitioner Development centered Management
Development 4. Strengthening Coaching 6. Sustainability of School
2. Vision for Environmental Practices Development
Learning Change 7. Final Learning Outcomes
Focus on self and classroom Focus on peer teachers Focus on school
improvement of Guru improvement of Guru improvement of Guru
Penggerak Candidate Penggerak Candidate Penggerak Candidate
Final outcomes and derivative indicators, as per MoECRT’s theory of change.
Major grouping Grouping Derivative Indicators
I. Teacher has the skills and I.1a Teacher can periodically interpret facts/events related
Teachers have the habits of reflection, then to the le arning expe rience in the de velopme nt of
moral, emotional, uses them as the basis self-competence by describing the feelings involved,
and spiritual maturity for the preparation of a describing the learning, and making improvement/
to behave according follow-up plan that reflects decision plans to create a student-centered learning
to the code of ethics. partiality to the students environment.
I.1b Teacher can periodically interpret facts/events related
to the teaching experience with students by describing
the feelings involved, describing learning, and making
improvement/decision plans to create a student-
centered learning environment.
Teacher analyzes their I.2a Teacher is able to study and infer their own position in
position in leadership the competency map of the Guru Penggerak
competencies and have
a self-development I.2b Teacher has documented plans and targets for self-
plan according to their development based on the identification of needs for
individual needs increased competence
I.2c Teacher carries out the prepared self-development
plans and targets
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 55
Major grouping Grouping Derivative Indicators
Teacher masters social- I.3a Teacher is able to identify the emotions felt in an event.
emotional competence
in teaching practice I.3b Teacher responds to emotions in a positive, healthy, and
in schools, both in productive way; build in the achievement of goals in the
the intracurricular, learning process of students
co-curricular and
extracurricular realms.
I.3c Teacher shows empathy regarding the student's point of
view
I.3d Teacher shows efforts to be a resilient in dealing with
challenging situations in social relations in the school
environment
I.3e Teacher carries out the decision-making process with
empathy, is aware of the consequences, and is student-
centered.
Teacher practices decision- 1.4a Teacher applies decision-making and testing measures
making based on virtue in favor of the student in situations of ethical dilemmas
values as a leader who using moral persuasion.
favors students
II. The teacher applies II.1a Teacher engages students in crafting class beliefs
Teachers plan, positive discipline in the
execute, reflect classroom and in school. II.1b Teachers apply monitors or managers positions consistently
on, and evaluate in the learning process in the classroom and school.
student-centered
learning in schools. II.1c Teacher applies the restitution process to the students
when making a mistake.
Teachers apply II.2a Teacher able to identify and meet students' learning
differentiation strategies needs based on data.
in the learning process in
the classroom and school II.2b Teacher able to apply content differentiation strategies,
(preparation; teaching, to process differentiation, and product differentiation in
assessment). learning (preparation, teaching and assessment)
Teachers practice II.3a Teacher structures learning to improve students' social
learning that is integrated emotional skills.
with social-emotional
competence. II.3b Teacher able to use various strategies in integrating social
and emotional learning.
III. Teacher develops learning III.1a Teacher establishes a learning community with colleagues
Teacher encourages communities with at his school.
peers/community to colleagues in their schools
implement student- to increase competence to III.1b Teacher and their peers have a map of needs within
centered learning. carry out student-centered the learning community at school to enhance student-
learning. centered learning.
III.1c Teachers and their peers have self-development plans
based on a map of needs.
III.1d Teacher and their peers run learning communities
according to development plans.
56 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Major grouping Grouping Derivative Indicators
Teacher applies coaching III.2a Teacher works with colleagues to determine goals in the
principles in empowering coaching process in order to support student-centered
their peers to create learning.
student-centered learning.
III.2b Teacher assists peers to identify student-centered develop-
ment needs
III.2c Teacher guides their peers in creating action plans for stu-
dent-centered learning development
III.2b Teacher uses coaching skills in carrying out academic
supervision
IV. Teacher has a vision of IV.1a Teacher involves various actors to reflect on the school's
Teacher initiates student-centered school vision to be more partial to students
student-centered development.
school development IV.1b Teacher, colleagues and principal have a vision of student-
with collaborative centered school development based on the results of joint
resource (asset) reflection
based.
Teacher initiates resource- IV.2a Teacher strives for a school environment that fosters
based student-centered student leadership (voice, choice, and ownership) in the
school development, development of school programs.
involving school residents.
IV.2b Teacher provides opportunities for students to demonstrate
voice, choice, and ownership in the development of school
programs
IV.2c Teache r able to identify and use a varie ty of assets (7
main capitals) according to the school context to initiate
sustainable and student-centered school development.
IV.2d Teacher engages a variety of actors in schools with clear
roles and responsibilities in structuring and carrying out
the initiation of sustainable and student-centered school
development programs.
C. Applicants and Selected Candidates, Batch 1 to 5
Participants Batch 1 Batch 2 Batch 3 Batch 4 Batch 5
Districts 56 56 56 166 166
Applicants 19,218 17,091 23,274 42,009 105,643
Verification and validation of Application 5,142 6,401 6,662 16,366 49,815
Stage 1 Selection: Essay 4,598 5,217 4,446 10,826 19,974
Stage 2 Selection: Interview and Teaching 2,800 2,800 2,801 8,053 8,000
simulation (expected)
Acceptance rate 15% 16% 12% 19% 8%
Completed the program 2395 3,004 2760 7948 7391
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 57
D. PGP Supporting Actors41
Mentor Facilitator Instructor
Participants Teachers, PGP Alumni, or school Civil servant trainer or school Former PGP facilitators with
principals with minimum of 5 supervisor with minimum of 5 minimum of 5 years teaching
years teaching experience and 1 years teaching experience experience
year of mentoring experience
Roles 1. Become mentor for CGP to: 1. Facilitate the online learning 1. Provide learning materials
a) Facilitate workshop activities: to CGP
b) Visit school a) Learning reflection
2. Sharing good practice
c) Assist in making follow-up b) Concept exploration
of applying the material
plans c) Collaboration Room
contained in PGP modules
2. Coordinate with working group 2. Provide feedback for all CGP’s
3. Become a resource person
assignments
3. Coordinate with facilitator for Mentor and Facilitator
3. Coordinate with working
4. Collect and analyze CGP
group
Principal and peer’s feedback
and communicate to CGP 4. Coordinate with mentor
5. Report CGP’s achievements 5. Make work report
Source. PGP Batch 5 Master Deck, Permendikbud No.26 (2022) on Pendidikan Guru Penggerak, and Sekolah Penggerak Website:
PGP Actors Selection Process
Notes. for the upcoming batches MoECRT plan to prioritize PGP alumni to become mentor and facilitator.
E. PGP Classroom Observation Instrument
1. 1. The teacher immediately starts the learning without mentioning the purpose of the learning.
Opening of the 2. The teacher only conveys the learning objectives and does not explain the activities.
Learning Process
3. The teacher ensures that students understand the expected learning objectives and the activities.
4. The teacher ensures that students understand the expected learning objectives and the activities
and discusses with students the relationship of topics discussed with other materials and with
daily life.
2. 1. Teacher does not show any modification of the learning environment according to learning
Comfortable objectives, characteristics of students and social emotional competence.
Learning 2. Teacher modifies the learning environment according to learning objectives (academic learning
Environment to needs).
Meet Student 3. Teacher modifies the learning environment according to learning objectives and understanding
Learning Needs student characteristics.
4. Teacher modifies the learning environment according to learning objectives, student
characteristics, and social and emotional learning objectives.
3. 1. Teacher dominates the interaction and gives few opportunities for students to express an opinion
Positive (one-way lecture method).
Communication 2. During one class hour, half of teacher communication is one-way and half of the time students
Usage: Interaction have the opportunity to express an opinion.
PGP Masterdeck PPT, Sekolah Penggerak Website
41
58 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
3. Teacher communicates reciprocally with student (teacher-student).
4. Teacher communicates reciprocally with students (teacher-student) and gives students the
opportunity to interact with each other (students).
4. 1. The teacher makes assumptions about the condition of the pupil without clarifying first.
Positive 2. The teacher listens to the opinions of students that are relevant to the topic or questions asked by
Communication the teacher alone.
Usage: Empathy
3. The teacher tries to figure out what their students are working on and thinking, but only manages
to do it with a few students.
4. The teacher finds out what their student is working on and thinking. The teacher listens actively to
the opinions expressed by all students.
5. 1. Teacher uses negatively charged words.
Positive 2. Teacher displays inconsistent expressions of emotions (between negative and positive emotions).
Communication
3. Teacher displays positive expressions of emotions and neutral emotions alternately.
Usage: Positive
emotions 4. Teacher displays a consistently positive expression of emotions (happy, enthusiastic, friendly
emotions).
6. 1. The Punisher Position: Teacher uses punishment, innuendo, and criticism to discipline students.
Application of The teacher will express anger when the student makes a mistake by rebuking, yelling, or
Discipline pointing.
2. Position which makes students feel guilty: The teacher silences the student or uses words that
make the student feel guilty (as if the teacher/parent/other students will suffer as a result of
their mistake).
2. Friend Position: Teacher uses friendship and humor to influence students. Teacher defends,
justifies or provides explanations for student behavior.
3. Monitor Position: Teacher applies discipline by using the rules and consequences agreed upon
with the pupil, using objective evidence. The consequences given relate to the rules that the
student violated.
4. Manager Position: Teacher uses the principle of restitution that reasonable mistakes occur, and
they accept the mistakes of students. The teacher asks the students why they made a mistake
and help them find a way out. This type of teacher sometimes uses the position of monitor and
manager in students in certain situations.
7. 1. Teacher pushes students to learn by providing negative punishments/consequences.
Motivation 2. Teacher encourages student learning behavior by conveying teacher expectations and providing
positive consequences.
3. Teacher encourages students by reminding them of the importance of learning.
4. Teacher fosters students' desire to learn by providing engaging and interesting learning sessions.
8. 1. The learning activities provided are only academically charged.
Socio-Emotional 2. There are activities that make students excited, having fun, or refocus on learning (such as: ice
Content in Learning breaking or energizers) but are not explicitly utilized to strengthen social-emotional competence
Activities and are separate from the academic content that is being learned.
3. Activities that reinforce students' socio-emotional competence are integrated into learning at: a
warm opening stage, a core involving-challenging-fun stage, and an optimistic but still separate
closing stage from the academic material that is being learned.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 59
4. Activities that reinforce students' socio-emotional competence have been integrated into learning
at: a warm opening stage, a core involving-challenging-fun stage, and an optimistic closing stage
by fusing it with academic content being studied.
9. 1. Teacher is silent and allows the students to do the assigned tasks themselves.
Approaches to 2. Teacher provides direction and solutions directly to students when they work on assignments.
Accompanying
3. Teacher helps students who are experiencing difficulties.
Students
4. Teacher helps students according to their learning needs. Students can do their assignments
independently as a result of teacher support.
10. 1. Teacher uses only one learning strategy throughout one session. OR Teacher uses learning
Use of Learning strategies but find it difficult to actively engage students in learning.
Strategies 2. Teacher uses several learning strategies and tries to encourage students to be actively involved in
learning and encourage skills development , but not consistently.
3. Teacher uses several relevant learning strategies to actively engage students throughout learning
and encourage the development of important skills.
4. Teacher smoothly modifies strategies, materials, and groupings to optimize students'
opportunities to learn and meet their learning needs. Students are actively involved throughout
learning and focused on work that not only develops skills but challenges them to solve problems.
11. 1. Teacher closes the class without giving learning conclusions.
Closing of the 2. Teacher only focuses on mentioning the teaching materials at the session meeting but does not
Learning Process attempt to conclude other learnings that were non-academic.
3. Teacher concludes and tells the student what other learning is gained besides the teaching
material (e.g., related to the social emotional skills learned).
4. Teacher invites students to conclude, reflect on what has been learned (both academically,
socially, and emotionally), and appreciate the positive progress or changes made by students.
12. 1. Teacher gives feedback that may offend the student personally (example: blaming the student for
Providing Feedback their nature).
2. Teacher only gives general feedback on the product of the whole student. OR The teacher only
gave a brief comment.
3. Teacher gives clear, concrete feedback) to the student's product that fits the learning objectives.
4. Teacher provides constructive feedback (clear and containing development advice) not only on the
student's product but also their learning process (e.g., how the student makes thinking strategy,
how the pupil solves the problem, communication techniques that was done by students).
13. 1. During the learning process, there is no assessment process.
The Functions of 2. During the learning process, there is an assessment process, but it is not closely related to the
the Assessment learning objectives.
3. During learning, an assessment process occurs and there is evidence of assessment according
with the learning objectives.
4. As learning progresses, an assessment process occurs that is according with the learning
objectives. There is evidence of teachers' efforts to differentiate to meet students' learning needs
and ensure students understand the assessment process.
60 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
APPENDIX 2 –
TEACH CLASSROOM OBSERVATION INSTRUMENT:
C. Applicants and Selected Candidates, Batch 1 to 5
Area Element (Score 1-5) Behavior (Score Low, Medium, High)
Time on Task 0. Time on Learning 0.1 Teacher provide activities to most students
0.2 Students are on task
A. Classroom 1. Supportive 1.1 Teacher treats all students respectfully
culture Learning 1.2 Teacher uses positive language with students
Environment
1.3 Teacher responds to students’ needs
1.4 Teacher does not exhibit bias and challenges stereotypes in the
classroom
2. Positive 2.1 Teacher sets clear behavioral expectations for classroom
Behavioral activities
Expectations 2.2 Teacher acknowledges positive student behavior
2.3 Teacher redirects misbehavior and focuses on the expected
behavior, rather than the undesired behavior
B. Instruction 3. Lesson Facilitation 3.1 Teacher explicitly articulates the objectives of the lesson and
relates classroom activities to the objectives
3.2 Teacher explains content using multiple forms of representation
3.3 Teacher makes connections in the lesson that related to other
content knowledge or students’ daily lives
3.4 Teacher models by enacting or thinking aloud
4. Checks for 4.1 Teacher uses questions, prompts or other strategies to determine
Understanding students’ level of understanding
4.2 Teacher monitors most students during independent/group work
4.3 Teacher adjusts teaching to the level of students
5. Feedback 5.1 Teacher provides specific comments or prompts that help clarify
students’ misunderstandings
5.2 Teacher provides specific comments or prompts that help identify
students’ successes
6. Critical thinking 6.1. Teacher asks open-ended questions
6.2. Teacher provides thinking tasks
6.3. Students ask open-ended questions or perform thinking tasks
C. Socio-emotional 7.Autonomy 7.1. Teacher provides students with choices
skills 7.2. Teacher provides students with opportunities to take on roles in
the classroom
7.3. Students volunteer to participate in the classroom
8.Perseverance 8.1. Teacher acknowledges students’ efforts
8.2. Teacher has a positive attitude towards students’ challenges
8.3. Teacher encourages goal setting
9.Social and 9.1. Teacher promotes students’ collaboration through peer
interaction
collaborative skills
9.2. Teacher promotes students’ interpersonal skills
9.3. Students collaborate with one another through peer interaction
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 61
TEACH Overall Result
This section will explain the teaching practices score of the entire sample using the Teach Primary observation tool. The results
will combine both the treated and control groups of our sample, as the aim is to assess the overall result rather than making
comparisons. It is important to note that the sample in this research are limited to teachers who were applying for participating
in Guru Penggerak program – Batch 5, and hence most likely consists of high performers and highly motivated teachers in Java,
the most populated island in Indonesia. Therefore, the Teach Primary score below does not represent the teaching practices of
Indonesian teachers because the sample is not nationally representative.
Figure 24. TEACH Result - Time on learning
No Yes
Teachers provide
learning activity
Time on Learning
5% 95%
Low Medium High
6 or more 2 to 5 0 or 1
Students are on task students are students students
off task are off task are off task
7% 23% 65%
Teachers in our sample provide learning activities to Figure 25. Teach Primary Score of All Areas
stude nts 95% of the time . The y engage stude nts with
various activities that related to learning such as providing 100%
lecture or worksheet, facilitating individual and group
work. However, when teachers provide task, the entire class
are on task only 65% of the time. The rest of the time, at
least two students did not participate by being distracted 75%
or causing disruption in the classroom.
63%
Data collected using Teach Tool indicates that 1 out of every
10 teachers struggle (score less than 2) in at least one area. 50%
Results from Figure 25 above indicates that teachers have
35%
strong ability in Classroom Culture (94% of teachers have
a score higher than 3) and in Instruction (64% of teachers
scored higher than 3); however, they exhibit slightly weaker 25%
ability in Socioemotional Skills (58% of teachers have a
score less than 3).
1% 1%
Overall, teachers performed well in the Classroom Culture 0%
area, achieving an average score of 3.6 out of 5 possible Medium Medium
Low Low High High
points. They demonstrated moderate performance in the
Instruction area, with an average score of 3.1, and weaker
performance in the Socioemotional Skills area, with an Distribution of scores
average score of 2.7.
62 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table 15. TEACH Results
Variable (1) (2) (3) (2)-(3)
Total Female Male Pairwise t-test
Mean/(SE)/SD Mean/(SE)/SD Mean/(SE)/SD Mean difference
Classroom Culture 3.569 3.565 3.578 -0.013
(0.019) (0.022) (0.038)
0.507 0.490 0.547
Supportive Learning Environment 3.660 3.672 3.631 0.041
(0.022) (0.026) (0.041)
0.572 0.568 0.584
Positive Behavioral Expectations 3.477 3.457 3.524 -0.067
(0.029) (0.033) (0.057)
0.765 0.741 0.819
Instruction 3.103 3.111 3.085 0.026
(0.022) (0.026) (0.040)
0.579 0.579 0.581
Lesson Facilitation 3.713 3.719 3.699 0.020
(0.029) (0.036) (0.048)
0.767 0.798 0.689
Checks for Understanding 3.347 3.356 3.325 0.031
(0.034) (0.041) (0.063)
0.911 0.918 0.898
Feedback 2.186 2.164 2.238 -0.074
(0.039) (0.047) (0.073)
1.041 1.039 1.048
Critical Thinking 3.167 3.204 3.078 0.127
(0.037) (0.043) (0.073)
0.990 0.964 1.047
Socioemotional Skills 2.678 2.677 2.678 -0.001
(0.026) (0.031) (0.044)
0.679 0.697 0.634
Autonomy 2.941 2.968 2.879 0.089
(0.036) (0.043) (0.064)
0.944 0.953 0.921
Perseverance 2.283 2.277 2.296 -0.019
(0.023) (0.028) (0.040)
0.600 0.613 0.571
Social and Collaborative Skills 2.809 2.787 2.859 -0.072
(0.051) (0.062) (0.087)
1.336 1.372 1.247
TEACH Global Score 3.116 3.118 3.114 0.004
(0.017) (0.020) (0.031)
0.438 0.436 0.441
Number of observations 700 494 206 700
Significance: ***=.01, **=.05, *=.1.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 63
Table 16. Overview of TEACH Instrument
Number of Schools 285
% of PGP school 10%
% of Non-PGP school 90%
% of Urban school 25%
% of Rural school 75%
Number of Teachers 350
# of TEACH Primary Observations 700
Figure 26. Distribution of Average Teach Primary Scores by Area and Element
Classroom A. Classroom Culture 3.6
Culture 1. Supportive Learning Environment 3.7
2. Positive Behavioral Expectations 3.5
Instruction B. Instruction 3.1
3. Lesson Facilitation 3.7
4. Checks for understanding 3.3
5. Feedback 2.2
6. Critical Thinking 3.2
Socioemotional C. Socioemotional Skills 2.7
Skills 7. Autonomy 2.9
8. Perseverance 2.3
9. Social and Collaborative Skills 2.8
1 2 3 4 5
Figure 26 shows that Indonesian teachers in our sample are relatively skilled at creating a supportive learning environment and
setting positive behavioral expectations. Moreover, these teachers are also skilled in facilitating the lesson. They score around
the medium range in checking for understanding and encouraging students to think critically; but are less skilled at providing
feedback. Lastly, teachers are around the medium range in promoting student autonomy and social and collaborative skills; but
are poor at fostering perseverance.
Overall, teachers performed well in the Classroom Culture area, achieving an average score of 3.6 out of 5 possible points. They
demonstrated moderate performance in the Instruction area, with an average score of 3.1, and weaker performance in the
Socioemotional Skills area, with an average score of 2.7.
64 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
APPENDIX 3 –
EMPIRICAL STRATEGY
We estimate the Local Average Treatment effect using a two-stage least squared approach. The first and second stage
equations are respectively:
2 2
PGPi,d = α0 + ∑ α1, x p
p
i ,d + α2 Abovei,d + ∑ α1,p (x × Above)p
i ,d + Xi + γd + ∈i,d (1)
p =1 p =1
2 2
yi,d = β0 + ∑ β1,p x p
i ,d + β2 PĜPi,d + ∑ β1, (x × Above)
p
p
i ,d + Xi + γd + ∈i,d (2)
p =1 p =1
Where yi,d is the outcome of interest for teacher i in district d; For robustne ss, we show diffe re nt e stimate s, varying the size
xi,d is the running variable—teachers’ score in the selection of the bandwidth and the level of polynomial relationship
tests, subtracted by the cut-off value for their district and between the running variable and the dependent variable.
contract type; and Abovei,d a dummy variable equal to one In particular, we include a quadratic fit when estimating
if a teacher crosses threshold for selection in their district; the model using the full sample of teachers, but a linear
PGPi,d is a dummy variable equal to one if a teacher was fit when we restrict the bandwidth. We show estimates for
selected to participate. X is a vector of teacher-level pre- bandwidths that range from just two values from the cutoff
treatment or time-invariant characteristics, such as age, to six values from the cutoff.
teacher contract type, teachers’ education level, and
gender. Control variables are selected using the adapted Lasso
procedure. We select control variables that are both
Our estimate of the local average treatment effect is β 2 . predictive of treatment and the dependent variable of
Since we have one-sided non-compliance, with a zero interest.
probability of teachers being selected if they fall just short
of the threshold, this estimate can also be interpreted as The F-statistic for the first-stage regression is very strong:
the Treatment Effect on the Treated: the impact of being 146.54.
selected to participate in PGP.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 65
APPENDIX 4 –
ADDITIONAL TABLES AND RESULTS
Table A.4.1: Teacher Characteristics (incl. recent school principals)—by PGP
(1) (2) (3) T-test
Variable PGP Control Total Difference
Mean/SE Mean/SE Mean/SE (1)-(2)
Administrative data
Selection score 100.152 91.385 96.990 8.794***
(0.240) (0.309) (0.288)
Female 0.697 0.796 0.732 -0.099**
(0.029) (0.035) (0.024)
Civil servant teacher 0.717 0.533 0.651 0.184***
(0.029) (0.043) (0.024)
Contract teacher 0.217 0.328 0.257 -0.111**
(0.026) (0.040) (0.022)
Sekolah Penggarak 0.045 0.015 0.034 0.030
(0.013) (0.010) (0.009)
Survey data
School Principal 0.111 0.036 0.084 0.074**
(0.020) (0.016) (0.014)
Masters degree 0.066 0.044 0.058 0.022
(0.016) (0.018) (0.012)
Teacher certificate 0.844 0.686 0.787 0.158***
(0.023) (0.040) (0.021)
Age 37.971 37.883 37.940 0.088
(0.376) (0.490) (0.298)
Years experience 15.922 15.657 15.827 0.265
(0.327) (0.432) (0.260)
PGP—batch 5 1.000 0.000 0.640 N/A
(0.000) (0.000) (0.025)
PGP—batch 7 or 8 0.000 0.015 0.005 -0.015*
(0.000) (0.010) (0.004)
Organisasi Penggerak 0.262 0.226 0.249 0.036
(0.028) (0.036) (0.022)
Sekolah Penggerak 0.086 0.073 0.081 0.013
(0.018) (0.022) (0.014)
Kurikulum Merdeka 0.041 0.007 0.029 0.034*
(0.013) (0.007) (0.009)
N 244 137 381
Notes: The value displayed for t-tests are the differences in the means across the groups. ***, **,and * indicate significance at the
1, 5, and 10 percent critical level.
66 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.2: TOC I. Self-reflection and self-development
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.70 0.14 0.03 -0.04
(0.03) (0.13) (0.12) (0.09)
Maps strength 0.90 -0.12 -0.16 0.01
(0.06) (0.22) (0.21) (0.16)
Show strength mapping document 0.20 0.56** 0.29 -0.06
(0.07) (0.28) (0.24) (0.18)
Can mention are for improvement 1.00 -0.03 -0.05 -0.07
(.) (0.05) (0.05) (0.05)
Observations 31 89 127 350
Notes: Row headings denote the dependent variable. x refers to running variable, centered at zero. The first column is the mean
for non-PGP teachers who scored one or two points below the cutoff for selection their district. Columns (2) to (4) show the
local average treatment effects, estimated using equations (1) and (2). In column (2) the sample is restricted to teachers who
scored between two points below and two points above the cutoff. In column (3) the sample is expanded to teachers who scored
between three below and three above the cutoff. In column (4) the same is expanded to all surveyed teachers. All estimations
control for the teacher’s contract type and x, interacted with treatment. Column (4) also controls for x2, interacted with treatment.
Each regression includes additional control variables that are selected using the adpative Lasso procedure. Baseline variables
that are either predictive of treatment or predictive of the dependent variable are included. Data from the teacher survey. All
dependent variables, except for the indices, are binary. “Index” is the arithmetic mean of the all the other variables in the panel.
“Map strength”=1 if a teacher mentioned that they have mapped their strengths and weaknesses. “Shows strength mapping
document”=1 if a teacher can show the document. ***, **, and * indicate significance at the 1, 5, and 10 percent critical level.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 67
Table A.4.3: II.1 Student-centered teaching—Positive discipline
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Panel A. Student-level
Index 0.63 0.25*** 0.15** 0.10**
(0.01) (0.09) (0.07) (0.05)
Has class agreement (social contract) 0.85 0.15 0.07 0.12
(0.02) (0.14) (0.13) (0.09)
Teacher always encourages us 0.63 0.20 0.11 0.06
(0.03) (0.13) (0.12) (0.09)
Always reminded of social contract 0.43 0.25 0.18 0.16
(0.03) (0.17) (0.14) (0.10)
NOT scolded by the teacher 0.62 0.42** 0.19 0.12
(0.03) (0.19) (0.16) (0.11)
NOT punished 0.82 0.22 0.12 0.11
(0.02) (0.17) (0.14) (0.09)
No-one afraid to ask 0.43 0.20 0.22 0.06
(0.03) (0.18) (0.16) (0.11)
Panel B. Teacher level
Index 0.47 0.19 0.04 0.19
(0.06) (0.22) (0.19) (0.14)
Teacher communicated reciprocally 0.70 0.26 0.26 0.25
(0.09) (0.34) (0.28) (0.22)
Manager position 0.10 0.50** 0.36* 0.34**
(0.06) (0.23) (0.19) (0.16)
Motivational Provision 0.60 -0.18 -0.49 -0.01
(0.09) (0.39) (0.33) (0.24)
Observations 31 89 127 350
Notes: See notes in table A.4.2. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable “Index”
is the arithmetic mean of the constituent binary indicators. See Table 5 for construction of the dependent variables.
68 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.4: II.1 Student-centered teaching—differentiated learning
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Panel A. Student-level
Index 0.53 0.02 0.05 0.02
(0.02) (0.07) (0.08) (0.05)
Always ask if student happy 0.56 0.06 0.09 0.06
(0.03) (0.13) (0.12) (0.08)
Always help the student 0.61 -0.01 -0.05 0.01
(0.03) (0.09) (0.11) (0.07)
Always provide different learning 0.54 -0.00 0.22* 0.08
(0.03) (0.11) (0.13) (0.09)
Assignments completed the way I like 0.36 -0.13 -0.19 -0.01
(0.03) (0.12) (0.15) (0.09)
Always patient when explaining 0.46 -0.06 0.12 -0.13
(0.03) (0.11) (0.12) (0.08)
Always helps when difficult with classwork 0.61 -0.01 -0.05 0.01
(0.03) (0.09) (0.11) (0.07)
Always explains difficult things clearly 0.55 0.09 0.18 0.05
(0.03) (0.12) (0.14) (0.09)
Panel B. Teacher-level
Index 0.32 0.10 0.16 0.17
(0.05) (0.18) (0.17) (0.13)
Adjust learning environment 0.40 0.15 -0.22 0.16
(0.09) (0.41) (0.33) (0.23)
Scaffolding & independent assignments 0.47 0.14 0.26 0.11
(0.09) (0.42) (0.35) (0.24)
Modify learning strategies 0.50 -0.26 -0.06 -0.13
(0.09) (0.41) (0.35) (0.25)
Concrete, clear, constructive feedback 0.13 0.19 0.46* 0.17
(0.06) (0.27) (0.24) (0.19)
Differentiated assessment 0.10 0.28 0.38 0.53***
(0.06) (0.26) (0.24) (0.19)
Observations 31 127 89 350
Notes: See notes in table A.4.2. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable “Index”
is the arithmetic mean of the constituent binary indicators. See Table 5 for construction of the dependent variables.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 69
Table A.4.5: II.1 Student-centered teaching—socio-emotional competencies
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Panel A. Student-level
Index 0.54 0.07 -0.01 -0.04
(0.01) (0.08) (0.07) (0.05)
Teacher can be asked anything (strongly agree) 0.50 0.20 0.11 -0.01
(0.03) (0.17) (0.14) (0.10)
Can tell teacher if bad day (strongly agree) 0.27 -0.00 -0.11 -0.09
(0.03) (0.13) (0.09) (0.08)
Like how treated by teacher (strongly agree) 0.55 -0.12 -0.23** -0.13
(0.03) (0.14) (0.12) (0.09)
Teacher always greets student with a smile 0.57 0.21 0.25* 0.13
(0.03) (0.17) (0.14) (0.10)
Teacher always kind to all students 0.84 -0.03 -0.13* -0.12**
(0.02) (0.09) (0.07) (0.06)
Panel B. Teacher-level
Index 0.50 -0.74*** -0.35 -0.05
(0.07) (0.28) (0.24) (0.17)
Empathetic 0.57 -0.74* -0.55* -0.45*
(0.09) (0.39) (0.32) (0.24)
Positive emotions 0.50 -1.20*** -0.71** -0.05
(0.09) (0.40) (0.29) (0.22)
Socio-Emotional Content in Learning Activities 0.43 -0.29 0.22 0.33
(0.09) (0.35) (0.34)) (0.24)
Observations 31 89 127 350
Notes: See notes in table A.4.2. Data in Panel A is at a student level; Panel B is at a teacher level. In each case the variable “Index”
is the arithmetic mean of the constituent binary indicators. See Table 5 for construction of the dependent variables.
70 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.6: Teach scores
Control Mean Impact
(1) (2) (3) (4)
| x| ≤ 3 | x| ≤ 4 Full
Index -0.26 1.12 1.08* 0.77*
(0.19) (0.71) (0.60) (0.45)
Classroom culture -0.36 1.33 1.66** 0.98**
(0.20) (0.85) (0.67) (0.48)
Supportive learning 3.48 0.66 0.57* 0.75***
(0.11) (0.41) (0.32) (0.25)
Positive behaviorial 3.35 0.44 0.81* 0.08
(0.12) (0.55) (0.44) (0.30)
Instruction -0.05 0.76 0.21 0.58
(0.19) (0.74) (0.64) (0.46)
Lesson facilitation 3.67 0.67* 0.34 0.16
(0.11) (0.40) (0.32) (0.26)
Check for understanding 3.35 0.34 0.04 0.17
(0.13) (0.54) (0.43) (0.32)
Feedback 2.23 -0.17 -0.16 0.02
(0.15) (0.63) (0.52) (0.37)
Critical thinking 3.07 0.57 0.16 0.72**
(0.13) (0.48) (0.45) (0.33)
Socio-emotional -0.20 0.58 0.73 0.26
(0.15) (0.53) (0.47) (0.41)
Autonomy 2.83 0.23 0.40 0.33
(0.13) (0.48) (0.38) (0.31)
Perseverance 2.25 0.44 0.29 0.06
(0.07) (0.29) (0.28) (0.21)
Social collaborative 2.65 0.18 0.37 -0.01
(0.15) (0.59) (0.48) (0.40)
Observations 31 89 127 350
Notes: See table A.4.2. Data from Teach classroom observations instrument. The dependent variables Classroom culture,
Instruction, and Socio-emotional are the arithmetic means of their constituent indicators, standardized to have have a mean of zero
and standard deviation of one. The dependent variable Overall is the arithmetic mean of these three indices, also standardized to
have a mean of zero and standard deviation of one.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 71
Table A.4.7: II.1 Create communities of learning (full sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.55 0.05 0.06 0.05
(0.04) (0.13) (0.17) (0.09)
(a) Learning community
Exists 0.87 -0.22 -0.04 -0.04
(0.06) (0.26) (0.20) (0.16)
—initiated by teacher or school principal 0.60 -0.58 -0.62* -0.26
(0.09) (0.39) (0.33) (0.25)
—initiated by teacher 0.23 -0.07 -0.27 -0.06
(0.08) (0.33) (0.28) (0.19)
—initiated by teacher† 0.13 -0.15 0.07 -0.08
(0.06) (0.31) (0.26) (0.19)
— peer teacher joined 0.77 -0.36 0.02 -0.02
(0.08) (0.30) (0.24) (0.18)
—helps improve teaching 0.80 -0.07 0.09 0.04
(0.07) (0.28) (0.24) (0.18)
(b) Self-development plan
Maps strengths 0.80 0.44 0.28 0.16
(0.07) (0.32) (0.27) (0.18)
—maps strengths (doc shown) 0.17 0.82*** 0.42** 0.02
(0.07) (0.24) (0.19) (0.17)
—Can mention area of improvement 0.77 0.59* 0.37 0.10
(0.08) (0.34) (0.28) (0.20)
Asked to self-reflect as a teacher 0.60 0.27 0.20 0.06
(0.09) (0.38) (0.32) (0.23)
—Initiated by PGP teacher 0.37 0.04 0.28 0.16
(0.09) (0.38) (0.31) (0.23)
Asked to self-reflect on learning process 0.53 0.10 0.02 0.26
(0.09) (0.39) (0.33) (0.24)
—initiated by PGP teacher 0.33 0.02 -0.16 0.23
(0.09) (0.31) (0.28) (0.23)
Asked to reflect on learning process† 0.67 -0.21 -0.15 0.20
(0.09) (0.33) (0.30) (0.22)
Observations 31 89 127 350
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey, with the exception of variables
denoted by †, which come from the school principal survey. Index is is the arithmetic mean of all the other dependent variables
shown in the table. All variables are binary.
72 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.8: III.1 Create communities of learning (restricted sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.49 0.38** 0.23 0.19
(0.06) (0.19) (0.15) (0.12)
(a) Learning community
Exists 0.81 -0.04 0.01 0.11
(0.10) (0.28) (0.23) (0.19)
—initiated by teacher or school principal 0.63 -0.24 -0.36 -0.13
(0.13) (0.46) (0.35) (0.28)
—initiated by teacher 0.13 0.33 0.05 0.29*
(0.09) (0.21) (0.19) (0.17)
—initiated by teacher† 0.06 -0.10 0.15 -0.17
(0.06) (0.26) (0.24) (0.19)
— peer teacher joined 0.69 0.21 0.35 0.25
(0.12) (0.37) (0.28) (0.21)
—helps improve teaching 0.69 0.24 0.34 0.30
(0.12) (0.39) (0.29) (0.22)
(b) Self-development plan
Maps strengths 0.63 1.07*** 0.81*** 0.39*
(0.13) (0.38) (0.31) (0.23)
—maps strengths (doc shown) 0.19 0.79*** 0.29 -0.13
(0.10) (0.26) (0.22) (0.20)
—Can mention area of improvement 0.56 1.40*** 1.00*** 0.43*
(0.13) (0.35) (0.28) (0.24)
Asked to self-reflect as a teacher 0.63 -0.35 -0.18 0.07
(0.13) (0.40) (0.35) (0.26)
—Initiated by PGP teacher 0.38 0.32 0.39 0.25
(0.13) (0.44) (0.37) (0.28)
Asked to self-reflect on learning process 0.56 0.72* 0.13 0.34
(0.13) (0.42) (0.36) (0.27)
—initiated by PGP teacher 0.31 0.81*** 0.18 0.37
(0.12) (0.25) (0.26) (0.24)
Asked to reflect on learning process† 0.56 0.18 0.07 0.29
(0.13) (0.43) (0.35) (0.27)
Observations 16 60 83 224
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey, with the exception of variables
denoted by †, which come from the school principal survey. Index is is the arithmetic mean of all the other dependent variables
shown in the table. All variables are binary.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 73
Table A.4.9: III.2 Coaching (full sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.48 -0.02 -0.10 0.18
(0.05) (0.19) (0.22) (0.14)
(a) Asked to discuss student-centered learning 0.67 0.49 0.43 0.29
(0.09) (0.37) (0.29) (0.22)
—initiated by PGP teacher 0.50 0.03 0.08 0.24
(0.09) (0.41) (0.33) (0.24)
(b) Maps out student needs 0.53 -0.13 0.04 0.35
(0.09) (0.37) (0.31) (0.24)
—initiated by PGP teacher 0.30 -0.06 -0.03 0.35
(0.09) (0.38) (0.32) (0.24)
(c) Observed teaching 0.80 -0.33 -0.21 -0.14
(0.07) (0.29) (0.26) (0.20)
Observed by PGP teacher 0.10 -0.57* -0.40* -0.01
(0.06) (0.29) (0.24) (0.16)
Observations 31 89 127 350
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey. Index is is the arithmetic mean of all
the other dependent variables shown in the table. All variables are binary.
Table A.4.10: III.2 Coaching (restricted sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.44 0.51** 0.35* 0.31**
(0.06) (0.20) (0.18) (0.14)
(a) Asked to discuss student-centered learning 0.63 0.77** 0.52 0.34
(0.13) (0.39) (0.34) (0.26)
—initiated by PGP teacher 0.50 0.77* 0.32 0.10
(0.13) (0.40) (0.35) (0.28)
(b) Maps out student needs 0.44 0.37 0.57 0.59**
(0.13) (0.43) (0.37) (0.27)
—initiated by PGP teacher 0.25 0.71** 0.53* 0.59**
(0.11) (0.36) (0.31) (0.24)
(c) Observed teaching 0.81 0.22 0.08 0.04
(0.10) (0.30) (0.26) (0.23)
Observed by PGP teacher 0.00 0.20 0.11 0.22**
(.) (0.15) (0.14) (0.10)
Observations 16 60 83 224
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey. Index is is the arithmetic mean of all
the other dependent variables shown in the table. All variables are binary.
74 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.11: IV.2 Vision of student-centered school development (full sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.618 -0.12 0.11 0.19
(0.05) (0.16) (0.14) (0.12)
Peer teacher knows vision 0.97 -0.03 0.06 -0.02
(0.03) (0.06) (0.07) (0.06)
—determined by joint discussion and reflection 0.77 -0.31 0.13 0.31*
(0.08) (0.26) (0.23) (0.18)
—initiated by PGP teacher 0.23 0.07 0.17 0.17
(0.08) (0.35) (0.28) (0.21)
—initiated by PGP teacher† 0.43 0.09 -0.06 0.12
(0.09) (0.30) (0.27) (0.22)
—Teachers and school community involved 0.63 -0.45 0.02 0.31
(0.09) (0.41) (0.33) (0.24)
Observations 31 89 127 350
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey, with the exception of variables
denoted by †, which come from the school principal survey. Index is is the arithmetic mean of all the other dependent variables
shown in the table. All variables are binary.
Table A.4.12: IV.2 Vision of student-centered school development (restricted sample)
Control Mean Impact
(1) (2) (3) (4)
−2 ≤ x < 0 | x| ≤ 2 | x| ≤ 3 Full sample
Index 0.44 0.27 0.15 0.25**
(0.05) (0.17) (0.15) (0.13)
—determined by joint discussion and reflection 0.75 0.25 0.22 0.49**
(0.11) (0.34) (0.30) (0.21)
—initiated by PGP teacher 0.25 0.40 0.11 0.17
(0.11) (0.33) (0.30) (0.24)
—initiated by PGP teacher† 0.31 0.86** 0.51* 0.37
(0.12) (0.34) (0.29) (0.24)
—Teachers and school community involved 0.69 -0.15 -0.08 0.22
(0.12) (0.41) (0.34) (0.25)
Observations 16 60 83 224
Notes: See table A.4.2. All dependent variables are constructed from the peer teacher survey, with the exception of variables
denoted by †, which come from the school principal survey. Index is is the arithmetic mean of all the other dependent variables
shown in the table. All variables are binary.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 75
Table A.4.13: Enjoyment of teaching
Control Mean Impact
(1) (2) (3) (4)
| x| ≤ 3 | x| ≤ 4 Full
Overall -0.38 2.30*** 1.94*** 1.22***
(0.15) (0.81) (0.67) (0.47)
I generally enjoy teaching 4.97 -0.12 -0.13 0.08
(0.03) (0.14) (0.13) (0.13)
Gladly prepare and teach my lessons 4.77 -0.27 0.21 0.14
(0.08) (0.32) (0.27) (0.20)
I often have reasons to be happy while I teach 4.57 1.17*** 0.86*** 0.36
(0.09) (0.40) (0.32) (0.24)
I generally teach with enthusiasm 4.67 0.24 0.35 0.15
(0.09) (0.40) (0.31) (0.22)
I often have reasons to be angry while I teach 2.37 -1.13* -0.70 -0.47
(0.15) (0.66) (0.67) (0.44)
I often feel annoyed while teaching 2.10 -1.77** -1.35** -1.16***
(0.15) (0.74) (0.56) (0.38)
Sometimes I get really mad while I teach 2.90 -1.13 -1.23* -0.75
(0.19) (0.85) (0.75) (0.51)
Teaching generally frustrates me 1.37 -0.32 -0.11 -0.22
(0.09) (0.37) (0.38) (0.25)
I generally feel tense and nervous while teaching 1.77 -1.65*** -1.22*** -0.39
(0.11) (0.56) (0.44) (0.31)
Observations 31 89 127 350
Notes: See table A.4.2. Data come from the teacher survey. All dependent variables (except for the index) are Liker scale questions
ranging from (1) “Strongly Disagree” to (5) “Strongly Agree”. Overall is the arithmetic mean, standardized to have a mean of zero
and standard deviation of one.
76 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL
Table A.4.14: Leadership confidence and aspirations
Control Mean Impact
(1) (2) (3) (4)
| x| ≤ 3 | x| ≤ 4 Full
Panel A. Leadership ambitions
Became a school principal 0.06 0.44** 0.14 0.02
(0.04) (0.17) (0.11) (0.09)
Plan to be a school principal or inspector 0.19 0.66** 0.84*** 0.50***
(0.07) (0.33) (0.26) (0.19)
Panel B. Self Confidence as a Leader
Index -0.70 2.64** 2.29*** 1.61***
(0.23) (1.07) (0.85) (0.57)
I have necessary ability to become a good leader of this school 3.68 1.26* 1.38** 0.66*
(0.16) (0.67) (0.54) (0.38)
I know what is needed to help teachers perform their duties 4.06 0.89 0.58 0.46
(0.11) (0.56) (0.43) (0.30)
I believe in my ability to influence teachers 3.71 1.54** 1.33** 0.80**
(0.14) (0.67) (0.54) (0.36)
I know how to encourage teachers 3.84 1.52** 1.22** 0.70*
(0.15) (0.68) (0.53) (0.37)
Trainings have equiped me to become a school principal 3.10 1.63** 1.57** 1.55***
(0.19) (0.79) (0.64) (0.49)
Observations 32 95 135 381
Notes: See table A.4.2. Data come from the teacher survey. The dependent variables in panel A are binary. All dependent variables
(except for the index) in Panel B are Likert scale questions ranging from (1) “Strongly Disagree” to (5) “Strongly Agree”. Index is the
arithmetic mean of these indicators.
IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL 77
Table A.4.15: Leadership confidence and aspirations (by gender)
Impact (| x| < 3) Impact (| x| < 4)
(1) (2) (3) (4)
Male Female Male female
Panel A. Leadership ambitions
Became a school principal 3.23 0.36** -0.08 0.20
(2.47) (0.17) (0.36) (0.12)
Plan to be a school principal or inspector 4.27 0.64** 1.15* 0.71***
(3.53) (0.27) (0.67) (0.24)
Panel B. Self Confidence as a Leader
Index — self-confidence as a leader 24.01 1.37** 6.32** 1.49**
(30.41) (0.68) (2.71) (0.65)
I have necessary ability to become a good leader of this school 6.27*** 0.72 2.90* 0.92*
(1.75) (0.63) (1.56) (0.50)
I know what is needed to help teachers perform their duties 16.23 0.64 1.98 0.48
(50.05) (0.45) (1.56) (0.44)
I believe in my ability to influence teachers 35.12 0.75 5.49* 0.73
(121.02) (0.47) (2.89) (0.47)
I know how to encourage teachers 13.71 0.55 4.57** 0.56
(18.11) (0.40) (1.98) (0.38)
Trainings have equiped me to become a school principal 13.96 1.03 2.51 1.29*
(19.33) (0.80) (1.56) (0.72)
Observations 20 75 33 102
Notes: See table A.4.2. Data come from the teacher survey. The dependent variables in panel A are binary. All dependent variables
(except for the index) in Panel B are Likert scale questions ranging from (1) “Strongly Disagree” to (5) “Strongly Agree”. Index is the
arithmetic mean of these indicators.
Table A.4.16: School Principal's Perception of the Teacher's Leadership Ability
Control Mean Impact
(1) (2) (3) (4)
| x| ≤ 3 | x| ≤ 4 Full
Overall 4.77 -0.44 -0.55** -0.48***
(0.05) (0.27) (0.24) (0.18)
Know how to be a good leader 4.80 -0.28 -0.43 -0.25
(0.07) (0.40) (0.38) (0.25)
Helps team accomplisih tasks 4.87 -0.43 -0.69** -0.63***
(0.06) (0.28) (0.29) (0.21)
Ability to influence and lead teacher 4.77 -0.70* -0.87*** -1.02***
(0.08) (0.37) (0.32) (0.26)
Encourage good team 4.63 -0.34 -0.19 -0.11
(0.10) (0.45) (0.35) (0.25)
Observations 31 89 127 350
Notes: See table A.4.2. Data come from the school principal survey. All dependent variables (except for the index) are Likert scale
questions ranging from (1) “Strongly Disagree” to (5) “Strongly Agree”. The school principal was thinking about the leadership
abilities of the target teacher when answering the questions. Overall is the arithmetic mean of these indicators.
78 IMPROVING TEACHERS AND SCHOOL LEADERSHIP IN INDONESIA: IMPACT EVALUATION OF GURU PENGGERAK PROGRAM AT THE PRIMARY LEVEL