66099 Report No. South Asia Human Development Sector 2011 Discussion Paper Series Report No. 49 South Asia: Human Development Unit Does Information Improve School Accountability? Results of a Large Randomized Trial Priyanka Pandey, Sangeeta Goyal and Venkatesh Sundararaman December 2011 Discussion Paper Series Discussion Papers are published to communicate the results of the World Bank‘s work to the development community with the least possible delay. The typescript manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to the formally edited texts. Some sources cited in the paper may be informal documents that are not readily available. The findings, interpretations, and conclusions expressed herein do not necessarily reflect the views of the International Bank for Reconstruction and Development/ The World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. 2 Abbreviations and Acronyms SSA Sarva Shiksha Abhiyan (Education for all) UP Uttar Pradesh MP Madhya Pradesh RCT Randomized Controlled Trial SBM School Based Management GP Gram Panchayat MLL Minimum Level of Learning VEC Village Education Committee PTA Parent Teacher Association SDMC School Development and Management Committee SC Scheduled Caste ST Scheduled Tribe 3 Acknowledgements This work has benefited from the generous financial support of the Education Program Development Fund and the Bank-Netherlands Partnership Program. We are extremely grateful to the Nike Foundation for their financial backing, and advice on the development of campaign tools. We are indebted to Savita Dhingra, Julie-Anne Graitge, Renu Gupta and Karthika Radhakrishnan for providing invaluable support and assistance. We thank Samuel Carlson, Nazmul Chaudhury, Amit Dar, Deon Filmer, Stuti Khemani, Reema Nayar, and Michelle Riboud for helpful comments. 4 Table of Contents Executive Summary ................................................................................................. 8 Chapter 1 Introduction ....................................................................................... 11 1. Context for Information Campaigns................................................................. 11 2. The Study ......................................................................................................... 13 3. Report Presentation .......................................................................................... 14 Chapter 2 Background: Decentralization in Basic Education in Indian States ....................................................................................................................... 16 1. The Constitutional Amendment for Devolution ............................................... 16 2. Specifics of Local Control in the Study States ................................................. 16 Chapter 3 Methods ............................................................................................ 19 1. Setting .............................................................................................................. 19 2. Timeline and Steps of the Study....................................................................... 20 3. Analysis ............................................................................................................ 24 Chapter 4 Impact of the Information Campaign in the Study States ............ 26 1. Baseline Characteristics ................................................................................... 26 2. Impact of the Information Campaign ............................................................... 26 2.1 State-Wise Impact (UP and MP) .................................................................... 27 2.2 Impact in Karnataka........................................................................................ 35 Chapter 5 Qualitative Findings on the Process of Change .............................. 38 1 Focus Group Discussions ................................................................................. 38 2 Interviews of School Committee Members ...................................................... 43 3. Summary .......................................................................................................... 49 Chapter 6 Discussion ........................................................................................... 51 References ............................................................................................................... 58 Appendix A: Chapter 4 Tables ............................................................................. 61 5 Tables (in text) Table 1: Characteristics of Sample Districts ........................................................... 19 Table 2: What were the Three Main Issues Discussed? .......................................... 39 Table 3: Approximate Cost-Benefit Calculations of the Information Campaign ................................................................................................................. 56 Figures (in text) Figure 1: Causal Links of Change Assumed ........................................................... 13 Figure 2: Percent Change in Participation and Awareness of School Committee Members (UP) ....................................................................................... 28 Figure 3: Percent Change in Participation and Awareness of School Committee Members (MP) ...................................................................................... 29 Figure 4: Percentage Change in Teacher Effort (UP).............................................. 30 Figure 5: Percent Change in Teacher Effort (MP) .................................................. 31 Figure 6: Percent Change in Percentage of Children Able to do a Competency (UP) .................................................................................................... 32 Figure 7: Percent Change in Percentage of Children Able to do a Competency (MP) .................................................................................................... 33 Figure 8: Percent Change in Percent of Children able to do a Competency (Karnataka) .............................................................................................................. 37 Boxes (in text) Box 1: ........................................................................................... School Based Management 16 Box 2: ............. Roles and Responsibilities of School Committees in Madhya Pradesh 22 Box 3: Qualitative Findings on the Process of Change (UP and MP)..................... 28 Box 4: Do Communities Perceive School Committee Members as Key Players? .................................................................................................................... 30 Box 5: ................................. Communities in Intervention Villages Monitored Teachers32 Box 6: Caste and Exclusion from the Accountability Process ................................ 34 Box 7: Post-Intervention Topics of Discussion ....................................................... 35 Box 8: Time Period over Which Discussions Continued ........................................ 38 6 Box 9: Percentage of Participants who Raised School Issues with Those Responsible for Service Delivery ............................................................................ 39 Box 10: Who Were the Issues Raised With? ........................................................... 40 Box 11: How Were Those Responsible for Delivery Approached by Parents?...... 40 Box 12: Experience of Raising an Issue with Those Responsible for Service Delivery.................................................................................................................... 41 Box 13: Use of Learning Assessment Tool by the Community .............................. 42 Box 14: Follow up after Using the Assessment Tool ............................................. 42 Box 15: Experience of Discussion ......................................................................... 43 Box 16: How Long did the Discussions Continue? ................................................ 44 Box 17: Main Themes Discussed ............................................................................ 44 Box 18: Percentage of Members who Raised School Issues with Those Responsible for Service Delivery ............................................................................ 45 Box 19: Who Were the Issues Raised With? ........................................................... 45 Box 20: How Were Those Responsible for Delivery Approached by Members? ................................................................................................................. 46 Box 21: Experience of Raising an Issue with Those Responsible for Service Delivery 46 Box 22: Use of Learning Assessment Tool by Members ........................................ 47 Box 23:Follow up after Using the Assessment Tool ............................................... 47 Box 24:Experience of Discussion ............................................................................ 48 Box 25: Perception of Improvement in Committee Meetings................................. 48 Box 26: Actions Taken by Members ....................................................................... 49 7 Executive Summary 1. Learning outcomes have gained increasing importance in the policy debate on basic education in developing countries. More than 90% percent of children between ages 6-11 are in school in India, but their learning achievements fall well below the expected standard. 2. The problem of poor quality outcomes is a more general one in various public services across many developing countries. Targeting resources efficiently to communities and getting public workers to perform have remained a challenge for many public services, including education. Weak mechanisms of accountability are a key reason for indifferent service delivery; workers rarely face any censure for their absences or below par performance. 3. Monitoring by the local community—who have a direct stake in the quantity and quality of public services that are available to them—can play a fundamental role in improving public service delivery. To this end, a number of developing countries, including India, are decentralizing control over local public services to local communities. The Sarva Shiksha Abhiyan (SSA), a nation-wide public scheme initiated in 2001 to universalize elementary quality education in India, aimed to increase accountability of schools to communities‘ through greater involvement of village education committees and parent-teacher associations. 4. Evidence suggests, however, that communities are too often uninformed of what public services they are entitled to and what controls they have over these services. Providing information that empowers stake holders may therefore be one way to stimulate public demand for quality services. In the case of public education, informed communities can better exercise their government given controls to exert pressure on teachers and schools to perform. 5. This study evaluates a community-based randomized controlled trial (RCT) to determine the impact of an information campaign on learning and other school outcomes. The study was conducted in three Indian states—Uttar Pradesh (UP), Madhya Pradesh (MP), and Karnataka. UP and MP are two large north and central Indian states, respectively, and lag behind in economic and social outcomes, while Karnataka in southern India is economically and socially more developed. Even though all three states have devolved oversight roles to the community with respect to government schools, they differ in the extent to which such devolution has taken place. The information campaign disseminated state specific information to the community on its oversight roles in schools and education services that parents are entitled to. Information was disseminated in 11-14 public meetings in each treatment village over a period of two and a half years. 8 6. The study finds that information matters. Providing information in an effective manner to communities leads to improved learning outcomes and substantial reduction in the wastage of public resources by reducing teacher absenteeism and increasing teacher effort in classrooms. Bigger gains from the information campaign were seen in the two lagging states, UP and MP, where the baseline level of efficiency in outcomes was much lower than Karnataka. 7. Findings from UP and MP: The information campaign led to significant improvements in the functioning of school committees and in teacher effort. School committees in both states became more active as a result of the information campaign. The number of school committee meetings, participation of members in meetings and school visits increased significantly. More members reported knowledge of school accounts and of committee‘s roles and responsibilities vis-à-vis teachers. With respect to teacher effort, in MP there was positive impact on teachers‘ attendance and engagement in teaching activities, and in UP, the impact was on teachers‘ attendance. In both states, the impact of the campaign was greatest for permanent civil service teachers, who had significantly lower effort than contract teachers at baseline. Among civil service teachers, the impact was greater for upper caste, male and more educated teachers. Higher absenteeism and low teacher effort for these categories of teachers are robust findings in the empirical literature for developing countries, and are attributed to their more powerful social status. In other words, results indicate greater impact of the campaign precisely where the accountability problem is more serious. Learning outcome gains from the campaign were less uniform, compared to the gain in teacher effort. A consistent result in both states was an increase in the share of children acquiring mathematics competencies. 8. Findings from Karnataka: The starting points for teacher effort and all other outcomes in Karnataka were much higher than either UP or MP, suggesting greater efficiency in delivery. In the baseline survey, nearly 90% of teachers were present and 80% of those present were teaching. These are much higher numbers when compared to MP and UP; they may leave little scope for information to have an impact. Moreover, the School Development Management Committees (SDMCs) in the state do not have any direct control over teacher effort unlike the other two states: they track neither teacher attendance nor classroom engagement. They do have indirect control over teachers and can report non-performing teachers to the education office which is like the other two states. The information campaign seems to have had an impact in activating the indirect channel: teachers with lower effort at baseline were more likely to be transferred out of schools in treatment villages. There is no impact on the functioning of SDMCs in terms of increase in participation of committee members. However, there was an increase in the knowledge of committee members between baseline and follow-up. The impact on learning occurs in mathematics, like in the other two states. This implies that classroom teaching practices, not measured by the survey, was likely to have been influenced by the campaign. 9 9. Focus group discussions conducted in UP and MP illustrate the channels that were activated by the presence of information, and the barriers to individual and collective action that remained. Change seems to have occurred through discussions within communities and communities bringing up issues with teachers and school committees. The main obstacle to change identified by focus groups was the powerful status of teachers who tend to enjoy protection from local elites. 10. The strengths of this study are that it used a structured intervention (the information campaign) that is easily replicable, had a rigorous cluster randomized controlled trial design and broad geographic coverage. Our findings clearly suggest that: (a) decentralization to communities is meaningless unless it is made real, i.e., communities at least need to know what oversight roles they have: and (b) providing information to communities that are unaware, through a structured campaign, can play a useful role in changing behavior and learning outcomes, particularly in lagging states. 11. A key feature of the SSA framework was increasing accountability of schools to the community through greater involvement of village education committees and parent- teacher associations. Resources have been allocated regularly through SSA for training of school committees. However, as the data show, effective training does not reach a majority of members. To strengthen local governance of schools, the states can revisit their training programs for communities to incorporate key components of the campaign:  Information should be disseminated repeatedly.  Complete information should be made available and left behind with the community.  Information should be imparted not only to school committee members, but also to the larger community.  Communication strategies should take into account literacy levels of the community and orient information accordingly.  School committees‘ and the larger communities‘ capacities to monitor learning outcomes should be strengthened. 10 Chapter 1 Introduction 1. Context for Information Campaigns In developing countries, learning outcomes have gained increasing significance in the policy debate on basic education. This is especially true for countries that have attained much progress in access, but continue to face the problem of low quality education in schools. In India, even though more than 90 percent of children between ages 6-11 are in school, learning achievements remain poor (Annual Status of Education Report, 2005). The problem of poor quality outcomes is a more general one and is not limited to the education sector; various public services across developing countries suffer from this (Global Corruption Report, 2006; World Development Report, 2004). Targeting resources efficiently to communities and ensuring that public service sector workers perform their duties continue to pose challenges. The primary reasons for this include: (a) lack of adequate resources; (b) inefficient and ineffective use of given resources; (c) corruption and capture of services by local elites1; and (d) unenforced accountability systems. When frontline workers such as doctors, health aides, and teachers don‘t deliver as expected, interventions that provide physical resources alone are often not enough to improve outcomes. Mechanisms of accountability are often weak in public service delivery; workers rarely face censure for abysmal performance or repeated absence from work.2 One way in which those responsible for delivery of public services can be made more accountable is by creating monitoring mechanisms for local communities who have a direct stake in the quantity and quality of public services that is available to them. If successful, such mechanisms can play a pivotal role in improving public service delivery (World Development Report, 2004). To this end, a number of developing countries, including India, have decentralized control over local public services and handing management over to communities. For example, as a result of the 73rd amendment to the Indian Constitution,3 many Indian states have devolved varying degrees of control over public schools and their finances to local communities. The Sarva Shiksha Abhiyan (SSA), a national scheme of the Government of India launched in 2001 to universalize 1 Capture refers to situations where certain groups based on their social status, class, asset-ownership, and political power etc capture public resources or prevent public action for their benefit at the cost of others in the community. 2 Chaudhury et al. (2006) document high absence rates among publicly funded health workers and school teachers in several developing countries, including India. 3 The 73rd amendment to the Indian constitution in 1992 made it mandatory for Indian states to devolve control over public services and finances for these services to the local village government called the gram panchayat. However it was left up to the states as to how much control to devolve. 11 quality elementary education, included in its design increased accountability of schools to the community through greater involvement of village education committees (VECs) and parent-teacher associations (PTAs). The impact of community monitoring on school functioning, however, has been lower than expected. This is highlighted, for example, by recent studies which report high teacher absence rates in a number of Indian states (pointing to low accountability levels and a lack of motivation among teachers) despite decentralized control to local communities. Moreover, both survey-based and anecdotal evidence indicate that communities are often uninformed of the services they are entitled to, as well as of the mandated controls that they can exert over these services (CAG report, 2001; Pandey et al. 2007; Banerjee et al., 2008); they may thus be unable to effectively demand accountability. Low levels of awareness among community members suggest that one way to stimulate local demand for accountability is to provide relevant information to stakeholders. Informed communities can exert pressure on teachers and schools to perform in powerful ways. For example, as a result of a newspaper campaign that publicized the diversion of primary school funds in Uganda and aimed to boost schools' and parents' ability to monitor local officials' handling of a large school-grant program, capture of funds by local government officials and politicians reduced from 80 percent to 20 percent (Reinikka and Svensson, 2004). The authors use an instrumental variable approach to identify the impact of public access to information in deterring capture at the local level. The gold standard for estimating the causal impact of any intervention is to conduct a randomized controlled trial (RCT). The evidence from past RCT evaluations of providing school related information to communities is albeit limited and mixed so far.4 A 2008 randomized study in the Indian state of Uttar Pradesh (UP), with a 5-6 month follow up period, found no impact of information on learning outcomes (Banerjee et al., 2008). In this study, information was imparted to communities in village meetings by the village head once. Another study also in UP, with a one year follow-up period, finds a positive impact of information on delivery of health and other social services, based on self-reported outcomes (Pandey et al., 2007). The information given was on oversight roles of communities with respect to health services, schools and village governance. The campaign design was standardized and structured to disseminate the information uniformly via tools such as audio tape and posters. The campaign involved repeat village meetings Even though results from these two studies differ, they are not strictly comparable due to differences in campaign structure, outcome measurement, and follow- up period. A comparison of the modalities of providing information across the studies, however, suggests that not only the content of the information, but who gives the information, who is present when the information is being received, how it is given and how frequently also 4 Moehler (2010) provides a review of findings from randomized controlled trials that evaluate various interventions, including information provision to citizens, aimed at improving public service delivery and other aspects of democracy and governance in developing countries. 12 matter makes a difference on the impact of the trial. A lesson from the experience of these studies is that for this research agenda to inform policy dialogue, research needs to be based on replicable means of information dissemination, objective outcome measurements and follow-up surveys over longer periods. 2. The Study The present study is an evaluation of a community-based randomized controlled trial (RCT) to determine the impact of an information campaign on learning and other school outcomes. The campaign disseminated information to the community on its oversight roles in schools and the education services they are entitled to. The study tested the hypothesis that school outcomes may improve if communities have detailed information about their oversight and about parents‘ entitlements from schools. The information campaign was designed so that it would be consistent across treatment sites and easily replicable, with outcomes being measured along the hypothesized chain of causality. The figure below illustrates the assumed causal links of change. Figure 1: Causal Links of Change Assumed Information Improvement in campaign targets school committee‗s and community‘s Improvement in Improvement in school teacher effort learning committee and awareness and outcomes larger participation in community school oversight The motivation behind providing information to communities was to make them more knowledgeable about their roles and responsibilities with respect to the local school which in turn could lead to better monitoring on their part. The actual magnitude of the impact of such an intervention, however, would depend on several factors. First, changing attitudes and patterns of habit and practice require a lot of time since they have been a way of life for decades. Secondly, the exact nature of the intervention, the extent of control communities have over schools, and the extent of public action in response to the campaign, would also be important. With respect to public action, the relative population proportion and interest in public services of the different groups in the community will be key determinants of change. For example, if local elites do not have a stake in public services, public action may be captured. Based on a field experiment on reducing corruption in infrastructure projects in Indonesia, Olken (2007) finds grassroots monitoring effective only in situations with relatively little free-riding and limited elite capture.5 Furthermore, group size and the length of interaction between individuals are also likely to affect participation (Olson, 1965). Smaller groups encourage participation while large groups can potentially discourage participation since non-participation may 5 Free riding refers to a situation where individuals benefit from a collective activity without participating in it. 13 involve no social cost, unlike in the former case where members risk possible social censure and retaliation. Additionally, repeated interactions between the same individuals in a group are likely to increase sustained cooperation, especially so if sufficiently credible punishments of non-cooperating are available.6 The study was conducted in three Indian states: Karnataka; Madhya Pradesh (MP), and Uttar Pradesh (UP). UP and MP are two large states in north and central India, respectively, that lag behind most of India in many social and economic indicators, while Karnataka, in southern India, is ahead on most development indicators. Even though the three states have devolved oversight roles to the community, they differ in the extent to which such devolution has taken place. In all the three study states, communities were randomly assigned to receive information on their oversight roles in local public schools. Baseline and follow-up surveys were administered to communities to measure outcomes. Additionally, we also conducted focus group discussions with community members in UP and MP.7 The objective of these discussions was identifying the channels of change. We find that providing information in an effective manner to communities leads to improved learning outcomes and substantial reduction in wastage of public resources by reducing teacher absenteeism and increasing teacher effort in classrooms. Bigger gains from the information campaign were seen in the two lagging states UP and MP, where the baseline level of efficiency in outcomes was much lower than Karnataka. Focus group discussions richly describe the manner in which the communities used the information provided, and the social mechanisms activated as a consequence of the presence of greater information in the public domain. Change seems to have occurred through discussions within communities, and communities actively bringing up issues of teacher and school quality with service providers. The nature of the intervention, in particular the structure, intensity and uniformity of the campaign, is central to explaining our results. The impact of information depends on the type and detail of information given, how it is communicated (i.e., the medium used) and whether those delivering it are perceived to be credible and reliable sources. 3. Report Presentation The report which presents findings based on two-year follow up is organized as follows. Chapter 2 provides a brief background on decentralization in basic education in India and details the structure of local control over schools in the three study states. Chapter 3 presents the methodological framework of the study. Chapter 4 details the findings on the impact of the information campaign in the three states and Chapter 5 presents the 6 This study is embedded in the larger research context of examining school-based management (SBM) reforms that have been introduced in many developing countries in recent years. It is pertinent to note however that the study does not assess a school-based management intervention per se, but evaluates the dissemination of information about a program (that has devolved control to the community) already in place. 7 The focus group was not conducted in Karnataka due to reasons of limited resources. 14 qualitative findings from focus group discussions. Chapter 6 presents a discussion on the findings and their policy relevance. Appendix A comprises the tables for Chapter 4. 15 Chapter 2 Background: Decentralization in Basic Education in Indian States 1. The Constitutional Amendment for Devolution For many states in India, the 73rd amendment (in1992) to the constitution flagged off community interventions in local public service delivery systems. It conferred constitutional status on local governance, with a view to giving communities key role in such governance through transfer of powers, authority and responsibilities in several matters including education and health. The amendment, which instructed states to devolve finances and control over public services to local communities, however left the onus on the structure and degree of control to individual states. Similar to other public services, in the area of basic education most states chose to work through a committee at the school or village level, typically comprising of head teachers, teachers, parents and other community members. This is not unlike the school-based management (SBM) approach followed in other countries (Box 1). Box 1: School Based Management School-based management has been in place in countries with diverse educational systems, such as Australia, Canada, Israel and US- going back 30 years in some cases to relatively recent reforms in several developing countries. SBM can be broadly defined as decentralization of authority from the central or state governments to the school level (Caldwell, 2005) and such reforms are far from uniform, shaped by the broader national policy and the social context in which they are created. Typically, the devolution of decision-making has two dimensions to it: the extent of autonomy devolved, and the people to whom the authority for decision-making is devolved. Most SBM reforms involve some sort of transfer of responsibility and decision making- usually the responsibility of school operations- to a combination of head teachers, teachers, parents and other community members. Most programs work through a school committee, which may: monitor school performance, for example, in test scores or teacher and student attendance; appoint or dismiss teachers; ensure that teacher salaries are paid on time; approve school budgets; and examine financial statements. Below, we describe how the devolution of control is organized to function through school committees in the respective study states. There are both similarities and differences across states in the structure and mandate of the institutions created for local governance of schools. 2. Specifics of Local Control in the Study States Madhya Pradesh 16 A parent teacher association (PTA) is mandatory in every school, membership of which is obligatory for parents of students. PTAs are expected to operate primarily through a 14-member Executive Committee, though all parents are supposed to participate in decision-making. The committee has a yearly term and it elects a parent as the chair from among its members, while the secretary is the head teacher. The PTA chair and the secretary (head teacher) jointly operate the school account. Schools receive annual government grants for repair and maintenance, school development, teacher learning materials (TLMs), school uniforms, civil works, and mid- day meals in their accounts. The PTA is supposed to monitor learning, and manage and oversee school funding. It is directed to monitor teacher performance and verify teacher attendance in order for the release of teachers‘ monthly salaries. It can make school visits and register complaints with district or block education offices Public schools in MP have two types of teachers—contract, and civil service teachers. The former, called samvida shikshak, are hired on a 3-year renewable contract by the block panchayat.8 Uttar Pradesh In every gram panchayat (GP)9, a village education committee (VEC) is mandatory; there is one VEC for all primary and junior government schools in the GP. The VEC comprises of 5 members, and is chaired by the elected head of the village government (Gram Pradhan, also referred to as Pradhan). Other members include the senior-most teacher and three parents of students. The VEC has a five-year term, except parent membership can cease if the child is no longer in school. New parent members then replace the old ones. The VEC chair and the school head teacher jointly operate the school account. As in MP, the school account receives annual government grants for repair and maintenance, school development, TLMs, school uniforms, and civil works. In addition, the gram panchayat account, which is co-signed by the VEC chair, receives stipends and mid-day meal funds. The VEC‘s job is to monitor learning and teacher performance, and to manage and oversee school funding coming to the school and gram panchayat accounts. It can visit schools and register any complaints with district/block education offices Public schools in UP have both contract and civil service teachers. The VEC is empowered to select contract teachers and to review their performance on an annual basis for deciding contract renewals. In selecting such teachers, VECs are required to follow 8 A block is the administrative unit that falls between a district and a gram (village) panchayat. The elected government at the block level is called a block panchayat. 9 A gram panchayat is the lowest administrative unit consisting of two to three revenue villages on average. The elected village government (gram panchayat council with the pradhan as its head) is formed at the gram panchayat level. 17 state guidelines that outline eligibility criterion. With a two-thirds majority, the VEC can remove a contract teacher any time during the contract period. Karnataka Every school has a School Development and Monitoring Committee (SDMC), which is mandatory. An SDMC has 9 members, including the head teacher of the primary school, parents, and elected representatives of the village government. A member‘s term is up to 3 years. One parent member is elected as the SDMC chair, while the head teacher is the secretary of the committee. The chair and the secretary (head teacher) jointly operate the school account (the type of funds received are similar to those in Madhya Pradesh schools). SDMC is mandated with monitoring learning, and managing and overseeing school funding. The committee is expected to hold a meeting of all parents every 3 months to discuss learning achievements. It is also expected to monitor teacher performance. Primary schools in Karnataka have only civil service teachers; there are no contract teachers. Unlike the other two states, SDMC does not have any explicit control over teachers; however as in the other two states it can inspect schools and register complaints with district/block education offices. The following is a broad summary of the differences in the structure and the mandate of school committees across the three states. - MP and Karnataka have put in place school level committees with relatively shorter terms (1 year and 3 year, respectively), while UP has a GP level committee with a longer term (5 year) that oversees all schools within the administrative territory of the GP. - In MP the committee is embedded in the general body of all parents in the school, unlike the other two states which have no explicitly defined general body other than the body of all voters in the GP (called the gram sabha). - The overall committee size is larger in MP and in Karnataka (9-14 members) compared to UP (5 members) as also is the number of parent members. - The committee is chaired by a parent in MP and in Karnataka and by the elected head of the village government in UP. - Relative to Karnataka, MP and UP have devolved greater control to these bodies with respect to teachers. In MP the committees verify teachers‘ attendance in order for teachers to receive their salary, and in UP they make decisions on hiring and renewal of contract teachers. 18 Chapter 3 Methods 1. Setting This study is a cluster randomized control trial (RCT) of 610 gram panchayats (GPs) in three states, Karnataka, MP and UP, randomly allocated to receive or not receive the treatment which is an information campaign. A GP is a cluster of approximately 1-3 adjacent villages and is the smallest unit of local government, which consists of an elected head and council members. The trial was conducted from 2006 to 2009. In each of the three states four districts were chosen purposefully, matched across states by literacy rates. This was done to make the districts comparable across states on at least one indicator of development. The districts in order from low to high literacy rates are: Sitapur, Pratapgarh, Hathras and Kanpur Dehat in UP, Dhar, Guna, Katni and Raisen in MP, Bellary, Gulberga, Kolar and Chikmagalur in Karnataka (Table 1). Table 1: Characteristics of Sample Districts UP MP Karnataka District Name (% literate, % population below poverty) * Kanpur rural (67, 43) Raisen (73, 38) Chikmagalur (73, 8) Hathras (63, 28) Katni (65, 27) Kolar (63, 22) Pratapgarh (59, 22) Guna (60, 32) Bellary (58, 43) Sitapur (49, 37) Dhar (53, 46) Gulbarga (51, 42) * Literacy rate is from Census of India (2001), poverty numbers are from Debroy and Bhandari (2003). Within a district, 50 GPs were selected from two randomly chosen blocks. We used a random number generator to randomly select the blocks and then GPs within the blocks. Half of the GPs within each block were then randomly assigned to intervention arm and the remaining half to control arm. Treatment and control GPs were evenly spread across the two blocks to reduce any potential contamination between intervention and control villages. This gives a total of 100 control GPs and 100 treatment GPs per state in MP and UP. In Karnataka the design was identical except an additional set of treatment villages was added that received a slightly different treatment called information and advocacy campaign.10 The number of GPs in each of the three cells in Karnataka (control, treatment 1 (information campaign), treatment 2 (information campaign plus advocacy)) totaled 70. 10 Due to reasons of limited resources, this second treatment was not expanded to the other two study states. 19 Sample sizes were determined using cluster randomized sample size calculations based on a 5 percent significance level and 80 percent power. 2. Timeline and Steps of the Study For MP and UP, baseline surveys were administered between February and April 2006. The first round of intervention was carried out from September 2006 to January 2007, and the first follow-up surveys administered between February and April 2007. A year after the baseline survey, focus group discussions were held with community members in select intervention GPs. A second round of intervention was from September 2008 to January 2009, with second follow-up surveys between February and April 2009. In Karnataka, baseline surveys were administered between July and August 2006.11 The first round of intervention was carried out from February to May 2007, with the first follow-up surveys administered between July and August 2007. The second round of intervention was from February until May 2009, and second follow-up surveys were administered between July and August 2009. Baseline survey One school was randomly selected from all public schools that had grades 1 to 5 in each GP. All grade 1-5 teachers in the school were in the sample, along with randomly selected students from specific grades. In MP and UP, 15 students were randomly selected from each of the grades 2, 3 and 4, totaling 45 per school. In Karnataka, 30 students were selected per school, 15 each from grades 4 and 5. School enrollment registers were used for random selection of students. In the instance a given grade had less than the required number of students all students were selected. The surveys were conducted via in-person interviews by a team of trained research assistants with prior experience in administering rural household surveys in the region. Surveys were conducted in local languages (Hindi in MP and UP and Kannada in Karnataka) and instruments were pilot tested prior to use. Outcomes measured at baseline  Teacher attendance and activity: Four unannounced visits were made, one every 2-3 weeks, to record attendance and activity. The latter was a measure of whether a teacher was actively engaged in teaching when the team arrived. Attendance takes the value 1 if teacher was present in school, 0 otherwise. Activity is 1 if the teacher was teaching, writing on the board, supervising written work, teaching by rote or another method, 0 if teacher was absent, chatting, sitting idle or standing outside the classroom, keeping order in the classroom but not teaching, engaged in non-teaching work. 11 Karnataka had a shifted timeline relative to MP and UP due to differences in the school year. The school year for Karnataka is from June to March, unlike in the other two states where school year is from July to May. 20 Teacher attendance and activity variables are constructed as averages over the four visits for a teacher and interpreted as fraction of visits the teacher was present (or engaged in teaching). Both variables take values between 0 and 1.  Students were assessed in school on a competency based language and mathematics test that lasted approximately 20 minutes. The language test had reading and writing competencies that involved reading simple 2-4 syllable words and short sentences, and writing simple 2-4 syllable words and short sentences that were dictated. The mathematics test included addition (with and without carryover), subtraction (with and without borrowing), multiplication and division. We used the minimum level of learning (MLL) framework recognized by the Government of India as a benchmark for the minimum that a child in a given grade should know. The competencies tested fall within or below those listed for the grade by the MLL framework.12 For example, the framework specifies that in language, a second grader should be able to read short paragraphs and write dictated sentences, and in math, two digit additions with carryover, and subtractions with borrowing. We define learning outcomes as the share of children who attain specific competencies: in language, outcomes are percent of children who can read sentence and words, and percent of children who can write sentence and words. In math, outcomes are percentages of children who can do: addition, subtraction and less (addition), multiplication and less (addition and subtraction), division and less (addition, subtraction and multiplication). In MP and UP tests took place at the end of the school year, while in Karnataka, testing was done 4 months later at the start of the school year. Thus, the test administered to grade 4 (grade 3) students at end of school year in MP and UP was given to grade 5 (grade 4) students at beginning of the school year in Karnataka.  Interviews of school committee members regarding their knowledge of roles and responsibilities and participation in school oversight related activities. Intervention We collaborated with the Nike Foundation in designing the intervention which was an information campaign. The Foundation provided financial support and creative advice in the development of campaign tools. The tools consisted of a short film of 6 minutes, poster, wall painting, take home calendar and a learning assessment booklet. All the tools used for the information campaign were the same across states except that the information they contained was state specific. The film, poster and calendar focused on the following information: details of roles and responsibilities of school oversight committees; rules for selection of members of these committees; rules for committee meetings; number of mandatory meetings, minimum 12 According to the framework, at least 80 percent of children are expected to be able to do at least 80 percent of the competencies for the grade. The framework can be found at www.education.nic.in/cd50years/r/2S/Book2S.htm 21 attendance requirements for meetings; record keeping of minutes; organization and funding of school accounts; right to information regarding the school including right to obtain copies of any school record; where to complain about any problems; benefits that students in primary grades are entitled to such as cash stipend, textbooks, mid day meal, school uniforms. The film and poster contained key information while the calendar contained all of the information in detail. Box 2: Roles and Responsibilities of School Committees in Madhya Pradesh A sample of the information provided to the community regarding the main roles and responsibilities of the PTA:  Ensure that the school is functioning well  Prepare schemes for improving schools  Ensure that all children in ages 5-14 are enrolled in school  Ensure that children attend school regularly  Look after any construction in the school, repairs and management of existing schools  Manage and monitor the funds coming into the school education account  Decide how money is to be spent based on the school‘s needs and give consent for use of funds  Monitor the distribution of textbooks, scholarships and uniforms  Implement the mid-day meal program. Monitor the quality of food served  Ensure that children are learning at appropriate levels for their grade  Ensure that teachers come to school regularly and teach properly  PTA chair verifies every teacher‘s attendance monthly by signing on teacher‘s attendance sheet. Can stop any teachers‘ salary by not signing on teachers‘ attendance sheet if teacher does not come regularly  Complain to the block or district education office or to the Jan Shiksha Kendra (cluster resource center) and recommend disciplinary action if dissatisfied with teacher (examples: if teachers do not come or discharge their duties appropriately)  Ensure at least 200 teaching days per school year, and at least 5 hours of teaching on an average per day The learning assessment booklet outlined the minimum levels of language and math skills that children are expected to acquire by grade, based on the MLL framework recognized by the government of India. Parents could use the assessment booklet on their child to find out whether the child knows the minimum expected for his or her grade. The information in all the tools was obtained from the basic education departments and tools were verified and approved by the state governments for their content before the campaign. Campaign teams were blind to baseline and follow up surveys. The information campaign was conducted in two rounds, 2006-07 and 2008-09. In the first round, the campaign was done three times, each time separated by a period of two- three weeks. Each time the campaign consisted of 2-3 meetings in different neighborhoods of the GP as well as distribution of posters and take home calendars. Residents were informed in advance about the dates and locations of meetings, and separate meetings were held in low and high caste neighborhoods. The primary target audiences were parents and members of school committees. A meeting lasted about 30- 40 minutes and consisted of a film presentation that was played twice, opportunities to 22 ask questions and for discussion to happen among the audience. People were notified that the information was collected from the government. To ensure that the information campaigns were uniform, research assistants read a scripted introduction and were only allowed to answer questions to which the answers were already written on the calendars. Any other questions or issues were not answered. The team was also not allowed to participate in any discussion that took place among the community members following the film presentation. In the second round, the campaign was also done three times, each time separated by a period of three-four weeks. We had conducted a focus group discussion in a very small sample of treatment villages (see below) between the two rounds and the following suggestions from participants were incorporated in the second round in the way meetings were organized without changing anything substantial in the intervention. One change was that meetings were now mainly held at the school instead of different locations in the village. If members from some caste groups (or women) were not willing to come to the common meeting, then the team would also hold another meeting in their neighborhood. A second change was that the film was excluded the last 2 times meetings were held. With the same film shown repeatedly, it was difficult to attract audience for the meetings. A third change was that in order to inform the illiterate audience, the key information in the calendar was read aloud exactly as written to preserve uniformity. In addition to the information campaign treatment that was identical to the treatment in the other two states, there was a second treatment done only in Karnataka. This was an additional 2 minute capsule at the end of the film which showed average wages for different levels of schooling to increase awareness about the economic benefits of schooling. It also advocated the audience to become involved in monitoring outcomes in the school. All other tools were the same as in the first treatment. The campaign in both treatments was conducted in an identical manner as in the other two states. First follow-up survey Baseline survey participants were re-interviewed after 12 months by research assistants who had no knowledge of the intervention. To maintain this blinding, the intervention group teachers, students and committee members were not asked if they had attended any informational meetings. In MP and UP, sample students (who were in grades 2, 3, 4 at baseline) were followed and they were now in grades 3, 4, and 5 respectively. Each cohort by grade was administered the same test form as in the baseline. For example, grade 3 students were given the same test form that had been administered to them when they were in grade 2 at baseline. The same process was followed in Karnataka: sample students at baseline, who were now in grades 5 and 6, were followed. Each cohort by grade was given the same test form as in the baseline (for example, grade 5 students who were in grade 4 at baseline were given the same test form). Focus group discussion (FGD) and qualitative interviews of committee members 23 One year after the first intervention round, we conducted FGDs in 10 randomly selected intervention GPs each in MP and UP.13 Two focus group meetings were held in each GP, one among residents from disadvantaged social classes—Scheduled Castes (SC) and Scheduled Tribes (ST)14 —and the other among non-SC/ST residents. Each meeting averaged 12 participants who were asked if they remembered the campaign, whether they had discussed the information with anyone else in the village post-campaign, whether they had raised school related issues with teachers or VEC/PTA, why bigger changes were not seen and what could be done to improve the campaign. Alongside focus group discussions, in the same GPs individual school committee members (32 in UP and 50 in MP) were interviewed one-on-one on the same questions that were put to the focus groups, with a few additional questions on their own participation in school oversight related activities. Second follow-up survey For the second follow-up survey, the first follow-up survey participants were re- interviewed after 24 months. In MP and UP, sample students in grades 3 and 4 at first follow-up could not be followed and tested since they were out of the primary school cycle at this time. A fresh sample of students in grades 3, 4 was chosen, while those currently in grade 5 could be followed from first follow-up (at which time they were in the grade 3). In case a grade 5 student from first follow-up was not in school anymore but remained in the village, the research team tracked the student in the village and administered them the test. Each grade was given the same test as in the first follow-up (for example, grade 5 was given the same test that grade 5 in first follow-up had been given). In Karnataka too, first follow-up survey participants were re-interviewed after 24 months. Sample students at baseline could not be followed and tested since they were out of the sample schools by this time. A fresh sample of students in grades 4, 5 was chosen. Each grade was given the same test as in the baseline (for example, grade 5 was given the same test that was given to grade 5 in baseline). 3. Analysis For outcomes from interviews of school committee members, the unit of analysis is either the individual member or the school as appropriate (and explained). The unit of analysis for teacher attendance and activity outcomes is the individual teacher. The unit of analysis for learning outcomes is the grade. In MP and UP, at second follow-up, grades 3 and 4 sample is a fresh sample of students while grade 5 sample had participated in baseline. For this reason, results on grade 5 are presented separately. 13 The focus group was not conducted in Karnataka due to reasons of limited resources. 14 SC and ST groups are officially recognized as disadvantaged populations due to extreme social, economic and educational background arising from the traditional practice of untouchability (SC) and geographic isolation and little social contact with the community at large (ST). 24 To measure the impact of the campaign, for each outcome we conduct a linear ―difference in differences‖ regression analysis comparing the change in intervention to the change in control group from baseline to second follow up after adjustment of standard errors for clustering.15 In MP and UP, for learning outcomes, we present two changes i.e., between baseline and first follow up, and between first and second follow ups. This is because the baseline and second follow up samples are not strictly comparable in terms of grades. In Karnataka, we present change in learning outcomes between baseline and second follow up since these samples are comparable in terms of grades. Focus groups and qualitative interviews of committee members are analyzed by percentage of respondents to each question. Responses to open-ended questions are presented as main themes mentioned. 15 The comparable changes between baseline and first follow-up are reported in a separate paper (Pandey, Goyal and Sundararaman, 2009). 25 Chapter 4 Impact of the Information Campaign in the Study States 1. Baseline Characteristics Treatment and control villages are similar in socio-economic characteristics of the sample (Appendix A: Table 1). Baseline survey indicates no significant difference between treatment and control groups with respect to outcomes (Appendix A: Tables 2-3). The following are some of the key findings of the baseline survey.16 � School committees do not participate much in exercising oversight and are poorly informed of their roles, especially in UP and MP. Headmasters are the most informed of the committees' roles and responsibilities. Parent members are the least informed and the least active in terms of participation when compared to the chairs or the headmasters. When asked to list the roles and responsibilities of the school committee, 52% of parent members in UP and 58% in MP could not list a single one. � A large proportion of school committee members, especially parent members, reported receiving no training from the state on their roles and responsibilities. Only 8% of parent members in MP and 2% in UP had received any training. � The survey highlights low teacher effort and poor student learning (Appendix A: Table 2). With respect to teacher effort: a) the average rate of teacher attendance was 65% in UP and 67% in MP; b) the average rate of teacher activity (i.e., active engagement in teaching-related activities) was 27% in UP and 31% in MP. No more than a third of the teachers were actively teaching during survey visits. � In terms of student learning: a) barely 12%-15% of sample students in MP and UP could read sentences and words, and an even lower 5%-7% could write sentences and words; b) in mathematics, only 12%-19% were able to do two-digit additions with and without carryover, 7%-11% could do subtraction and less, 4%-6% could do multiplication and less, and 4-5% were able to do division and less (Appendix A: Table 2). � There is substantial variation in outcomes across states, between Karnataka on one hand and MP and UP on the other. Karnataka had much higher levels of community awareness as well as better teacher and student outcomes than the other two states (Appendix A: Table 3). Average teacher attendance was 88 % and average rate of teaching activity was 68 %. Nearly 47 % of students could read simple sentences and words (compared to 12-15 % in the other two states). 2. Impact of the Information Campaign 16 Baseline outcomes are reported in detail in a separate paper (Pandey et al., 2010). 26 The analysis is based on a ―difference in differences‖ linear regression where the change in outcome from baseline to second follow-up is the dependent variable. For individual i in GP j in block k in district h, the regression equation is as follows: ΔΥijkh = a + b Χjkh + bh Χjkh * dh +dh +�ijkh ΔΥijkh is the change in outcome, Χjkh is a treatment dummy variable taking value 1 if village j in block k in district h belongs to the treatment group and 0 if it belongs to the control group. dh are district dummy variables. �ijkh is a random error term and a is a constant term in the regression equation. The treatment effect is allowed to vary across districts. The estimate of the average treatment effect is the sum of the coefficient estimates of b and ∑h bh*dh where dh takes the mean value in the sample. Because the baseline outcomes are at such different levels in Karnataka compared to UP and MP, the expected impacts are likely to be different. Therefore to maintain clarity in presentation and ease of reading, results from MP and UP are put in one subsection and from Karnataka in a following subsection. 2.1 State-Wise Impact (UP and MP) We report impact in three broad outcome categories: school committees, teacher effort, and learning achievements. 2.1.1 I) Impact on school committees Uttar Pradesh: The unit of analysis is the individual committee member. In UP, the intervention is associated with a 25 % (24 percent points) increase in the number of committee meetings and a 25 % (9 percent points) increase in member participation in school visits between baseline and second follow-up. There is an increase in committee members‘ awareness about school accounts and committee roles and responsibilities vis-à-vis the teachers (Table 4a and Fig 2 below). The impact is larger on upper caste (Non-SC) and women members (Table 4a). There is an increase in awareness though not in participation of low caste (SC) members, and no impact on male members. The impact is greater on the chairs than on parent members. This suggests that committee members other than the chair and the head teacher—and, in particular, socially disadvantaged low castes—are not part of the increase in participation. 27 Figure 2: Percent Change in Participation and Awareness of School Committee Members (UP) *indicates significance at 5 percent level or below, +indicates significance at 10 percent level. As a result of routine member turnover, 60 % of original committee members remained while the remaining 40 % were new members between the baseline and second follow- up. To check for the consistency of above results, we used data on all members to analyze responses at the GP level, using mean response by member category (i.e., whether a member is head teacher, chair, or parent). The results show an increase in the number of meetings, and an improvement in member participation in school visits and meetings (Table 3a, third column). There is also an increase in members‘ awareness regarding school accounts and roles and responsibilities vis-à-vis teachers. Box 3: Qualitative Findings on the Process of Change (UP and MP) Qualitative interviews of committee members in a few treatment GPs demonstrate that a large share of committee members reported discussing campaign information with others in the village (87%), and that the discussions went on for weeks. More than one-third reported raising teacher attendance and learning issues with the teacher or the chairperson of the school committee, and 75% reported using the learning assessment tool to assess learning in schools. A majority of the committee members perceived a post-intervention ‗improvement‘ in committee meetings. When asked what improved about the meetings, main responses given were ―now meetings are held more frequently‖, ―learning is discussed in meetings‖, ―more members attend meetings‖ and ―all are informed of upcoming meetings‖. When probed further on specific actions taken by committee members, many more reported that they personally attempted to find out learning levels and verify teacher presence in schools than talked to the teacher directly on these issues. This may be indicative of the social distance between teachers and the community in general. Chapter 5 details the findings from these interviews. 28 Madhya Pradesh: In MP, analysis was not possible at the individual member level because new PTA election rules came into effect during the course of the first round of the intervention. Since then new school committees have been re-elected annually. Therefore, the difference in differences analysis was done using mean response at the GP level by member category. There is an 11 % (21 percent points) increase in the number of school visits by the committee; 15 % (9 percent points) increase in member participation in school visits; and an increase in awareness regarding school accounts (Table 4b and Fig. 3 below). Figure 3: Percent Change in Participation and Awareness of School Committee Members (MP) *indicates significance at 5 percent level or below, +indicates significance at 10 percent level. As evident from Table 4b, in most cases the impact is larger in PTAs with high proportion of upper castes (i.e., non-SC/ST) and male members. This indicates exclusion of low caste and women members from increased participation. Like in UP, the impact is greater on the chairs of school committees, who report an increase in the number of meetings and school visits, increased participation in visits and greater awareness about school accounts and PTA‘s responsibilities vis-à-vis the teachers. 29 Box 4: Do Communities Perceive School Committee Members as Key Players? In both states, results on the campaign‘s impact on school committees (Section 2.1.1) suggest that the increase in participation was greater for the chairs or the head teachers compared to other members. In focus group discussions with parents, we find that a large proportion reported going to the school or the chair, rather than to other committee members, to discuss school related issues. Of the parents who reported bringing up school issues such as learning or teacher presence, a large percent (70%) approached the school, and the rest (30%) went to the chair. Almost none contacted other committee members. This indicates that the community bypassed committee members other than the chair or the teacher. The other members are possibly not seen as voices or stakeholders in the school (see Chapter 5 for details of focus II) Teacher Effort group findings). Uttar Pradesh: In UP, the intervention is associated with an increase of 12 % (8 percent points) in teacher attendance between baseline and second follow-up (Appendix A, Table 5a; Fig.4). The impact on attendance at first follow-up was 11 percent, similar to the second follow-up. When we separate the sample by tenure --into contract teachers and civil service teachers who have permanent jobs--the impact is primarily on civil service teachers. Their attendance goes up by 23 % (14 percent points) between baseline and second follow-up. When we analyze the results by civil service teachers‘ characteristics, the impact is significant and greater on upper caste, male, more educated and less experienced teachers (Fig. 4). Figure 4: Percentage Change in Teacher Effort (UP) *indicates significance at 5 percent level or below, +indicates significance at 10 percent level. Madhya Pradesh: In MP, the intervention is associated with increases of 9 % (6 percent points) in teacher attendance and 27 % (8 percent points) in teacher activity (Appendix A, Table 5a; Fig. 5). At first follow-up, there was no impact on attendance while the impact on activity was similar to that in the second follow-up, at about 30 %. 30 Like in UP, the impact is primarily on the civil service teachers. Their attendance goes up by 16 % (11 percent points) and activity by 45 % (13 percent points). When we analyze the results by characteristics of civil service teachers, the impact on teacher activity is significant and larger on upper caste, male, more experienced and more educated teachers (Fig. 5). Figure 5: Percent Change in Teacher Effort (MP) *indicates significance at 5 percent level or below, +indicates significance at 10 percent level. Overall, for both states the baseline survey results showed certain types of teachers displayed lower effort than others: civil service teachers had lower attendance and activity rates than contract teachers; in UP more educated and male teachers, and in MP, more educated and experienced teachers had lower attendance and/or activity rates. A recent study from six countries (including India) underlines this trend. Teachers with more powerful social status (male, more educated and experienced) had higher absenteeism (Chaudhury et al, 2006). Teachers belonging to higher socioeconomic strata commonly enjoy protection from local elites, who share common caste and class backgrounds; absenteeism is often facilitated by the existence of this nexus (Dreze and Saran, 1995). Our results, therefore, indicate greater impact of the campaign precisely where the accountability problem is more serious. How can we explain these results? Greater participation of school committee members in meetings and inspections, and their increased awareness of committee roles and responsibilities vis-à-vis teachers is likely to have enhanced the scrutiny of teachers. Furthermore, focus groups indicate the larger parent community monitored the schools (Box 5). This extra scrutiny is likely to have affected those civil service teachers more whose motivation and effort levels were lower at baseline. Contract teachers, on the other hand, had higher effort rates (16-22 % higher attendance and 37-64 % higher activity rates) and much lower salary levels (between 31 1/4th and 1/5th of civil service teachers).17 It may not have been worthwhile for contract teachers to increase effort levels further, as not putting in any more effort would not invite social censure because in the community‘s perception, they may have been reasonably ‗maxed‘ out. Box 5: Communities in Intervention Villages Monitored Teachers Focus group discussions with parents reveal that a significant percent raised learning and teacher related issues, and when asked who they raised these issues with, a large percent stated going to the school or the teacher. This direct scrutiny by the community may have exerted pressure on teachers to increase effort. See chapter 5 for details. III) Learning Achievement The impacts on learning are seen mainly in mathematics competencies, in grades 3 and 4 in UP and in grade 5 in MP (Appendix A, Table 6; Figs. 6 and 7). Figure 6: Percent Change in Percentage of Children Able to do a Competency (UP) *indicates significance at 5 percent level or below. Uttar Pradesh: In addition to the impact on mathematics, there is an increase of 37 % in the share of children who are able to write words and sentences in grades 3 and 4 in UP. There are no other impacts in language competencies. Note the percent impacts are large because of the low levels of absolute learning outcomes at baseline (see section 1). Additionally, the impact between baseline and first follow-up is marginal (Appendix 1, Table 6). 17 Attendance and activity rate numbers are based on the baseline survey. Salary estimates are based on Goyal and Pandey (2011). 32 Madhya Pradesh: As is evident from Figure 7, even though MP shows an impact on learning at second follow-up, it is limited to mathematics and to grade 5. Figure 7: Percent Change in Percentage of Children Able to do a Competency (MP) *indicates significance at 5 percent level or below. For both states even though we observe an impact on learning at second follow-up, it occurs mainly in mathematics, does not occur in all grades or much in language. This may be due to several reasons. First, the greater impact on math compared to language may be because acquiring basic math skills can be easier if it is not significantly dependent on the language skills of the student. The math test had questions that were based on numbers and mathematical operation symbols, and not on words. While we do not have results from other studies to confirm our argument, we posit that national or state level mathematics tests which often include written problems may create a barrier for children to negotiate them. Second, lack of consistent impact on learning across grades and competencies can be due to poor teaching skills. Teacher effectiveness is a combination of teacher skills and time spent by them teaching. The campaign had a positive impact mainly on teacher time through enhanced community monitoring. If teachers are under- skilled, extra time spent in the classroom may not translate into large increases in learning outcomes. Third, another reason for low impact on learning may be insufficient time between the start of the intervention and follow-up surveys. Learning is a cumulative process and students need exposure to the intervention for a longer period to enjoy greater gains. Research elsewhere suggests that in general only after about 5-8 years of implementation of school based management type of intervention, changes are seen in the more difficult to modify indicators such as learning achievement (Bryk et al, 1998). 2.1.2 Does Impact Differ by Village Characteristics? In order to examine whether village characteristics are associated with impact, we redid the above analyses by the proportion of socially disadvantaged population groups (SC and ST) and the 33 proportion of literate population in the GP (Appendix A, Tables 7a-7b). Since we did not randomize the selection of treatment and control villages by these characteristics, the results can only be interpreted as correlations between impact and characteristics. In both MP and UP, there are greater numbers of significant impacts in villages with low fractions of SC/ST populations (i.e., with high fractions of non-SC/ST or upper caste populations), compared to villages with high fractions of SC/ST populations. This suggests that capture is more likely when the disadvantaged population is larger and the advantaged population is smaller. SC/ST parents are a weaker voice and likely to face the wrath of teachers and school committees according to focus groups (Box 6). Significant impacts are also more in number in villages with low literacy rates, particularly in MP. This distinction, although present, is less so in UP. An explanation for this result can be that villages with low literacy rates have a greater demand for schooling. Box 6: Caste and Exclusion from the Accountability Process Focus groups suggest that SC/ST parents were more hesitant (compared to non-SC/ST parents) to raise a school issue that would challenge the teacher. When they did bring up a concern, they were indeed more likely to face a hostile reaction from the teacher or the chair. FGD findings reveal that a smaller percent of SC/ST parents reported raising concerns related to teacher attendance or learning proficiencies (as a follow up after using the learning assessment tool on their child) with the teacher or the committee. When probed about the reaction of the person with whom the issue was raised, a substantially higher percent of SC/ST participants stated being subject to the person‘s anger. (See chapter 5 for more details). 2.1.3 Focus Groups As many as 73 % of focus group participants had discussed the disseminated information with others in the village, and 59 % said the discussion went on for weeks. Learning and teaching issues were mentioned most frequently as topics of discussion: all the groups mentioned learning and teaching as main themes discussed, with 76 % listing these before other topics. Additional topics of discussion included student entitlements from the school, student attendance, and school infrastructure. 34 Box 7: Post-Intervention Topics of Discussion When focus groups were asked to list the three main themes discussed with others in the community, learning was mentioned foremost and most frequently, followed by teacher issues. Thirty three percent of participants in the 40 focus groups stated raising teacher attendance as an issue with the school, the school committee, or an education official at the block level. Nearly 40 % of participants also mentioned raising learning as a concern. When raising teacher or learning issues, most participants reported having raised these with teachers or chairs (of committees), and almost none approached other VEC/PTA members or education officials. Communities seem to have bypassed other committee members who were possibly perceived as peripheral or without influence (see also Box 4). More than 70 % of participants stated going with other parents, rather than alone, to discuss the issue. This suggests individual parents are apprehensive about bringing up an issue alone because teachers or chairs are perceived as somewhat unapproachable. In fact, when inquired about the reaction of the service provider or the chair whom the issue was brought up with, a large proportion of focus group participants reported experiencing an angry reaction from the person and difficulty in discussing the issue. When asked why bigger changes did not occur in learning or teacher effort, prominent themes were that ―teacher is dominating‖, ―it is difficult to talk to the teacher‖, ―teacher does not listen‖ and teacher does not care‖. Among SC/ST groups, ―people are afraid to talk to the teacher‖ was also mentioned. When asked how the information campaign could be made more effective, the dominant response (mentioned by 73 % of the groups) was ―have more frequent meetings‖. Other suggestions were to focus the information campaign towards illiterate parents and to call all parents to a common meeting. 2.2 Impact in Karnataka 35 Since the two treatments differ in one dimension and are otherwise identical, we report the average impact of the two treatments (pooled) compared to the control group, and the additional impact of the second treatment compared to the first. 2.2.1 I) Impact on School Committees As 75 % of SDMC members were newly elected between the baseline and second follow- up surveys, difference-in-difference analysis was done using mean response at the school level by member type (i.e., whether a member is a head teacher, chair, or parent). Results show no impact on member participation; this can be due to new member elections that took place soon after the second round of intervention, and did not leave enough time for participation to change. There is a significant increase, however, in awareness regarding school accounts and the roles and responsibilities of SDMC, such as preparing schemes to improve school quality and undertaking school visits (Appendix A: Table 8). II) Teacher Effort There is no impact on teacher attendance and activity. This can be expected because teacher effort was high at baseline; nearly 90 % of teachers were present, and 80 % of those present were teaching. However, at both the first and second follow-ups we observed that teachers with low attendance in the previous survey round were more likely to have been transferred from their schools in treatment villages compared to control villages. This was possibly a result of complaints filed by school committees to the education office. Attendance at first follow-up is inversely and significantly related to whether a teacher was found transferred out of the school at second follow-up in treatment villages. The relationship between attendance at baseline and transfer at first follow-up is also inverse and significant. (Appendix B: Table 9b). This relationship between attendance and subsequent transfer is insignificant in control villages. When we analyze by teacher characteristics, the inverse relationship in treatment villages between attendance and transfer is significant for upper caste, male, less educated, and less experienced teachers (Appendix B: Table 9c). At baseline, we observed significantly lower effort on part of teachers who were male, less educated and less experienced (Pandey et. al., 2008). This implies that teachers slacking to a greater degree at baseline were more likely to have been shifted out of schools in treatment villages, suggesting that information impacted the mechanism of community control over teachers. III) Learning Achievement The impacts on learning between baseline and second follow-up occur mainly in mathematics competencies (Appendix B: Table 10; Fig.8). There are no impacts on language competencies. 36 Figure 8: Percent Change in Percent of Children able to do a Competency (Karnataka) *indicates significance at 5 percent level or below. 2.2.2 Does Impact Differ by Village Characteristics? We redid the above analyses by the proportion of socially disadvantaged population groups (SC and ST) and the proportion of literate population in the GP (Table 11). There are greater numbers of significant impacts in villages with high fractions of SC/ST populations and high literacy rates, the opposite of what we found in the other two states. One reason for this reversal in results can be that Karnataka has a longer history of affirmative action in favor of disadvantaged caste groups who may, therefore, have a greater political voice. The state also has a longer history of decentralization (much before the 73rd amendment came into being); high literacy rates are more likely to translate into efficient and active use of information when local governance structures are functional. 37 Chapter 5 Qualitative Findings on the Process of Change 1 Focus Group Discussions A year after the first round of intervention, we conducted focus group discussions in randomly selected treatment GPs —10 each in MP and UP. Two focus group meetings were conducted in each GP, one among residents from SC/ST social classes, and the other among residents belonging to non-SC/ST social classes. Each meeting averaged 12 participants, who were asked if they remembered the campaign, whether they had discussed the information with anyone else in the village afterwards, if they had raised school-related issues with teachers or VEC/PTA, why bigger changes were not seen and what could be done to improve the campaign. 1.1 Did the Community Discuss the Information that was provided? Among focus group participants, 73% had discussed the disseminated information with others in the village, and 59% said the discussion went on for several weeks (Box 8). The most frequently-mentioned topics of discussion were learning and teaching issues: all of the groups mentioned these among the main themes discussed, and 76 % listed them ahead of other topics (Table 2 and Box 7). Also discussed were entitlements, school infrastructure, and student attendance. Box 8: Time Period over Which Discussions Continued 73 percent of the focus group participants discussed the information received with others in the neighborhood or the village. 38 Table 2: What were the Three Main Issues Discussed? Percentage of focus groups that mention 1st issue 2nd issue 3rd issue Learning 59 24 0 Teacher attendance or teaching 17 32 15 Scholarship, mid-day meal, uniform, school 15 32 26 maintenance Student attendance 2 0 8 No issue mentioned 0 12 47 1.2 Were any Teaching or Learning Concerns Raised with Those Responsible for Service Delivery? On average, 35% of focus group participants reported bringing up teacher attendance as a concern with either the school or the committee (VEC/PTA) or an education official at the block level. In non-SC/ST groups, 42% had raised teacher attendance issues, compared to 25% in SC/ST groups (Box 9). Almost 40% of focus group participants raised learning as an issue; the percentages for non-SC/ST and SC/ST participants are not very different. This suggests that SC/ST parents may be more hesitant raising issues of teacher effort that directly challenge the teachers (who are often influential) than raising learning issues. Box 9: Percentage of Participants who Raised School Issues with Those Responsible for Service Delivery Participants were asked: After the campaign, did you ever raise teacher attendance (learning levels) as an issue with either the teacher or the chair or the VEC/PTA or a block education official? 39 Of the participants who raised school issues, more than 70% talked to the teachers, and 30% or less talked to the chairs (Box 10). Almost none talked to other members of school committees (1 % or less), or block education officials (3 % or less). In other words, communities approached committee chairs and teachers directly, bypassing other committee members who may have been perceived as marginal and without voice. Box 10: Who Were the Issues Raised With? Participants were asked (following a positive response to the question in Box 9): If you did raise an issue, with whom did you raise it? More than 70% of those who raised an issue went with other parents, rather than alone, to talk to teachers, chairs or other committee members (Box 11) which suggests that individual parents were apprehensive about raising an issue alone. This may be because teachers and Pradhans or chairs are seen as powerful and difficult to approach. Box 11: How Were Those Responsible for Delivery Approached by Parents? Participants were asked (following a positive response to the question in Box 9): If you did raise an issue, did you go alone or with others to talk to this person? 40 In fact, when asked about the reaction of the person with whom the issue had been raised, 40% encountered an angry reaction when the issue was teacher attendance and 35% reported the same when the issue was learning (Box 12). Almost a third or more of the participants reported difficulty in discussing these concerns with the person they had raised them with. Box 12: Experience of Raising an Issue with Those Responsible for Service Delivery Participants were asked (following a positive response to the question in Box 9): Did the person you raised an issue with get angry and was it difficult to talk to her/him? 1.3 Was the Learning Assessment Booklet used in Assessing Learning Levels? 1.3 Was the Learning Assessment Booklet Used by the Community to Assess Learning Levels? More than 60% of focus group participants reported using the learning assessment booklet to assess learning levels of their children. Of the ones who did, one third of non- SC/ST participants (and 20% of SC/ST participants) discussed it as an issue (Box 13). The majority discussed it with the teacher and a much smaller number did so with the chair or the Pradhan (Box 14). 41 Box 13: Use of Learning Assessment Tool by the Community Participants were asked: Did you ever use the learning assessment booklet on your child to assess her learning levels? If so, did you ever discuss it as an issue with either the teacher/chair or pradhan/VEC/PTA/block education official? Box 14: Follow up after Using the Assessment Tool Participants were asked (following a positive response to the second question in Box 13): If you did discuss after assessing your child, whom did you discuss with? More than half of the participants felt the person with whom they discussed the issue got angry and was difficult to talk to. A much higher percent (67%) of SC/ST participants compared to non-SC/ST participants (46%) reported experiencing anger from the teacher or the chair with whom they discussed the issue (Box 15). 42 Box 15: Experience of Discussion Participants were asked (following a positive response to the second question in Box 13): If you did discuss, what did you experience? Did that person get angry and was it difficult for you to talk to him/her? 1.4 Obstacles in the Way of Change When asked why bigger changes did not occur in learning or teacher effort, prominent themes mentioned by focus group participants were that ―teacher is dominating,‖ ―it is difficult to talk to the teacher,‖ ―teacher does not listen,‖ and ―teacher does not care.‖ Among SC/ST groups, it was also mentioned that ―people are afraid to talk to the teacher.‖ When asked how the information campaign can be made more effective, the dominant response (73%) was to have more frequent meetings to disseminate information. Other suggestions included focusing the information campaign on illiterate parents, and to call all parents to the same meeting. 2 Interviews of School Committee Members Alongside conducting focus group discussions, we conducted one-on-one interviews with individual school committee members in the same GPs. The interviews included questions that were put to the focus groups and some additional questions on participation of members in school oversight. 32 VEC members in UP and 50 PTA members in MP participated. All of the members remembered the campaign. 2.1 Did Committee Members Discuss the Information provided? Eighty seven percent of the committee members who participated in the interviews had discussed the disseminated information with others in the village and 72% said the discussion went on for weeks (Box 16). Learning and teaching issues were mentioned 43 most frequently as topics of discussion (Box 17): all of the members mentioned learning and teaching among the discussion main themes, and 83% of committee members listed them ahead of other topics. Other topics of discussion were entitlements, school infrastructure and student attendance. Box 16: How Long did the Discussions Continue? 87 percent of school committee members discussed the information received with others in the neighborhood or village Box 17: Main Themes Discussed When asked to list the three main themes discussed, learning was mentioned foremost and most frequently followed by teacher issues. 2.2 Did Committee Members Raise Teaching or Learning Issues with Service Providers? On average, 27% of the committee members interviewed had raised teacher attendance issues, and 40% had raised learning issues with either the school or the chair (Pradhan) 44 (Box 18). Most committee members who raised these issues talked to teachers, 25% or less talked to the chairs (Pradhans), and none talked to education officials (Box 19). More than 70% went alone, rather than with others, to talk to teachers or Pradhans (Box 19). This is unlike the findings from the focus groups discussion with parents— committee members are likely to have more influence compared to individual parents, and therefore may be less fearful of authority (teachers, chairs, Pradhans). Box 18: Percentage of Members who Raised School Issues with Those Responsible for Service Delivery Members were asked: After the campaign, did you ever raise teacher attendance (learning) as an issue with either the teacher/chair or pradhan/other VEC or PTA members/block education official? Box 19: Who Were the Issues Raised With? Members were asked (following a positive response to the question in Box 18): If you did raise an issue, with whom did you raise it? 45 Box 20: How Were Those Responsible for Delivery Approached by Members? Members were asked (following a positive response to the question in Box 18): If you did raise an issue, did you go alone or with others to talk to this person? When asked about the reaction of the person with whom the issue was raised, most committee members report neither experiencing neither an angry reaction nor any difficulty in discussing the issue (Box 21). This again contrasts with the findings from the discussions with parent focus groups and is possibly due to greater influence of committee members in comparison to individual parents. Box 21 : Experience of Raising an Issue with Those Responsible for Service Delivery Members were asked (following a positive response to the question in Box 18): If you did raise an issue, did that person get angry and was it difficult for you to talk to him/her? 46 2.3 Was the Learning Assessment Booklet Used by School Committee Members to Assess Learning Levels? More than 75% of the school committee members reported using the learning assessment booklet to assess learning levels of their children (Box 22). Of the ones who assessed, 43% discussed it with the service provider or school committee (Box 22) and of the ones who did, most talked with the school and a much smaller number with the chair (Pradhan) (Box 23). Box 22: Use of Learning Assessment Tool by Members Members were asked: Did you ever use the learning assessment booklet on your child to assess her learning levels? If you did, did you ever discuss it as an issue with the teacher/chair or pradhan/other VEC or PTA members/block education official? Box 23:Follow up after Using the Assessment Tool Members were asked (following a positive response to the second question in Box 22): If you did raise the assessment issue, whom did you raise it with? 47 Again unlike the focus groups, most members neither found it difficult to discuss the assessment issue with the person they went to, nor did they experience any anger from him/her (Box 24). Box 24:Experience of Discussion Members were asked (following a positive response to the second question in Box 22): If you did raise the assessment issue, what did you experience? Did the person get angry, and was it difficult for you to talk to him/her? 2.4 Did Committee Members’ Feel Participation Changed after the Campaign? A large percentage of members felt an improvement in school meetings came about due to the campaign (Box 25). When asked what improved about the meetings, main responses given were: ―meetings are held more frequently,‖ ―learning is discussed in meetings,‖ ―more members attend meetings,‖ and ―all are informed of when a meeting will take place.‖ Box 25: Perception of Improvement in Committee Meetings Members were asked: Did the campaign make a difference to committee meetings? 48 Members were asked specific questions about participation (Box 26). More than 70% of members attempted to find out learning levels in schools, 36% or more tried to verify teacher presence in schools or actual school timings, while a much smaller percent talked directly to the teacher about their presence or teaching. Box 26: Actions Taken by Members Members were asked following questions about specific actions they took post-campaign to monitor the school. 3. Summary A large percent of focus group participants had discussed the disseminated information with others in the village, and the discussions had lasted at least a few weeks. Learning and teacher issues were the most frequently discussed topics. Communities brought up issues of learning and teacher absence with service providers and school committees. The learning assessment tool was widely used to assess grade-appropriate learning levels of children, and any learning concerns were thereafter discussed with the teachers. Interviews of school committee members revealed similar findings. The information was widely discussed, learning and teacher issues were discussed with the teachers and attempts were made to assess learning levels via the tool. Members perceived an improvement in committee meetings, which were now more inclusive and held more frequently after the campaign. They reported taking specific actions to monitor the schools such as checking school timings and teacher presence, and assessing grade- appropriate learning levels. These findings indicate three important highlights. First, the information provided through the campaign was valued given the degree of discussion and monitoring that it engendered. Second, learning was an important concern to communities given the 49 involvement of the parents in using the assessment tool and conferring with service providers. Third, the campaign spurred community action. Many parents actively brought up school-related issues with teachers and VEC/PTA as a community as well as individually. Obstacles to implementing change were also experienced by parents. When asked why larger changes did not occur in learning or teacher effort, main themes put forward included teacher dominance and indifference, and fear of the teacher. 50 Chapter 6 Discussion This study shows that information matters in improving the oversight capability of communities over the local school, leading to stronger school incentives. Providing information in an effective manner to communities leads to improved learning outcomes and a substantial reduction in wastage of public resources by reducing teacher absenteeism and increasing teacher effort in classrooms. 100 million of the 134 million primary school-going children in India attend a public school (NUEPA, 2010) in a system that has struggled to strengthen teacher and school accountability and produce learning outcomes. The costs of a weak system have been low levels of learning for generations of students. So far the decentralized governance model to oversee school performance, with communities having oversight powers over schools and teachers through school committees, has not been effective in extracting accountability and making schools deliver acceptable learning outcomes. One of the key reasons contributing to ineffective functioning of school-based management committees has been lack of knowledge among communities—committee members themselves are unaware of how committees are formed, who the members are, and what controls they have over the school (Banerjee et al., 2008; Pandey et al., 2007). Village level school committees provide a coordination mechanism for collective action in the rural Indian context with strong caste and socio-economic divides. Information gaps in the public services domain undermine the potential for such collective action. This study used a randomized controlled trial methodology to examine the effects of providing information to communities on school committees and services that parents are entitled to from schools. Key outcomes were the functioning of school committee, teacher effort, and student learning. Information was provided through a structured campaign in a uniform manner, conducted in two rounds over a period of two and a half years. During each round, efforts were made to reach the entire community through many public meetings in the GP, repeated every few weeks. The information was provided through a film and leave-behind items like posters, wall paintings and calendars which were distributed to households. To enable parents to assess whether their children were acquiring government prescribed minimum levels of learning, booklets identifying the same by grade were distributed to households.18 The information campaign led to improvements in the functioning of school committees, teacher effort, and learning. The impacts were larger in UP and MP and much smaller in Karnataka perhaps because the states had different starting points. Teacher effort and all other outcomes at baseline were much higher in Karnataka than UP and MP, indicating greater efficiency in delivery. Findings in UP and MP: 18 Use of the booklet was demonstrated in the meetings. 51 School committees in both states became more active as a result of the campaign. The number of committee meetings and participation of members in meetings and school inspections increased significantly. More members reported knowledge of school accounts, and of committee roles and responsibilities vis-a-vis teachers. Increase in participation, however, varied by caste, gender, and position in the school committee. Impact on chair members of the committees was larger as compared to parent members. Upper caste members reported increase in participation while lower caste members appeared excluded from the gains in both states. With respect to teacher effort, in MP there was positive impact on teachers‘ attendance and engagement in teaching and in UP, the impact was on teachers‘ attendance. In both states, the impact of the campaign was greatest for civil service permanent teachers. In MP, both attendance and activity rates improved for civil service permanent teachers. Among civil service teachers, impact was greater for upper caste, male and more educated teachers in both states. Higher absenteeism and low teacher effort for these categories of teachers are robust findings in the empirical literature for developing countries and are attributed to their more powerful social status (Chaudhury et al., 2006; Pandey et al., 2008). In other words, results indicate greater impact of the campaign precisely where the accountability problem is more serious. Learning outcome gains from the campaign were less uniform compared to the gain in teacher effort. A consistent result was an increase in the share of children acquiring mathematics competencies. There was insignificant impact on language; noticeable impact occurred only in writing competency in one state. These results may be explained as below. First, we conjecture that improving math skills may be easier in the short-term, more so if they are not significantly dependent on the language skills of the student. The math test had questions that were based on numbers and mathematical operation symbols, and not on words. While we do not have results from other studies to confirm our argument, we posit that national or state level mathematics tests which often include written problems may create a barrier for children with poor language skills to negotiate them. Second, lack of a more consistent impact on learning can be due to inadequate teaching skills and the fact that learning is a cumulative process where students need exposure to the intervention for a longer period; assessing impact may require more time. Differences in where the impact occurs can be partly due to a divergence in the structure, roles and responsibilities of school committees. PTAs in MP are school-level committees, relatively large in size and elected for a year; VECs in UP are smaller, village-level bodies elected for a term of 5 years. Group size and the length of interaction between members are likely to matter for participation. Coordination is difficult in larger teams where the share of influence of each team member is lower, thus reducing motivation for individual effort. Where the committee has authority for a shorter period, members may not find it worth their while to put their weight behind issues that will find resolution beyond their term of tenure. These could explain the greater number of changes in outcomes measuring participation and awareness of school committees in UP relative to MP. There are also important differences in the extent of oversight vis-à-vis teachers. For a teacher to receive her salary, school committees are obligated to verify teacher 52 presence in MP, while in UP committees control the tenure of contract teachers only. This can explain why the impact on teacher effort was wider in MP i.e., it was on both attendance and activity, while in UP it was on attendance alone. In both states, impacts on committee outcomes, teacher effort, and learning differed by GP characteristics, the share of socially and economically disadvantaged populations (SC and ST groups) and literacy rate. Impacts were higher in GPs with a high share of non- SC/ST groups (i.e., low SC/ST presence). Public goods may get delivered easily if the non-SC/STs, especially the more powerful castes, form a large share.19 When the disadvantaged population has a large share, capture by powerful groups is likely. Impact was also higher in GPs with low literacy rates which can be explained by greater demand for education in such areas. Findings in Karnataka: In Karnataka there was no impact on the functioning of school committees in terms of member participation. This could be a result of committee elections being too close to the follow-up survey, not leaving enough time for participation rates to change. Nevertheless, there was an increase in the awareness of committee members. There was no impact on teacher effort either. Nearly 90 % of teachers were present and 80 % of those present were teaching at baseline, leaving little scope for impact. Moreover, unlike the other two states, school committees (SDMCs) do not have direct control over teacher effort– they neither track teacher attendance/classroom engagement nor control teacher tenure. They have indirect control over teachers i.e., committees can report non-performing teachers to the district education office (committees in the other two states can also do the same). The information campaign seems to have had an impact on this indirect channel. We find that in treatment villages (while not in control villages), teachers with lower effort at baseline were more likely to have been transferred out of schools by the time of follow-up.20 The impact on learning occurred in mathematics competencies, as in the other two states. It is noteworthy that while teacher effort and school committee participation as measured by the survey did not change, there was an improvement in learning. This implies that classroom teaching is likely to have been impacted by the campaign. The impact of the campaign varies by literacy rate and share of SC/ST population in the GP but not in the same direction as in the other two states. Impacts are larger in GPs with high literacy rates and high share of SC/STs. This finding is in direct contrast with UP and MP, where impacts were greater in GPs with low literacy rates and low SC/ST populations. One explanation for this reversal in results is that high literacy GPs may make more efficient use of information than low literacy GPs, particularly when local governance mechanisms are functional. Karnataka has a longer history of decentralization than other states (going back to 1959-60); in fact the state was 19 Banerjee and Somanathan (2007) find some evidence of this. 20 This relationship was not significant in the other two states. 53 considered a model for the formulation of the 73rd Constitutional amendment. The baseline survey indeed confirms that there is substantially greater community involvement in schools in the state. A second explanation is that the low caste groups are less marginalized in Karnataka. The state has a longer history of affirmative action in favor of low castes.21 Yet another explanation can be that in UP and MP, where public schools perform poorly, a higher share of parents in high literacy GPs have exited from these schools, choosing to enroll their children in private schools.22 This would lower the stake of the community in public schools, particularly of the elite groups who tend to exit first. Overall, our findings clearly suggest that (a) decentralization to communities is meaningless unless made real i.e., communities at least need to know what oversight roles they have; and (b) providing information to communities that are unaware, through a structured campaign, can play a useful role in changing behavior (community participation and teacher effort) and learning outcomes, particularly in lagging states. In a state like Karnataka, where teachers are less absent and communities actively participate in schools, the campaign can be helpful in turning the focus on learning. Focus group discussions provide some indication of the mechanisms of change and the barriers that remained. Change seems to have occurred through discussions within communities, and communities bringing up issues with teachers and school committees. Two critical suggestions made by the focus groups are indicative: organize a large meeting where the entire community is present and secondly, provide information regularly. Information that becomes ‘common knowledge‘ and less exclusive and privileged is more likely to empower citizens; the presence of information in the public domain can help citizens exert pressure to demand services. Government schemes that depend on community monitoring generally do not have effective mechanisms for transmission of necessary information for communities to exercise their role as client. Moreover, careful thought also needs to be given to the fidelity of these mechanisms when communities are largely poor and illiterate. Prominent themes mentioned by focus groups as obstacles in the way of change included ―teacher is dominating‖, ―teacher does not listen‖ and ―people are afraid to talk to the teacher‖. Teachers tend to enjoy protection from local elites and therefore it is important that any information provided is not appropriated. Some of the previous work on this topic did not necessarily find a positive impact of information dissemination (Banerjee et al, 2008). The nature of our intervention, in particular the structure, intensity and uniformity of the campaign, may explain the differences in results observed here. All information campaigns are not the same just 21 A study of south Indian states, that included Karnataka, finds greater participation of SC/STs in village meetings compared to non-SC/STs, and more so in villages with high literacy rates (Besley, Pande, and Rao, 2007). 22 This is supported by estimates provided by Muralidharan and Kremer (2007) on the presence of private schools in rural areas of Indian states, i.e. percentage of villages with at least one private school. Private school prevalence is much lower in Karnataka villages (12%) compared to MP (23%) and UP (57%). 54 because they are disseminating information. They differ based on the type and detail of information given, how it is communicated (i.e., the medium used) and whether those delivering it are perceived to be credible and reliable sources.23 Changing behavior to change school outcomes requires time. Barriers to collective action are apparent from focus group discussions and from the findings that villages with a higher fraction of disadvantaged populations saw smaller changes. Research over a longer time period is needed to examine whether behavioral changes translate into greater changes in learning and whether a campaign sustained over a longer time generates greater impact. A caveat in interpreting our findings is that the impact of an intervention in the realm of school-based management on learning outcomes is mediated by curriculum, pedagogical practices and professional competency of teachers which are outside the remit of information provided or skills that the community in general can bring to bear in their roles as monitors. Even if teachers were present and teaching 100 percent of the time, learning outcomes would improve only by so much if teaching skills are inadequate. Currently knowledge of teacher-student engagement inside classrooms and what works and what does not here is still at a rudimentary level. Research is needed to evaluate innovations in strengthening the core activities of the school —teaching and learning. Providing information requires resources but is still a very cost-effective way to improve the efficiency of public resources. The cost of running the campaign was approximately $149 per year for each gram panchayat (GP). About 70% of India‘s education budget is spent on teacher salaries; high rates of teacher absenteeism and low teacher effort in classrooms result in huge wastage of public resources. With the 6th Pay Commission24, primary teachers are expected to get a 50%-70% hike in salary, potentially resulting in even greater waste with given rates of teacher absenteeism and effort. The table below presents rough calculations of how much the increased attendance is worth when valued at teacher costs per day, using a conservative estimate of the monthly salary of a regular public school teacher (Rs 15,000). As is clear from the numbers in the last column of the table, the value of reclaimed public resources is substantial and much more than the cost of the campaign, even without taking into account benefits from better learning outcomes for children. 23 For instance, in the focus groups from an earlier study, also in UP (Pandey et al., 2007), participants reported being afraid of the village head and not trusting him. 24 The Pay Commission is an administrative system of the government of India to determine the salaries of government employees. 55 Table 3: Approximate Cost-Benefit Calculations of the Information Campaign State (number Number of Baseline Increase in Value of Cost of the of days in a primary civil attendance attendance increased campaign per school year) service rate rate due to the attendance per year26 teachers campaign year 2007-0825 UP (200 school 3,28,212 61% 23% (14 Rs. 8.27 billion Rs. 0.36 billion days in a year) percentage (US$ 176 (US$ 7.7 points) million27) million) MP (220 school 3,04,268 64% 17% (11 Rs. 6.02 billion Rs. 0.16 billion days in a year) percentage (US$ 128 (US$ 3.4 points) million) million) To provide quality education, a key component of the SSA framework was increasing accountability of schools to the community through greater involvement of village education committees and parent-teacher associations. For the first time since the committees were created, resources were allocated through SSA for regular training of school committees. However as the data show effective training does not reach a majority of members. Based on the findings reported here, to strengthen local governance of schools the states can revisit their training programs for communities to incorporate key features of the campaign in design and implementation:  Provide information repeatedly: We provided information several times in every GP. Focus group discussions reveal that communities see one-time information as insufficient.  Provide complete information that is left behind with the community: We provided complete details of information, roles and responsibilities of school committees and services that parents are entitled to from the school, in tools (take-home calendar, poster, and booklet) that were left behind after the meetings.  Provide information not only to school committee members, but also to the larger community: We provided information not only to school committees, but also to the entire village. Since change in school outcomes seems to have occurred through discussions among parents and parents bringing up issues with teachers and school committees, it is a useful lesson for policy that information is provided to the entire community. This will prevent information from being captured by the powerful.  Communication strategies should take into account literacy levels of the community and orient information accordingly: Our study used tools (film and posters with pictures)—that were not entirely dependent on written words—for parents to know the rules.  Strengthen school committees’ and the larger communities’ capacities to monitor learning outcomes: All three states list ―ensuring children are learning at grade- appropriate levels‖ among the responsibilities of school committees; however 25 Source: DISE 2007-08, NUEPA 2009. 26 Cost for the state is calculated using the total number of GPs (52002 in UP, 22931 in MP). 27 1 US$ = Rs. 47 as of March 2010 56 committees or communities often have no idea of grade-appropriate learning levels. Communities can be taught easy ways to assess children‘s learning. This comes out clearly in our study where we used simple booklets based on the government‘s own benchmarks for minimum learning levels. A large percent of parents and committee members in the focus groups reported using the booklets distributed, either on their own or through someone else if they were illiterate, to assess learning levels. The booklet also allowed them to approach the teachers in a less personal way (it was easier to discuss children‘s learning levels with teachers than their late arrival to school or absence). 57 REFERENCES 1. Annual Status of Education Report (2005). India: Pratham. http://images2.asercentre.org/ASER_REPORTS/Aser_2005_Report.pdf 2. Banerjee, A., Banerji, R., Duflo, E., Glennerster, R., and Khemani, S. (2008). Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India. Policy Research Working Paper Series 4584. Washington, D.C.: World Bank. 3. Banerjee, A., and Somanathan, R. (2007). The Political Economy of Public Goods: Some Evidence from India. Journal of Development Economics; 82 (2). 4. Besley, T., Pande, R., and Rao, V. (2007). Political Economy of Panchayats in South India. Mumbai: Economic and Political Weekly; February 24. 5. Bryk, A.S., Thurn, Y.M., Eaton, J.Q., and Luppescu S. (1998) Academic Productivity of Chicago Public Elementary Schools. Chicago: Consortium on Chicago School Research. http://ccsr.uchicago.edu/content/publications.php?pub_id=48 6. Caldwell, B. (2005). School-Based Management. Paris: IIEP-UNESCO. 7. Chaudhary N., Hammer J., Kremer M., Muralidharan K., & Rogers H. (2006). Missing in Action: Teacher and Health Worker Absence in Developing Countries. Journal of Economic Perspectives; 20 (1): 91-116. 8. Comptroller and Auditor General of India (2001). Report of the Comptroller and Auditor General of India on the Union Government (CAG). 9. Constitution of India. 73rd amendment, 1992. http:// indiacode.nic.in/coiweb/amend/amend73.htm 10. Debroy, B., and Bhandari, L. (Eds.) (2003). District-level Deprivation in the New Millennium. Delhi, India: Konark Publishers. 58 11. Dreze, J., and Saran, M. (1995). Primary Education and Economic Development in China and India: Overview and Two Case Studies. In Basu, K., Pattanaik, P., & Suzumura, K. (Eds.). Choice, Welfare and Development. Oxford: Clarendon Press. 12. Duflo, E., Dupas P., & Kremer, M. (2007). Peer Effects, Pupil-Teacher Ratios, and Teacher Incentives: Evidence from a Randomization Evaluation in Kenya. MIT mimeo. Massachusetts: Massachusetts Institute of Technology. 13. Government of India (2001). Census of India 2001. http://www.censusindia.net 14. Goyal, S., and Pandey P. (2011). Contract Teachers in India. Education Economics. 15. Moehler, Devra C. (2010). Democracy, Governance, and Randomized Development Assistance. Annals of the American Academy of Political and Social Science. 16. Muralidharan, K., & Kremer, M. (2007). Public and Private Schools in Rural India. Mimeo. San Diego: University of San Diego. 17. Olken, B.A. (2007). Monitoring Corruption: Evidence from a Field Experiment in Indonesia. Journal of Political Economy; 115 (2). 18. Olson M. (1965). The Logic of Collective Action: Public Goods and the Theory of Groups. Harvard University Press. 19. Pandey, P., Sehgal, A., Riboud, M., Levine, D., & Goyal, M. (2007). Informing Resource-Poor Populations and the Delivery of Entitled Health and Social Services in Rural India. A Cluster Randomized Controlled Trial. Journal of American Medical Association; Oct 24/31; 298 (16): 1867-1875. 20. Pandey P., Goyal S., & Sundararaman, V. (2010). Public Participation, Teacher Accountability and School Outcomes in Three States. Economic and Political Weekly; June 12. 59 21. Pandey P., Goyal S., & Sundararaman, V. (2009). Community Participation in Public Schools: Impact of Information Campaigns in Three Indian states. Education Economics; 17(3). 22. Pandey P., Goyal S., & Sundararaman, V. (2008). Public Participation, Teacher Accountability and School Outcomes: Findings from Baseline Surveys in Three Indian States. Policy Research Working Paper No. 4777. Washington, D.C.: World Bank. 23. Transparency International. Global corruption report 2006: corruption and health. http://www.transparency.org/publications/gcr/download_gcr/download_gcr_2006. 24. World Bank (2004). World Development Report. Washington, D.C.: World Bank. 25. Reinnika R., and Svensson, J. (2004). Local Capture: Evidence from a Central Government Transfer Program in Uganda. Quarterly Journal of Economics;119 (2): 679-705. 60 Appendix A: Chapter 4 Tables Table 1: Mean Sample Characteristics at Baseline UP MP Karnataka Student Characteristics Treatment Control p++ Treatment Control p++ Treatments Control p++ 1&2 Age (years) 8.69 8.76 .36 8.89 9.01 .08 10 10 .77 % Male .49 .49 .47 .50 .52 .59 .53 .50 .06 General caste (neither SC/ST, .15 .15 .72 .22 .17 .14 .54 .56 .63 nor OBC) OBC .40 .41 .83 .31 .31 .91 - - SC .45 .44 .64 .15 .16 .60 .34 .30 .36 ST - - - .32 .36 .44 .12 .14 .52 % Mother literate .20 .21 .49 .12 .14 .32 .36 .37 .90 % Father literate .55 .58 .10 .45 .46 .74 .55 .57 .63 Land (in acres) 1.38 1.24 .51 2.53 2.49 .86 3.46 3.40 .85 School Characteristics Enrollment 170 189 .12 113 124 .27 141 128 .42 Pupil-teacher ratio 62 70 .15 57 55 .71 26 27 .70 % Schools with toilet .28 .39 .15 .37 .38 .83 .74 67 .55 % Schools with drinking water .84 .82 .68 .74 .70 .55 .65 67 .73 % Schools with playground .84 .88 .46 .58 .48 .14 .49 38 .24 % Schools with electricity .01 0 .35 .04 .09 .28 .62 55 .31 Number of blackboards 3.5 4 .11 3.19 3.45 .28 6 5.5 .38 Teacher Characteristics 38 38 .66 38.84 38.36 .67 39.14 38.68 .49 Age (years) % Male .61 .60 .72 .80 .81 .77 .58 .61 .40 % Contract teachers .39 .43 .20 .16 .14 .61 - - % With high school education .43 .41 .67 .48 .47 .84 .71 .74 .58 % With pre-service training .59 .58 .70 .35 .37 .77 .94 .90 .11 Experience (years) 10.94 10.86 .90 14.19 13.33 .41 12 12 .51 Distance traveled to school (in 6.68 5.88 .40 9.30 8.99 .84 9.92 9.84 .95 km.) ++ p-values are based on clustered standard errors. 61 Table 2: Baseline School Committee, Teacher and Learning Outcomes (UP and MP) UP MP Percent (unless stated otherwise) Treatment Control Δ+* Treatment Control Δ+* School Committee Number of meetings in past 1 yr .90 .99 -.09 2.40 2.34 0.06 % Members attended .39 .44 -.05 .64 .63 0.01 Number of school visits in past 1 yr 1.1 1.2 -.04 1.91 1.94 -0.03 % Participated in inspections .34 .37 -.02 .61 .64 -0.03 % With knowledge of school accounts .41 .45 -.04 .59 .61 -.02 % With knowledge of stipend account .37 .44 -.07 .30 .36 -.06 % With knowledge of mid-day meal .29 .34 -.05 .36 .39 account -.03 % With knowledge of teacher aspects of .26 .24 .02 .32 .33 -.01 roles and responsibilities of school committee; mentioned at least one: monitoring attendance/ensuring teacher teaches/complain if teacher does not perform/any other % With knowledge of other aspects of .40 .42 -.02 .26 .30 -.04 roles and responsibilities of school committee; mentioned at least one: inspecting schools, managing civil works, preparing schemes for school improvement, managing school funds % With knowledge of entitlements .38 .35 .03 .35 .40 -.05 aspects of roles and responsibilities of school committee; mentioned at least one: ensuring textbooks, scholarships, quality mid-day meals, uniforms reach the students Teachers Teacher attendance .63 .68 -.05 .64 .69 -.05 Teacher activity .26 .27 -.02 .29 .32 -.03 Learning (percent who can) Read sentences and words .14 .12 0.02 .15 .13 0.02 Write sentences and words .05 .06 -0.01 .07 .05 0.02 Mathematics (addition) .19 .18 0.01 .15 .12 -0.07 Subtraction, and less .11 .10 0.01 .08 .07 -0.03 Multiply, and less .06 .06 0.00 .05 .04 0.00 Divide, and less .04 .05 -0.01 .05 .04 0.00 + Δ =Treatment-control; * indicates this difference is significant at 5 percent or below based on p values that are based on clustered standard errors. 62 Table 3: Baseline School Committee, Teacher and Learning Outcomes (Karnataka) Percent (unless stated otherwise) Treatments Control Δ+* 1&2 School Committee Number of committee meetings in past 1 yr 1.54 1.51 0.03 % Who attended .54 .51 0.03 Number of school inspections in past 1 yr 1.47 1.40 0.07 % Who participated in inspections .53 .47 0.05 % Who have knowledge of school accounts .68 .71 -.03 % Who have knowledge of stipend account .70 .68 .02 % Who have knowledge of mid-day meal .70 .68 .02 account % Who have knowledge of teacher aspects .55 .52 .03 of roles and responsibilities of school committee; mentioned at least one: monitoring attendance/ensuring teacher teaches/complain if teacher does not perform/ any other % Who have knowledge of other aspects of .80 .79 .01 roles and responsibilities of school committee; mentioned at least one: school visits, manage civil works, prepare schemes for school improvement, manage school funds % Who have knowledge of entitlements .67 .65 .02 aspects of roles and responsibilities of school committee; mentioned at least one: ensuring textbooks, scholarships, quality mid-day meals, uniforms reach the students Teachers Teacher attendance .88 .87 .01 Teacher activity .67 .69 -.02 Learning (percent who can) Read sentences and words .45 .47 -0.02 Write sentences and words .32 .34 -0.02 Mathematics .73 .77 Addition -0.04 Subtraction, and less .51 .55 -0.05 Multiply, and less .32 .34 -0.02 Divide, and less .26 .30 -0.04 + = Δ Treatment-control; * indicates this difference is significant at 5 percent or below based on p values that are based on clustered standard errors. 63 Table 4a: OLS Regression: Difference-in-Differences in School Committee Participation and Knowledge (UP) 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow up- up- up- up- up- up- up- up-baseline up- baseline baseline baseline baseline baseline baseline baseline baseline All All, GP Non-SC SC Men Women Chair Headmaster Parents level Number of Meetings .24* .14* .32** .15 .15 .46+ .31* .55** .14 Attended meeting .07 .04** .10+ .04 .03 .17+ .08* .16 .05 Number of Inspections .13 .15 .17 .18 .01 .43* .40 .15 .02 Attended inspection .09+ .08* .11* .09 .06 .18** .19* -.02 .07 Persons present at inspection .19 .25+ .47** -.21 .05 .61** .63* -.22 .10 Knowledge of school accounts .11** .12** .18** .01 .08 .15* .18** .21** .05 Knowledge of stipend account .08* .09* .05 .14** .01 .18* .14* .09 .02 Knowledge of mid-day meal account .06** .08** .01 .16* .03 .11* .11+ .16* .00 Knowledge of roles and responsibilities: Roles and responsibilities vis-à-vis .08* .11** .05 .12* .06 .14* .09 .14 .12** teachers; mentioned at least one: ensuring teacher attendance/ensuring teacher teaches/complain if teacher does not perform Other roles and responsibilities; .11* .03** .04+ .11* .06+ .03* .08 -.10 .05* mentioned at least one: inspect schools, manage civil works, prepare schemes for school improvement, manage school funds Roles and responsibilities vis-à-vis -.04 -.05 .08 .10+ -.06 .05* -.02 -.14 .02 entitlements; mentioned at least one: ensuring textbooks, scholarships, quality mid-day meals, uniforms reach the students **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 64 Table 4b: OLS Regression: Difference-in-Differences in School Committee Participation and Knowledge (MP) 2nd follow 2nd follow 2nd follow 2nd follow 2nd 2nd follow 2nd follow up- 2nd follow up- GP level→ up- up- up-baseline up- follow up-baseline baseline baseline baseline baseline baseline up- baseline All Non-SC SC Women Men Chair Head master Parents Number of Meetings .05 -.02 .21* .40* .11 .27+ .01 .08 Attended meeting .00 .06* -.06 -.15 .00 .02 -.03 .03 Number of Inspections .21* .57** .14 .01 .37** .52** .20 .25+ Attended inspection .09** .16* .11 .00 .15* .16* .13** .08+ Persons present at inspection .41 .59 .87 -.48 .98 .23 1.4 .04 Knowledge of school accounts -.01 .07 -.09 -.15 .01 -.06 .03+ -.03 + Knowledge of stipend account .07 .06 .12 .14 .11 .23 .01 .10* Knowledge of mid-day meal account .07* .17+ -.03 -.14 .08* .20** .00 .10** Knowledge of roles and responsibilities: Roles and responsibilities vis-à-vis teachers; .02 .09* .07 .20 .06* .22** -.06 .09* mentioned at least one: ensuring teacher attendance/ensuring teacher teaches/complain if teacher does not perform Other roles and responsibilities; mentioned .06 .04 .01 .10 .03 .02 .10 .04 at least one: inspect schools, manage civil works, prepare schemes for school improvement, manage school funds Roles and responsibilities vis-à-vis .05 .05 .03* .06+ .04 .06 .05 .04+ entitlements; mentioned at least one: ensuring textbooks, scholarships, quality mid-day meals, uniforms reach the students **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 65 Table 5a: OLS: Difference-in-Differences in Teacher Effort (UP and MP) UP MP 2nd follow up-baseline 2nd follow up-baseline * All teachers Teacher attendance .08 .06* All teachers Teacher activity .03 .08* Civil service teachers Teacher attendance .14** .11** Teacher activity .01 .13** Contract teachers Teacher attendance .01 -.14 Teacher activity .04 -.14 **Indicates sig at 1%, *indicates sig at 5%; based on p values that are based on clustered standard errors. Table 5b: OLS: Are Transfers Related to Past Attendance? (UP and MP) UP MP Coefficient Coefficient Coefficient on Coefficient on on teacher on teacher teacher teacher attendance at attendance at attendance at attendance at 1st follow-up baseline 1st follow-up baseline Dependent variable: 1 if teacher was transferred out of the school at 2nd follow-up, 0 otherwise Control sample -.13 - -.12 - Treatment sample -.12 - -.24* - Dependent variable: 1 if teacher was transferred out of the school at 1st follow-up, 0 otherwise Control sample - -.18 -.07 Treatment sample - -.09 -.04 * Indicates sig at 5 % based on p values that are based on clustered standard errors. Table 6: OLS: Difference-in-Differences in Learning (UP and MP) UP MP Change in Treatment- Change in Control Change in Treatment- Change in Control 2nd -1st follow up 1st follow up-baseline 2nd -1st follow up 1st follow up-baseline Percent children Grade 5 Grade 3/4 Grade 5 Grade Grade 5 Grade Grade 5 Grade 3/4 3/4 3/4 Divide, and less .01 .03 * .00 .00 .02** -.01 -.01 .00 * Multiply, and less .01 .03 .00 -.01 .05** -.01 -.01 -.01 Subtraction, and less .00 .03* .02 .00 .08* .00 -.01 .00 Addition -.01 .05** .04* -.02 .11* -.01 .00 .01 Writing sentence, and less -.01 .01 .00 .03* .01 .00 .02 .03 Reading sentence, and words .01 .04* .01 .00 .01 -.02 .01 .01 **Indicates sig at 1%, *indicates sig at 5%; based on p values that are based on clustered standard errors. 66 Table 7a: OLS: Difference-in-Differences by GP characteristics (UP) Low Proportion SC/ST High Proportion Low Literacy High Literacy (below median) SC/ST (below median) (above median) (above median) Teacher attendance .12** .03 .09 .09+ Teacher activity .09* -.04 .06* .03 Grades 3 and 4: .05** .02* .05** .01 Divide, and less Multiply, and less .03* .03* .05* .01 * ** Subtraction, and less .03 .03 .05 .00 Addition .04** .05* .08** .001 ** * Writing .05 .01 .05 .01 ** Reading .01 .00 .05 -.03 Grade 5: .02 .00 .03 .01 Divide, and less Multiply, and less .01 .002 .01 .02 Subtraction, and less .02 .00 .01 .00 Addition .02 .00 -.01 .00 Writing .02 -.01 .04 -.02 Reading -.02 .00 .02 -.02 School Committee Meetings .46** .18 .43 .28+ Attended meeting .13* .04 .13 .05 Inspections .21 .36 .23 .29+ Attended inspection .12* .03 .14* .13** + * Persons present at inspection .27 -.19 .31 .35 **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 67 Table 7b: OLS Regression: Difference-in-Differences by GP Characteristics (MP) Low Proportion High Proportion Low Literacy High Literacy SC/ST SC/ST (below median) (above median) (below median) (above median) Teacher attendance .07* .08 .09* .05 Teacher activity .08* .07 .10** .03 Grades 3 and 4: .00 -.02 .00 -.01 Divide, and less Multiply, and less .00 -.01 .00 -.01 Subtraction, and less .01 .00 .02 -.03 Addition .02 .01 -.01 -.01 Writing -.04 .00 .00 .03 Reading .03 -.03 .00 .00 Grade 5: .01 .05* .02+ .03* Divide, and less Multiply, and less .03 .07* .05* .05* * Subtraction, and less .05 .09 .07 .08** * Addition .13 .08 .09 .13* Writing .02 .01 .02 .01 Reading .01 .04 .03 -.01 School Committee Meetings .15** .00 .04 .04 Attended meeting .01 .00 -.03 .04 Inspections .36** .10 .27* .11 Attended inspection .10* .10* .13** .03 Persons present at inspection .32 .69+ .57 .31 **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 68 Table 8: OLS Regression: Difference-in-Differences in School Committee Participation and Knowledge (Karnataka) 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow 2nd follow GP level→ up-baseline up-baseline up-baseline up-baseline up-baseline up-baseline up-baseline All Non-SC SC Women Men Chair Parents Number of Meetings -.04 -.04 -.04 -.12 .02 -1.5 .04 Attended meeting -.06 -.04 -.13 -.06 -.10 -.04 -.04 Number of inspections .01 .00 .04 .00 .18+ -.01 .04 Attended inspection -.01 -.04 .01 .01 -.04 -.02 -.02 Persons present at inspection -.38 -.99* -.21 -.72 -.41 -.38 -.33 Knowledge of school accounts .13** .15** .11* .13+ .14* .20** .06+ Knowledge of stipend account .04 .06 .03 .06 .02 .12+ -.02 Knowledge of mid-day meal account .00 .02 -.02 .02 -.01 .02 .01 Knowledge of roles and responsibilities: Roles and responsibilities vis-à-vis teachers; mentioned at .04 .10* -.02 .05 -.02 .05 .03 least one: ensuring teacher attendance/ensuring teacher teaches/complain if teacher does not perform Other roles and responsibilities; mentioned at least one: .16* .21** .12 .22** .09 .25* .10** inspect schools, manage civil works, prepare schemes for school improvement, manage school funds Roles and responsibilities vis-à-vis entitlements; -.03 .02 -.09 .09 -.07 -.07 .00 mentioned at least one: ensuring textbooks, scholarships, quality mid-day meals, uniforms reach the students **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 69 Table 9a: OLS Regression: Difference-in-Differences in Teacher Effort (Karnataka) Change in Treatment- Change in Control 2nd follow-up-baseline All teachers Teacher attendance -.01 All teachers Teacher activity -.02 Table 9b: OLS regression: Are transfers related to past attendance? (Karnataka) Coefficient on Teacher Coefficient on Teacher Attendance at 1st follow- Attendance at baseline up Dependent variable: 1 if teacher was transferred out of the school at 2nd follow-up, 0 otherwise Control sample -.30 - Treatments 1 and 2 sample -.18* - Additional effect of treatment 2 no Dependent variable: 1 if teacher was transferred out of the school at 1st follow-up, 0 otherwise Control sample - -.04 Treatments 1 and 2 sample - -.30* Additional effect of treatment 2 no * Indicates sig at 5 % based on p values that are based on clustered standard errors. Table 9c: OLS Regression: Are Transfers related to Past Attendance? (Karnataka) Coefficient on teacher attendance at 1st follow-up Dependent variable: 1 if Male Female Upper Non- College Less than Experience Experience teacher was transferred Caste Upper Degree or College above below out of the school at 2nd Caste Higher Education Median Median follow-up, 0 otherwise Treatment sample -.38* -.06 -.21* -.10 -.10 -.21* -.05 -.30* * Indicates sig at 5 % based on p values that are based on clustered standard errors. Table 10: OLS Regression: Difference-in-Differences in Learning (Karnataka) Change in Treatment- Change in control 2nd follow up-baseline Percent children Divide, and less .07* Multiply, and less .08* Subtraction, and less .08* Addition .04 Writing sentences, and less .00 Reading sentences and words -.03 * Indicates sig at 5 % based on p values that are based on clustered standard errors. 70 Table 11: OLS Regression: Difference-in-Differences by GP Characteristics (Karnataka) Low Proportion SC/ST High Proportion Low Literacy High Literacy (below median) SC/ST (below median) (above median) (above median) Teacher attendance -.02 .01 .00 -.02 Teacher activity -.05 .03* -.01 -.02 Grades 4 and 5: .27 .24* .08 .44* Math Divide, and less .07 .08** .05 .11** Multiply, and less .11* .04+ .04 .12** Subtraction, and less .09+ .06* .02 .14* Addition .01 .05 .01 .07 Writing -.02 .01 .02 -.03 Reading -.02 -.03 -.03 -.02 School Committee Meetings .00 -.09 .14 -.21 Attended meeting -.05 -.12 -.06 -.10 * Inspections .02 -.01 -.36 .30** Attended inspection -.06 .01 -.11 .06 Persons present at inspection -1.00 -.35 -.80 -.40 **Indicates sig at 1%, *indicates sig at 5%, +indicates sig at 10%; based on p values that are based on clustered standard errors. 71