Participatory Budgeting in Kenya: Piloting New Techniques for Project Monitoring Principal Investigators: Michael Touchton, Ph. D., University of Miami, Brian Wampler, Ph.D., Boise State University For the Kenya Participatory Budgeting Initiative (KBPI) Kenya Accountable Devolution Program, World Bank. Acknowledgements The lead authors for this report were, Michael Touchton, Professor of Political Science, University of Miami and Brian Wampler, Professor of Political Science, Boise State University. The quantitative study was conducted by the Innovations for Poverty Action, contracted by the Kenya Accountable Devolution Program. The research benefited from the guidance and contributions of Annette Omolo (Social Development Specialist and Task Team Leader), Rose Wanjiru (Participatory Budgeting Consultant, World Bank), Tiago Peixoto (Senior Governance Specialist, World Bank) and Stephen Davenport (Senior Governance Specialist). The report benefited from the insight of peer reviewers, Michael Anthony Roscitt (Public Sector Specialist, EA1G1) and (Markus Goldstein (Lead Economist, AFRCE). The team acknowledges the support of the county governments of Baringo, Elgeyo Marakwet and Makueni in facilitating the conduct of the study. The team acknowledges the strategic guidance and support of Camille Lampert Nuamah (Operations Manager, Kenya) and Robin Mearns (Practice Manager, Social Development Africa, SAFS 2). The Kenya Participatory Budgeting Initiative (KBPI) is implemented under the World Bank’s Kenya Accountable Devolution Program led by Helene Carlsson Rex. The team acknowledges the critical financial support provided by KADP donors, namely Denmark, DFID, European Union (EU), Finland, Sweden and USAID. 1 Executive Summary The implementation of Participatory Budgeting (PB) programs in three Kenyan counties is helping to build the foundations for accountability at the subnational level. Kenya’s experience with devolution and the ensuing fiscal autonomy is relatively new (since 2013) and PB, as a policymaking tool, is even newer (since 2015). Nevertheless, we identify several ways that the program is making positive contributions. We draw from a large survey of PB participants, a control group of non-PB participants, participant observation, and over 40 interviews with key stakeholders. Our evidence points to several encouraging developments regarding citizens’ attitudes and their behaviors. We find that PB participants generally believe that their participation is worthwhile; they believe that PB is working well and that it is creating opportunities for citizens’ engagement in policymaking. However, we also note that some of the results of our pilot evaluation are less encouraging and suggest that progress toward accountability through PB is not uniform. To be sure, Kenya’s PB programs are new and we would not expect them to show impacts in all areas or to function the same way as others around the world that have been developed and reformed for almost 30 years. Finally, we conclude this policy note by recommending specific policies and programmatic features that county governments could use to improve how these programs function. It is our hope that these recommendations will help to consolidate the initial advances we identify in our evaluation as well as to move beyond the programs’ most significant limitations. PB Programs in Kenya Participatory Budgeting is a policymaking institution in which citizens are directly involved in deciding how local governments spend their resources. Citizens have the opportunity to attend a series of meetings in which they first deliberate and then vote on which policy projects or social programs the local governments will implement. In Kenya, PB is adopted at the county level as one way to comply with the constitutional requirement for public participation in budget processes. County governments implement the process at various administrative levels, with some counties starting at the village, cluster, sub-ward, and others at ward levels, where citizens attend meetings to propose, discuss, and select development projects from up to half of the counties’ development budgets. We focus on Makueni, Baringo, and Elgeyo-Marakwet Counties, where the World Bank provides technical assistance to county governments. Methods The core data for this policy report comes from a survey of PB participants as well as a control group of non-participants. In the survey, we assess respondents’ attitudes on important PB-related themes, such as social inclusion, the relative power of their voice, and a general assessment of the program. In addition, we also asked a series of questions about citizens’ activities within and beyond PB. The resulting data thus emphasizes both attitudes and behaviors to inform a 2 comprehensive evaluation of PB in Kenya. The survey was administered by Innovations for Poverty Action, an international firm dedicated to impact evaluations. Our survey first asks basic demographic questions surrounding respondents’ gender, age, education, and income. We also want to know about our survey population’s activities related to government processes and civil society. We ask questions on whether respondents have previously attended budget forums, whether they belong to civil society organizations, and whether they consider themselves leaders in these organizations. Finally, we ask questions to gauge respondents’ opinions on what development sector should receive more money among the top four the county funds. (Water, Health, Education, and Roads). 1 Next, we randomly assigned respondents to experimental treatment and control groups as part of a survey experiment. This is designed to counter challenges to inference from selection bias (see Duflo et al. 2007; Bannerjee et al. 2015; Duflo and Kremer 2005). The mechanics of a survey experiment are as follows: respondents assigned to a treatment groups for a particular question are read a prompt designed to prime them to consider a specific issue area or to reveal information about their county government’s activities. Then, we ask these respondents to express their opinions on the thematic issue area. For example, the extent to which they think PB gives people like them voice. We then compare these responses with those from a control group that responded to an identical question without hearing the treatment prompt or receiving the information. Because of random assignment, we view responses that differ between treatment and control groups as evidence of treatment effects, which in turn reveal underlying information about respondents’ opinions and program performance. We also administered surveys to non-participants in counties using PB. This is designed to address the possibility that participants’ very positive views of PB stem from high levels of support for the government, which would raise concerns for participants’ ability to hold public officials accountable through PB. Our results are generally consistent across basic questions and experimental treatments, which alleviates the gravest concerns in this area. Participants and non- participants express high levels of support for PB programs and respond very similarly to the experimental treatments described below, which suggests that PB participants are fairly representative of the broader population. In many cases, there are no statistically significant differences between treatment and control groups– either for PB participants or non-participants. Furthermore, the magnitude of the treatment effects are relatively small in most of the cases where they do emerge in the data. Nevertheless, we argue that these differences allow for a rank-order of respondents’ attitudes and behaviors across different issue-areas surrounding PB. In the first phase, we administered 1,000 village, village-cluster, and sub-ward surveys in Makueni County. In the second phase, we administered 1,000 ward-level surveys in two counties (Makueni and Baringo; N=2,000). Our initial analysis of the survey results suggested unusually high levels 1 Table I in the Annex presents the results of basic demographic questions below. 3 of support for the government, even as our analysis of the programs suggest that these governments were struggling to carry out basic components of the programs (e.g., project implementation). We then expanded our survey population to include non-participants to gain a better understanding of how PB participants and non-participants might hold different attitudes and engage in different behaviors. We therefore surveyed 500 additional respondents who did not participate in PB meetings in each of three counties (Makueni, Baringo, and Elgeyo-Marakwet) as well as 500 PB participants in Elgeyo-Marakwet. The samples of PB participants were doubly-random: we randomly selected ward PB meetings where surveys were administered to randomly-selected individuals. We randomly selected villages from among 3600 total in Makueni County. We selected from Makueni’s 30 wards, Baringo’s 29, and Elgeyo-Marakwet’s 20. The sample of non-participants was randomly-selected from individuals in randomly-selected households, marketplaces, and other public venues in the same wards where we collected earlier data. This suggests that the samples of PB participants are at least plausibly representative of all PB participants. We argue that the same can be said for non- participants. These samples are also very similar to one another, with the following exceptions: PB participants are systematically older, more likely to be male, and have attended more budget forums than non-participants. 2 We do not have an optimal control group in the sense that we did not measure respondents partisan affiliation, political activism, support for the local government, sense of civic duty, or other unobservables that might condition respondents’ selection into participation through PB. We also did not administer surveys to citizens in counties that do not use PB. Nevertheless, our samples are relatively balanced across important demographic and behavioral indicators, which offers the best control group possible under the circumstances. Overall, we completed more than 5,000 surveys across the three counties. The survey results capture different moments of the PB cycle (village vs. ward), along with the attitudes and behaviors of over 3,500 PB participants, and 1,500 non-participants. Beyond the survey, our research team carried out 40 structured interviews with government officials and civil-society leaders over a 15-month period. In addition, we engaged in participant- observation of 15 different PB meetings. These data provide extremely rich material that drive significant advances our understanding of how PB works in Kenya. Key Findings We identify positive, ambiguous, and potentially problematic outcomes based largely on the results of our surveys. This is common across similar programs around the world as well as within specific programs, where positive, ambiguous, and potentially negative outcomes coexist in the same institutional space. We also draw from qualitative data to better understand and contextualize the quantitative outputs to provide actionable recommendations to improve Kenya’s PB programs. 2 Tables I (a) and II (a) present difference of means tests for demographic and behavioral indicators for PB participants and non-participants. 4 Positive Signals The first treatment corresponds to citizens’ perceptions of voice in decision-making processes in the development budget. We reminded respondents that PB programs result in the selection of specific projects and then prompted them with statements with variations on whose support may be considered important to have projects selected: citizens, who are the central participants in PB, MCAs, who are formally involved at the end of the PB process, and chiefs, who have no formal role in PB. Respondents were evenly assigned to each treatment condition and a control group, which did not include a statement about support. Then we ask respondents the extent to which they agree with the following statements using a five- point Likert scale (strongly agree, agree, no opinion, disagree, strongly disagree). 1. PB is a program that allows citizens like me to influence project selection. 2. PB allows for a wide range of viewpoints to be considered in project selection 3. PB allows disadvantaged groups to influence project selection. We find that a large majority of PB participants hold very positive views of the PB process. PB participants also believe that they have voice within the programs. The very high levels of support for these programs is highly unusual in the global PB context, and raised concerns that PB participants were selected from a pool of strong supporters of the county government. To better understand if there were differences in the attitudes of participants and non-participants, we expanded our survey coverage to include 1,500 respondents who did not participate in PB. We find that many non-participants had heard of PB and had very positive opinions of the programs. We infer from these results that there is broad support among PB and non-PB participants for citizen-oriented, participatory programs like PB in Kenya. Second, we find that both PB participants and non-participants report being actively engaged with civil society organizations (CSOs). Although we do not attribute this civil society engagement to PB, we believe that these results suggest that there is fertile ground for incorporating citizens and CSOs into the PB process. Relatedly, the vast majority of PB participants and non-participants indicate that community groups play a positive role in the PB process. This suggests that community groups are beginning to carve out a role for themselves in the new participatory programs. Similarly, survey respondents consistently believe that the Members of the County Assembly also play a positive role in the PB process. Our next treatment addresses issues of collective action in support of PB projects. We gave half of respondents the following prompt about project success: 5 PB projects are more likely to be successful if community members provide labor/contribute money to help build the project. The other half of the respondents received a control prompt: PB projects are no more likely to be successful if community members provide labor/contribute money to help build the project. We then asked respondents about their willingness to contribute labor and financial support for the projects. In each case, we also requested respondents’ phone numbers (which we immediately encrypted and stored anonymously) as a way to separate disingenuous respondents from those truly willing to give labor or money. 3 Both PB and non-participants are willing to provide their labor and contribute to a community funding pool to support project implementation. PB participants appear to be committed to this process because it provides an opportunity for them to work with government officials to identify and then implement projects. In the survey, we first asked if respondents would be willing to volunteer, donate, or contribute money to projects. We then followed up with a question asking for participants’ phone numbers to test the credibility of their claims. Importantly, we found that a sizable majority of respondents volunteered their labor, their financial support, and provided their phone number– both among PB participants and non-participants. We interpret these results to indicate an encouraging commitment to project implementation through PB. We do not know if these phone numbers are accurate, but the number of digits respondents gave corresponds to plausible numbers for Kenyan phone customers in all but 11 cases. Fourth, PB respondents generally believe that the program addresses the needs of marginalized groups. This result suggests that PB programs are promoting core values associated with social inclusion. Finally, our research went beyond the surveys as we conducted roughly 40 interviews with key informants and collected data from participant-observation at 15 PB meetings. We found that PB participants were very vocal in many meetings, especially surrounding the implementation status of projects selected in previous years. In this context, PB participants pursued retrospective accountability, whereby they consistently pressured county government officials in the executive branch to explain why specific projects had not been implemented or completed. Additionally, we found that citizens were using PB forums in Baringo county to pressure county government officials to respond to a regional food crisis. These claims fall outside the bounds of our survey, but they do suggest some positive collateral benefits of citizen engagement as PB spills over into other critical areas. 3 Mobile phone penetration is quite high in our survey population; our survey enumerators asked respondents who refused to give phone numbers if they were willing to give other contact information, such as an address, or a relative’s phone number. Only 84 out of more than 5,000 respondents declined to give a phone number because neither they nor their relatives had a phone. 6 In sum, the most positive results from our pilot study suggest that PB expands citizens’ voice, focuses on bringing development projects to underserved areas, and helps to establish the foundations for accountability. We also see that PB participants are engaged in PB as citizens as well as members of community groups, which suggests greater potential for spill-over effects beyond PB. Interviews with community leaders demonstrate that they discuss PB programs and projects in their communities. Citizens believe that they have voice in the PB process, which is additionally supported by their willingness to donate time and personal resources to support project implementation. These results are all very encouraging and unusually positive in a global context, especially at such an early stage in PB’s development in Kenya. However, not all of our results are as positive and it would be very strange indeed if they were. We therefore turn now to the more ambiguous results from our study. Ambiguous Outcomes Our next treatment addresses questions of who has power in terms of which projects are implemented in the following year (among those that have already been selected). Respondents see the following statements, rotated evenly among participants, based on county government information on which projects have been completed and where they are located. Each year Participatory Budgeting programs select specific projects that the government is supposed to complete. Last year, in your ward, the county government built many/some/few of these Participatory Budgeting projects. A control group of respondents did not see any prompt covering how many projects were built in the last year. Respondents were evenly assigned to each treatment condition and control. Then we ask respondents the extent to which they agree with the following statements using the same five-point Likert scale as above. 1. The support of community groups (for example, self-help, CBO, NGOs) in civil society is necessary to complete projects through Participatory Budgeting. 2. The support of Members of County Assembly is necessary to complete projects through Participatory Budgeting. 3. The support of chiefs is necessary to complete projects through Participatory Budgeting. We find strong support for PB among participants and non-participants even when primed with negative information about project implementation: support for PB is just as high in wards where the county government reported low implementation rates (due to funding delays, understaffed county governments) as well as in places with much higher rates. This is worrisome from an accountability perspective because citizens are less likely to demand accountability if they maintain strong support for government activities and policies even when the governments struggle to fulfill their commitments. There are three plausible interpretations for this seeming disconnect between participants’ attitudes and project completion rates. One explanation is that PB participants are invested in building a new type of policymaking process. They remain supportive in the face of low project implementation rates because they are optimistic about how the new program may bring 7 development projects to the community. From this perspective, citizens’ support for PB suggests that they can differentiate between innovative democratic processes, which they support, and ongoing challenges of local capacity to implement development projects. Respondents may also understand that delayed disbursements from the national government limit the counties’ capacity; the prolonged election stalemate in 2017 further compounded the lack of county resources for project completion. The second plausible explanation is that high levels of support for county governments in general may explain consistent support for PB among participants and non-participants, even when presented with information on slow implementation of projects that citizens selected. A very large percentage of voters in each county voted for the current governors. County populations may therefore remain supportive in the face of lower implementation rates. Voters’ strong loyalty to their elected government may potentially limit accountability through PB because these strong supporters may not strongly contest the local governments’ policies. Finally, a third possible explanation is that our treatment and the subsequent questions are not picking up anything meaningful. This is a possibility for this and other differences between expressed preferences or observed experiences and support for PB. For example, support for PB also has no relationship with the distance between preferences for spending in one development sector (water, education) and government development spending in the previous year. It could be that citizens’ expressed preferences in surveys are simply not meaningful. Thus, the high level of support for PB among participants and non-participants, even in the face of negative information, likely stems from the hope associated with a new democratic, participatory institution and their loyalty to the elected county government. This creates an ambiguity around PB because we are uncertain if respondents’ strong support is due to how the program is functioning or if it is due to factors not directly related to PB performance. Administering surveys to non-participants (N is 1,500) in PB processes alleviates some of the gravest concerns discussed above. In these three counties, broad support for elected officials among non-PB participants implies that support for incumbent county governments is not limited to PB participants, but is relatively consistent across local populations. In other words, both PB participants and non-participants strongly support their elected governments, which may make it relatively more difficult to hold county government officials accountable; citizens have greater incentives to accept official government policies and priorities from governments they support. We do see participants holding governments accountable for delays with specific projects, but we do not know if these efforts will extend to larger projects or other county government activities. The role of the Members of the County Assembly (MCAs) is also ambiguous. MCAs have a formal role approving the overall budget allocation to PB, as well as approving the final slate of projects in county-wide budget-making processes. A large majority of respondents have a positive view of the MCAs’ participation in PB. In our qualitative work, we found that MCAs spoke at the beginnings of many PB meetings, thus giving them a de facto voice in the process. The MCAs’ involvement could suggest the establishment of horizontal accountability, whereby elected officials who are formally independent from the county government are willing to advocate for citizens in these spaces. 8 However, it is also plausible that PB participants do not distinguish between executive and legislative branches of the county government. Rather, the MCAs’ presence at some PB meetings offers an opportunity for ordinary citizens to connect with government officials and permits them to develop government connections. This suggests that MCAs may better link citizens to their governments, which could help citizens and county governments address pressing problems. This might help to strengthen horizontal accountability as the MCAs are empowered to work on behalf of their constituents. However, MCAs could also use these interactions to create new networks for political patronage. Additional research is needed to better understand how the MCAs’ involvement in PB affects the process. Ongoing Challenges for PB in Kenya Our results point to several key challenges for PB in Kenya. The survey evidence suggests that PB respondents may think differently about the informal authority that chiefs wield in their villages relative to citizens and MCAs. Survey respondents show less agreement with the idea that a wide range of viewpoints emerge through PB when they are primed to consider chiefs as opposed to MCAs and citizens. Similarly, Respondents decrease their support for the statement that chiefs’ support is necessary for project implementation when they believe that few projects have been implemented in their ward. Chiefs, of course, have no formal role in the PB programs and we do not expect them to be formally involved. Yet, survey results and qualitative interviews reflect chiefs’ participation in village affairs in ways that respondents believe impacts PB differently than citizens, community groups, or MCAs. This finding surrounding chiefs is more pronounced in the respondents we surveyed at the village level and among women respondents. The differences across priming conditions are not great, but that they still present a rank-order of different relevant actors for PB. Next, the organizational structure that integrated citizens into the process from the village, to the village-cluster, to the sub-ward, to the ward, and, finally, to the county level appears to have stretched the county government’s administrative units thin. Our qualitative research suggests wide variation in how these programs actually function. Simplifying the institutional processes might be a strategy to improve budget knowledge, preference formation, and accountability. Our final treatment corresponds to questions of social inclusion through participatory budgeting. We want to know how participants and non-participants think of PB as a program that meets challenges facing traditionally marginalized communities. We want the respondents to think carefully about which populations PB is or is not actively serving. We prompted respondents with the following three treatments, divided evenly among the population: The challenges facing youth/women/people with disabilities in this county means they deserve additional development funds through Participatory Budgeting. 9 The remaining respondents did not see any prompt. We then asked respondents to gauge the extent to which they agreed with the following statements on social inclusion through PB using the same five-point Likert scale as above. “Participatory Budgeting projects provide support for disadvantaged communities” “PB allows people from disadvantaged groups to influence the spending of development funds.” There is broad support for the statements that PB supports disadvantaged communities and that disadvantaged groups influence spending through PB. Priming respondents to think about youths or women does not change perceptions that PB provides support for disadvantaged communities or gives disadvantaged groups power in spending development funds to meet their needs. However, priming them to think about people with disabilities decreases their perceptions that PB benefits disadvantaged communities. This is just one significant result, but the implication is that respondents may be less inclined to believe that PB supports social justice goals when they are induced to think about the likely beneficiaries of PB projects. Again, the overall levels of results are high here: respondents do believe that PB supports disadvantaged communities and that disadvantaged groups influence spending through PB. However, the treatment also reveals a statistically significant rank-order, where people with disabilities may benefit less than other groups. There are no demographic differences in these response patterns. Once again, the control group and treatment group responses are already high: respondents broadly agree that PB serves disadvantaged communities and includes disadvantaged groups in decision-making. Yet, they agree less with this statement when given additional priming to consider people with disabilities. The implication here is that social inclusion must be discussed and included as part of the PB program design if it is to be achieved in practice. Finally, our qualitative interviews suggest that some community groups feel excluded from the process. Several community leaders felt that the county government deliberately excluded them from the meetings, by not providing timely information on the venues and meeting schedule, as well as cancelling meetings at short notice in some cases. County government officials have a mandate to hold and operate PB meetings, which is similar to other PB programs around the world. However, the degree to which these meetings are open to all participants and facilitate the inclusion of all community voices can vary. In this case, county officials tightly controlled ward- level PB meetings, which may limit some CSOs’ access as well as potentially slow the development of an independent civil society. Implications and Next steps We end this pilot report by identifying several key areas where county governments and the World Bank could invest their efforts to improve PB in Kenya. The following overarching areas would benefit from additional attention and resources: 10 • Information and Knowledge o Citizens continue to lack basic information about the budget, the budget process, and the county government’s policy preferences. This is a problem in most PB programs around the world, but it is particularly acute in Kenyan counties due to the relatively recent creation of county governments. We recommend: Additional budget workshop meetings targeted toward the most active participants. We see facilitating more informed participation as potentially occurring through several channels. A good start would be to post the budget on the well. In terms of the workshops, we found that PB programs in Brazil worked better when community leaders and PB delegates, who tend to be the most active participants, received extra education and training on the budget (Wampler and Touchton 2019). We believe this result applies to PB in many other middle- and low-income contexts, like Kenya.  Developing institutional processes to share information with participants about project implementation over the course of the budget cycle. • Organizational structure o The county governments’ efforts to hold meetings at multiple levels is challenging because their administrative capacity is yet underdeveloped. The governments could streamline PB programs by focusing participants’ efforts at key moments of decision-making. We recommend: Creating separate rural and semi-urban tracks. It might be beneficial to delegate resources to rural villages to permit them to implement projects. The semi-urban and urban areas could begin their efforts at the sub-ward levels. Decentralizing into rural and semi-urban tracks is already underway for Kenyan PB programs that the World Bank supports. We see decentralization as a possible pathway to using hyperlocal knowledge of community needs and claims made on public officials to complete projects in their villages.  Strengthening the knowledge and autonomy of the Community Resource Volunteers (CRVs), who now operate at the village level. These CRVs play a key role bridging the gap between village-level discussions and meetings held at the sub-ward and ward levels. It would be beneficial to strengthen the capacity and knowledge of the CRVs. • Project Implementation o County governments continue to face financial and administrative challenges that make it difficult for them to implement the projects selected by citizens.  Additional attention should be focused on building administrative capacity to efficiently translate citizens’ demands into specific projects. • Civil Society o Kenyan PB programs are run by the county government and have a bottom-up decision making process that engages citizens at the local level. However, it 11 would be beneficial for counties to expand the role of CBOs and CSOs and partner with them for more impactful results. We recommend: Partnering directly with a wider range of CBOs and CSOs to ensure that a broader range of citizens and community groups are included in the process. Most of our respondents report being members and leaders of CBO/CSOs. Yet, these may be organizations that lack information or autonomy, are not policy-oriented, or are uncritical of the government. Key informant interviews revealed much dissatisfaction among CBO/CSOs and their inability to participate in budget discussions.  Simultaneously, CSOs should be encouraged to open local offices and develop stronger connections with local CBOs and citizens.  Partnering with CSOs or NGOs to provide trainings and educational workshops. These could be joint government-CSO projects, for example. • Research We identify four areas that would allow the World Bank to develop a more comprehensive assessment of PB in Kenya o Project implementation  Rigorous assessment of projects selected since 2015. Ideally, county governments would provide each year’s list of selected projects as well as a list (and costing) of implemented projects. Kenyan-based research teams could then verify if these projects were actually implemented. Visits to a randomly selected subset of project sites would allow a qualified team to assess the quality of the project and its perceived impact. o Process-oriented analysis  For the next round of PB, it would very beneficial to engage in qualitative research to develop a more comprehensive understanding of how PB works. Interviews with a wider range of key informants, ready access to county government documents, and more extensive participant observation would comprise more comprehensive qualitative case studies in 10-15 wards. o Data collection from counties without PB.  It would be helpful to know how the attitudes and behaviors of respondents in counties with PB compare to those in counties without PB. There are theoretical reasons to believe that PB will have spillover effects and improve broader governance and accountability within the county. Examining this hypothesis by comparing our data with those from other counties would provide leverage on this question and potentially inform counties’ decisions to adopt PB. o RCT at village level. Randomly assign PB rules to assess what types of rules influence project selection, citizens’ attitudes and behaviors, and longer term impacts. An RCT surrounding PB program design and 12 operational rules will answer the most important policy questions, including about how to incentivize participation and inclusion of marginalized populations. We currently know extremely little about which PB program rules are associated with what PB outcomes. This is a critical omission because any organization promoting and facilitating PB adoption, like the World Bank, and any government considering PB does not have rigorous evidence to inform their decisions about what type of PB to implement and why. Dissemination and Adaptive Learning: We propose several dissemination channels for this report. First, the report should inform the next cycle of PB programs in Kenya. In particular, programs could add “information and knowledge” knowledge features to deliver education on the budget, budget processes, and governments’ policy preferences. Distributing this report and/or its findings directly to citizens would also demonstrate the impact of their participation in PB; citizens have shifted government behavior in some important ways that increased access to development projects and improved citizens’ lives. This will have the effect of closing the feedback loop from citizens to governments and back, as well as to increase momentum for future PB cycles. Finally, we advocate comparing these initial findings from Kenya with findings from countries around the world that have used PB for longer time-frames. Comparative lessons from these evaluations will then inform future rounds of PB design. 13 Annex: Participatory Budgeting in Kenya: Piloting New Techniques for Project Monitoring Table I Summary Statistics for Demographic Responses* Question Makueni Sub- Makueni Wards Baringo Wards Elgeyo-Marakwet Kenya** Wards Gender 43% Men 55% Men 68% Men 68% Men 50% Male Median Age 45 48 43 39 18 Education 48% completed 28% completed 53% completed 35% completed 74% completed primary school primary school or primary school or primary school or primary school or less. less. less. less or less. 27% completed 20% completed 19% completed 18% completed 55% completed secondary secondary school. secondary school. secondary school. secondary school. school Household 67% earn less 75% earn less than 79% earn less 69% earn less 51% earn less Income than $100 per $100 per month. than $100 per than $100 per than $100 per month. month. month. month. 33% earn less 50% earn less than 52% earn less 50% earn less 34% earn less than $50 per $50. than $50. than $50. than $50 month. Respondents 1001 1649 1692 1184 Not Applicable 14 Table I (a) Comparing PB Participants to Non-participants. T-tests reflect difference of means across each sample Question Makueni Sub- Makueni Wards Baringo Wards Elgeyo-Marakwet Wards Gender (PB 43% Men 55% Men 68% Men 78% Men Participants) Gender NA 55% Men 63% Men* 60% Men** (Non- participants) Mean Age 45 48 43 44 (PB Participants) Mean Age NA 35** 33** 34** (Non- participants) Education 48% completed 28% completed 53% completed 35% completed (PB primary school primary school or primary school or primary school or Participants) or less. less. less. less 27% completed 20% completed 19% completed 21% completed secondary secondary school. secondary school. secondary school. school. Education NA 27% completed 42% completed 40% completed (Non primary school or primary school or primary school or participants) less. less.** less* 21% completed 22% completed 18% completed secondary school. secondary school. secondary school. Household 67% earn less 75% earn less than 79% earn less 62% earn less Income (PB than $100 per $100 per month. than $100 per than $100 per Participants) month. month. month. 33% earn less 50% earn less than 52% earn less 50% earn less than $50 per $50. than $50. than $50. month. Household 67% earn less 75% earn less than 79% earn less 69% earn less Income than $100 per $100 per month. than $100 per than $100 per (Non- month. month. month.* participants) 15 33% earn less 50% earn less than 60% earn less 52% earn less than $50 per $50. than $50.* than $50. month. Respondents 1021 1035 1040 483 (PB Participants) Respondents NA 654 672 621 (Non- participants) * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 (Kenya data, presented in the final column, is from the Kenya National Statistics Bureau) Table II Summary Statistics for Behavioral and Opinion Responses for all Respondents. Question Makueni Sub- Makueni Baringo Elgeyo- Wards Wards Marakwet Previous Attendance 64% 75% 65% 95% at Budget Forums Belong to CSOs 75% 72% 27% 38% Self-identified as 57% 65% 67% 66% leaders, among those who belong to CSOs Agree with County 79% 82% 68% 74% Spending Decisions Which Sector 48% Water 65% Water 60% Water 30% Health, Deserves more 30% Education 20% Education 15% Education 27% Water Funding? 15% Health 10% Health 15% Health 23% Roads, 8% Roads 5% Roads 10% Roads 19% Education Preferred distribution 32% Education 34% Water 29% Water 30% Health of the development 26% Health 26% Education 27% Education 27% Education budget 26% Water 24% Health 26% Health 21% Water 16% Roads 16% Roads 19% Roads 20% Roads Respondents 1001 1649 1692 1184 16 Table II (a) Comparing PB Participants to Non-participants: Behavioral and Opinion Responses for Respondents. T-tests reflect difference of means across each sample Question Makueni Sub- Makueni Baringo Elgeyo- Wards Wards Marakwet Previous Attendance 64% 75% 65% 95% at Budget Forums (PB Participants) Previous Attendance NA 17%** 17%** _ at Budget Forums (Non-participants) Belong to CSOs (PB 75% 72% 27% 38% Participants) Belong to CSOs NA 58%* 20%* 20%* (Non-participants) Self-identified as 57% 65% 67% 66% leaders, among those who belong to CSOs (PB Participants) Self-identified as NA 25%** 33%** 38%** leaders, among those who belong to CSOs (Non-participants) Agree with County 79% 82% 68% 74% Spending Decisions (PB Participants) Agree with County NA 58%** 43%** 57%** Spending Decisions (Non-participants) Which Sector 48% Water 65% Water 60% Water 30% Health, Deserves more 30% Education 20% Education 15% Education 27% Water Funding? (PB 15% Health 10% Health 15% Health 23% Roads, Participants) 8% Roads 5% Roads 10% Roads 19% Education Which Sector NA 35% Education 33% Water 33% Health, Deserves more 29% Water 30% Health 27% Water Funding? (Non- 19% Health 19% Roads 22% Roads, Participants) 16% Roads 17% Education 21% Education ** ** ** Preferred distribution 32% Education 34% Water 29% Water 30% Health of the development 26% Health 26% Education 27% Education 27% Education budget (PB 26% Water 24% Health 26% Health 21% Water Participants) 16% Roads 16% Roads 19% Roads 20% Roads 17 Preferred distribution NA 35% Water 28% Water 32% Health of the development 25% Education 28% Education 26% Education budget (Non- 22% Health 26% Health 22% Water participants) 18% Roads 18% Roads 20% Roads Respondents (PB 998 1004 1007 480 Participants) Respondents (Non- NA 627 645 596 participants) * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 Treatment 1: Voice Table III: Voice Effects of Treatment 1 on Responses Surrounding Voice Lower scores on the Likert scale indicate stronger agreement with baseline statements. We use - Holm-Bonferroni corrections to address the prospect for cross-contamination across multiple treatments per respondent. Treatment Condition Average Citizens Wide Ranges of Disadvantaged Have Voice Viewpoints emerge Groups Have Voice through PB through PB through PB Mean Likert Scale Mean Likert Scale Mean Likert Scale Response Response Response (SE) (SE) (SE) Control 1.70 1.67 2.00 (0.05) (0.03) (0.06) Citizens 1.73 1.71 2.00 (0.07) (0.08) (0.06) Chiefs 1.80 1.77** 1.99 (0.05) (0.04) (0.03) MCAs 1.72 1.69 1.97 (0.04) (0.06) (0.09) N: 5497 5503 5488 18 * indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.05. ** indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.01. Table III (a): PB participants vs. Non-Participants Responses Surrounding Voice. T-tests compare PB participants to Non-participants. Lower scores on the Likert scale indicate stronger agreement with baseline statements. Treatment Condition Average Citizens Wide Ranges of Disadvantaged Have Voice Viewpoints emerge Groups Have Voice through PB through PB through PB Mean Likert Scale Mean Likert Scale Mean Likert Scale Response Response Response (SE) (SE) (SE) Control (PB 1.72 1.70 1.98 Participants) (0.05) (0.06) (0.08) Control (Non- 1.69 1.73 2.01 participants) (0.07) (0.09) (0.07) Citizens (PB 1.69 1.74 2.01 Participants) (0.07) (0.04) (0.06) Citizens (Non- 1.75 1.68 2.00 participants) (0.08) (0.08) (0.04) Chiefs (PB 1.81 1.79 2.00 Participants) (0.07) (0.05) (0.05) Chiefs (Non- 1.77 1.73 1.97 participants) (0.07) (0.08) (0.05) MCAs (PB 1.74 1.71 1.98 Participants) (0.06) (0.07) (0.09) MCAs (Non- 1.70 1.67 1.95 participants) (0.06) (0.08) (0.09) N: PB Participants 3648 3665 3666 19 N: Non-Participants 1849 1838 1822 * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 The results for treatment 1 show that PB participants and non-participants strongly agree with the statements that PB gives people like them voice, that a wide range of viewpoints emerge through PB, and that disadvantaged groups have voice through PB. Priming respondents to think about citizens or MCAs does not change perceptions that PB gives people like them voice, incorporates a wide range of viewpoints, including those from disadvantaged groups. Priming respondents to think about chiefs does not change perceptions that PB gives people like them voice, but it decreases perceptions that PB incorporates a wide range of viewpoints. It does not change opinions about viewpoints from disadvantaged groups. The lack of effect for MCAs is to be expected, at some level, because the process is designed for the MCAs to be involved. However, the constitutional reforms fostering participatory processes marginalize chiefs, to some extent, create unclear lines of authority, and risks a power struggle over local development projects. This finding surrounding chiefs is more pronounced in the respondents we surveyed at the village level and among women respondents. The differences across priming conditions are not great, but that they still present a rank-order of different relevant actors for PB. Chiefs may thus play a role in shaping public opinion, but respondents see them as less important than MCAs and community groups to the PB processes, which suggests that PB is working the way that it is intended. We believe that respondents’ institutional analysis is generally correct here, which indicates that rural villagers have some knowledge of local power distribution and responsibility. Treatment 2: Authority Table IV: Authority Effects of Treatment 2 on Responses Surrounding Authority in Project Implementation. Lower scores on the Likert scale indicate stronger agreement with baseline statements. We use Holm-Bonferroni corrections to address the prospect for cross-contamination across multiple treatments per respondent. Treatment Condition Community Chief’ Support is MCAs’ Support is Groups’ Support is Necessary for Necessary for Necessary for Implementation Implementation Implementation Mean Likert Scale Mean Likert Scale Mean Likert Scale Response Response Response (SE) (SE) (SE) 20 Control (no prompt) 2.00 2.40 1.83 (0.06) (0.05) (0.07) Most PB Projects 1.99 2.43 1.82 were Implemented (0.09) (0.07) (0.08) Some PB Projects 1.94 2.34 1.84 were Implemented (0.10) (0.09) (0.07) Few PB Projects 2.02 2.54* 1.93 were Implemented (0.08) (0.08) (0.09) N: 5512 5470 5518 * indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.05. ** indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.01. Table IV (a): Comparing PB Participants to Non-participants Responses Surrounding Authority in Project Implementation. T-tests compare PB participants to Non-participants. Lower scores on the Likert scale indicate stronger agreement with baseline statements. Treatment Condition Community Chief’ Support is MCAs’ Support is Groups’ Support is Necessary for Necessary for Necessary for Implementation Implementation Implementation Mean Likert Scale Mean Likert Scale Mean Likert Scale Response Response Response (SE) (SE) (SE) Control (no prompt): 2.01 2.42 1.82 PB Participants (0.05) (0.06) (0.06) Control (no prompt): 1.98 2.39 1.85 Non-participants (0.06) (0.06) (0.08) 21 Most PB Projects 1.98 2.41 1.83 were Implemented (0.08) (0.12) (0.09) (PB Participants) Most PB Projects 2.02 2.45 1.80 were Implemented (0.10) (0.14) (0.10) (Non-participants) Some PB Projects 1.95 2.33 1.85 were Implemented (0.11) (0.10) (0.13) (PB Participants) Some PB Projects 1.92 2.36 1.82 were Implemented (0.08) (0.07) (0.09) (Non-participants) Few PB Projects 2.04 2.54 1.94 were Implemented (0.09) (0.08) (0.07) (PB Participants) Few PB Projects 2.00 2.53 1.92 were Implemented (0.08) (0.06) (0.08) (Non-partricipants) N: (PB Participants) 3672 3651 3703 N: (Non-participants) 1840 1819 1815 * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 Respondents tend to agree that MCAs’, community groups’, and chiefs’ support are all necessary to complete projects. The greatest agreement is for MCAs’ support for implementation and the least for chiefs’ support. Results for treatment 2 show few treatment effects. Priming respondents to think about how projects get implemented in places with high, middling, and low rates of implementation does not change perceptions that community groups or MCAs are necessary to complete projects. There is variation in project implementation across the wards in the survey, yet there is no relationship between responses from citizens in places with higher levels of project implementation and those with lower levels. Thus, participants’ opinions on government authority and community groups’ influence do not appear to depend on whether projects are implemented or not. At first glance, the lack of connection here may suggest weaker prospects for holding MCAs accountable through PB in the face of lack of development project implementation. 22 If citizens have limited information about how implementation works, they may agree with what the government says regardless of what is occurring on the ground. However, the response to the “few projects implemented” treatment may show that PB participants have a sense of where authority lies for project implementation and who is at fault when projects are not completed. Respondents decrease their support for the statement that chiefs’ support is necessary for project implementation when they believe that few projects have been implemented in their ward. Many respondents believe chiefs’ support to be somewhat necessary for project implementation in general (with a mean score between “agree” and “no opinion”). This is an interesting finding because chiefs do not have a formal role in the process, but there is the perception that they informally engage in project selection and implementation. Respondents also rate chiefs’ support as less important than community groups’ and MCAs support. This shows respondents’ perceptions of the difference between local elected officials with influence over the development budget and project implementation (MCAs) and nationally-appointed officials with traditional authority, but few formal roles or power in county financing or service delivery and no direct responsibility for completing projects. Respondents therefore have their opinions on chiefs’ more limited role for project implementation confirmed when they believe that few projects have been implemented in their wards. Coupled with the responses to the first treatments above, the results suggest that chiefs have the power to potentially cause problems by excluding marginalized populations from deliberations, but not to solve problems by completing PB projects. Treatment 3: Collective Action Our third treatment addresses issues of collective action in support of PB projects. We gave half of respondents the following prompt about project success: PB projects are more likely to be successful if community members provide labor/contribute money to help build the project. The other half of the respondents received a control prompt: PB projects are no more likely to be successful if community members provide labor/contribute money to help build the project. We then asked respondents the following questions about their willingness to contribute labor and financial support for the projects. In each case, we also requested respondents’ phone numbers (which we immediately encrypted and stored anonymously) as a way to separate disingenuous respondents from those truly willing to give labor or money. 1. Are you willing to volunteer 1 day a week for 3 months to help complete the project? Yes NO 29A. If yes, please provide your phone number___________________ 23 2. Are you willing to contribute 100 KSH per month for 3 months to help complete the project? Yes NO 30A. If yes, please provide your phone number___________________ Table V: Collective Action Effects of Treatment 3 on Willingness to Contribute Labor or Money to PB Projects. Higher scores indicate greater willingness to contribute labor or money. We use Holm- Bonferroni corrections to address the prospect for cross-contamination across multiple treatments per respondent. Treatment Condition Willingness to Phone Willingness to Phone Contribute Number? Contribute Number? Labor Money Mean Mean Mean Mean Response (SE) Response (SE) Response (SE) Response (SE) Control (PB Projects 0.79 1.0 0.69 1.0 are Not more successful (0.13) (0.00) (0.08) (0.00) with community labor and money) PB Projects are more 0.79 1.0 0.67 1.0 successful with (0.15) (0.00) (0.07) (0.00) community labor and money N: 5465 5428 5457 5418 * indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.05. ** indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.01. 24 Table V (a): Comparing PB Participants to Non-participants Willingness to Contribute Labor or Money to PB Projects. T-tests compare PB participants to Non-participants. Higher scores indicate greater willingness to contribute labor or money. Treatment Condition Willingness to Phone Willingness to Phone Contribute Number? Contribute Number? Labor Money Mean Mean Mean Mean Response (SE) Response (SE) Response (SE) Response (SE) Control (PB Projects 0.80 1.0 0.70 1.0 are Not more successful (0.02) (0.00) (0.03) (0.00) with community labor and money): PB Participants Control (PB Projects 0.79 1.0 0.69 1.0 are Not more successful (0.03) (0.00) (0.04) (0.00) with community labor and money): Non- participants PB Projects are more 0.81 1.0 0.67 1.0 successful with (0.06) (0.00) (0.06) (0.00) community labor and money (PB Participants) PB Projects are more 0.80 1.0 0.69 1.0 successful with (0.03) (0.00) (0.05) (0.00) community labor and money (Non- participants) N: (PB Participants) 3602 3597 3591 3584 N: (Non-participants) 1849 1838 1822 1806 * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 25 There are no statistically significant results for treatment three. Respondents primed with the idea that PB projects are more likely to be successful if community members provide labor/contribute money to help build the project did not increase their willingness to volunteer, give their phone number, and contribute resources. The lack of result here could potentially stem from the high rates of willingness to volunteer in the first place: 80% of respondents were already willing to volunteer, which corresponds to the high rates of agreement with questions about voice and social inclusion through PB among this survey population. Other respondents may find the idea of new development projects in their communities attractive and thus attend meetings to steer resources in their directions. However, they may not be willing to contribute labor or money to these projects and are unconvinced by prompts on project success. There are no differences for treatment three across different populations within our survey. Higher SES individuals, members of community groups, community leaders, etc. are no more likely to contribute labor or money as a result of the treatment than any other groups in the sample. Treatment 4: Social inclusion Our final treatment corresponds to questions of social inclusion through participatory budgeting. We want to know how participants and non-participants think of PB as a program that meets challenges facing traditionally marginalized communities. We want the respondents to think carefully about which populations PB is or is not actively serving. We prompted respondents with the following three treatments, divided evenly among the population: The challenges facing youth/women/people with disabilities in this county means they deserve additional development funds through Participatory Budgeting. The remaining respondents did not see any prompt. We then asked respondents to gauge the extent to which they agreed with the following statements on social inclusion through PB using the same five-point Likert scale as above. “Participatory Budgeting projects provide support for disadvantaged communities” “PB allows people from disadvantaged groups to influence the spending of development funds.” 26 Table VI: Social Inclusion Effects of Treatment 4 on Responses Surrounding Social Inclusion. Lower scores on the Likert scale indicate stronger agreement with baseline statements. We use Holm-Bonferroni corrections to address the prospect for cross-contamination across multiple treatments per respondent. Treatment Condition PB Supports Disadvantaged Groups Disadvantaged Influence Spending Communities Through PB Mean Likert Scale Mean Likert Scale Response (SE) Response (SE) Control (no prompt) 2.15 2.08 (0.16) (0.18) Youths 2.19 2.01 (0.12) (0.15) Women 2.21 2.13 (0.13) (0.18) People with Disabilities 2.33* 2.04 (0.10) (0.15) N: 5448 5439 * indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.05. ** indicates P> Chi2 for Holm-Bonferroni adjusted significance of difference in means between treatment and control less than 0.01. 27 Table VI (a): Comparing PB Participants to Non-participants Responses Surrounding Social Inclusion. T-tests compare PB participants to Non-participants. Lower scores on the Likert scale indicate stronger agreement with baseline statements. Treatment Condition PB Supports Disadvantaged Groups Disadvantaged Influence Spending Communities Through PB Mean Likert Scale Mean Likert Scale Response (SE) Response (SE) Control (no prompt): 2.19 2.10 PB Participants (0.08) (0.10) Control (no prompt): 2.10 2.07 Non-participants (0.06) (0.13) Youths (PB 2.20 2.06 Participants) (0.05) (0.07) Youths (Non- 2.17 2.00 participants) (0.05) (0.08) Women (PB 2.18 2.15 Participants) (0.07) (0.07) Women (Non- 2.21 2.13 participants) (0.07) (0.05) People with Disabilities 2.30 2.03 (PB Participants) (0.09) (0.07) People with Disabilities 2.36 2.08 (Non-participants) (0.06) (0.09) N: PB Participants 3602 3597 N: Non-participants 1849 1838 * indicates P> t for difference in means between treatment and control less than 0.05. ** indicates P> t for difference in means between treatment and control at less than 0.01 For treatment four, there is broad support for the statements that PB supports disadvantaged communities and that disadvantaged groups influence spending through PB. Priming respondents to think about youths or women does not change perceptions that PB provides support for disadvantaged communities or gives disadvantaged groups power in 28 spending development funds to meet their needs. However, priming them to think about people with disabilities decreases their perceptions that PB benefits disadvantaged communities. This is just one significant result, but the implication is that respondents may be less inclined to believe that PB supports social justice goals when they are induced to think about the likely beneficiaries of PB projects. Again, the overall levels of results are high here: respondents do believe that PB supports disadvantaged communities and that disadvantaged groups influence spending through PB. However, the treatment also reveals a statistically significant rank-order, where people with disabilities may benefit less than other groups. There are no demographic differences in these response patterns. Once again, the control group and treatment group responses are already high: respondents broadly agree that PB serves disadvantaged communities and includes disadvantaged groups in decision-making. Yet, they agree less with this statement when given additional priming to consider people with disabilities. The implication here is that social inclusion must be discussed and included as part of the PB program design if it is to be achieved in practice. References Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of development economics, 4, 3895-3962. Banerjee, A., Duflo, E., Glennerster, R., & Kinnan, C. (2015). The miracle of microfinance? Evidence from a randomized evaluation. American Economic Journal: Applied Economics, 7(1), 22-53. Duflo, E., & Kremer, M. (2005). Use of randomization in the evaluation of development effectiveness. Evaluating development effectiveness, 7, 205-231. 29