GPSA NOTE 15 CAPACITY AND IMPLEMENTATION SUPPORT AREA PORTFOLIO PERFORMANCE REVIEW MARIA POLI AND FLORENCIA GUERZOVICH1 The objective of this report is to inform the Secretariat of the Global Partnership for Social Accountability (GPSA) regarding its fiscal year 2017 (FY17) work plan on capacity building, knowledge and learning, and monitoring and evaluation as well as to provide inputs for the program’s overall strategy, particularly institutional development and fundraising. The report takes stock of prior and ongoing work by presenting the initial setup of the Capacity and Implementation Support (CIS) area, including rationale, areas of focus, institutional arrangements, and resourcing. The document discusses lessons learned to date and presents an overview of scale-up and course correction measures under way. It concludes by putting forth a proposed Action Plan for FY17. This work is based on a desk review of program documents including, but not limited to, annual technical progress reports from grantee civil society organizations (CSOs), back-to-office reports, and dissemination notes as well as a systematic capacity-build- ing analysis of internal qualitative progress reviews carried out by the CIS team.2 1 This work is a product of the staff of The World Bank with external contributions. It was written by Maria Poli and Florencia Guer- zovich of the Global Partnership for Social Accountability (GPSA), World Bank, in October 2016 and published in March 2020. Only the sources of GPSA’s most recent grant application-related documents referenced in the note have been updated. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. Please cite the work as follows: Poli, Maria, and Maria F. Guerzovich. 2020. “Capacity and Implementation Support Area: Portfolio Per- formance Review.” Global Partnership for Social Accountability Note 15. World Bank, Washington, DC. It can be accessed at https:// gpsaknowledge.org/knowledge-repository/capacity-and-implementation-support-area-portfolio-performance-review/. The authors thank Emilie Fokkelman for her comments, Marine Perron for her support, Barbara Rice for edition and Claudio Mendonca for the design. 2 This work is distinct from, and intended to complement, “mentoring partnerships” as described in the GPSA Operations Manu- al. They may be proposed in grant applications with the aim of targeting the needs of nascent civil society organizations (CSOs) 1 Capacity and Implementation Support Area: Portfolio Performance Review This report takes stock of prior and ongoing work by presenting the initial setup of the CIS area, including rationale, areas of focus, institutional arrangements, and resourcing. The doc- ument discusses lessons learned to date and presents an overview of scale-up and course correction measures under way. It concludes by putting forth a proposed Action Plan for FY17. Rationale: Capacity-Building in the Context of the GPSA The GPSA was created in 2012 by the World Bank to contribute to country-level gov- ernance reforms and improved service delivery. As a platform for global grant making and knowledge sharing, it provides sustained and strategic support to CSOs to work together with governments to solve critical governance challenges. The GPSA aims to foster enabling environments in which citizen feedback and participation is used to solve fundamental problems in service delivery and to strengthen the performance of public institutions. In the GPSA’s approach, social accountability is an element of collaborative governance whereby citizen feedback is needed to better understand public problems and develop appropriate solutions. Often, these solutions are co-produced between civil society and public sector institutions and may also be jointly implemented. The GPSA’s specific objectives are (World Bank 2012, v):  To generate knowledge, networking, and financing to build civil society‘s capacity to engage in evidence-based social accountability;  To support government counterparts and World Bank teams in embedding social accountability more strategically in their programs; and and CSOs with little or no experience with social accountability. Mentor CSOs must be legal entities in the country where the project will be implemented or in any of GPSA’s opted-in countries. They work in partnership with inexperienced CSOs to on- grant and mentor them through clear agreements that spell out the contributions and roles of each. In these cases, applicants must link the institutional strengthening and capacity-building needs of the mentee CSO(s) to the implementation of social accountability activities, which enables the mentee CSOs to learn by doing. See GPSA Operations Manual (World Bank 2014). 2 Capacity and Implementation Support Area: Portfolio Performance Review  To draw on the experience, knowledge, and resources of external partners to enable the scale-up of social accountability. To meet these objectives, the GPSA program is divided into two main components:  Programmatic support to CSOs for social accountability (SAcc). This component awards grants to CSOs working in countries that have “opted-in” to the GPSA to implement projects that engage governments collaboratively and with political savvy in their social accountability efforts.  Knowledge activities, including a global platform for knowledge exchange and research. At the core of the GPSA is the mandate to contribute to improving development re- sults by supporting capacity building for enhanced citizen feedback and participation. The technical paper (Board Paper) presented to the World Bank’s Board of Directors proposing the establishment of the GPSA put this goal at the center of the program in response to two main findings (World Bank 2012, iii, 4). First, years of World Bank experience supporting governance and development reforms and the results of evalua- tion concurred on the need “to pay attention to civil society capacity building as part of an urgent update of the Bank‘s approach to institution building for better in-country economic governance.” Second, large knowledge gaps were found on the interplay be- tween social accountability, supply-side reforms, and the political economy of specific country contexts. In this regard, the paper put forward that addressing these gaps “re- quires learning by doing, rather than passive research,” and proposed the GPSA follow this approach in helping to tackle them. In response to this mandate, funding must prioritize sustained support for “larger grants for periods of 3–5 years” coupled with mentoring and technical assistance to CSOs and government counterparts (World Bank 2012, v). The assumption in the GPSA Board Paper and its Results Framework is that civil society groups need to develop cer- tain capacities to leverage social accountability in order to deliver solutions to critical governance challenges. 3 Capacity and Implementation Support Area: Portfolio Performance Review Moreover, the GPSA Results Framework includes a specific output regarding the ca- pacity of grantee CSOs to apply a collaborative social accountability approach — en- compassing applied political economy analysis and other elements that are considered relevant for the success of SAcc initiatives (GPSA 2015). The GPSA is fully responsible for delivering this output. Evolution of the GPSA Capacity-Building Area Since its inception, the GPSA has always put the accent on capacity-building, dedi- cating a large part of its efforts and funds to CSOs’ institutional strengthening and capacity development, knowledge and learning, as well as technical support during the different phases of the projects. As the program grew and welcomed more grantees, it formalized and strengthened its capacity-building work with the creation of a special- ized team entirely dedicated to providing timely and quality support to grantees. With time, it became clear that this team should also ensure the articulating of monitoring, evaluation, knowledge, and learning. MANDATE The GPSA Board Paper placed capacity building as a central program mandate. It identified the need to provide predictable core funding that can sustain the capacity-building efforts of civil society groups over 3- to 5-year periods of time. It also identified the necessity to offer other capacity options for smaller or start-up CSOs which do not yet have strong expertise on social accountability by encouraging on-granting or support for capacity-building and mentoring arrangements from intermediary CSOs awarded GPSA funding (see box 1). 4 Capacity and Implementation Support Area: Portfolio Performance Review INITIAL CAPACITY BUILDING WORK The first phase of the GPSA’s capacity-building work focused on three priorities. The initial priority was setting up a grant-making process in line with the GPSA Board Paper’s twofold objective of (i) providing “strategic and sustained” support to CSOs for social accountability projects that (ii) may advance knowledge on social accountability’s contribution to improv- ing governance and development outcomes. To this end, the GPSA Secretariat developed a grant application process that embedded the program’s key design features, while integrat- ing lessons from the latest research and evidence on the impact and effectiveness in the field of social accountability, including on transparency, accountability, and participation. 5 Capacity and Implementation Support Area: Portfolio Performance Review Box 1. Main Program Design Features in GPSA Grant-Making Process and Grant Application Sustained and predictable funding Targeted social accountability approach  3- to 5-year periods of time  Problem-driven, value-added  $500,000 to $1 million rationale to support ongoing public sector reforms and decision makers Country ownership Holistic approach, including:  Explicit government decision to “opt-in”  Systemic, political economy analyses  Priority themes defined in-country of context, actors, institutions, and  Applicant civil society organizations strategic options (CSOs) based in-country (legal  Scalability embedded in choice of status, local budget, local board) strategy, process, and “tools”  10-day (nonbinding) government  Adaptive management and learning and 5-day public comment period on (with earmarked funding) pre-selected proposals  Institutional strengthening and World Bank’s comparative advantage capacity development  Issues and sectors supported through  Unrestricted funding applicable the Bank’s country strategy and to institutional strengthening and portfolio operational expenses  Grant projects supervised by  Mentoring: lead applicant CSOs to sector specialists (World Bank mentor smaller, less experienced project supervisors, called “task CSOs, community-based team leaders” or “TTLs”) and organizations, or civil society groups complementing active or pipeline operations where applicable  Partnerships: funding shared with partner CSOs according to functional distribution of capacities Source: GPSA Operations Manual (World Bank 2014), GPSA Grant Application Guidelines (World Bank 2019a) and Fourth Call for Proposals’ Orientation Session presentation (World Bank 2019b) 6 Capacity and Implementation Support Area: Portfolio Performance Review The application template required applicants to justify their proposals on the basis of a “collaborative social accountability” approach (see box 2). Technical review criteria were also defined consistently with this approach in addition to other institutional cri- teria intended to direct funding toward locally-managed grant projects. In coordination with World Bank country offices, the GPSA Secretariat organized informative sessions for potential applicant CSOs ahead of three global calls for proposals carried out so far. 7 Capacity and Implementation Support Area: Portfolio Performance Review Box 2. Collaborative Social Accountability For the GPSA, collaborative social accountability is a process encompassing four main elements:  Problem-driven approach. The starting point to address specific governance and service delivery problems affecting citizens’ well-being is the identification of problems whereby citizen feedback is needed to better understand their causes and to develop appropriate solutions.  Context based. The problem is understood within the context of the actors, institutions, and processes that are already involved in resolving it, including those stakeholders who have an interest in or are affected by the issue.  Constructive engagement. A problem-driven political economy analysis is coupled with an assessment of the existing and potential space for engaging with public sector institutions. It also considers how the World Bank may help to foster such space.3 Both the process and the actual feedback generated through social accountability must be shared, discussed, and coordinated with the public sector institutions involved, primarily those that have the decision-making power to translate the feedback obtained into actual changes aimed at improving governance and development processes.  Multi-stakeholder coalitions and partnerships. The complexity of governance and service delivery problems calls for the concerted action of actors that have direct and indirect interests in supporting their resolution. It also requires combining diverse types of expertise, outreach capacities, and influence in order to work simultaneously on the range of dimensions involved. Ideally, all these stakeholders learn from each other and work together to figure out options regarding strategies and solutions and put them to work. Civil society organi- zations drive collaborative social accountability interventions but are not lone actors. Source: GPSA Grant Application Guidelines (World Bank 2019a) 3 “GPSA would focus on areas of Bank comparative advantage and value addition arising from a combination of the Bank‘s official and unique relationship with governments, the range and reach of its partnership and knowledge services, its convening power, and its ability to complement and reinforce supply-side GAC interventions with demand-side efforts for better development re- sults. The Bank can use its convening power and leverage its traditional engagements with governments to create more space for constructive engagement between CSOs and governments to enhance development effectiveness” (World Bank 2012, iv). 8 Capacity and Implementation Support Area: Portfolio Performance Review The second priority of GPSA’s capacity-building work was to conceptualize and opera- tionalize a knowledge and learning component. A concept note informed the prepara- tion of terms of reference to select and award a knowledge grant to manage a global knowledge and learning (K&L) online platform. It was conceived to target a range of stakeholders — primarily, practitioners in civil society, academia, and governments as well as future GPSA grantee CSOs and government counterparts — looking for relevant SAcc materials and knowledge exchange opportunities. Moreover, both the grant application and grant projects were required to include a K&L component, for which the GPSA Secretariat provided guidance. This component should link and use the proposed interventions as opportunities for learning about strategic pathways. The requirement built on the available literature at the time of the GPSA’s first operational setup and was intended to “nudge” CSOs to shift knowledge strategies from a traditional focus on the production of generic learning for external audiences (e.g. through “best practices” and “case studies”) to a more internally-focused pro- cess based on iterative learning and adaptive management. 9 Capacity and Implementation Support Area: Portfolio Performance Review Box 3. Adaptive management and learning in GPSA-supported projects GPSA aims to support grantees to develop their own systems and processes for adaptive learning so that they can effectively contribute to a growing body of knowledge on social accountability tools and practices. This means, among other things, ensuring that experiences and lessons flow from the project to the wider organizational environment of the grantee and its stakeholders for reflection and feedback, leading to adaptations in the strategies that inform project implementation. GPSA will also assist grantees in disseminating such adaptive learning widely among practitioners and policy-makers in order to enhance the effectiveness of SAcc interventions. The GPSA online Knowledge Platform provides access to knowledge, support sharing of experiences, facilitate learning, and networking.4 GPSA requires that grant proposals include a K&L Component, whereby applicants develop a plan that links strategies for project-specific, organizational and peer learning to processes for adapting the project and its implementation based on documented learning. The K&L component should also reflect on how the proposed interventions include opportunities for advancing knowledge about strategies and pathways for promoting transparency, accountability and civic engagement that could be shared with target audiences within the wider community. Some key questions to answer in designing the K&L Component are:  How do we ensure that learning from this project is informing our organizational strategies as a social accountability actor?  What mechanisms will we use to get feedback on emerging lessons/issues from our stakeholders and peers (please specify who they are) and how will our learning be shared with them? How do we ensure that such sharing is iterative and continuous? 4 http://gpsaknowledge.org 10 Capacity and Implementation Support Area: Portfolio Performance Review  The K&L component, as other project components, can and should be adjusted to context and learning during the course of the grant, including budgetary course-correction.  How do we make sure that we continuously will learn from and adapt to our specific operating context?  What particular contribution to K&L on SAcc do we expect to make that can be shared with the broader community of actors interested in SAcc?  What are our K&L needs and knowledge gaps? What are our organizational strengths and weaknesses in terms of effectively using K&L in our work? While proposals are being assessed on their strengths, the proponent’s ability to recognize needs and weaknesses is an important aspect as well.  What K&L resources do we have? Are they effective in achieving the objectives for which they were developed or do we need to improve them? Are we prepared to share these resources?  How do we include project participants/beneficiaries in learning and ‘sense making’ of lessons throughout the project? How do we ensure that their learning is sustained when the project is over? Source: Third Global Call for Proposals’ Application Template (World Bank 2015) Regarding the third priority, the GPSA Secretariat focused its efforts on supporting the first group of 11-selected grantee CSOs during project preparation, approval, and implementation. This phase formally began in July 2013, with the first collective ca- pacity-building activity for grantees delivered through a one-week workshop aimed to discuss projects’ design and support preparations for implementation. It was fol- lowed by targeted support to grantee CSOs during project implementation. This sup- port focused on providing high-quality, rapid response guidance to grantees and World 11 Capacity and Implementation Support Area: Portfolio Performance Review Bank TTLs. Special emphasis was given to integrate political economy analysis and collaborative social accountability into project strategies and operations, in line with the GPSA’s Results Framework and Theory of Change. To this end, the Secretariat fa- cilitated the dialogue between new grantees and World Bank TTLs to forge a common understanding of projects’ designs and operational roadmaps as well as to leverage opportunities offered by pipeline or active World Bank sector operations. This type of implementation support entailed different forms of accompaniment and technical assistance. Initially, rapid needs assessments were carried out with all proj- ect teams to identify priority needs in terms of knowledge and operational support. Some teams required support in brokering the dialogue with government counterparts; others needed specific expertise, for instance, on understanding public procurement of education inputs, or designing participatory processes involving different types of “tools”, such as community scorecards and citizen report cards. From a portfolio per- spective, the Secretariat also oversaw the technical quality across all grant projects regarding social accountability approaches and risk mitigation measures. As the portfolio grew from 11 to 19 projects after the second round of grants was awarded, the GPSA Secretariat formally set up a Capacity and Implementation Support (CIS) team aimed to ensure timely and quality support on social accountability across the portfolio. An important lesson learned from the experience with the first round of grant projects was that CSOs appreciated a type of accompaniment beyond funding, primarily focused on helping them to navigate the political economy of projects’ implementation. Critical in this process was (i) the relationship with World Bank sector specialists and the synergies with broader sector work, which sometimes also included other development partners, and (ii) the rela- tionship with government, which entailed dynamic collaborative frameworks that required political savviness and learning-by-doing. Between October and December 2014, additional human resources were identified and recruited based on country and regional expertise. A dedicated full-time consultant was assigned to lead capacity-building activities with sup- port from a cadre of GPSA advisors covering all the pre-selected and active projects. 12 Capacity and Implementation Support Area: Portfolio Performance Review Until early FY16, research, monitoring, evaluation, and learning associated with grant- ee operations were supported by other GPSA areas; however, the experience from pro- viding implementation support pointed to the need to better integrate these functions. As a result, in October 2015, they were moved to the capacity-building area of respon- sibility. Some K&L functions targeting general audiences remained separate, though in coordination with activities within the CIS area. 13 Capacity and Implementation Support Area: Portfolio Performance Review Capacity and Implementation Support Review The GPSA’s CIS team rolled out a series of tailored activities to strengthen the capac- ities of grantee CSOs. Different activities fit in the different capacity gaps of grantees and other groups and are designed to support the development of concrete capacities during the course of design and implementation of GPSA grants. TYPES OF IMPLEMENTATION SUPPORT Core activities combined different types of support: one-on-one mentoring and advice to grantee CSOs, World Bank TTLs and country offices, and Government counterparts; technical assistance; funding to build capacity collectively; and funding for K&L (see table 1 for more details).  One-on-one mentoring and advice to grantee CSOs, World Bank TTLs and country offices, and Government counterparts. The focus is on integrated support on core capacities through joint reflection and action to increase implementation effectiveness while contributing to complex change processes that reach beyond grantees’ own direct influence.  Technical assistance. Targeted support is given by expert consultants to meet specific project and organizational needs that may not be funded through project funds.  The GPSA, in partnership with the Aga Khan Foundation and the Aga Khan Graduate School of Media and Communications (based in Nairobi), supports communications and media capacity building through individual mentoring and collective learning.  GPSA finds high-level expertise to meet specific technical needs in areas such as procurement in education and design of data collection instruments, among others. 14 Capacity and Implementation Support Area: Portfolio Performance Review  Funding to build capacity collectively. Support targets all or subgroups of grantees (or potential grantees), networks, and other collaborative efforts.  The GPSA supported or participated in activities to promote joint reflection and capacity building for multiple stakeholders working in a particular grant or context.  The GPSA funds events to develop capacities of multiple grantees at once. It also supports activities to build capacities of grant applicants, such as a pilot online course.  The GPSA also supports the annual Global Partners Forum to incentivize broader field collaboration and knowledge exchange.  Funding for Knowledge and Learning. All GPSA grants include a K&L component. Grantees identify areas in which they would like to invest resources.  A majority of grantees have used K&L funds to organize local knowledge sharing and reflection events with local partner CSOs and other key stakeholders.  K&L funds are also being used to write reflection pieces shared through the GPSA Knowledge Platform.  Funding for technical service providers, intermediaries, or researchers. Grants or contracts facilitate global knowledge and learning exchange and support research.  Fundar, a CSO in Mexico, received a knowledge grant to develop a knowledge platform offering access to social accountability materials and virtual K&L exchange events.  The GPSA and selected grantees partnered with the MIT Govlab to produce collaborative research and, in so doing, increase grantee monitoring and evaluation capacities. 15 Capacity and Implementation Support Area: Portfolio Performance Review Table 1. Types of GPSA Support and Activities Type of Support One-on-one mentoring and advice for World Bank task team leaders and country offices, and Government counterparts Benefits  Targeted, ongoing relationship building, matchmaking, brokering, and support through high-quality, rapid response guidance for grantees in cooperation with TTLs.  Mentors can be critical friends of the project, providing an informed external view to help understand and receive feedback from the broader context.  Mentors can also provide honest feedback. Their advice can facilitate learning and course correction to adjust to this broader context and its changes.  Mentors can act as grantee “translators” conveying their needs to other project stakeholders and vice-versa as well as focusing on creating a shared collaborative approach among them.  Support grantees to apply political economy and collaborative social accountability to their projects and to customize technical tools to their political and organizational contexts.  Support TTLs so that they can also support grantees effectively. Limitations  Customized support is time and resource intensive. It is not possible to provide similar levels of support across the portfolio.  Mentor-grantee fit is crucial for these relationships to work. It is difficult to identify the right mentor who can build the right type of relationship with grantees and have the technical, political, and collaborative skills to work with a range of partners. 16 Capacity and Implementation Support Area: Portfolio Performance Review Type of Support Technical assistance Benefits  Tailored to specific grantee need or designed transversally when more than one grantee shares a need.  CSO accesses high-level expertise.  Transversal TA allows for both individual mentoring and collective learning.  Economies of scale for collective needs and sharing resources with other donors or global partners.  Technical assistance from an outside provider can allow for a more objective approach. Limitations  Absorption capacity varies depending on prioritization of need in relation to implementation.  Technical assistance that is too funder-driven will be less effective. Input from grantees is critical.  Off-the-shelf capacity-building interventions can be less effective. Type of Support Funding to build capacity collectively Benefits  Complements individual support in areas where project partners may benefit from collective learning and capacity building.  Economies of scale in areas of common needs.  Encourages grantees and partners to work together, including government counterparts.  Recognizes the need for multiple actors working to address social issues.  Provides critical funding to help strengthen collaborative efforts. Limitations  It can be difficult to determine how best to structure the support.  Outcomes may be unclear given multiple actors and efforts.  Funders must make multiyear commitments for the support to be meaningful. 17 Capacity and Implementation Support Area: Portfolio Performance Review Type of Support Funding for knowledge and learning Benefits  Targeted support to meet specific needs that may not be funded from other sources.  Offers funding and space for iterative learning and adaptive management of grant project.  May help set the stage for broader organizational growth and development. Limitations  Most grantees prioritize technical know-how for staff and volunteers, rather than building organizational capacities.  Requires one-on-one mentoring to build adaptive learning capacities. Type of Support Funding for technical service providers, intermediaries, or researchers Benefits  Helps ensure civil society groups have access to knowledge, experience, and resources to best build their capacity.  Can offer economies of scale.  Can offer expertise the funder doesn’t have in-house. Limitations Limited tailoring for individual needs — more suited to collective needs across themes and sectors. Note: CSO = civil society organization; TA = technical assistance; TTL = task team leader. ALLOCATION OF RESOURCES BASED ON THE EVOLVING CAPACITIES OF GRANTEES At the request of the GPSA Steering Committee, a capacity needs assessment based on applications to the first and second global calls for proposals found significant capac- ity gaps relative to the program’s focus on collaborative and politically smart social accountability (Guerzovich and Poli 2014). The analysis revealed that both applicants and grantees have overall low capacities to apply a collaborative social accountability approach to targeted problems. 18 Capacity and Implementation Support Area: Portfolio Performance Review This assessment was consistent with the GPSA’s experience supporting the first two rounds of grantee CSOs in the preparation and implementation of grant projects. The GPSA’s CIS team allocated its resources for both first and second round grantees based on the early assessment of applications mentioned above. This assessment also included the initial experience with the first round of grantee CSOs’ needs assessment complet- ed in December 2014 jointly with grantee CSOs and TTLs and a series of follow-up site visits. Ongoing one-on-one mentoring was used to inform and update this assessment. LEVEL OF GRANTEE CAPACITY BUILDING NEEDS Inductively, after more than 2 years of GPSA experience, the CIS team has identified a group of core capacities for the implementation of collaborative social accountability processes. These capacities are summarized in the figure 1 and include political savviness and adaptability as well as technical (social accountability and sectoral) competence. The GPSA has zeroed in on these capacities, rather than others because they seem to affect GPSA grantees’ ability to make progress across regions and sectors but also be- cause they impinge on these or other capacities. For instance, a politically savvy orga- nization may realize it needs to change its political strategy to make its programming responsive to its context while increasing the likelihood of achieving impact. However, unless the group is willing to adapt, knows how to adjust its programming, and has the technical competence to redesign its operations, it is unlikely to achieve impact. 19 Capacity and Implementation Support Area: Portfolio Performance Review Figure 1. Critical Capacities for Collaborative Social Accountability Note: CSO = civil society organization; WB = World Bank; Govt = government. Table 2 summarizes the definition of these interrelated capacities. 20 Capacity and Implementation Support Area: Portfolio Performance Review Table 2. Interrelated Capacities for Collaborative Social Accountability Type of Capacity In Practice Organizational Project management Integrates financial, human resources, and technical management into (fit for GPSA-type management) coherent, quality, and adaptive management. Adaptability Adjusts integrated management in a rapid and timely manner based on iterative assessment of organizational, analytical, and operational progress. Civic Ability to problem-solve and work with others: Inter-partner coordination  Strategy is validated with partners and based on functional distribution and coalition building according to comparative capabilities of all involved.  Roles, responsibilities, resources (human, financial), and social and political capital are explicitly discussed.  Mechanism for periodic reflection and performance monitoring is agreed upon.  When there is a CSO leading the process, it exhibits collaborative leadership capabilities.  Is able to understand its own role in the context of a broader coalition – and to coordinate or negotiate formal and informal agreements across formal and informal, state, and nonstate actors. Constructive engagement  Understands engagement as a continuous process shaped by terms of agreements with government actors and sharing of information and resources to operationalize SAcc process.  Demonstrates effective communications, negotiation, and consensus- building skills throughout process, including proactive anticipation and planning as well as dynamic adaptation to changing political events. Operational Applied technical  Technical quality of participatory data collection. competences  Efficient and effective management of participatory cycles.  Capacity to use data to fill information gaps and to inform service delivery chain bottlenecks and corrective measures.  Capacity for follow-up at different government levels — combining both technical and political analysis. 21 Capacity and Implementation Support Area: Portfolio Performance Review Type of Capacity In Practice Analytical Technical competences  Can diagnose or understand sector diagnostics, key issues, and approaches to solutions.  As a result, selects focus areas for SAcc process strategically.  Defines SAcc process based on political, cost-efficiency, and sustainability analysis.  Technical quality of data analysis.  Capacity for M&E: qualitative and quantitative results definition; sequencing process and final results realistically; use of M&E for adaptive management. Problem-driven work and  Applies political analysis to sector assessment and SAcc process and results outcomes definitions.  As a result, selects focus areas for SAcc process strategically. Note: CSO = civil society organization; GPSA = Global Partnership for Social Accountability; M&E = monitoring and evaluation; SAcc = social accountability. Based on this assessment, grantees were grouped from very low to high.  Very low. These CSOs have limited social accountability experience and deficits along the four key capacities – analytical, civic, organizational, and operational.  Low. These CSOs have some experience implementing social accountability tools but are lacking in collaborative social accountability experience. They lack at least one, and often more than one, of the key capacities.  Medium. These grantees have experience with using social accountability tools and some experience with collaborative social accountability. They have capacities in all four key areas albeit with some significant voids in some important aspects of these capacities.  High. They have experience with tools and tactics, and often with collaborative social accountability. They are strong along all four key capacities with the ability to rapidly identify capacity gaps and learn and integrate lessons into their operations. 22 Capacity and Implementation Support Area: Portfolio Performance Review Figure 2 maps the status of round 1 and 2 grantees as of May 2016. It also includes a preliminary assessment of round 3 grantees. Figure 2. Distribution of GPSA Grantees by Capacity Level and Region Capacity Level High Medium Capacity Level High Medium Low Capacity Level High Medium Low Capacity Level Very-Low Medium Low Note: Blue = Round 1 | Green = Round 2 | Orange = Round 3. Round 3 CSOs are placed based on the information available at the time of proposal selection; placement might change. Figure 2 doesn’t include CSOs funded through Open Society Foundations’ parallel funding mechanism. CSO = civil society organization | GPSA = Global Partnership for Social Accountability 23 Capacity and Implementation Support Area: Portfolio Performance Review The distribution in figure 2 is based on the direct experience of the GPSA team with grantee CSOs, and it may not necessarily reflect the capacity levels of the organizations as a whole. However, some indicators of capacities, such as those related to project management and technical competence, may reflect the existence or lack of standard operating procedures. Almost 50 percent (13) of the grantees are placed in the medium capacity level. This placement is consistent with the GPSA’s grant application requirements, which set a minimum of at least 3 years of experience implementing social accountability projects. It is also consistent with the qualitative analysis of grant proposals. They showed a very limited understanding of collaborative approaches, even when the CSO had ex- tensive experience in mobilizing citizens, gathering feedback through typical social accountability instruments or tools (e.g., citizen report cards, community scorecards, and social audits), or organizing participatory processes and events (e.g., participatory planning, public hearings, and citizen forums). Overall, at the time this analysis was done for the first time, Round 1 projects had been under implementation for almost 2 years, and Round 2 projects for 1 year. Three proj- ects had moved from lower to higher capacity levels. In these cases, the change could be partly attributed to the GPSA’s mentoring of project teams since project inception, and most importantly, to evidence of the gradual improvement of teams’ capacities to ana- lyze the context and to respond proactively by adjusting their strategies and operations. In the case of projects classified in the medium capacity level, the analysis also iden- tified indicators of progress that demonstrate the increased capacities of the teams to assess their performance and introduce adjustments in the implementation strategy. For instance, these projects begin with rigid designs in their operational sequences and without sound validations of such designs with key stakeholders or in political economy terms. However, over time, teams demonstrate their openness to engage collaboratively with other CSOs as well as with government counterparts. Likewise, they are willing to “learn by doing” by adjusting their original plans on the basis of operational experience. As a result, these projects have shown concrete improvements in managing relations 24 Capacity and Implementation Support Area: Portfolio Performance Review with government and their partners as well as continuous qualitative improvements in the implementation of feedback instruments and participatory mechanisms. Grantees classified as high capacity have required minimum support from the GPSA’s CIS team, usually in the dialogue with TTLs only. Progress reports and periodic updates with project teams demonstrate the existence of articulated capacities — technical, civic, man- agerial, etc.-. These projects, in addition to providing peer learning and exchange with other GPSA grantees, are relevant insofar they offer insights into some of the determining factors of success. As such, they need to be considered to better understand such factors in cross-country comparisons of GPSA projects (and for future GPSA evaluations). The analysis drew from project documents and technical reports, and from the CIS team’s observations as a result of the advisory services provided to project teams and TTLs. This type of analysis is a useful tool to assess the types of support and follow-up that the GPSA can provide to grantee CSOs, as well as to guide the allocation of re- sources for this purpose. The following section provides broader conclusions from this analysis, including from a previous analysis of grant applications. 25 Capacity and Implementation Support Area: Portfolio Performance Review Lessons Learned and Insights to Date The GPSA team has identified several lessons from its capacity-building component that could help it take its work to the next level. In terms of the field, the need is to:  Increase the capacity of the grant applicants to produce collaborative social accountability proposals to solve critical governance problems. Candid reflection on capacity needs and limits remains a challenge.  Improve the linkages between applicant proposals and ongoing state reforms. The CSOs’ capacity for dialogue, consensus building, and co-production is limited. The field is only starting to experiment with these approaches, and experiences remain siloed. Much more can be done to help scale up multi-stakeholder collaboration for problem solving on the ground.  Increase grantees’ capacities to apply political economy analysis to targeted problems. Social accountability approaches are ill-informed, are not politically savvy in prioritizing viable entry points, and tend to be a combination of tools plus advocacy campaigns rather than driven by sound problem- and sector-driven political thinking and action. This is reflected in grantees’ operational plans and budgets (and their revision over time) and puts the chances of projects to deliver at risk.  Build on and scale experiences, processes, and incentives for joint decision making for programmatic-collective action among civil society groups. Regarding the structure of capacity building, these insights can help navigate this challenge:  The linkage between grant making and capacity building increases the GPSA’s ability to design activities that offer clear, practical value to partners. The GPSA’s capacity building is targeted to practitioners working together on concrete 26 Capacity and Implementation Support Area: Portfolio Performance Review projects. Its work involves much more than technical knowledge associated with a particular task and political know-how.  Certain capacity-building strands entail relationship building between funder and grantee that gradually seem to contribute to common ways of doing and approaching things. An example is mentorships.  Grantees increasingly prioritize collaborative capacity building and learning. This is happening as the result of joint work on projects, i.e. learning by doing, rather than in original plans or passive knowledge transition.  Tailored support enables to deliver fit for purpose capacity building when it comes to implementation challenges. Capacity needs differ according to a variety of factors such as project cycle stage, geographic location, organizational and partnership characteristics, governance landscape and sectoral political economy.  Structural capacity building strives to gradually create abilities within politically informed, collaborative social accountability projects among civil society, governments, and World Bank staff, among others. This kind of collaboration— which can range from information exchanges to co-production — requires that different stakeholders, included but not limited to grantees, build capacities and relationships to take on this joint approach. To support and strengthen local ownership, partnerships and joint capacities for multistakeholder constructive engagement have emerged as a related, underlying goal of capacity-building activities.  To implement GPSA’s innovative approach to social accountability, continuous capacity building is better suited than an episodic one. The GPSA and its partners seek to solve complex, heavily context-dependent governance problems that likely require experimentation, reflection, and refinement or course correction as part of the capacity building for the implementation process. Time also allows for the interval and effort needed for changes associated with developing collaborative social accountability and relationship and trust building to take hold. 27 Capacity and Implementation Support Area: Portfolio Performance Review  Under these conditions, it is challenging to rely too heavily on in-kind support from the GPSA’s network of global partners. They are better suited to provide ad hoc support when they can also get concrete benefits for their contributions.  Creating the conditions for adaptation and course correction also requires time and energy. Grantees have had difficulties identifying adaptive learning in their operations. GPSA support in terms of research, monitoring, evaluation, and learning has had deficits, particularly lack of integration with operations. The deficits are embedded in broader funding contexts that often limit incentives for these kinds of changes. A key aspect is ensuring that continued exchanges of information and openness adapt in a timely manner.  The implementation of adaptive learning and management projects requires adequate investment in and integration of research, learning, monitoring, and evaluation activities, both at the grantee and the portfolio levels. Lack of articulation undermines individual operations as well as the GPSA’s ability to leverage its operations for field building, learning, and resource mobilization.  Individual capacity-building support should be complemented with small groups, multistakeholder activities to build collective capacities. The GPSA’s approach to social accountability is innovative and tackles complex problems. Projects require knowledge about a large number of issues and practices. No single grantee or GPSA staff member can claim full knowledge and competence across these issues. Key stakeholders other than civil society grantees (e.g., government officials and World Bank staff) can fill critical knowledge and competency gaps. Collective learning by doing and reflection with other stakeholders, peers, and critical friends can help enhance grantees’ understanding and adaptation to specific contexts. Small groups can enable individuals to go deeper in bounded reflection on facilitating operational adaptation and uptake. Useful subgroups may include geographic, sectoral, or common concerns. 28 Capacity and Implementation Support Area: Portfolio Performance Review Scale-Up and Course Correction Measures The capacity gaps in the social accountability field are large, particularly when it comes to leveraging the approach through constructive engagement with governments. The GPSA has experimented and learned about ways through which these capacity gaps can be addressed. As the grant and capacity-building portfolio grows, the capacity-build- ing component is considering or implementing a series of course correction measures to implement the GPSA’s mandate. MAIN FOCUS FOR CAPACITY BUILDING WORK The GPSA’s CIS team will maintain a strong focus on hands-on, one-on-one support for grantees and key project stakeholders in the design and implementation of projects, but will adapt to better approach the challenge of a larger portfolio and the lack of capacities of actors that CSOs engage in their work, such as public officials. Key adaptations include:  Reinforce the work of capacity-building advisors with pre-selected grantees and their TTLs to establish relationships and redesign projects upfront, based on lessons learned in the prior rounds of grants.  Shift capacity development and grant making toward tailored country and regional support through a “hubs” approach.  Integrate research, monitoring, evaluation, and learning needs into the capacity- building component.  Invest additional resources to capture lessons from operations for field building, learning, and resource mobilization. CAPACITY BUILDING FOR ECOSYSTEM The CIS team, in dialogue with internal and external stakeholders, has prepared a concept note to guide the GPSA’s capacity development strategy, one of the pillars of its strategic review. 29 Capacity and Implementation Support Area: Portfolio Performance Review References Guerzovich, Maria F., and Maria Poli. 2014. “Are We Ready for Strategic Social Accountability?” Global Partnership for Social Accountability Dissemination Note. World Bank, Washington, DC. http:// gpsaknowledge.org/knowledge-repository/creating-space-for-social-accountability-the-global-partnership- for-social-accountability/#.VtSJHpwrLWI. Tsai, Lily, and Maria F. Guerzovich. 2015. “Results Framework.” Global Partnership for Social Accountability, Washington, DC. https://www.thegpsa.org/Data/gpsa/files/field/documents/gpsa_revised_ results_framework_10december2015.pdf World Bank. 2012. “Global Partnership for Social Accountability and Establishment of a Multidonor Trust Fund.” Technical Paper prepared for the World Bank Board of Directors, Washington, DC. http:// documents.worldbank.org/curated/en/772371468325192991/pdf/675810BR0REVIS0Official0Use0Only090.pdf. ———. 2014. Global Partnership for Social Accountability: Operations Manual. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/438531483708114727/Global-Partnership-for-Social- Accountability-GPSA-operational-manual. ———. 2015. “Third Global Call for Proposals. Part 2: Main Application Form.” Global Partnership for Social Accountability, Washington, DC. Available at: https://www.thegpsa.org/grants/3rd-call-proposals-closed ———. 2019a. “Grant Application Guidelines.” Global Partnership for Social Accountability, Washington, DC. https://www.thegpsa.org/Data/gpsa/files/field/documents/guidelines_gpsa_cfp4_applicationf.pdf ———. 2019b. “Global Orientation Session. Fourth Global Call for Proposals.” Presentation prepared for the online Global Orientation Session for the GPSA’s Fourth Call for Proposals on April 24th, 2019, Washington, DC. https://www.thegpsa.org/Data/gpsa/files/field/documents/orientation_session_4th_cfp_gpsa_final.pdf 30 Capacity and Implementation Support Area: Portfolio Performance Review © 2020 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 www.worldbank.org Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Suggested Citation Poli, Maria, and Maria F. Guerzovich. 2020. “Capacity and Implementation Support Area: Portfolio Performance Review.” Global Partnership for Social Accountability Note 15. World Bank, Washington, DC. Design Claudio Mendonca For more information on the Global Partnership for Social Accountability, visit https://www.thegpsa.org, the GPSA Knowledge Platform at https://gpsaknowledge.org/, or contact us at gpsa@worldbank.org. 31