POVERTY & EQUITY NOTES
TOOLS & METHODS OCTOBER 2020 · NUMBER 35
Monitoring Poverty and Equity for Development Effectiveness
Utz Johann Pape, Nobuo Yoshida, Silvia Malgioglio
Monitoring progress towards the World Bank Group’s (WBG) goals, eradicating poverty and boosting shared prosperity, is
essential to adjust the Bank’s portfolio investment decisions and to effectively implement projects. The Poverty & Equity
Global Practice has developed a suite of tools to allow cost-effective monitoring of portfolios and projects to enhance their
development effectiveness.
Having reliable, timely data on poverty and to the midterm point of the project by 40 percent.
inequality is critical for economic and Raimondo (2016) 2 similarly found that the quality
distributional analyses, but also for creating and of monitoring systems significantly correlates with
implementing effective development positive project outcomes. Especially in shock
strategies. Tracking microeconomic data can contexts, it is important to continuously collect
provide an accurate picture of a given development crucial data to ensure that projects continue
context by measuring changes in key reaching beneficiaries and quickly adapt to
socioeconomic indicators in near-real time. changing circumstances—for example, by
Tracking data is also critical for making evidence- including new beneficiaries.
based strategic decisions about resource allocation
and to help investments and projects reach The World Bank has developed an innovative
intended beneficiaries—often the poorest and rapid data collection and evidence-based
most vulnerable segments of the population. policymaking toolbox. The first suite of tools aims
to collect poverty and inequality data rapidly, and
Data collection is traditionally expensive and includes Rapid Response Phone Surveys (RRPS)
requires face-to-face access to respondents. Thus, and the Survey of Well-being via Instant and
it is often unattainable, especially when shocks due Frequent Tracking (SWIFT) methodology. The
to conflict, natural disasters, or even pandemics second suite of tools helps managers make
such as COVID-19 occur—situations where informed decisions related to portfolio allocation,
monitoring is most critical. project targeting, and project monitoring. These
include the Portfolio Footprint (PFF), the Project
Data collection is highly beneficial for project Targeting Index (PTI), and Iterative Beneficiary
performance and should not be neglected even Monitoring (IBM). These cost-effective tools utilize
if challenges arise. Legovini et al. (2015) 1 new technologies to monitor in real-time the
examined World Bank projects between 2005 and country context as well as project implementations.
2011 and found that having an impact evaluation The high-quality data and analysis generated by
increased the average quarterly disbursements up
1
Legovini, Arianna; Di Maro, Vincenzo; Piza, Caio. 2
Raimondo, Estelle. What difference does good
2015. Impact evaluation helps deliver development monitoring & evaluation make to World Bank project
projects (English). Policy Research working paper; no. performance? The World Bank, 2016.
WPS 7157; Impact Evaluation series. Washington, DC:
World Bank Group.
the tools help increasing accountability and changing socio-economic conditions due to
effectiveness of projects and portfolios. COVID-19.
Rapid Monitoring Tools Project and Portfolio Management Tools
Rapid Response Phone Surveys (RRPSs) can be The Portfolio Footprint (PF) examines the
implemented quickly by telephone to access at- subnational distribution of World Bank (WB)
risk populations during a crisis. RRPSs allow portfolio commitments and their relationship
stakeholders to swiftly collect household and firm- with indicators or need. The approach estimates
level data using phone interviews (Computer- WB portfolio amounts spent or committed at the
Assisted Telephone Interviews) to monitor crisis sub-national level to understand spatial
impact, inform and assess mitigation measures. distribution of investments. It largely focuses on
RRPSs allow for near real-time survey data investments that can be disaggregated at low
collection and can reach respondents in remote geographic levels, usually leaving out centralized
areas. The main limitation of RRPSs is data spending or budget support projects. Once that
representativeness, as they can exclude poorest estimation is completed, the commitments are
and hardest-to-reach populations. This risk can be overlaid with different welfare and sectoral
mitigated by using statistical sampling weights, or indicators. Portfolio Footprints have been used to
by establishing statistical and physical help identify potential disconnects between WB
infrastructure to deploy RRPSs. For COVID-19, investments and needs, and informs discussions
RRPSs have proven invaluable to monitor socio- about future geographic investment targeting.
economic impacts, which emphasizes the need to
establish such infrastructure. The Project Targeting Index (PTI) is an
evidence-based targeting method that maps
The Survey of Well-being via Instant and priority geographical areas based on
Frequent Tracking (SWIFT) is a poverty subnational poverty and socioeconomic
projection approach that allows users to indicators. The PTI method consists of identifying
estimate poverty through 10-minuete appropriate, quantifiable criteria— poverty rates,
interviews. SWIFT does not collect direct income number of poor, and customizable, quantifiable
or consumption data, but rather collects poverty key country indicators such as food insecurity,
correlates—such as household size, asset agriculture, education, incidence of violence, or
ownership, or education levels—and converts them service accessibility—and aggregating these
into poverty statistics using estimation models. multiple statistics into a single indicator. The
SWIFT estimations use machine learning framework is flexible and can accommodate
techniques and survey-to-survey imputation based specific project objectives by including other
on the latest available household budget survey criteria as needed. After calculating a single PTI
data. These techniques make SWIFT-imputed data index, geographical areas (such as districts or
estimates comparable with official poverty, counties) can be ranked in order of priority using
inequality, and income data. The WBG has already the PTI score. PTI analysis has been used to help
applied this cost-effective and timely methodology guide project teams in selecting project sites, and
in a large number of countries and contexts. The can introduce greater transparency and objectivity
SWIFT methodology is being adapted for use in into the selection process.
high frequency surveys to monitor rapidly
October 2020 · Number 35 2
Iterative beneficiary monitoring (IBM) is a from administrative data, but it is highly valuable
method for gathering low-cost, iterative information for future project teams. Therefore,
feedback on project performance directly from identifying priority areas should not be based not
beneficiaries. IBM aims to bridge the disconnect solely on the PTI. Project teams should record any
between project planning and what happens on challenges or issues they face in statistical and
econometric methods to inform future project
the ground. In contrast to traditional project
design and targeting decisions.
effectiveness measurement tools, IBM produces
short reports and focuses on diagnosing specific
barriers to effective project implementation. The Examples of Tool Usage
IBM methodology is designed around phone
collection of multiple rounds of small-scale data
from a sample of project beneficiaries. It uses fewer RRPS proved to be critical during and after the
research questions and smaller samples than 2014 Ebola outbreak, as well as for the COVID-
standard project M&E, making data collection 19 pandemic. To measure the economic impact of
costs minimal. IBM offers an agile, problem- Ebola on Liberian households, the World Bank, with
oriented feedback loop that allows management the Liberian Institute of Statistics and Geo-
teams to adjust implementation activities on-the- Information Services and the Gallup Organization,
conducted several rounds of high frequency
fly.
mobile phone surveys. RRPS made it possible to
collect critical data about the economic situation to
Conclusions and Limitations
inform policy decisions. For example, it was found
that more than 90 percent of households were
forced to employ economic coping strategies
(Figure 3), and that this number did not increase
The PTI can serve as a powerful tool for project
between rounds 4 and 5 of the survey. Similarly, the
planning and monitoring. It draws on a battery of
World Bank deployed more than 100 RRPSs around
objective indicators to provide insights into
the world to measure behavioral responses to
geographical priority areas based on certain
COVID-19 as well as socio-economic impacts on
Country Management Unit (CMU) criteria or
households, firms, and communities.
objectives. Applying this evidence-based approach
to inform spatial targeting helps improve cost
Figure 1: Percent of households using economic
effectiveness of the Bank portfolio and maximize its
coping strategies, Liberia, March 2020
intended benefits.
The PTI should not be a static tool but rather
should be adjusted according to changing
circumstances on the ground or as project
objectives evolve. In addition, after project teams
identify priority areas, they may face many
unexpected challenges and unknown issues in a
project site. Future project teams should easily be
able to draw lessons from these past experiences.
These critical pieces of contextual information will
be analyzed through case studies and, if
appropriate, through those priority areas. Such
context-specific information cannot be The COVID-19 outbreak threatens to push as
systematically collected by surveys or compiled many as 100 million people into poverty, but
October 2020 · Number 35 3
taking rapid pulse of these shifts is extremely Under each pillar, users can adapt the tool to
challenging. SWIFT is being used in COVID-19 projects by choosing indicators and defining
RRPS to measure the impact of COVID-19 on the weights. Providing an equal weight to each
poor and vulnerable, and to continuously estimate indicator shows that Morocco's priority areas
rapidly changing poverty and income status. As of concentrate in the southwest provinces and the
August 2020, around 15 countries are expected to coastal region around Casablanca. An online
include SWIFT modules in their questionnaires. A dashboard was created to allow project teams to
recent survey in St Lucia shows important overlay priority target areas with reported COVID-
differences in access to basic services between the 19 cases as well as regional socio-economic
poor and the non-poor (Figure 2) that can indicators.
otherwise be overlooked when the population is
not observed through a poverty lens. In addition, an IBM intervention was
implemented to improve the development
Figure 2: Access to basic needs by poverty effectiveness of an apprenticeship training for
status, St Lucia 2020 the Supporting the Economic Inclusion of Youth
project. The objective of this activity was to
increase the number of beneficiaries who complete
the training to optimize the existing apprenticeship
system. The Word Bank team developed the IBM
questionnaire to identify monitoring and
evaluation obstacles, such as difficulty in
identification of beneficiaries, lack of contextual
information, and poor quality of the data. The IBM
report was shared with the project team, which
took measures to address the issues. Another
In Morocco, World Bank country management
round of data collection will assess improvements
shifted to a living strategy (Learning and
and continue to monitor project implementation.
Adaptive CPF) to adapt to unforeseen and
changing circumstances by using several
By collecting data using phones, the IBM
integrated tools from the toolbox. A PF analysis
answers COVID-19 monitoring challenges,
was implemented to inform management on
helping to maintain the link between
provincial allocations of portfolio commitments
institutions and beneficiaries. The first COVID-19
relative to different indicators of need like poverty.
IBM was conducted in Morocco for the Municipal
The PF highlighted that despite positive correlation
Performance Program, which aims to improve the
between the number of poor and total expenditure
delivery of services and infrastructure for residents
per province, the portfolio allocation varied across
of Moroccan cities. The IBM assessed disruptions
provinces with similar poverty levels, with a few
and delays in the delivery of services, as well as
provinces receiving substantially lower allocations.
municipal responses, to help understand project
challenges and improve crisis preparation.
A PTI for Morocco complemented the PF by
identifying priority areas based on quantitative
analysis. In Morocco, the PTI was constructed
Conclusions
around 3 pillars:
(a) Promoting private sector job creation.
(b) Strengthening human capital.
The World Bank’s Poverty and Equity Global
(c) Promoting inclusive and resilient territorial
Practice’s agile and responsive tools for poverty
development.
data collection and analysis allow development
actors to make evidence-based decisions even
October 2020 · Number 35 4
when resources are limited. The expansion of However, the tools can be deployed even more
mobile network coverage and advent of new efficiently and effectively when statistical
technologies, including cloud-based computing, infrastructure is already in place. While most
continue to multiply the possibilities for data tools can also be deployed rapidly in a crisis, data
gathering and analysis. High cost and complexity quality can benefit, and costs further lowered, if
were often the cause of low frequency monitoring physical and statistical infrastructure for data
of portfolios and projects. Development of this new collection already exists. For example, building
toolbox addresses these obstacles, allowing representative phone sampling frames based on
frequent, high-quality, low-cost monitoring. These national household surveys will improve the
tested methodologies allow for evidence-based representativeness of RRPS. Establishing a survey
decision-making and employ a poverty and equity infrastructure for SWIFT, for example, can speed
lens to help investment portfolios and projects data collection to gain insights on poverty and
work more effectively. equity impacts when a crisis such as COVID-19
occurs. IBM can be readily adjusted to a new crisis-
The tools are particularly applicable in context once it is already in place.
challenging situations and fragile contexts. With
traditional methodologies often requiring face-to- ABOUT THE AUTHORS
face interaction, the new toolbox can be largely Utz Johann Pape is a Senior Economist in the
deployed remotely. In addition to reducing costs World Bank’s Poverty and Equity Global Practice.
and allowing more frequent monitoring, we can
use the methodologies in countries with severe Nobuo Yoshida is a Lead Economist in the World
access restrictions; for example, due to insecurity or Bank’s Poverty and Equity Global Practice.
risk of contagion during shocks such as the COVID-
19 pandemic. Silvia Malgioglio is a Social Scientist in the World
Bank’s Poverty and Equity Global Practice.
Figure 3: Tool implementation by country (darker color = more projects) as of FY20
This note series is intended to summarize good practices and key policy findings on Poverty-related topics. The views
expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank, its board or its
member countries. Copies of these notes series are available on www.worldbank.org/poverty
October 2020 · Number 35 5