Hungary:
Measuring
Inclusive Growth
for Enhanced Development Impact
Targeting and Monitoring of EU Co-funded Social Inclusion Activities
at the Subregional Level
Disclaimer
This report is a product of the International Bank for Reconstruction and Development / the World
Bank. The findings, interpretation, and conclusions expressed in this paper do not necessarily reflect
the views of the Executive Directors of the World Bank or the governments they represent. The
World Bank does not guarantee the accuracy of the data included in this work.
This report does not necessarily represent the position of the European Union or the Hungarian
Government.
Copyright Statement
The material in this publication is copyrighted. Copying and/or transmitting portions of this work
without permission may be a violation of applicable laws.
For permission to photocopy or reprint any part of this work, please send a request with the
complete information to either: (i) Ministry of Human Capacities (Arany János u. 6-8, 1051, Budapest,
Hungary); or (ii) the World Bank Group Romania (Vasile Lascăr Street, No 31, Et 6, Sector 2,
Bucharest, Romania)
This report was prepared under the Advisory Services Agreement for Policy Advice to Support the
Implementation of the National Social Inclusion Strategy of Hungary signed between the World Bank
and the Ministry of Human Capacities on March 17, 2015.
1
Acknowledgments
The team would like to express its gratitude to peer reviewers Joost de Laat, Katalin
Szatmári, Maria Beatriz Orlando, and Andor Ürmös for their technical inputs and comments
during the review process, as well as to Christian Bodewig, Alina Nona Petric, Manuel
Salazar, Kenneth Simler, and Rob Swinkels for their helpful comments. The team is
particularly grateful to counterparts from the Government of Hungary, in particular Csaba
Andor and László Ulicska from the Ministry of Human Capacities, and András Kezán from the
Hungarian Central Statistical Office, for their guidance, support, and helpful inputs
throughout the preparation of this report. The team was supported by András Tamás Torkos,
Isadora Nouel, and Andrei Zambor. Editorial work for this report was provided by Lauri
Scherer.
This report was prepared by a World Bank team composed of Céline Ferré, Sándor
Karácsony, Ádám Kullmann, Valerie Morrica, and Nóra Teller, under the guidance and
supervision of practice managers Nina Bhatt and Andrew D. Mason.
2
3
Table of Contents
Acknowledgments ............................................................................................................ 2
Abbreviations and acronyms ............................................................................................. 6
Executive summary ........................................................................................................... 8
1. Introduction............................................................................................................. 10
2. Social exclusion: Frozen in a state of lagging behind ................................................. 12
2.1. Monetary poverty and social exclusion ........................................................................ 12
2.2. Capturing the dimensions of social exclusion ................................................................ 13
BOX 1. Using a multidimensional approach to compare social inclusion outcomes between
Roma and non-Roma in Central Europe ................................................................................. 13
A recent report published by UNDP (Ivanov and Kagin, 2014) analyzes the difference in outcomes
between the Regional Roma Survey (RRS) and the UNDP vulnerable groups survey of 2004.
Instead of using a single indicator of poverty (such as income or consumption), the report uses a
multidimensional approach to poverty (the MPI). Among Roma, the MPI decreased substantially
between 2004 and 2011 in Bulgaria and Romania, primarily due to a decline in the number of
“poor” households (that is, those with 5 to 7 deprivations in the MPI). “Severe poverty” (more
than 7 deprivations) decreased significantly only in Romania................................................... 13
Looking at the different dimensions of poverty allowed Ivanov and Kagin to identify the key
drivers of poverty reduction between 2004 and 2011; education and living conditions have
improved, while fundamental rights and access to the labor market have worsened. The analysis
of multidimensional versus monetary poverty metrics also reveals that the decline is similar in
both metrics in Bulgaria, while the decline in monetary poverty in Romania is stronger than in
multidimensional poverty. ...................................................................................................... 13
Source: Ivanov and Kagin (2014). ............................................................................................ 14
2.3. What are these dimensions? ........................................................................................ 14
2.4. Can the “depth” of social exclusion be measured? ........................................................ 16
3. Measuring social exclusion: Global and local lessons for Hungary ............................. 17
3.1. What has been done in Hungary so far? ....................................................................... 17
3.1.1. The HCSO used census-based indicators on education and unemployment to map
marginalized communities ............................................................................................................ 17
3.1.2. Targeting tool for complex social inclusion interventions designed by HCSO .............. 18
BOX 2. Identifying pockets of poverty at the subregional level: Mapping Romania’s
marginalized communities .................................................................................................... 22
3.1.3. Monitoring indicators from NSIS ................................................................................... 23
BOX 3. Europe-wide efforts to monitor national Roma inclusion strategies .......................... 24
3.1.4. Best predictors of AROP ................................................................................................ 25
3.2. International experience with measuring social inclusion outcomes at the national level
28
3.2.1. Choice of dimensions .................................................................................................... 29
3.2.2. Number of indicators..................................................................................................... 29
3.2.3. Composite versus non-unified indicators...................................................................... 33
3.2.4. Disaggregation level and frequency at which indicators are collected......................... 33
4
4. What indicators should be used in Hungary? ............................................................ 34
4.1. Indicators that are suited to track progress toward social inclusion ............................... 34
4.2. Issues and dilemmas regarding collecting and tracking indicators to monitor progress on
social inclusion in Hungary ...................................................................................................... 35
4.3. Policy objectives and considerations for developing an indicator set ............................. 40
4.4. The proposed indicator set for Hungary........................................................................ 41
4.5. Confirming the validity of indicators............................................................................. 43
5. How to design a tracking tool that combines social inclusion indicators with project
data? .............................................................................................................................. 45
5.1. Proposal for a tool to track development policy interventions in a social inclusion context
45
5.2. Topics and hypothesized impact mechanism of selected social inclusion measures ....... 46
5.3. Next steps toward an improved tracking tool ............................................................... 53
5.4. Recommendations for targeting and monitoring EU–co-funded social inclusion
investments in the 2014–2020 period...................................................................................... 55
6. References ............................................................................................................... 57
Annex I. Summary of TÁMOP 5 measures and hypothesized impact mechanisms ............ 59
Annex II. Relative variability of selected indicators .......................................................... 67
Annex III. Mapping for results in Hungary........................................................................ 69
5
Abbreviations and acronyms
AROP At risk of poverty
AROPE At risk of poverty and social exclusion
CLLD community-led local development
ECA Europe and Central Asia
ECD early childhood development
ESF European Social Fund
ESIF European Structural and Investment Funds
EU European Union
EU-SILC EU Survey for Income and Living Conditions
FRA Fundamental Rights Agency
GIS geographic information system
HCSO Hungarian Central Statistical Office
HDI Human Development Index
LEP Local Equal Opportunity Plan
M&E monitoring and evaluation
NGO nongovernmental organization
NRIS National Roma Inclusion Strategy
NSIS National Social Inclusion Strategy
OECD Organisation for Economic Co-operation and Development
OP Operational Program
OPHI Oxford Policy and Human Development Initiative
PISA Programme for International Student Assessment
TÁMOP Társadalmi Megújulás Operatív Program (Social Renewal Operative Program)
TEIR Területfejlesztési és Területrendezési Információs Rendszer (Territorial
Development and Spatial Planning Information System)
6
UNDP United Nations Development Programme
7
Executive summary
Social exclusion is the process by which individuals or entire communities of people are
systematically blocked from or denied full access to various rights, opportunities, and
resources that are normally available to members of a different group, and which are
fundamental to being socially integrated into that particular group—housing, employment,
education, health care, civil participation, and due process. It includes (but is not limited to)
poverty and material deprivation, and can be reflected in multiple dimensions of everyday
life.
There have been many attempts to measure the extent to which a community or an
individual is socially excluded, both in Hungary and in other countries. For example, the
Central Statistical Office of Hungary (HCSO) used the 2001 and 2011 population censuses to
produce a map of segregated census blocks using two indicators related to unemployment
and education. The Hungarian government defined the 33 most disadvantaged microregions
so as to facilitate the targeting of European funds; the government has also introduced a
monitoring framework of the National Social Inclusion Strategy (NSIS). In addition, HCSO and
the World Bank have jointly identified the 2005 at-risk-of-poverty (AROP) rates at the
microregional level. There have also been numerous global attempts that provide lessons on
(i) how to choose which dimensions of social exclusion to measure; (ii) the number of
indicators to collect; (iii) whether to use composite versus non-unified indicators; and (iv)
the level of disaggregation and frequency of measurement.
Against this background, this report offers an approach to developing a set of indicators that
should be collected to track progress on social inclusion at the subregional level. These
indicators should (i) be available at a highly disaggregated geographical level (ideally the
microregion or district); (ii) be affected relatively quickly by changes in social
inclusion/exclusion; and (iii) be available on a regular basis (ideally annually). The report
identifies 22 such indicators that cover the thematic areas of (i) monetary poverty and
material resources; (ii) employment/labor; (iii) education and health; and (iv) housing and
living conditions.
The report also explores whether there is an opportunity to leverage this approach to
develop a tracking/monitoring tool that could combine locally available social exclusion data
with project information from EU co-funded social inclusion projects. Such a tool could serve
as a feedback mechanism for policy makers on whether funds are spent in the highest-need
areas. The underlying analysis—which is based on the evaluation of impact mechanisms of
selected EU co-funded measures within the 5th Chapter of the Social Renewal Operational
Program (TÁMOP 5) between 2007 and 2012—demonstrates that it is possible to develop
this feedback mechanism based on currently available data in the case of social inclusion
challenges and investments aimed at responding to them. However, the analysis also finds
that such tracking opportunities could be substantially improved by regularly collecting
project information on (i) the social challenge that the call for proposal or measure intends
8
to address; (ii) the time frame in which the call for proposal is launched and the projects are
financed; (iii) number of beneficiaries in the project; and (iv) the exact location of the
implementation.
The report encourages policy makers to add additional dimensions to routinely collected
data when designing the calls for proposals for future EU co-funded social inclusion
measures. These dimensions should correspond to the proposed indicator set and should
also be linked to EU 2020 targets. At a minimum, these dimensions should include: (i)
constrained school careers and/or low education levels; (ii) child poverty; (iii) crime or
deviance; (iv) low employment or activity levels; (v) gaps in/lack of (selected) quality
social/human service delivery for various target groups (for example, at the local level, home
care, social, child protection, youth welfare service, services for people with disabilities); (vi)
indebtedness or excessive housing costs; (vii) housing segregation in Roma/poor
neighborhoods; and (viii) discrimination of vulnerable groups.
9
1. Introduction
The objective of this paper is to develop a way to monitor and track progress on social
inclusion of vulnerable groups in Hungary, particularly among marginalized Roma
communities. This approach will enable stakeholders to track the status of social inclusion at
the subregional level, and can serve as a feedback mechanism on whether projects
cosponsored by the European Structural and Investment Funds (ESIF) are sufficiently
targeted to disadvantaged areas.
The paper builds on various Hungarian attempts to draft indicator sets to find and
subsequently gear EU-funded projects toward areas with the poorest social inclusion
outcomes. The Central Statistical Office of Hungary (HCSO) used the 2001 and 2011
population censuses to produce a map of segregated census blocks using a combination of
two indicators related to unemployment and education. In 2007 the Hungarian government
defined the 33 most depressed microregions based on a set of approximately 30 economic,
social, and infrastructure indicators, and used a similar approach, with minor changes, for
the 2014–2020 period as well. Finally, after determining the best set of poverty rate
predictors, World Bank and HCSO experts have jointly developed a poverty map at the
microregional level using 2005 data. While these modeling exercises all tried to link poverty,
segregation, or regional disparities with a set of predictors so as to identify factors that
influence these outcomes, none were aimed at tracking progress toward poverty reduction
or social inclusion at a subregional level.
The second part of this report takes stock of different exercises undertaken with Hungarian
data to map, target, track, and monitor some aspects of social exclusion at different levels of
disaggregation. We present four such attempts to: (i) map marginalized communities; (ii)
target the most disadvantaged microregions; (iii) track selected social inclusion goals; and
(iv) model at risk of poverty (AROP) rates at the microregional level. The report then
examines what has been done in international practice in terms of selecting and collecting
indicators that measure social inclusion. This part relies heavily on Labonté et al. (2011), who
conducted a meta-analysis of such attempts in the United Kingdom, France, Australia, and
across Europe. These examples from around the world focused primarily on collecting
indicators of social exclusion at the national level, and not at a highly disaggregated
subnational level (which is the objective of this exercise). In addition, international attempts
to measure progress toward social inclusion did not attempt to track progress on a regular
basis, be it annually or even every few years. Finally, the report describes the method and
process of indicator selection for Hungary. We also identify gaps and articulate remaining
research needs.
In particular, this report aims to design the best possible set of indicators to be collected on
a regular basis (ideally annually) at the lowest geographical level to monitor progress toward
social inclusion. The optimal set of indicators should (i) accurately identify the different
dimensions of social exclusion; (ii) be available at a geographically disaggregated level; (iii)
10
be collected regularly, preferably on an annual basis; and (iv) be “dynamic”—that is, respond
to local development dynamics as demonstrated by relative variability.
The concluding section summarizes the dilemmas associated with dynamically measuring
social change in the Hungarian context, and proposes development project parameters that
should be continuously followed in order to enable tracking and (limited) monitoring. In
addition to providing inputs and making suggestions for future discussions on how to match
local level information on social development and development projects co-funded by the
European Union (EU) in the 2014–2020 period, the report also outlines a logical approach
and proposes a tool to enable the parallel tracking of development activities in conjunction
with social inclusion trends at the local level through a geographic information system (GIS).
Such a tool can provide continuous feedback for policy makers on whether investments are
geared toward the highest need areas. The approach underlying the proposed tool draws on
the recently completed evaluation of social inclusion investments conducted under the
TÁMOP 5.
11
2. Social exclusion: Frozen in a state of lagging behind
Social exclusion is the process by which individuals or entire communities are systematically
blocked from or denied full access to various rights, opportunities, and resources that are
normally available to members of a different group, and which are fundamental to being
socially integrated into that particular group—housing, employment, education, health care,
civil participation, and due process (Levitas et al., 2007). The term “social exclusion,” which
describes social disadvantage and being relegated to the fringes of society, first arose in
France in the 1970s in a context of radical economic restructuring and subsequent concerns
over risks to social cohesion and stability (Silver, 1994).
Social inclusion, on the other hand, consists of improving the ways individuals and groups
can take part in society, and especially pertains to improving the ability, opportunity, and
dignity of people who are disadvantaged on the basis of their identity (for example, ethnic
affiliation). Social inclusion and exclusion require an analytical framework that uncovers the
underlying causes of poverty and deprivation. It exposes the multidimensional nature of
deprivation and scrutinizes the correlates of poverty, be they lack of schooling, constrained
labor market participation, poor health, or residential segregation. It underscores that
deprivation arising from social exclusion tends to occur along multiple axes at once, so that
policies that improve just one of these axes of deprivation—such as improved access to
education—will not release the grip of others (World Bank, 2013).
2.1. Monetary poverty and social exclusion
It is important to note that while social inclusion may well be about reducing poverty, it is
often about much more; in some cases, it is not even about poverty at all. For example, the
Middle East protest movements that have been grouped together as the Arab Spring have
been fueled in part by middle-class citizens’ demands for greater inclusion in public decision
making and that political leaders be held more accountable (World Bank, 2013).
Nevertheless, given the fact that extreme and chronic poverty often overlap with social
exclusion, monetary poverty has long been used as a proxy for social exclusion, and there
has been much work done to standardize and improve poverty measures, mainly driven by
work conducted by the World Bank (see Deaton, 1997; Ravallion and Bidani, 1994;
and Ravallion, 1992). Poverty in most cases is considered in absolute terms: the share of
people living below a set threshold that would ensure households have access to basic
minimums—one- and two-dollar-a-day poverty lines, for instance.
In other cases, including for Hungary and all other EU member states, poverty is considered
in “relative terms”—that is, the share of people living with less than a percentage of the
mean or medium income. According to the EU 2020 1 indicators on poverty and social
1
Europe 2020 is the EU’s growth strategy. It is based on three priorities: employment, productivity, and social
cohesion. The EU has set five objectives related to employment, innovation, education, social inclusion, and
12
exclusion, EU member states are required to measure a headline indicator of people at risk
of poverty and social exclusion (AROPE) (EUROSTAT, 2015). This composite indicator shows
the number of people affected by at least one of three forms of poverty: monetary poverty
(inadequate income); material deprivation; or low work intensity (limited participation in the
labor market). People are AROPE if they suffer from at least one of these. Monetary poverty
is defined as earning less than 60 percent of the median income in the country of residence.
The material deprivation rate is an indicator that expresses the inability to afford items
considered by most people to be desirable or even necessary for an adequate life. Finally,
low work intensity is defined as the number of persons living in a household with a work
intensity below a threshold set at 20 percent.
2.2. Capturing the dimensions of social exclusion
More recently, additional measures of social inclusion have been developed to include
dimensions beyond economic status. Amartya Sen has developed the Human Development
Index (HDI), which gives equal weight to three non-monetary indicators (life expectancy,
literacy, and infant mortality). The HDI was later modified to include additional aspects of
poverty, in particular standards of living, which led to the Multidimensional Poverty Index
(MPI) that was developed by the Oxford Policy and Human Development Initiative (OPHI)
and the United Nations Development Programme (UNDP). Like the HDI, the MPI divides
poverty along three dimensions—health, education, and living standards—but uses 10
indicators instead of 3. Health is measured by child mortality and nutrition; education is
measured by years of schooling and school attendance; and living standards are measured
through access to cooking fuel, toilets, water, electricity, floors, and asset ownership. The
MPI also gives equal weight to the three main dimensions (health, education, and living
standards), which means that each of the two indicators for health have a weight of 1/6;
each of the two indicators for education have a weight of 1/6; and each of the six indicators
for living standards have a weight of 1/18.
BOX 1. Using a multidimensional approach to compare social inclusion outcomes between
Roma and non-Roma in Central Europe
A recent report published by UNDP (Ivanov and Kagin, 2014) analyzes the difference in outcomes
between the Regional Roma Survey (RRS) and the UNDP vulnerable groups survey of 2004. Instead of
using a single indicator of poverty (such as income or consumption), the report uses a
multidimensional approach to poverty (the MPI). Among Roma, the MPI decreased substantially
between 2004 and 2011 in Bulgaria and Romania, primarily due to a decline in the number of “poor”
households (that is, those with 5 to 7 deprivations in the MPI). “Severe poverty” (more than 7
deprivations) decreased significantly only in Romania.
Looking at the different dimensions of poverty allowed Ivanov and Kagin to identify the key drivers of
poverty reduction between 2004 and 2011; education and living conditions have improved, while
climate/energy to be reached by 2020. Each member state has adopted its own national targets in each of
these areas. The strategy is underpinned by concrete actions at EU and national levels.
13
fundamental rights and access to the labor market have worsened. The analysis of multidimensional
versus monetary poverty metrics also reveals that the decline is similar in both metrics in Bulgaria,
while the decline in monetary poverty in Romania is stronger than in multidimensional poverty.
Source: Ivanov and Kagin (2014).
2.3. What are these dimensions?
The concept of poverty—whether absolute, relative, or multidimensional—does not fully
capture the different facets of social exclusion. One can be relatively wealthy (and not
captured by a poverty index) and still experience social exclusion; for example, as a result of
discrimination.
A measure to more fully capture all dimensions of social exclusion involved including all
potential dimensions in which social exclusion may appear, be it material resources, income,
housing, education, access to employment, or opportunities to participate in the local
economy. To this end, Atkinson and Marlier (2010) and Labonté et al. (2011) have reviewed
the existing theoretical and empirical literature on social exclusion. Labonté et al. (2011)
have defined eight core dimensions along which social exclusion is likely to happen:
1. Income and material resources
2. Employment
3. Education and skills
4. Affordable and adequate housing
5. Health
6. Social resources and networks
7. Community resources and civic participation
8. Personal safety2
Material resources are the bedrock of social exclusion (when they are lacking) or inclusion
(when they are adequate for both basic needs and normative social activities). The
employment domain is particularly relevant, since its lack has adverse effects on material
resources as well as on social participation (social networks, support). Education and skills
provide capacity to participate socially and to engage in more secure forms of employment.
Housing adequacy, affordability, and security together emerge as an important subset of
material resources that extend beyond the basic need for shelter.
The health domain is also relevant, since adverse health conditions can affect productivity
levels, while having a disability can lead to social exclusion. Regarding the social resources
2
While earlier versions of this list included discrimination, it was dropped in the most recent version.
14
domain, institutionalized persons or those separated from their families generally have
reduced social resources and are at greater risk of exclusion. On the other hand, having
social resources (social networks, or the density of one’s social life; social support, or the
quality of what those networks offer; and opportunities for different forms of social
participation) are considered processes of inclusion. Community resources are identified as a
separate domain and encompass access to community services and opportunities for
political and civic participation. Finally, the personal safety domain refers to exposure to
crime or discrimination that may affect or prevent individuals from participating in
economic, social, civic, or political activities.
The conceptual framework proposed by Labonté et al. (2011) overlaps with the framework
developed by the World Bank (2013) to identify paths toward social inclusion. Individuals
and groups want to be included in three interrelated domains: markets, services, and spaces
(see Figure 1). The three domains represent both barriers to and opportunities for inclusion.
Just as different dimensions of an individual’s life intersect, so do the three domains.
Intervening in one domain without considering the others is likely to hamper the success of
inclusion policies and programs.
Figure 1. Working toward social inclusion—a framework
Source: World Bank (2013).
These two frameworks define the core set of expectations and priorities that should be
included in an indicator set aiming to measure the various dimensions of social exclusion.
Therefore, the final set of indicators that will be presented in Section 3 relies on these two
analytical frameworks and divides indicators that measure progress toward social inclusion
15
between various dimensions: monetary poverty and material resources, access to the labor
market, education and health, housing and living conditions, and community setting.3
2.4. Can the “depth” of social exclusion be measured?
In addition to social exclusion’s “lateral” aspects discussed in the previous section, intensity
or “depth” of social exclusion is also important. Miliband (2006) recommends considering
social exclusion in three “dimensions”—width, depth, and concentration. Wide exclusion
refers to the large number of people excluded on a single or small number of indicator(s).
Concentrated exclusion refers to the geographic concentration of problems and to area
exclusion. Deep exclusion refers to those excluded on multiple and overlapping dimensions.
For example, the Regional Roma Survey (UNDP, World Bank, and EC, 2011) finds that the
labor market outcomes among Hungarian Roma women are significantly worse than those
among Roma men: only 13 percent of Roma women have a job, compared to 34 percent of
Roma males and 51 percent of non-Roma females living nearby. This highlights a significant
gender gap in terms of labor market access by Roma jobseekers. 4
Any choice of indicator set should be able to address these three aspects of social exclusion;
that is, the indicators should capture a broad swath of dimensions (social processes) that
characterize social exclusion; be capable of disaggregation to geographic areas (ideally local
areas or even neighborhoods); and be linkable to specific individuals or groups.
3
Due to data limitations, the final set of indicators does not include social, political, and cultural resources, or
civic and political participation.
4
The accompanying handbook, The People Behind the Numbers: Developing a Framework for the Effective
Implementation of Local Equal Opportunity Plans, elaborates how complex social problems emanating from
overlapping disadvantages can be broken down into individual elements, and which of these elements are best
addressed at the local level.
16
3. Measuring social exclusion: Global and local lessons for
Hungary
Any method of measuring social inclusion needs to take into consideration the experience of
earlier attempts to do so, both in Hungary and elsewhere in the world. These experiences
include attempts to target (identifying a country’s lagging areas that should be emphasized
when developing interventions) and monitor (determining what data should be regularly
collected to identify whether the gap is closing between the most disadvantaged and the
general population) social inclusion measures.
3.1. What has been done in Hungary so far?
Over the past few years, various entities in Hungary have developed various sets of
indicators to target those areas with the worst social inclusion outcomes. The HCSO used the
2001 and more recently the 2011 population census to produce a map of segregated census
blocks using two indicators related to unemployment and education. In parallel, the
Hungarian government defined the 33 most depressed microregions—based on economic,
social, and infrastructure indicators—and used a similar approach, with minor changes, for
the 2014–2020 period as well.
3.1.1. The HCSO used census-based indicators on education and unemployment to
map marginalized communities
Identifying marginalized areas based on data and evidence is the precondition for effectively
targeting marginalized groups, including Roma. Similar to the World Bank–led exercise in
Romania (see Box 2), socioeconomic indicators reflecting deprivation have also been shown
to effectively grasp the most excluded social strata in Hungary. The HCSO was tasked with
producing a map of segregated neighborhoods using census-block information from the
2001 population census, and then the 2011 census.
Segregated neighborhoods were defined as parts of settlements where segregation is
significantly higher than in other neighborhoods, and people face poor housing conditions
and have limited access to basic services (water, sewage, social and healthcare services, and
so on). The HCSO focused on two indicators that were closely linked to segregation: (i) a lack
of work-able adults with regular income from employment; and (ii) work-able individuals
with their highest school qualification below the 8th grade.
For the first full EU financial period (2007–2013), 5 data from the 2001 population census
were used, along with the following methodology: Urban areas were considered segregated
if at least 50 percent of active-age residents (between ages 15 and 59) were unemployed
and their highest school qualification did not exceed the 8th grade. A second tier consisted
of urban areas threatened by segregation, in which the above-mentioned indicator took up a
5
Abridged Version of the Urban Development Manual (State Secretariat for Regional Development and
Construction Ministry for National Development and Economy, March 2009).
17
value between 40 and 50 percent. In the case of the Budapest conurbation, urban areas
were considered segregated if the value of the above-mentioned indicator attained or
exceeded 35 percent; areas were considered to be threatened by segregation if the value of
the indicator fell within 25–35 percent.
In addition, data on social assistance uptake was used to more precisely define the scale of
segregation. Areas where the proportion of regular social aid beneficiaries attained twice
that of the city average (or the district average in Budapest) were considered to be
segregated. Areas where the proportion of regular social aid beneficiaries attained 1.7 to 2
times the city’s (district’s) average were considered threatened by segregation (EC, 2011).
For the current EU planning period (2014–2020), 6 data from the 2011 population census
were used, along with the same methodology, only the thresholds were modified. Urban
areas were considered segregated when at least 35 percent of the active-age residents (15–
59 years old) were unemployed and their highest school qualification did not exceed the 8th
grade. Urban areas where the aforementioned indicator took up a value between 30 and 35
percent were considered threatened by segregation. In the case of the Budapest
conurbation, urban areas were considered segregated if the indicator value attained or
exceeded 20 or 25 percent for inner and outer districts, respectively. Areas were considered
threatened by segregation if the indicator value fell within 15–20 or 20–25 percent for inner
and outer districts, respectively. Settlements with 200 to 2,000 inhabitants kept the same
threshold as before: 50 percent to be classified as a segregated neighborhood, and 40 to 50
percent to be considered as threatened by segregation.
Overview of indicators used
- One indicator related to low intensity of labor and one related to low level of education
- Household-level data from the 2011 population census
- Indicators available at the census-block level, but only every decade (10 years)
3.1.2. Targeting tool for complex social inclusion interventions designed by HCSO
Increasing territorial disparities at the microregional level (LAU1, formerly NUTS4) was
recognized as a major problem in the mid-2000s, partly as a result of social and ethnic
tensions in some of the most depressed microregions, especially in northeast Hungary. The
implementation of EU funds in the 2007–2013 financing period brought forth the need to
target lagging areas where social inclusion programs should be focused. In these areas,
“complex programs” were foreseen to be developed in groups of cca. 10–30 projects per
microregion, covering human services and infrastructure development, economic
development, transport and environmental infrastructure development, and urban
development. For example, in the area of human development, a designated pool of funds
6
See http://net.jogtar.hu/jr/gen/hjegy_doc.cgi?docid=A1200314.KOR.
18
and thematic calls from the Human Resource Development Operational Program (OP)
targeted these microregions in the 2007–2013 period.
As a result, the HCSO was tasked in 2007 with defining a new set of indicators that would
help target the most disadvantaged microregions. The HCSO had the opportunity to build on
already elaborated-upon sets of indicators to identify lagging regions since the end of the
1980s. It initially used discriminant analysis to determine the best set of indicators for
identifying depressed areas (1986–1993); subsequently switched to scoring models (1993–
2007); and finally moved to feature scaling (normalization) in 2014. While the set of
indicators was limited to about 10 until 1996, more recent years have seen an expansion in
the number and dimensions of indicators, reaching 20–30 indicators as of 2007 (Kezán and
Szilágyi, 2015).
During the 2007 exercise, a set of 31 economic, social, and infrastructure indicators were
defined. Some were drawn from the previous census (6 years earlier, in 2001); others came
from more recent data sources. The composite indicator was aggregated at the
microregional level, for all 175 microregions, which were in turn ranked according to their
final score and categorized into four groups of intervention (complex, medium, simple, and
none). Indicators were chosen according to the following criteria: (i) indicators needed to be
transparent, simple, reliable, and reproducible over time; and (ii) measurements needed to
be consistent, so that the new methodology was comparable to earlier ones. There was no
particular need to find indicators that would allow for tracking progress over time, as the
goal of the composite indicator was to identify ex ante the most disadvantaged regions.
In 2013, the 175 microregions were replaced by 198 districts (including the 23 districts of
Budapest), and the HCSO undertook once more the task of identifying Hungary’s most
depressed areas. The final set of indicators largely overlapped with those selected in 2007
(see Table 1) and identified 24 variables related to economic, social, and infrastructure
dimensions. 7
Table 1. List of indicators targeting the most disadvantaged microregions/districts—2007
and 2014
ECONOMIC INDICATORS 2007 2014
Number of active business organizations, per 1,000 inhabitants X X
Number of guest nights in tourism, per 1,000 inhabitants X
7
The legislative framework for these indicators is comprised of the following regulatory acts: (i) 290/2014 (XI.
26.) Government decree on the classification of the beneficiary districts; (ii) 105/2015 (IV. 23.) Government
decree on the classification of the beneficiary communities and the classification system conditions; (iii)
1247/2015 (IV. 23.) Government resolution on measuring changes in regional differences in development of
dynamic scorecard. An important aspect of the latter is the fact that it imposes the obligation to develop an
indicator system aimed at tracking social changes at different territorial levels on an annual basis.
19
Number of retail shops, per 1,000 inhabitants X X
Share of employment in agriculture, per total employment X
Share of employment in services, per total employment X
Growth rate of number of active business organizations X
Amount of local tax revenues of municipalities, per inhabitant X
Share of local tax revenues of municipalities, per total revenues X
Number of researchers and developers, per 1,000 inhabitants X
INFRASTRUCTURAL INDICATORS
Share of flats supplied with the potable water pipes network X
Share of flats supplied with the gas pipes network X
Share of flats supplied with regular waste collection X X
Length of sewer pipes, per potable water pipes X
Share of flats supplied with the sewer pipes network X
Road accessibility of closest county seat and microregional seat (minute) X
Road accessibility of closest motorway (minute) X
Share of paved roads, among all municipal roads X
Number of phone subscribers, per 1,000 inhabitants X
Number of cable TV subscribers, per 1,000 inhabitants X
Number of broadband internet subscribers, per 1,000 inhabitants X X
SOCIAL INDICATORS (1/2)
Share of flats with at least 3 rooms and built in the past 5 years X X
Share of non-comfort/substandard flats X
Average price of used flats X
Number of cars, per 1,000 inhabitants X
Number of migrants, per 1,000 inhabitants X X
Number of deaths, per 1,000 inhabitants X X
Amount of personal income tax basis, per inhabitant X X
X X
Share of population in municipalities with density above 120 persons per sq
20
km
Number of nursery and day care places, per 1,000 inhabitants below 2 years X
Average life expectancy at birth, for men X
Average life expectancy at birth, for women X
SOCIAL INDICATORS (2/2)
Rate of number of persons below 15 years, per number of persons above 60
X
years
Share of households without employment X
Share of persons with secondary school graduation, among persons above 18
X X
years
Number of persons receiving regular social aid from the municipality, per
X
1,000 inhabitants
Share of persons receiving regular social aid from the municipality or
X
unemployment
Share of persons receiving regular child protection aid, among persons below
X X
24 years
EMPLOYMENT INDICATORS
Share of registered jobseekers, among working-age population X X
Share of long-term registered jobseekers, among working-age population X X
Share of low-educated (not more than primary school) registered jobseekers X
Activity rate X
Source: HCSO (2007; 2014).
The final list of areas to be developed with complex programs changed significantly from
2007–2011 to 2014. Of the 33 microregions to be developed with complex programs in
2007, 9 were not identified as districts to be developed with a complex program. At the
regional level, the number of areas to be developed with a complex program decreased
most in South Transdanubia (from 8 to 4), and increased most in Northern Great Plain (from
8 to 12). At the county level, the number of areas to be developed with complex programs
decreased most in Somogy (from 4 to 2), and increased most in Hajdú-Bihar (from 1 to 4).
In addition, HCSO produced a map overlapping most disadvantaged municipalities and high
unemployment rates (see Figure 2), and the latter proves to be extremely correlated with
the former. Preferred districts and municipalities are defined according to very similar sets of
indicators as those defined for the disadvantaged microregions, with minor differences (for
example, life expectancy is used at the district level but not the municipality level, as data
21
would not be reliable). Therefore, the reason for having both is to be able to target
disadvantaged municipalities in districts that would not be classified as disadvantaged.
Figure 2. Most disadvantaged municipalities and unemployment rates
Legend
other
high unemployment rate
underdeveloped
Source: HCSO (2015); Kezán and Szilágyi (2015).
Overview of indicators used
- Set of 24–31 indicators related to four dimensions of economic and social depression: economic,
social, infrastructure, and education
- Different sources of data, including population census (2001 and 2011), local administrative data,
registers (for example, for tax revenues)
- Indicators available at the microregional (or district) level, with varying regularity (census every 10
years, tax revenues every year, and so on)
BOX 2. Identifying pockets of poverty at the subregional level: Mapping Romania’s
marginalized communities
In Romania, official municipal maps used by local government decision makers often do not show all
of the marginalized (Roma) settlements, as these are typically of an informal nature. As a result,
these communities’ needs are frequently overlooked in local development plans, including those
funded with EU structural funds. However, a new approach for spending EU funds—community-led
local development (CLLD)—allows EU-funded activities to be explicitly targeted to pockets of
22
deprived communities. To help the Romanian authorities design their CLLD program, the World Bank
developed a methodology that highlighted the location of severely marginalized communities for
each town and city in Romania—the Atlas of Urban Marginalized Areas in Romania, which presents
maps that are based on data from the 2011 population and housing census. This tool uses a typology
and corresponding indicators that are based on qualitative research and a review of earlier analysis
and indices of urban marginalization. The maps use indicators at individual, household, and dwelling
levels (such as education, employment, access to electricity, and so on) from the 2011 census. For
each of these indicators, the values at the urban census sector level (areas of typically about 200
people) are determined for all urban census sectors and an urban threshold is then defined as the
80th percentile. For each urban census sector, it is subsequently determined whether its value is
above the threshold for that indicator. If a census sector has a particular combination of indicators
that are above their threshold, it is regarded as disadvantaged or marginalized. For a number of
cities, maps at the census sector level were produced, displaying the typology of urban marginalized
areas as determined by applying this methodology to the census data. An additional series of maps
were produced that reflect information collected directly from urban authorities in Romania on
whether marginalized communities existed in their municipality, and if so, where. For a subset of
cities, maps were available from both the census-based method and the information gathered
directly from urban authorities. Using census data to identify urban marginalized communities is a
promising approach. However, further work is needed to assess its validity, including beyond urban
areas.
While maps of marginalized communities in urban areas can provide more finely tuned information
about subnational or within-city variations in poverty and marginalization, and improve resource
allocation, they cannot solve all development problems. To make the most of the information, it
must be complemented with local, context-specific knowledge that draws on local expertise and
community demand. In other words, after identifying the areas or populations in greatest need, it is
necessary to understand why these places are poor. The reasons are likely to vary from place to
place, and may include inadequate infrastructure, lack of economic activity, an insufficiently skilled
workforce, or other factors. While the right combination of approaches will vary by country, the
maps provide important information to help improve policies and programs to combat poverty and
social exclusion.
Source: World Bank (2015).
3.1.3. Monitoring indicators from NSIS
The Government of Hungary is the first EU member state to develop and submit a national
Roma inclusion strategy, which is titled National Social Inclusion Strategy: Extreme Poverty,
Child Poverty, Roma. Subsequently, the government developed a monitoring system
intended to track progress of Hungary’s National Social Inclusion Strategy (NSIS) in each of
the areas identified as priority development dimensions: (i) poverty and social exclusion,
with particular emphasis on the Roma population; (ii) reproduction of social exclusion; and
(iii) equal access to economic opportunities. Indicators are further classified into second- and
third-tier indicators, with each tier offering more in-depth understanding of each dimension
of social exclusion.
23
The core social inclusion indicators—which largely overlap with the EU 2020 social
indicators—are as follows:
1. Share of households living in poverty and social exclusion, or AROPE (EU 2020)
2. Share of households living in financial deprivation
3. Share of households with low work intensity
4. Employment rate
5. Share of children living in deep poverty
6. Share of 3–5 year olds enrolled in kindergarten
7. Share of children in the 6th grade that have parents with a maximum of 8 years of
completed primary school who have obtained a score of 1 or below on standardized
performance tests, compared to all 6th graders
8. Number of school dropouts by 10th grade, compared to the full cohort
Beyond social inclusion indicators, a program monitoring system has also been applied and is
being run by the state administration.
Overview of indicators used:
- Set of 8 core indicators related to social exclusion (poor economic conditions, employment,
education)
- Different sources of data, but many missing, and the monitoring system is under revision; data
collection by the HCSO is ongoing
- Few indicators available
BOX 3. Europe-wide efforts to monitor national Roma inclusion strategies
In addition to the progress the Government of Hungary has made in monitoring national social
inclusion outcomes on the basis of the NSIS, considerable efforts are underway to improve results-
based monitoring and evaluation (M&E) on Roma inclusion in other EU member states as well. Led by
the EU Fundamental Rights Agency (FRA), a working group of member states is developing a model
aimed at adopting a set of common rights-based indicators that can comprehensively assess Roma
inclusion efforts at the EU level. It applies a so-called structure-process-outcome (S-P-O) indicator
model that assesses (i) the legal and policy framework (structural indicators); (ii) the concrete
measures to implement it (process indicators); and (iii) the achievements as observed for the target
group(s), for example, Roma (outcome indicators).
A recent transnational workshop on M&E organized by the European Social Fund (ESF)—Roma
Inclusion Network in Madrid (November 13–14, 2014)—has also provided an opportunity for
24
member state participants to share past Roma inclusion investments, M&E experiences, and plans
for the next programming period. The workshop has highlighted the fact that all countries are aware
of the past programming period’s weak M&E systems, and that some progress is being made with
regards to M&E in the 2014–2020 programming period, particularly in the areas of (i) better
targeting of resources (such as using information on disadvantaged localities—the Czech Republic,
Slovakia, Hungary); and (ii) monitoring whether projects are reaching Roma (in Hungary, Bulgaria)
using self-identification. Participants discussed concrete ways to improve results targeting and
monitoring using modern IT technology, and by making M&E systems much more inclusive toward
the implementing organizations and final beneficiaries.
Source: World Bank (2015).
3.1.4. Best predictors of AROP
Finally, in an effort to provide geographically disaggregated estimates of poverty, the World
Bank and the HCSO constructed a map of AROP estimates—one of the key indicators of
poverty and social exclusion in the EU—at the microregional (“kistérség”, local
administrative unit or LAU1) level in 2014 (World Bank, 2014). While this exercise only
looked at one dimension of social exclusion—namely, monetary poverty based on income—
it identified the best predictors of poverty from a subset of variables included both in the
2005 microcensus and the simultaneously collected 2005 EU Survey for Income and Living
Conditions (EU-SILC). 8
Table 2. Best predictors of AROP at the microregion level—2005
Variables predicting log per capita disposable income Coefficient
Employment
No active members in the household -0.56
Two active members in the household -0.10
One or two employees, if household head is employer 0.08
More than two employees, if household head is employer 0.24
Number of employees in the household -0.28
8
Formally launched in 2004 in 15 countries and expanded in 2005 (to all 25 member states, Norway, Iceland),
2006 (to Bulgaria), and 2007 (to Romania, Switzerland, Turkey), the EU-SILC provides two types of annual data:
(i) cross-sectional data pertaining to a given time or time period with variables on income, poverty, social
exclusion, and other living conditions; (ii) longitudinal data pertaining to individual-level changes over time,
observed periodically over a four-year period. EU-SILC is a multipurpose instrument that focuses mainly on
income. Detailed data are collected on income components, mostly on personal income. Information on social
exclusion, housing conditions, labor, education, and health is also collected.
25
Household head working in chemical manufacturing 0.23
Household head working in retail, trade, or repair of personal and household
goods -0.10
Household head working in post or telecommunications 0.23
Household head without a contract at work -0.21
Household head with a contract at work for several months -0.16
Household head occupation (education & administration) 0.06
Education
Maximum education (continuous) 0.02
Proportion of people with no education/adults in household -0.29
Proportion of people with primary education/adults in household -0.28
Proportion of people with secondary education, no final exam/adults in
household -0.35
Proportion of people with secondary education/adults in household -0.23
Household composition
Household head is woman -0.12
Household head divorced 0.07
Proportion of children between 0–5 years old -0.18
Number of individuals between 13–18 years old 0.11
Proportion of elderly members above 60 0.32
Household size of one or two members 0.06
Housing conditions
No hot water in the dwelling -0.12
26
Paneled walls9 0.05
Income/monetary indicator
Proportion of total amount of personal income tax, inhabitants 18+ 0.09
Geography
County=9 -0.09
County=15 -0.07
County=18 -0.10
Region_1*Proportion of unemployed adults in the household/total no. of
adults -0.23
Region_2*Proportion of unemployed adults in the household/total no. of
adults -0.22
Region_3*Maximum education -0.01
Region_3*Household size 0.04
Region_4*Proportion of unemployed adults in the household/total no. of
adults -0.35
Region_5*Proportion of unemployed adults in the household/total no. of
adults -0.33
Region_7*Maximum education -0.01
Source: World Bank and HCSO staff calculations.
The poverty mapping exercise relied on the model based on methodology from Elbers et al.
(2003), using household-level data from the 2005 microcensus—a representative survey
covering 2 percent of the Hungarian dwelling stock, conducted by the HCSO in April 2005—
and the 2005 EU-SILC. The microcensus data covered a number of key household and
individual characteristics, including (i) demography (age/sex profiles, marital status,
household composition); (ii) employment (employment status, occupation, and industry);
(iii) educational attainment; and (iv) information on dwellings (type of ownership, amenities,
number and surface of rooms, type of sewage, type of walls).
The poverty mapping model relied on data regarding total disposable household income
(after transfers) per equivalent adult, which was available for 6,927 households. The
disposable household income per equivalent adult formed the dependent variable of the
models. This variable was chosen because it is also used to calculate the official
measurement of poverty reported to the European Commission (AROP indicator). The
9
As used in prefabricated housing modules.
27
microcensus was only representative at the county level: for the purposes of constructing a
unit-level model, the exercise relied on EU-SILC data on a number of household and personal
characteristics (household composition, age, gender, level of education, employment status,
and type of employment), as well as dwelling characteristics.
Variables were selected for the modeling stage using comparable variables between the
microcensus and EU-SILC, as well as auxiliary data at the settlement level (such as average
schooling at the settlement and district level, average household size, average age of
household head, among others). Several individual variables collected in the microcensus
were used for creating means of geographical partitions; for instance, average schooling at
the settlement and district level, average household size, and average household age,
among other variables. The final list of variables is presented in Table 2 and the final map is
shown in Figure 3.
Figure 3. AROP rates at the microregion level, 2005
Source: World Bank (2014).
Overview of indicators used
- Set of 35 indicators related to 6 dimensions of poverty—employment, education, family
composition, housing conditions, income, and geography
- Household-level data from the 2005 microcensus and 2005 EU-SILC
- Indicator available at the census-block level, but only every decade (10 years); or at the regional
level every year from EU-SILC
3.2. International experience with measuring social inclusion outcomes at
the national level
There is a broad and diverging range of views on how to select indicators to examine social
exclusion. Most methodologies include various dimensions under the terminology social
28
exclusion, and select different indicators for each field. The following sections are mostly
based on a review of Labonté et al. (2011), and Levitas et al. (2007). Both reports present a
meta-analysis of the different frameworks designed to analyze social inclusion/exclusion in
Australia, Canada, France, the United Kingdom, and the EU.
3.2.1. Choice of dimensions
A review of existing social exclusion frameworks, indicators, and measures led to the
identification of eight main principle domains that capture processes of social
exclusion/inclusion, as conceptualized by Labonté et al. (2011), and 10 topic areas in Levitas
et al. (2007), most of which overlap:
Table 3. Dimensions of social exclusion
Labonté et al. (2011) Levitas et al. (2007)10
Income and material resources Material/economic resources
Employment Economic participation
Education and skills Culture, education, and skills
Affordable and adequate housing Living environment
Access to public and private services
Health Health and well-being
Social resources and networks Social participation
Social resources
Community resources and civic participation Political and civic participation
Personal safety Crime, harm, and criminalization
Source: Labonté et al. (2011); Levitas et al. (2007).
The review by Labonté et al. (2011) of international evidence on choosing indicators to
measure social exclusion shows that, in most cases, the set of indicators chosen falls within
the aforementioned categories. Furthermore, in most cases, due to data availability and
reliability of indicators, it is restricted to the following five categories: (i) monetary poverty;
(ii) access to the labor market; (iii) education and health; (iv) housing and living conditions;
and (v) macroeconomic conditions of place of residence.
3.2.2. Number of indicators
Most countries finalize a set of 20 to 30 indicators organized in layers, or tiers, of indicators.
The first tier is usually a restricted number of lead indicators, which cover the broad fields
10
This matrix is called the Bristol Social Exclusion Matrix, or B-SEM.
29
that have been considered the most important elements that lead to social exclusion. The
second—and sometimes third—tiers of indicators support Tier I indicators and describe
other dimensions of the problem in more depth.
Atkinson, Marlier, and Nolan (2004) designed an indicator set for the EU that incorporates
EU-specific social exclusion dimensions, and keeps in mind availability of data from EU-SILC
and other datasets. The final set of indicators is presented in Table 4. It consists of two tiers
of indicators, which were chosen for the EU as a whole. An additional third tier is left blank
to be country specific, and depends on data availability as well as the specific focus
researchers want to take regarding social inclusion/exclusion.
Table 4. Atkinson et al. indicators for the EU
Level 1/Tier I
1 The risk of financial poverty as measured by 50% and 60% of national median income
2 Income inequality as measured by the quintile share ratio; that is, the ratio of the share
of national income received by the top 20% of households relative to the bottom 20% of
households
3 The proportion of those aged 18–24 with only lower secondary education (and not in
education or training)
4 Overall and long-term unemployment rates measured on an International Labor
Organization basis
5 Proportion of population living in jobless households
6 Proportion of population dying before the age of 65, or the ratio of those in bottom and
top quintile groups who classify their health as bad or very bad according to the World
Health Organization definition
7 Proportion of people living in households lacking specified amenities or with specified
housing faults
Level 2/Tier II
8 Proportion of people in households below 40% and below 70% of median income, and
proportion below 60% of the median fixed in real terms
9 Value of 60% of median threshold in purchasing power for two- and four-person
households
10 Proportion of the population living in households permanently at risk of financial
poverty
11
Mean and median equivalized poverty gap for a poverty line set at 60% median income
(this measures depth of poverty by calculating the extent to which those in poverty fall
30
below the poverty line)
12 Income inequality as measured by the decile ration and the Gini coefficient
13 Proportion of the population aged 18–59 with only lower secondary education or less
14 Proportion of discouraged workers, proportion non-employed and proportion in
involuntary part-time work, as a percentage of the total 18–64 population, excluding
those in full-time education
15 Proportion of people living in jobless households with current income below 60%
median
16 Proportion of employees living in households at risk of poverty (60% median)
17 Proportion of people who are low paid
18 Proportion of people unable to obtain medical treatment for financial reasons or
because of waiting lists
19 Proportion of the population living in overcrowded housing
20 Proportion of people who have been in arrears on rent or mortgage payments
21 Proportion of people living in households unable to raise a specified sum in an
emergency
Indicators to be developed
22 Non-monetary indicators of deprivation
23 Differential access to education
24 Housing of poor environmental quality
25 Housing cost
26 Homelessness and precarious housing
27 Literacy and numeracy
28 Access to public and essential private services
29 Social participation and access to Internet
Source: Atkinson and Marlier (2010).
The indicator set developed by Atkinson et al. for the EU is not to be confused with the EU
2020 headline indicators, discussed in Section 1.2. The EU 2020 indicators express the five
key objectives of the EU 2020 strategy (increasing the employment rate; increasing
investments in R&D; attaining certain climate change and energy targets; reducing school
dropout rates and increasing the share of people completing tertiary education; and lifting
Europeans out of poverty and social exclusion) that have been determined at the EU level
31
and translated into national targets (and incorporated in national reform programs). Table 5
summarizes the headline indicators.
Table 5. EU 2020 headline indicators
Topic Headline indicator
Employment Employment rate—age group 20–64
- females
- males
R&D Gross domestic expenditure on R&D
Climate change and energy Greenhouse gas emission
Share of renewable energy in gross final
energy
Primary energy consumption
Final energy consumption
Education Early leavers from education and training
- females
- males
Tertiary educational attainment
- females
- males
Poverty and social exclusion People at risk of poverty or social exclusion
(share)
People at risk of poverty or social exclusion
(total)
People living in household with very low
work intensity
People at risk of poverty after social
transfers
Severely materially deprived people
Source: EUROSTAT (http://ec.europa.eu/eurostat/web/europe-2020-indicators).
Most importantly, and closely related to the findings of HCSO presented in Figure 2, access
to the labor market is shown to be a very good predictor of social exclusion. Furthermore,
32
most indicators are shown to be rather time invariant. This has been demonstrated by a
longitudinal analysis of panel data in the United Kingdom. Burchardt et al. (1999) use the
British Household Panel Survey, which has information on the same households spanning
from 1991 to 1995, to investigate social exclusion. Using five dimensions of social
exclusion—low income, low wealth, low production activity, political disengagement, and
social isolation—Burchardt et al. show that there was little change in the value of the
indicators in the five-year period studied. 11
3.2.3. Composite versus non-unified indicators
Many researchers pool the indicators into a single social exclusion index (the BIP-40
indicator for France, for instance), 12 while others choose not to unify the various facets of
exclusion and keep all indicators separately (Burchardt et al., 1999 for the United Kingdom,
for instance). The advantage of composite indicators is that they allow for a single reading of
the results: each unit of observation—county, year, and so on—is associated with just one
number that is tracked across geographical units or over time, while single indicators need to
be presented, compared, and tracked one by one.
It is important to note that composite indicators come with two major caveats: (i)
interpretation; and (ii) choice of aggregation methodology. When using composite indicators
it is, in most cases, quite difficult to interpret the value associated with the composite
indicator. For example, while a poverty rate of 60 percent means that 6 in 10 households live
below the poverty line, there is not clear and simple explanation for an HDI of 60 percent. A
composite indicator allows for ranking (60 percent is less than 70 and more than 50 percent)
but not for interpretation. In addition, it may be difficult to justify the choice of a particular
methodology to aggregate indicators into a single composite indicator, as there are no
scientific guidelines regarding how to aggregate indicators into a single unified measure
(thus different aggregation measures using the same set of indicators may yield different
composite values, and different rankings across geographic areas, or over time). Because we
want to retain the multidimensional picture of the phenomenon of social exclusion, the final
set of indicators presented in Section 3 does not offer a composite—or unified—measure.
3.2.4. Disaggregation level and frequency at which indicators are collected
A review of the different international examples mentioned above (Australia, Canada,
France, the United Kingdom) shows that in most cases data is seldom collected on a regular
basis, or at a highly disaggregated level. As part of the South Australian government’s social
inclusion initiative, the Australian Workplace Innovation and Social Research Centre
developed a model that provided both qualitative and quantitative indicators. The Northern
11
Belgium used the French BIP-40 framework. It recorded little change in the indicator values in the short term
(Labonté et al., 2011).
12
The BIP-40 indicator was developed in 1999 and used until 2005 in France. It used a battery of 58 indicators,
each of them normalized and then aggregated into a bundled “barometer.” It was, among others, criticized for
being a “black box” (Hétfa and Alapítvány, 2013).
33
Adelaide Survey of Social Inclusion was implemented in 2005 but remained a one-off
exercise, with measures that were unlikely to be integrated into routine data gathering or be
administered in subsequent studies. In France, the central statistical office (INSEE) collected
the BIP-40 barometer between 1999 and 2005 and produced statistics at the national level
on an annual basis. Levitas et al. (2007) discuss the matrix of indicators to follow progress
toward social inclusion, but rely on the UK’s Poverty and Social Exclusion survey, which was
last fielded in 1999. Finally, the EU indicators developed by Laeken (Marlier, Atkinson,
Cantillon, and Nolan, 2007) and refined by Atkinson and Marlier (2010) were not fully used
to assess performance toward social inclusion across EU member states until 2004. Atkinson
et al. (2005) report that the indicators have been used at the national level to (i) explain
differences among EU member states; (ii) assist individual states in their policy development;
(iii) promote “joined up government” by identifying where intersectoral work is required;
and (iv) target setting. It is unclear the extent to which these four intents have been
operationalized.
4. What indicators should be used in Hungary?
The overview of international literature finds that there is no consensus about how to
measure social exclusion, apart from the need to capture its multifaceted nature. What
further complicates the matter is the tendency to use available quantitative measures, with
social exclusion being defined retrospectively (by the choice of available indicators) rather
than prospectively (ex ante through social theory). The construction of social indicators
necessarily entails a compromise between the theoretical definition (demand-driven
indicators) and what is empirically possible (supply-driven indicators). Data may simply not
be available, may not be of adequate quality, or may not be sufficiently comparable across
geographic areas or time.
4.1. Indicators that are suited to track progress toward social inclusion
How often data is collected, at which level of disaggregation, and whether indicators are
quantitative or qualitative depends on the reason why policy makers want to measure social
exclusion. For targeting new policy programs, researchers may use data available up to a few
years before a proposed program’s implementation, and may include indicators that are
quite static. For outcome monitoring, or how “socially inclusive” Hungary is becoming, one
would need a set of dynamic indicators, with data aggregated at a small geographical level.
On the other hand, for policy monitoring, or identifying which policies are associated with
increasing or decreasing social exclusion, one may rely on a mixture of qualitative and
quantitative data, with detailed information to capture the breadth and depth of social
exclusion. For policy and program planning, or for the evaluation of policies and programs,
one would need highly disaggregated data, but not necessarily with high collection
frequency.
34
The different sets of indicators defined for the various exercises undertaken by HCSO—
mapping, targeting, and monitoring, as presented in Section 2—mostly overlap when it
comes to themes/dimensions of social exclusion. Most include monetary poverty, access to
employment, education, and sometimes housing and access to services. At the same time,
the geographical level at which data is available, and the year(s) for which that information is
available and meaningful, varies quite substantially. Mapping segregated census blocks relies
on information available every decade, but at the census-block level. Targeting most
disadvantaged areas requires more frequent data that is available annually and at the
regional level, such as from information from EU-SILC or the latest census (every decade) at
the microregional level, or administrative data collected with various frequencies at various
levels of disaggregation.
4.2. Issues and dilemmas regarding collecting and tracking indicators to
monitor progress on social inclusion in Hungary
During the team’s discussions with HCSO in the course of 2015, various dilemmas and
questions were raised regarding the selection of dynamic social inclusion monitoring
indicators. In this section we reflect on these dilemmas, the lessons of which will be
incorporated into a recommendation for a measurement tool that could serve as a “follow-
up mechanism” for tracking development policy interventions in a social inclusion context.
How can the final set of indicators developed by different government agencies be
harmonized with indicators of general social development?
Most of the core indicators presented in the final set are collected by local municipalities,
the national household survey, the population census, or come from administrative data
collected by different line ministries. The harmonization of the diversity of data requires
strategic decisions on the one hand and considerable statistical and methodological efforts
on the other. One must acknowledge that trade-offs between a statistically less robust
public policy–related registry data (for example, number of visits to doctors or people
receiving selected local subsidies) to hard social indicators (such as unemployment figures)
are not always necessary, as both can inform policy making—though at other levels of
robustness and relevance. An example of this dilemma is the case of urban segregation
indicators. Altogether only two robust data components (education and employment) are
enough to define the most problematic areas, but if the local level wants to have a deeper
understanding of the local processes, several local registry-based indicators can be useful.
Therefore, while the mainstream solution is to opt for sets of indicators (to ensure a robust
analytical basis and a sensitive policy tracking tool), at the same time, qualitative information
may better explain the field experience in cases of extremely poor and fragmented
communities. For example, in a community where social development starts by helping
people understand why it is important to engage with the social worker or spend a full day
at school or work, sometimes very soft changes need to be captured, and these can only be
achieved through qualitative information.
35
Which territorial level is the most relevant when analyzing different social phenomena and
progress?
Balancing the depth of information and the level of disaggregation is a complex exercise.
Accurate information may only be available at the regional level, while microregions or
districts and municipalities may have low capacity to collect information that would be
relevant for tracking progress toward social inclusion. On the other hand, local-level
information is sometimes aggregated to a larger territorial level, where its primary purpose
is to enable comparisons between different geographical areas. In the Hungarian context,
the core of the dilemma is related to the fact that the Hungarian public administration
system is one of the most fragmented in Europe. This explains why the local level must often
deal with very few inhabitants and operate at a very low capacity, covering a limited set of
tasks in terms of service delivery, with a small delegated budget and few revenues. The size
of the municipalities is also highly heterogeneous (from less than a hundred to as many as
1.7 million inhabitants). As a result, the concentration of problems and complexity of issues
at the local level also diverge considerably. A more balanced analysis is possible if the
microregions or districts are taken as the units of observation. At this level, the average
population is 50,000, still with relatively large variations: Bélapátfalva has less than 9,000
and Miskolc has more than 250,000 inhabitants.13
In terms of policy analysis, considering the trade-off between reverting to exploring the
social dynamics at the local level versus choosing larger (and hence more comparable)
population size observation units depends on whether we consider the smallest local
administrative unit as a competent stakeholder in the formation of
inclusion/exclusion/development processes. So far, in Hungary, it is “local competence” to
tackle social exclusion issues (as per the obligation to develop Local Equal Opportunity Plans
(LEPs) and intervention strategies in every municipality), for example by designing an
effective local social benefit system. More recently, however, the allocation mechanism for a
considerable amount of social transfers and many social services has been shifted to the
district level. In the realm of the benefits, those under district supervision are categorized as
so-called “income-compensation benefits,” whereas it is the local level’s responsibility to
design and raise funds for “expenditure-compensation benefits.” 14 While it can be
meaningfully analyzed at the local level how the face and depth of social exclusion is shaping
up, the phenomena leading to changing the levels of social exclusion—including changes
deriving from the dynamics of the social transfer system—cannot be limited to the local
level. Another important public policy realm limited in terms of meaningful analysis at the
very local level is related to the labor market: despite the fact that indicators regarding
unemployment have proved to be the most precise dynamic measures to track social change
at the local level, the effects of “local” labor market policy interventions will necessarily go
13
See http://www.jaras.info.hu/jarasok-nepessege.
14
See http://www.kormanyhivatal.hu/download/1/39/d1000/szoc_tam_tajekoztato.docx.
36
beyond the boundaries of the local administrative units, and often also beyond the district
level. 15
The education system also has “gaps.” For example, in 2010, close to 1,000 (every third)
municipalities had no primary school.16 Thus, it is less meaningful to see whether school
service is available at the local level as an indicator for access to services (and hence social
inclusion) than it is to lift the geographical level of analysis to a (school) district level. This
allows one to explore early school leaving, school performance survey results, absenteeism,
and so on. Moreover, there are areas that do not have secondary education opportunities at
all. 17 Thus, improvements to education completion will necessarily depend on other factors
such as transportation options and scholarship availability.
The effects of location and displacement can be also critical when defining the territorial
level of analysis. For example, successful training and education programs—especially in
substantially lagging regions—can either contribute to a decline in unemployment or can
have the opposite effect. They can result in increased unemployment, as a trained labor
force would be more mobile and able to move to regions with a higher demand for labor.
The same can happen when production investments attract a labor force from another
region, but the newly created jobs do not affect the local labor supply (for example, if people
are not trained accordingly, lack necessary skills, and so on). Hence, local unemployment
levels would not improve as a result of the investment; active labor would be moved from
elsewhere and relatively lower activity rates would characterize their “sending”
municipalities after the investment.18
In summary, while income levels can be measured (proxied) at the local level, broader social
inclusion processes can only be tracked in larger geographic units. However, the analysis of
the processes (and potential causalities) is hardly meaningful at a single level. For example,
people may commute for jobs, but if there are issues regarding accessibility, they will be
unable to take advantage of job opportunities. Moreover, social inclusion programs can be
implemented in territorial arrangements that are disconnected from the territorial units in
which the public administration system operates, especially if they are run by the
nongovernmental organization (NGO) sector.
Do all indicators require the same time frame, taking into account the potential lag in the
impact of specific interventions on outcomes of interest, such as early childhood development
15
For more concrete examples of geographic relevance, see
http://www.ksh.hu/interaktiv/terkepek/mo/ingc.html, “Helyben lakó és dolgozó a helyben foglalkoztatottak
százalékában, 2011”—ratio of local inhabitants who are locally employed as a percentage of all locally
employed, 2011, or “Naponta bejáró a helyben foglalkoztatottak százalékában, 2011”—daily commuters to the
municipality as a percentage of the locally employed, 2011.
16
See http://www.ksh.hu/docs/hun/xftp/idoszaki/pdf/tarsatlasz.pdf.
17
For example, in northeast Hungary, in the formerly Bodrogközi microregion, which had 17,000 inhabitants,
there were no secondary schools.
18
For a recent analysis on this and a critical review of impact measurement methods, see Dusek et al. (2014).
37
(ECD) or education and training interventions that have long-term impacts on the labor
market?
As demonstrated in the UK example using panel data (Burchardt et al., 1999), few indicators
that track progress toward social inclusion improve within a five-year time frame. Depending
on the types of indicators, some may be affected quickly, while others may take time to
record improvement. Thus, tracking the latter and not seeing any changes would not
necessarily point to an absence of progress toward social inclusion, but would just indicate
the fact that one should wait longer to start measuring impact on those indicators. For
example, in the case of ECD, kindergarten attendance among young children is an indicator
that should be affected quickly, and could thus be measured annually, while cognitive
improvements and long-term labor market inclusion would not be affected immediately.
According to OECD data, investment in early childhood development will only be rewarding
after several years, and at the latest by age 15 (OECD, n.d.). Trainings for adults who have
been out of the labor market for a long time will have a very diverging impact regarding the
time and geographic location where changes emerge; whereas, for example, job search
assistance and job brokerage may produce more immediate results (OECD, 2005).
In the Hungarian context, several evaluations commissioned by the former Hungarian
Development Agency on measures financed in the 2007–2013 EU funding period have
piloted various methodologies to adjust the assessments to time-related impact
measurement constraints. For example, the evaluation of the developments in higher
education, 19 the (quantitative) evaluation of health investments, 20 and the evaluation of
measures targeting Roma integration 21 all point to the fact that in the case of impact
assessments, the time frame of the measurement of results is critical. Sustainability issues
would also very much depend on whether appropriate conditions would still exist in the long
term to ensure that the results can unfold, and hence be measured at all.
What level of heterogeneity is there when discussing different policies, such as education,
labor, or social issues?
As a result of the above, in the case of different sectoral policies, different time frames and
territorial levels may be relevant to track progress toward social inclusion in Hungary. At the
same time, the development in these subsectors is interrelated; thus, caveats at the local
level may already highlight meaningful information for policy design at higher levels.
Stakeholders should design a meaningful combination of geographical units to observe
(including taking into account the center–periphery issue) and a timespan in which to track
19
See
http://palyazat.gov.hu/download/48112/Fels%C5%91oktat%C3%A1si_%20%C3%A9rt%C3%A9kel%C3%A9s_z%
C3%A1r%C3%B3jelent%C3%A9s_I.pdf.
20
See
http://palyazat.gov.hu/download/48436/Eg%C3%A9szs%C3%A9g%C3%BCgyi_kvantitat%C3%ADv_%C3%A9rt%
C3%A9kel%C3%A9s_Budapest_Int%C3%A9zet.pdf.
21
See http://palyazat.gov.hu/download/39813/Roma_ertekelesi_zarojelentes_V.pdf.
38
the processes. These should be designed according to the priorities set by the given inclusion
policy agenda; for example, whether the focus should be on children whose life trajectories
would unfold later; on the long-term unemployed; or on development of lagging regions
with a multitude of challenges.
What difficulties stem from establishing a composite/synthetic indicator of social exclusion,
and what difficulties are associated with presenting a large set of single indicators, each of
which reflects a different dimension of social exclusion?
It may be difficult to justify the choice of a particular methodology to aggregate indicators
into a single composite indicator, as there are no scientific guidelines for how to do this.
Thus, different aggregation measures (including weighting) using the same set of indicators
may give different composite values, and different rankings across geographic areas, or over
time. The development of the Canadian Index of Well-being (CIW), for example, has taken
almost eight years of work, involving scores of researchers and countless meetings before
attaining a level of quality, quantity, and consensus regarded sufficient for its release—
including selection of subindicators, choice of model, weighting indicators, and how to treat
missing values (Levitas et al., 2007). On the other hand, presenting a large set of indicators
may be confusing for policy makers, as many indicators are bound to worsen in the very
same localities where other indicators drastically improve.
To what extent may the data collected by the different line ministries, research institutes,
various registers and the HCSO be combined or complement each other; or, for example, to
what extent can they be appropriately used for disaggregation (to get local-level data)?
As mentioned earlier, a sound framework for analyzing progress toward social inclusion
should include systematic data collection; information from tailor-made surveys should also
be included to more deeply understand some phenomena, on an ad hoc basis. Hence, data
collected systematically by the HCSO or the line ministries should be preferred, and
additional information collected by research institutes, tailor-made surveys, and qualitative
assessments should be used as secondary data sources. Beyond the costs associated with
data validation (it is expensive and time consuming to validate registry and other survey-
based data), the variations of alternative data sources in territorial design and
representation can further challenge the combination of data.22
At the same time, as highlighted above, there are some useful techniques for combining
data resources to disaggregate data to lower territorial levels; the poverty mapping exercise
discussed in Section 2.1.2 has applied such an approach.
22
For example, in education, the data collected in KIR-STAT (the national information system on education) is
organized according to schools (operators and school buildings); hence, commuters are accounted for in their
respective schools.
39
What tools are needed to explore whether policy interventions applied locally in the most
disadvantaged settlements are the most adequate, taking into account that the list of
settlements at the lowest end of the spectrum is roughly constant?
Evaluating the relative impact of a certain social inclusion policy versus another set of
decisions—for example, investing in access to labor markets through subsidized employment
for the youth versus soft-skills development for prime-age adults, or constructing
kindergarten facilities versus improving the transition from primary to secondary school—
would require using robust impact evaluation methods, carefully comparing one set of
disadvantaged settlements that benefit from the first policy option with another set of
settlements benefiting from the second. Such experiments are few and far between,
especially when it comes to comparing two options: most impact evaluations compare a
situation where nothing is done to one where a program is put in place.23 Therefore, there is
also considerable room in Hungary to design and implement targeted impact assessment
exercises to pilot the adequacy of selected methods.
4.3. Policy objectives and considerations for developing an indicator set
In order to allow the proposed indicator set to respond to the aforementioned
requirements, issues, and dilemmas, not only is readily available geographically
disaggregated and high quality data required; in addition, specific policy objectives should be
established, and the indicator set should be aligned with these.
The first policy objective is to identify whether certain groups (geographic, ethnic,
demographic, and so on) are systematically excluded along the core dimensions of social
inclusion (outcome monitoring). Viewed from this policy objective, the government should
have in place a data collection system that captures most of the core dimensions, broken
down by background characteristics such as geography, ethnicity, and demography. This
type of outcome monitoring does not necessarily require a full census, as a representative
sample of core groups (geographic, ethnic, demographic, and so on) may be used. Here, the
EU-SILC goes a long way, as it captures most (if not all) of the core dimensions, and allows
for some disaggregation (by demography, gender, and region). High-quality administrative
data will also support the objectives of outcome monitoring, and this data source (at least
for some data) can be accessible at highly disaggregated geographical levels.
The second policy objective is to have an M&E system in place that enables decision makers
to target their programs to socially excluded groups. Here, representative samples are
clearly not sufficient, as data should be available at a highly disaggregated geographical
level. However, while outcome monitoring should include at least some indicators that can
be affected relatively quickly by changes in social inclusion/exclusion, some indicators used
for targeting may not need to be affected relatively quickly or need to be available on a
23
For a summary of methods and numerous case studies, see for example Baker (2000).
40
regular basis. For example, the map of most disadvantaged regions or list of most
disadvantaged groups is likely fairly stable over time.
4.4. The proposed indicator set for Hungary
From the large set of indicators that are available within the Government of Hungary’s
Territorial Development and Spatial Planning Information System (TEIR), the team selected
indicators that (i) respond to the issues, challenges, and considerations identified in Sections
4.1, 4.2, and 4.3; and (ii) incorporate lessons from the international examples mentioned in
Section 2. Moreover, the team specified (iii) what primary policy objective the indicator may
serve (targeting versus outcome monitoring)—while keeping in mind that some indicators
may be suited for both objectives.
For the analysis, the team collected the existing data from TEIR and selected a subset of
indicators directly from what was already collected. This set includes the majority of so-
called objective indicators of non-monetary exclusion, such as the possession of material
goods and facilities and physical conditions of life. At the same time, what may be called
subjective indicators—such as self-assessment of general health conditions, economic
hardship, and social isolation, or the expressed degree of satisfaction with various aspects of
work and life—have not been included in the indicator set due to lack of available data, even
though such data would also provide relevant information on social inclusion. Against this
background, the team proposes using the set of social inclusion indicators summarized in
Table 6.
Considering the size of the Hungarian Roma population and the persistent gaps between
Roma and non-Roma in Hungary and elsewhere in the region, it would be important for data
to be collected in an ethnically disaggregated manner. While it is currently not possible to do
so, the introduction of an ethnic identifier in upcoming surveys (the EU-SILC in particular)
will make this feature available in the coming years.
Table 6. Set of proposed indicators for Hungary
Indicator Primary Source Level Frequency Coefficient
policy of
objective (after 2000) 24 variation 25
1. Monetary poverty and
material resources
1.1 AROP rates (using Targeting EU-SILC Regiona Yearly N/A
EUROSTAT’s methodology
24
There might be some variation in methodologies from year to year—such as incentives and sanctions
regarding the registration of unemployed—which might impact the figures.
25
The average coefficient of variation is computed like so: we first take the standard deviation of each indicator
over time at the settlement level. Then we normalize it by the average value of the indicator for each
settlement over time. Finally, we take the mean over all settlements.
41
of 60% of median income) l 26
1.2 Income inequality Targeting EU-SILC Regiona
l
(or share of high-income TEIR Yearly 36%
taxpayers on low-income Local
taxpayers) (2005–2013)
1.3 Income growth of Outcome EU-SILC Regiona Yearly N/A
bottom 40% monitoring l
1.4 Persistent risk of Outcome EU-SILC Regiona Yearly N/A
poverty monitoring l
1.5 Intensity of poverty Outcome EU-SILC Regiona Yearly N/A
(poverty gap) monitoring l
1.6 Share of households Outcome TEIR Yearly 23%
receiving regular child monitoring
benefit (2006–2013)
1.7 Financial inclusion/debt Outcome N/A
monitoring
1.8 Dependency ratio Outcome TEIR Local Yearly 7%
monitoring
(2005–2013)
2. Employment/labor
2.1 Long-term Outcome TEIR Local Yearly 42%
unemployment rate (180 monitoring
days) (2003–2013)
2.2 Very long-term Outcome TEIR Local Yearly N/A
unemployment rate monitoring
2.3 Share of population Targeting EU-SILC Regiona Yearly N/A
living in jobless households l
with current income below
60% of median income
2.4 Working poor (share of Targeting EU-SILC Regiona Yearly N/A
working adults living in poor l
households)
2.5 Share of households Targeting EU-SILC Regiona Yearly N/A
with very low work intensity l
26
Or microregional if small area estimation (as discussed in Section 3.1) can be updated annually.
42
3. Education and health
3.1 Early school Outcome Admin Local Yearly N/A
leavers/drop-out rates monitoring
3.2 Adults with low Outcome TEIR Local Census
educational attainment monitoring
N/A
- 15–24 year-olds
N/A
- 15–59 year-olds
3.3 Low reading/literacy Outcome Admin Local Yearly N/A
skills of pupils monitoring
3.4 Low birth weight Targeting TEIR Local Yearly N/A
4. Housing and living
conditions
4.1 Water/sewage/gas Targeting TEIR Local Yearly 5%-11%-
connection 18%
(2000–2013)
4.2 Overcrowded flats Outcome TEIR Local Yearly 6%
monitoring
(2000–2013)
4.3 Households receiving Outcome TEIR Local Yearly 91%
housing allowance monitoring
(2003–2010)
4.4 Crime/violence Targeting TEIR Local Yearly 54%
(2002–2013)
4.5. Confirming the validity of indicators
As described by the literature on social exclusion indicators, many measures are not very
sensitive to changes in the short term. The last column of Table 6 displays the coefficient of
variation of all available indicators.
The most “dynamic” indicators—that is, indicators whose average coefficient of variation at
the settlement level over time is high—are the number of households receiving a housing
allowance (91 percent); crime rate (54 percent); and long-term unemployment (42 percent).
Income inequality (measured by the share of high-income taxpayers on low-income
taxpayers) and share of households receiving children benefits show moderate dynamism,
with respective coefficients of variation of 36 and 23 percent.
43
Finally, and unsurprisingly, indicators related to water, sewage, and gas, flat overcrowding,
and dependency ratio show very little variability over time.27 The figures in Annex 2 show
the difference between a steady indicator such as access to sewage, and a highly dynamic
indicator such as long-term unemployment at the settlement level, through indicators of a
random sample of 9 Hungarian settlements (Figures A2.1 and A2.2) as well the settlements
with the highest and lowest relative variability (Figures A2.3 and A2.4, respectively).
27
Unfortunately, some of the more sensitive poverty-related dynamic indicators will become unavailable in the
future, due to changes in administrative regulations and data collection. For example, the housing allowance
scheme was redesigned in March 2015, allowing municipalities more freedom to decide whether to distribute
this transfer. As a result, any data relating to this local-level subsidy that had been targeting income poor
families will necessarily be biased, because some municipalities will report about it as a housing allowance,
others as a local allowance, and still others will not grant any subsidies of this kind. Similarly, the regular child
protection benefit that had been one of the best targeted social allowances for extremely poor families was
changed as of 2013; thus, the administrative data on number of beneficiaries and costs relating to this benefit
cannot be used for later years, either.
44
5. How to design a tracking tool that combines social inclusion
indicators with project data?
In previous sections we have identified the key components of an approach that enables
policy makers to track the status of social inclusion at the subregional level. The following
section explores ways to turn this approach into a tool that could combine locally available
social exclusion data with project information so as to allow projects to be continuously
tracked, with a view toward developing a feedback mechanism regarding whether funds are
spent in areas of highest need.
5.1. Proposal for a tool to track development policy interventions in a social
inclusion context
In Section 4.4 we introduced a brief set of social indicators that would be appropriate to (i)
accurately identify the different dimensions of social exclusion; (ii) be available at a
geographically disaggregated level; (iii) be collected regularly, preferably on an annual basis;
and (iv) be “dynamic”—that is, respond to local development dynamics as demonstrated by
relative variability. The proposed indicators were listed in Table 6 in Section 4.4.
Furthermore, as discussed in Section 4.1, some of these indicators would be less dynamic
than others, but still relevant for tracking policy measures, and can be illustrative for looking
at public policy outcomes. Moreover, some of the indicators would only be available at a
level higher than the local one, which is often a more relevant territorial level for
observation than the local level.
In this section we explore the possibility of linking some of the above social indicators to
development policy measures’ potential indicators, with a view toward delivering a
meaningful tool for describing, targeting, and tracking the social processes in places where
social inclusion interventions have taken place. We revert to the findings of earlier
evaluation exercises that analyzed selected EU–co-funded measures’ impact mechanisms.
Specifically, we draw on lessons from the recently completed evaluation of social inclusion
measures implemented within the TÁMOP 5 between 2007 and 2012 28 (the Evaluation
Report).
In the 2007–2013 period, TÁMOP 5 included most of the measures for social inclusion
activities and developments targeting the poorest and most disadvantaged communities in
Hungary, among them children and the long-term unemployed. 29 Our analysis includes
28
The report is available at
http://www.nfu.hu/download/48414/T%C3%A1rsadalmi_%20befogad%C3%A1s_%C3%A9rt%C3%A9kel%C3%A
9si_jelent%C3%A9s.pdf and its annexes are available at
http://www.nfu.hu/download/48413/T%C3%A1rsadalmi%20befogad%C3%A1s_esettanulmanyok.pdf.
29
The evaluation was based on document review, qualitative analyses (including case studies), and quantitative
analyses. One quantitative analysis was based on a tailor-made data collection where a representative sample
from all beneficiaries was compiled. An online survey was organized, with a 30 percent to 35 percent response
rate. The number of project-level responses was close to 1,000. The survey and analysis of results was carried
45
measures that are relevant to the current exercise, and excludes many that are not.30
Therefore, our findings do not paint a full picture and are meant to be illustrative inputs for
designing similar future exercises, with a view toward encouraging the design of a more
comprehensive tracking and monitoring system for the new (2014–2020) EU funding period.
The measures discussed in the Evaluation Report cover several thematic areas, and the
analysis relies on a hypothesized impact mechanism: the thematic areas and impact
mechanisms are discussed in Annex I of this report. Projects supported between 2007 and
2012 show considerable heterogeneity; moreover, the target organizations are also
diverse—national-level institutions, microregions, municipalities, and NGOs, as well as
consortia of various compositions were eligible for funding under the themes. The
mechanisms behind the desired project outcomes are also diverse, and the differences in
project design (for example, launch period, size of the beneficiary group, required intensity
of work with beneficiaries, multiplication of effects) display a variety of ways to address
selected social problems with TÁMOP 5 funding.
5.2. Topics and hypothesized impact mechanism of selected social inclusion
measures
Table 7 demonstrates a way to link the EU–co-funded project with tracking (local) social
processes, based on the content analysis delivered by the Evaluation Report. Based on the
results of the content analysis, we develop a general list of social challenges addressed. Since
the social problems to be addressed can be defined based on a simple analysis, if necessary,
their coherence with various strategic documents (such as relevant OPs, NSIS, or sectoral
strategies) can be easily checked and updated. Second, we link these with the list of calls
that directly or indirectly address the social challenge in question. (Some calls designed to
approach complex social situations would match more than one social challenge.) Third, we
select one or more proxy indicators that would best serve to track the development of the
given social challenge. Finally, we list the indicators that are available and have proven to be
dynamic and measureable at the lowest possible geographic level (as discussed in Section
3.1) while keeping in mind the limitations of this approach (as discussed in Section 4.2).
out by REVITA Foundation; the evaluation was led by Hétfa Research Institute and Metropolitan Research
Institute, Budapest.
30
For example, TÁMOP 5 had financed, among other initiatives, drug prevention programs, rehabilitation of ex-
convicts, victim protection, methodological developments in a background institution of the Ministry of Human
Capacities (MHC), programs for people living with disabilities, and so on.
46
Table 7. Matching social challenges with measures and tracking indicators
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
children with share of TÁMOP 5.1.1.- Number of 3.1 Early school
constrained population with 09/1-2, TÁMOP people without leavers/drop-
school careers low education 5.2.2-08/1, 2, having out rates (L) a
(i.e. 8th TÁMOP 5.2.2- completed the Available in the
grade)/Early 10/1, TÁMOP first class of framework of
school leavers / 5.2.3-A-11 primary school the EU2020
dropouts TÁMOP 5.2.3- among those data collection
A-12/1 over 10 years by the
(L, Census) background
institute to the
Ministry of
Human
Capacities
child poverty share of children TÁMOP 5.2.2- 1.6 Share of
living under the 08/1, 2, households
poverty line TÁMOP 5.2.2- receiving
10/1, TÁMOP regular child
5.2.3-A-11, benefit from
TÁMOP 5.2.3- among those
A-12/1 under 18 years
(L)
2.5 Share of
very low work
intensity
households (L,
meaning
households
with no
employed
person) among
all households
deviances crime rates TÁMOP 5.1.1.- 5.1 Number of
09/1-2, TÁMOP registered
5.2.5-08/1/C crimes/violence
per 100
persons (L)
low share of young TÁMOP 5.1.1.- Share of
participation of unemployed 09/1-2 unemployed
young in the among the
47
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
labour market labour market
entrants (L)
digital illiteracy digital illiteracy TÁMOP 5.1.1.- X
09/3 measurable
e.g. by nr of
households
connected to
broad band
internet (L)
low activity activity rate, TÁMOP 5.1.1.- 2.1 Long-term
rates/large unemployment 09/1-2, TÁMOP unemployment
unemployment rate, long-term 5.1.1.-09/3, rate (180 days)
unemployment TÁMOP 5.1.1- (L), 2.6 Share of
rate 09/6-7, households
TÁMOP- with very low
5.3.1/08/1, 2, work intensity
TÁMOP-5.3.1- among all
C-09/2, households (L)
TÁMOP-
5.3.1/08/2,
TÁMOP-5.3.9-
11/1
gaps in / lack of share of local TÁMOP 5.1.1- Segregation additionally:
(selected) residents 09/4-5, TÁMOP index produced measurable
quality receiving social 5.1.1-09/8-9, by the HCSO e.g. by
social/human benefits, share TÁMOP 5.1.3- based on 2011 various sub-
service delivery of local 09/1,2, TÁMOP Census data sectoral data
for various residents on 5.2.2-08/1, 2, on service gaps
target groups selected TÁMOP 5.2.2- via SZOCIR (L)
(e.g. at local transfers, nr of 10/1, TÁMOP
level, home clients per staff 5.2.3-A-11,
care, social, employed in the TÁMOP 5.2.3-
child social A-12/1, TÁMOP
protection, assistance/home 5.2.5-08/1/A,
youth welfare care/welfare TÁMOP
service, services service sector 5.2.5.A-10/1,
for people with TÁMOP
disabilities) 5.2.5.A-10/2,
TÁMOP-
5.3.8.A2-12/1,
TÁMOP-
48
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
5.3.8.A2-12/2,
TÁMOP-
5.3.8.A3-12/1,
TÁMOP-
5.3.8.A3-12/2,
TÁMOP -
5.4.1/08/,
TÁMOP -
5.4.2/08/1,
TÁMOP 5.4.3-
09/1, TÁMOP
5.4.3-09/2,
TÁMOP-5.4.3-
10/1, TÁMOP-
5.4.3-10/2,
TÁMOP -5.4.4-
09/1/A,
TÁMOP -5.4.4-
09/1/B,
TÁMOP -5.4.4-
09/1/C, TÁMOP
-5.4.4-09/2/A,
TÁMOP -5.4.4-
09/2/B,
TÁMOP -5.4.4-
09/2/C,
TÁMOP-
5.4.5/07/1 és
5.4.5-09/1,
TÁMOP-
5.4.6.A-12/2,
TÁMOP-5.4.9-
11/1, TÁMOP -
5.5.2 /09/2,
TÁMOP - 5.5.2
/09/3, TÁMOP-
5.5.2/10/4,
TÁMOP-
5.5.3/08/01,
TÁMOP-
5.5.3/08/02,
TÁMOP-5.5.3-
49
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
09/1
low community TÁMOP 5.2.5- X
cohesion 08/1/B
homelessness nr of homeless TÁMOP- X
5.3.3/08/1, measurable
TÁMOP- e.g. by
5.3.3/08/2, available beds
TÁMOP-5.3.3- in homeless
10/1, TÁMOP- service
5.3.3-10/2 provision (L)
indebtedness / level of TAMOP-5.3.5- 4.3 Share of
housing cost household debt, 09/1 people
overburden housing cost receiving
overburden housing
allowance per
100 residents
(L)
housing share of people TÁMOP-5.3.6- 4.1 Share of
segregation of living in 11/1 dwellings
Roma / people inadequate connected to
living in poverty housing of Roma sewage (L)
neighbourhoods origin Nr of
households as
gas consumers
among all
dwellings (L),
Amount of
electricity
provided to
households per
residents (L)
Share of
dwellings
connected to
the water
system (L)
4.2
Overcrowded
flats: Nr of
50
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
dwellings in /
number of
resident
population at
the end of the
year based on
the Census
data (L)
low labour activity rate of TÁMOP- 2.1 Long-term X
market people with 5.3.8.A2-12/1, unemployment measurable
participation of disabilities TÁMOP- rate (180 days) e.g. by nr of
people with 5.3.8.A2-12/2, (L) people on
disabilities TÁMOP-
Number of disability
5.3.8.A3-12/1,
people pension (L)
TÁMOP-
receiving
5.3.8.A3-12/2,
disability
TÁMOP-
benefits (L)
5.4.5/07/1 és
5.4.5-09/1,
TÁMOP-
5.4.6.A-12/2,
TÁMOP-
5.4.7/08/1,
TÁMOP-
5.4.7/08/2,
TÁMOP-
5.4.7/09/1,
TÁMOP-
5.4.8/08/1
discrimination statistics on TÁMOP– Share of Roma X
of Roma and cases at the 5.5.4.A-09/1, at local level (L, can be
people living Equal Treatment TÁMOP– Census data) obtained from
with disabilities Body and in 5.5.4.B-09/1, the Equal
and other court TAMOP – Treatment
vulnerable 5.5.5/08/1, Authority
groups TÁMOP
5.5.7/08/1
low consumer TÁMOP- X
awareness 5.5.6/08/1,
TÁMOP-
5.5.6/08/2
51
Social problem Proxy social Examples for Closest Missing
addressed indicator to measures in matching indicator/notes
track relevance TÁMOP 5 tracking
of measure indicator
exclusion of nr of people TAMOP – 5.1 Number of
people with with criminal 5.6.1.A-11/1., registered
criminal records records TAMOP – crimes/violence
5.6.1.A-11/3, per 100
TAMOP – persons (L)
5.6.1.A-11/ 4.,
TÁMOP 5.6.2-
10/1
high crime rates crime rates TAMOP – 5.1 Number of
among young 5.6.1.A-11/1, registered
TÁMOP- crimes/violence
5.6.1.B-12/1, per 100
TÁMOP- persons (L)
5.6.2/08/1,
TÁMOP 5.6.2-
10/1
*collected at (L)=local; (R)=regional level; **=the benefit’s figures are not available after 2013, but since the
analysis concerns the period until 2013, this seems to be an appropriate indicator; *** the benefit was cut and
redesigned after 2015, but since the measures concern the period until 2013, this seems to be an appropriate
indicator
The analysis demonstrates that most of the challenges and associated activities could be
tracked with an indicator from the proposed indicator set. It is therefore possible to develop
a feedback mechanism that is based on whether thematic activities are undertaken in the
geographic areas with the highest need in a certain theme. This feedback mechanism would
be based on tracking the indicators at the lowest possible geographical level and comparing
them with projects’ geographic locations. For example, an analysis of how TÁMOP 5 funding
sources had reached localities or districts (or any higher geographical level) that experience
child poverty could be undertaken using a map with the rate of beneficiaries of the regular
child benefit, including the information of the location of projects funded under the relevant
calls (in this case, TÁMOP 5.2.2-08/1 and 2, TÁMOP 5.2.2-10/1, TÁMOP 5.2.3-A-11, TÁMOP
5.2.3-A-12/1). 31 For another example on unemployment rates, displaying projects under
TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3, TÁMOP 5.1.1-09/6-7, TÁMOP-5.3.1-08/1 and 2,
TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1 could serve to track where this
31
For the 2014–2020 period, an alternative indicator will have to be defined for this topic.
52
problem has been addressed via labor market inclusion projects funded under TÁMOP 5.
These examples are elaborated in more detail in Annex 2.
5.3. Next steps toward an improved tracking tool
As demonstrated by the example of selected TÁMOP 5 2007–2012 period calls, the inclusion
of “social challenge” dimensions in the database of calls/measures can be a useful tool for:
1. displaying the dynamics of (selected) social challenges as proxied through low
administrative-level social indicators; and subsequently
2. tracking whether any funding is addressing these social challenges in the given
locality/district/and so on in a given time period.
To this end, routinely combining maps of subregional social exclusion indicators with project
data would require the regular collection of at least the following project information:
• the social challenge the call/measure intends to address via an ex post content
analysis of the calls or via an ex ante solution, as proposed below for the 2014–2020
period;
• the time frame in which the call is launched and the projects are financed (from and
until what year);
• number of project beneficiaries; and
• the location(s) of the implementation.
Additionally, the Cohesion Policy Regulations 2014–2020 entail strengthened monitoring and
evaluation arrangements, with the goal of ensuring that (i) robust and reliable data are
available; (ii) these data can be aggregated at the EU level; and (iii) evaluation activities focus
on assessing the effectiveness and impact of ESF support. To this end, member states are
now required to ensure that data collection systems provide electronically structured data
about the participants for each priority axis, broken down by investment category. Annex B
and C of the Monitoring and Evaluation of Cohesion Policy guidance note (EC, 2015) offers a
list and definition of common and youth employment indicators (output and results). The
tracking tool can accommodate and make this information available as well.
As the tool is not designed for robust impact analysis, but rather for simply visualizing
information on social processes and linking it with data on funding and expenditure, no
conclusions can be made regarding the causalities and the level of change. However, the tool
does allow one to make observations regarding what changes were ongoing at the time
selected projects in the given geographic area were funded, and whether the locally
implemented projects responded to local challenges. Furthermore, the analysis underlying
the proposed tool builds on the Evaluation Report’s content analysis of all related calls. The
53
analysis helped identify the social challenges, and many of these could be directly linked to
the proposed set of dynamic indicators. Due to the limited data offered by the management
authority’s database on EU–co-funded projects (EMIR), it is unrealistic to re-elaborate the
content of all past calls related to social inclusion. 32
At the same time, for the 2014–2020 period, the relevant administrative body or bodies (the
Prime Minister’s Office or the respective departments in the line ministries) should be
encouraged to add additional dimensions to routinely collected data when they administer
or design the call. These dimensions should correspond to the dynamic indicator set and
could allow for a more sophisticated classification of EU–co-funded projects, thereby making
it possible to track the local challenges addressed. These dimensions should also be linked to
EU 2020 targets. The dimensions should, at a minimum, include:
1. constrained school careers/low education
2. child poverty
3. crime or deviance
4. low employment/activity levels
5. gaps in/lack of (selected) quality social/human service delivery for various target
groups (at the local level, home care, social, child protection, youth welfare service,
services for people with disabilities)
6. indebtedness/housing cost overburden
7. housing segregation of Roma/people living in impoverished neighborhoods
8. discrimination of vulnerable groups (such as Roma, people with disabilities, and so
on) 33
The above list should be matched and complemented with dimensions included as relevant
dimension/indicators in the forthcoming Operational Programs (OPs), which include a social
inclusion dimension and other dimensions that are being defined as a basis for indicators of
the forthcoming revision of the NSIS II in the course of 2015. It is important to ensure that
indicator definitions are consistent across policy objectives. This means choosing, whenever
possible, the same definitions for project monitoring indicators that are also part of the
national-level outcome monitoring. Such consistency will help explain movements—or lack
thereof—in outcome indicators.
32
Currently, EMIR only includes information about whether the project is implemented in a lagging region;
whether it targets Roma, people with disabilities, or any vulnerable group; and whether it has contributed to
an improvement in their quality of life. Information appears to be incomplete, and hence its use is limited.
33
The proposed dimensions are not necessarily complementary.
54
It is also important that national key projects co-funded by EU resources also report project
data in a territorially disaggregated manner. To this end, it is important that government
agencies that are beneficiaries of EU-funded interventions track resource allocation and
project indicators in a territorially disaggregated manner, and that the Prime Minister’s
Office enables the EU project database to accommodate this reporting mechanism.
5.4. Recommendations for targeting and monitoring EU–co-funded social
inclusion investments in the 2014–2020 period
1. Establish an institutional/operational framework for the regular collection of social
inclusion indicators in a territorially and ethnically disaggregated manner—a
proposed set of indicators can be found in Table 6.
Responsible agency: Ministry of Human Capacities and Hungarian Central Statistical
Office
Time frame: 2016 and onwards
Expected costs: low (most of the indicators are based on existing data)
2. Establish a collection framework of project data (including for national key projects)
for the 2014–2020 project period, corresponding to and consistent with the dynamic
indicator set. It should include (but not necessarily be limited to) the following
dimensions:
o constrained school careers/low education
o child poverty
o crime or deviance
o low employment/activity levels
o gaps in/lack of (selected) quality social/human service delivery for various
target groups (at the local level, home care, social, child protection, youth
welfare service, services for people with disabilities)
o indebtedness/housing cost overburden
o housing segregation of Roma/people living in impoverished
neighborhoods
o discrimination of vulnerable groups (such as Roma, people with
disabilities, and so on) 34
Responsible agencies: Ministry of Human Capacities and Prime Minister’s Office
34
The proposed dimensions are not necessarily complementary.
55
Time frame: January–June 2016
Expected costs: low (requires the addition of a few variables to the EU project database)
3. Complement the project data collection framework with
o further relevant dimensions based on the forthcoming OPs, which have a
social inclusion dimension;
o dimensions that are being defined as a basis for indicators of the forthcoming
revision of the NSIS II; and
o indicators aimed at monitoring the performance of each relevant OP priority
axis, broken down by investment category.
Responsible agencies: Ministry of Human Capacities and Prime Minister’s Office
Time frame: January–June 2016
Expected costs: medium (requires closer coordination between government agencies)
4. Map and publish the social inclusion indicators by using a GIS software application or
the national mapping platform for administrative data, TEIR TETA
Responsible agencies: Ministry of Human Capacities and Lechner Lajos Knowledge Center
Time frame: 2016 and onwards
Expected costs: low (requires the use of the existing GIS platform)
5. Continuously map and publish project data by using a GIS software application or the
national mapping platform for administrative data, TEIR TETA
Responsible agencies: Ministry of Human Capacities, Prime Minister’s Office, and Lechner
Lajos Knowledge Center
Time frame: 2016 and onwards
Expected costs: low (requires the continued use of the existing GIS platform)
56
6. References
Atkinson, A., B. Cantillon, E. Marlier, and B. Nolan. 2005. “Taking Forward the EU Social
Inclusion Process.” Independent Report commissioned by the Luxembourg
Presidency of the Council of the European Union, Government of Luxembourg,
Luxembourg.
Atkinson, A. B., and E. Marlier. 2010. “Analyzing and Measuring Social Inclusion in a Global
Context.” United Nations, Department of Economic and Social Affairs (DESA), New
York, ST/ESA/325.
Atkinson, A. B., E. Marlier, and B. Nolan. 2004. “Indicators and Targets for Social Inclusion in
the European Union.” Journal of Common Market Studies 42 (1): 47–75.
Baker, J. L. 2000. Evaluating the Impact of Development Projects on Poverty. Washington,
DC: World Bank.
Burchardt, T., J. Le Grand, and D. Piachaud. 1999. “Social Exclusion in Britain 1991–1995.”
Social Policy and Administration 33 (3): 227–44.
Deaton, A. 1997. The Analysis of Household Surveys: A Microeconometric Approach to
Development Policy. Washington, DC: World Bank.
Dusek, T., and M. Lukovics. 2014. “Az ELI és az ELI Science Park gazdasági hatásvizsgálata.”
Területi Statisztika 54 (3): 202–19.
Elbers C., J. O. Lanjouw, and P. Lanjouw. 2003. “Micro-Level Estimation of Poverty and
Inequality.” Econometrica 71 (1): 355–64.
EUROSTAT. 2015. “Europe 2020 Indicators—Poverty and Social
Exclusion.” http://ec.europa.eu/eurostat/statistics-
explained/index.php/Europe_2020_indicators_-_poverty_and_social_exclusion.
EC (European Commission). 2011. “On the Territorial Aspects of Extreme Poverty: Drawing
Up a European Extreme Poverty Map.” European Platform for Roma
Inclusion. http://ec.europa.eu/justice/discrimination/files/romaplatform_discussion
_paper_poverty_2011_en.pdf.
EC. 2015. “Programming Period 2014-2020. Monitoring and Evaluation European Cohesion
Policy.” Guidance Document, European Social Fund.
Hétfa, and R. Alapítvány. 2013. “A társadalmi befogadást szolgáló fejlesztések (TÁMOP 5.
prioritás) értékelése.” Budapest: Hétfa.
Ivanov, A., and J. Kagin. 2014. “Roma Poverty from a Human Development Perspective.”
Roma Inclusion Working Papers, United Nations Development Programme, Istanbul.
57
Kezán, A., and D. Szilágyi. 2015. “Measuring the Disparities in the State of Development of
LAU1 and LAU2 Units.” Presentation at the Hungarian Central Statistical Office.
Labonté, R., A. Hadi, and X. E. Kauffmann. 2011. “Indicators of Social Exclusion and Inclusion:
A Critical and Comparative Analysis of the Literature.” É/Exchange Working Papers,
vol. 2, no. 8, University of Ottawa.
Levitas, R., C. Pantazis, E. Fahmy, D. Gordon, E. Lloyd, and D. Patsios, 2007. The Multi-
dimensional Analysis of Social Exclusion. London: Department for Communities and
Local Government (DCLG).
Marlier, E., A. B. Atkinson, B. Cantillon, and B. Nolan. 2007. The EU and Social Inclusion:
Facing the Challenges. Bristol: Policy Press.
Miliband, D. 2006. Social Exclusion: The Next Steps Forward. London: Centre for Analysis of
Social Exclusion. London: LSE and ESRC Research Centre.
OECD (Organisation for Economic Co-operation and Development). n.d.
“Investing in High-Quality Early Childhood Education and Care
(ECEC).” http://www.oecd.org/education/school/48980282.pdf.
OECD. 2005. “Labour Market Programmes and Activation Strategies: Evaluating the
Impacts.” In OECD Employment Outlook, 173–
208. http://www.oecd.org/els/emp/36780874.pdf.
Ravallion, M., and B. Bidani. 1994. “How Robust Is a Poverty Profile?” World Bank Economic
Review 8 (1): 75–102.
Ravallion, M. 1992. “Poverty Comparisons: A Guide to Concepts and Methods.” Living
Standards Measurement Study (LSMS) Working Paper, No. LSM 88, World Bank,
Washington, DC.
Silver, H. 1994. “Social Exclusion and Social Solidarity.” International Labour Review 133 (5–
6): 531–78.
UNDP (United Nations Development Programme), World Bank, and EC. 2011. Regional Roma
Survey.
World Bank. 2013. Inclusion Matters: The Foundation for Shared Prosperity (Advance
Edition). Washington, DC: World Bank.
World Bank. 2014. “Small Area Estimates of Poverty in Hungary: A Methodological Report.”
Mimeo, World Bank, Washington, DC.
World Bank. 2015. Equality of Opportunity for Marginalized Roma: A Pathway to Shared
Prosperity in Europe. Washington, DC: World Bank.
58
Annex I. Summary of TÁMOP 5 measures and hypothesized impact mechanisms
Code of the measure Topic Goal Hypothesized impact mechanism
Child and youth protection Through preventative and Reducing and preventing deviances, children
TÁMOP 5.1.1-09/1-2 (funding available only for the intervention programs, to help will improve their school careers and
most disadvantaged microregions) youth enter the labor market facilitate their later labor market integration.
Development of digital
Developing digital literacy competence will
TÁMOP 5.1.1-09/3 competences (funding available
To develop digital literacy help people use the Internet to, among
only for the most disadvantaged
other things, search for jobs.
microregions)
Local community development Capacity of organizations that work with the
TÁMOP 5.1.1-09/4-5 programs (funding available only To develop community programs poor will strengthen, and new services will
for the most disadvantaged and services emerge that will help reduce social
microregions) exclusion.
Training and employment for To help disadvantaged groups
The chances of participants in the labor
TÁMOP 5.1.1-09/6-7 vulnerable groups (funding integrate into the labor market
market will improve and they will obtain
available only for the most and access services, training, and
new skills and on-the-job training.
disadvantaged microregions) programs
Training and employment for
Developing service delivery and the NGO
TÁMOP 5.1.1-09/8-9 professionals (funding available To strengthen local human
sector will improve access to services and
only for the most disadvantaged capacities and civil society
reduce social exclusion.
microregions)
Community-based development To integrate those living in deep
TÁMOP 5.1.3-09/1 activities for the integration of poverty via the tools of social Access to social services for the poor will
and 2 people living in deep poverty— and community work, and to improve, and hence the poorest will increase
professional coordination design selected public services their community activity.
that match local demand
59
Code of the measure Topic Goal Hypothesized impact mechanism
Establishment of the methodology To elaborate program elements
for scaling up of the Chance for that target vulnerable children
TÁMOP 5.2.1-07/1 Children program (ECD) and aged 0–7 with special attention Ensuring good quality project calls will
TÁMOP 5.2.1-09/1 follow-up of local projects paid to children 0–5 who are not improve the implementation’s effectiveness.
in preschool, to enhance their
chances in society
Scaling up the ECD-focused To develop skills and complex
TÁMOP 5.2.2-08/1 Chance for Children program to programs for vulnerable children ECD facilitates later school careers of
and 2 the country level, focusing on the aged 0–7, with special attention children, and hence their chances in the
TÁMOP 5.2.2-10/1 most disadvantaged microregions paid to children 0–5 who are not labor market will improve.
in preschool
TÁMOP 5.2.3-A-11 Scaling up the ECD-focused To reduce child poverty and Improved ECD, community development,
TÁMOP 5.2.3-A-12/1 Chance for Children to the country prevent the reproduction of and local human service delivery will
level poverty improve access to quality services.
TÁMOP 5.2.5-08/1/A Integration program of children To compensate for the
Facilitating access to child protection
TÁMOP 5.2.5-A-10/1 and youth—child protection disadvantages experienced by
services will help make families less
TÁMOP 5.2.5-A-10/2 component affected school-aged children
vulnerable.
and youth
Integration program of children To improve the quality of life of
TÁMOP 5.2.5-08/1/B Youth activities in the community will
and youth—youth-focused disadvantaged youth and
increase with the help of the service.
component develop their community skills
Integration program of children To compensate for the
TÁMOP 5.2.5-08/1/C and youth—drug consumption– disadvantages experienced by Through preventative programs, deviant
related component affected school-age children and behavior will decrease.
youth, preventative programs
TÁMOP 5.3.1-08/1 “First Step”—enabling and Programs to enable and prepare Improving the employability of the target
and 2 preparatory programs for for independent living, and group and helping them acquire new
TÁMOP 5.3.1-C-09/2 independent living for people with programs for preparing the steps competencies and knowledge will facilitate
60
Code of the measure Topic Goal Hypothesized impact mechanism
TÁMOP 5.3.1-08/2 low reintegration chances into the and motivating to enter the labor their labor market integration.
labor market market, supporting services
Targeted (single beneficiary)
To develop homeless
project for the professional and
TÁMOP 5.3.3-08/1 organizations and related Educating organizations that work with the
methodological elaboration of the
TÁMOP 5.3.3-08/2 services that would serve the homeless and offering complex labor market
program for social and labor
TÁMOP 5.3.3-10/1 social and labor market and housing programs will improve the
market integration of homeless
TÁMOP 5.3.3-10/2 integration of homeless people, chances for social reintegration of homeless
people and programs for social
enhancing their professional, people.
and labor market integration of
housing, and social situations
homeless people
Pilot program for prevention of Scaling up debt management programs and
To launch debt management
TÁMOP 5.3.5-09/1 arrears traps introducing alternative services will help
programs and related
stabilize indebted beneficiaries’ financial
preventative programs
situation.
Complex poverty/Roma The labor market and social integration
settlement program (ensuring chances of people living in segregated
To help the integration of people
TÁMOP 5.3.6-11/1 access to complex human neighborhoods will improve through
living in segregated
services) complex programs like social and community
environments
development, education, health, training,
and employment actions.
Supporting motivation trainings Through needs assessments and tailor-made
TÁMOP 5.3.8-A2-12/1 and related events at employers, To help the labor market
development, and by creating the
TÁMOP 5.3.8-A2-12/2 targeting the most vulnerable integration of people with
nationwide supporting network, the target
TÁMOP 5.3.8-A3-12/1 groups in order to facilitate their reduced workability and access
group will have a better chance to find
labor rehabilitation services
chances in the labor market employment on the open labor market.
61
Code of the measure Topic Goal Hypothesized impact mechanism
Supporting motivation trainings
To help the labor market Through needs assessments and tailor-made
and related events at labor
integration of people with development, and by creating the
TÁMOP 5.3.8-A3-12/2 market services, targeting the reduced workability and help nationwide supporting network, the target
most vulnerable groups in order
them receive labor rehabilitation group will have a better chance to find
to facilitate their chances in the
services employment on the open labor market.
labor market
Learning partnerships for To develop competencies for
Tailor-made trainings will improve the
TÁMOP 5.3.9-11/1 enhancing employability vulnerable people and trainings
potential labor market participation of the
according to their special
target group.
education needs
Modernization of social services,
fostering the capacity of central To develop social, child welfare,
TÁMOP 5.4.1-08/1 Based on methodological development,
and regional strategic planning, child protection, and drug
services will become more effective.
elaborating decisions related to prevention services
social policy
Central developments of social To modernize social services via
TÁMOP 5.4.2-08/1 Based on methodological development,
informatics the development of a centralized
services will become more effective.
electronic service
Development of home care To develop services related to
TÁMOP 5.4.3-09/1 The quality of life of beneficiaries who
home care
TÁMOP 5.4.3-09/2 receive home care will improve, and
TÁMOP 5.4.3-10/1 recipients of home care transfer will have
TÁMOP 5.4.3-10/2 more chance to find employment.
62
Code of the measure Topic Goal Hypothesized impact mechanism
Development of social trainings,
professional trainings (including
skills and higher level trainings),
TÁMOP 5.4.4-09/1/A fostering local training capacities
TÁMOP 5.4.4-09/1/B To train social, child welfare, and
Developing social protection, child welfare,
TÁMOP 5.4.4-09/1/C child protection staff, and to
and child protection services will improve
TÁMOP 5.4.4-09/2/A develop trainings (and training
beneficiaries’ access to quality services.
TÁMOP 5.4.4-09/2/B materials
TÁMOP 5.4.4-09/2/C
TÁMOP 5.4.5-07/1 Creating a barrier-free
To develop methodological and Raising professional standards will improve
TÁMOP 5.4.5-09/1 environment via physical and info-
training materials the quality of service delivery.
communication tools
Scaling up knowledge and service
development relating to the To disseminate the methods and
TÁMOP 5.4.6-A-12/2 Trainings will prepare providers for
creation of barrier-free the training materials elaborated
investments in equal treatment.
environments via physical and in TÁMOP 5.3.4
info-communication tools
TÁMOP 5.4.7-08/1 Development of basic The rehabilitation program will help visually
TÁMOP 5.4.7-08/2 rehabilitation services for the To provide basic rehabilitation
impaired people more easily integrate into
TÁMOP 5.4.7-09/1 visually impaired services for the visually impaired
the labor market.
Enhancing the professional To modernize the services Modernizing the system and creating a well-
TÁMOP 5.4.8-08/1 background for complex relating to people living with educated institution will make active labor
rehabilitation disabilities market policies more effective.
63
Code of the measure Topic Goal Hypothesized impact mechanism
Pilot program for the functional To develop the most essential
TÁMOP 5.4.9-11/1 Basic service delivery will be more effective
combining of basic services social and child welfare services
based on the combination of services.
and coordinate service delivery
Supporting local programs and Implementing the horizontal goals will
TÁMOP 5.5.1-A-10/1 To implement programs that
initiatives to implement horizontal improve the local conditions for the social
facilitate TÁMOP horizontal goals
goals* integration of vulnerable groups.
TÁMOP 5.5.2-09/1 Scaling up voluntary work— To establish the professional Standardizing voluntary work will improve
central coordination foundation of voluntary work service quality.
Scaling up voluntary work—local To scale up voluntary work by Scaling up voluntary work will improve the
TÁMOP 5.5.2-09/2 projects implementing voluntary capacities of CSOs and state- and
TÁMOP 5.5.2-09/3 programs and fostering civil municipality-run institutions;
TÁMOP 5.5.2-10/4 society organizations that deal standardization will also help develop
with voluntary work programs’ professional quality.
TÁMOP 5.5.3-08/01 Supporting organizations to CSOs will more easily participate in service
TÁMOP 5.5.3-08/02 develop and service civil society To develop civil society delivery; thus, capacities of such services will
TÁMOP 5.5.3-09/1 organizations organizations improve, and in lagging regions they might
fill gaps in service delivery.
Supporting antidiscrimination
programs in the media— Discriminated and vulnerable people will
TÁMOP 5.5.4-A-09/1 component A: media-related To train and employ people at become employed, hence their
training and employment of risk of discrimination representation will improve, which will
people with Roma background foster their impact on opinion building.
and people living with disabilities
Supporting antidiscrimination
To produce media broadcasts
programs in the media— Media representation of discriminated
TÁMOP 5.5.4-B-09/1 that establish a positive
component B: enhancing the people will improve and reduce negative
perception of discriminated
reduction of discrimination stereotypes about them.
groups
through the media
64
Code of the measure Topic Goal Hypothesized impact mechanism
Fighting discrimination—forming Uncovering discriminatory actions will
To foster actions against
public perception and fostering improve the advocacy capacities of affected
TÁMOP 5.5.5-08/1 discrimination and strengthen
the authorities’ work (single groups; awareness-raising programs will
the advocacy capacity of
beneficiary: Equal Treatment reduce the prevalence of discriminatory
discriminated groups
Authority) actions.
Dissemination of the importance Fostering the capacities of organizations that
TÁMOP 5.5.6-08/1 of consumer protection by To promote the importance of deal with consumer protection and
TÁMOP 5.5.6-08/2 forming awareness-based consumer protection disseminating knowledge will make
consumer behavior advocacy activities more effective.
Development of the network of
To develop advocacy work
advocacy organizations from
relating to making legal
TÁMOP 5.5.7-08/1 among the NGO/legal The effectiveness of legal protection will
representatives available for sick,
representatives for sick or improve.
institutional clients and in the
institutionalized clients and in the
realm of child protection
realm of child protection
Enhancing the chances of social
The programs will improve the life chances
integration for incarcerated To reintegrate people with
TÁMOP 5.6.1-A-11/1 of those who have criminal records, and
people and those under criminal records, prevent crime
preventative programs will reduce crime
patronage based on training and and victimization
levels, especially among young offenders.
employment programs
Special integration and The chance to attend trainings and obtain
TÁMOP 5.6.1-A-11/3 reintegration activities for To promote the social and labor marketable professional skills, along with
TÁMOP 5.6.1-A-11/4 incarcerated persons and those market integration of people incentives to the business sector, will
under patronage and living in with criminal records improve the target group’s chances of
corrective institutions integrating into the labor market.
65
Code of the measure Topic Goal Hypothesized impact mechanism
Assisting groups of minors and To foster the cooperation of the
youth via crime-prevention police and the receiving Curricular and extracurricular activities will
TÁMOP 5.6.1-B-12/1 programs who are especially institutions to address the needs improve youth’s knowledge and self-esteem
exposed to committing crimes and of minors and youth especially and professionals’ methodological toolkit,
becoming victims prone to committing crimes and which will in turn decrease crime levels.
becoming victims
Assisting the methodological To reduce crimes among minors Needs assessment and professional trainings
TÁMOP 5.6.2-08/1 development crime-prevention and youth, prevent victimization, will enhance more effective implementation
and reintegration programs and assist the reintegration of of crime-prevention and reintegration
fostering social cohesion offenders programs.
Assisting the methodological Training professionals and offering
development crime-prevention information and support services for victims,
To reduce crimes among minors
and reintegration programs and elaborating a stepwise reintegration
TÁMOP 5.6.2-10/1 and youth, prevent victimization,
fostering social cohesion—Phase II program for people with criminal records,
and assist the reintegration of
will enhance their reintegration to society
offenders
and into the labor market, which will also
reduce victimization.
Source: Hétfa and Alapítvány, 2013, pp. 46–49. Hypothesized impact mechanisms were elaborated based on content analysis of the calls as indicated in the report.
*The horizontal goals are: facilitating equal chances, sustainability, facilitating territorial cohesion, international and interregional cooperation, scaling up of social
innovation, and transfer of experiences.
66
Annex II. Relative variability of selected indicators
Figure A2.1. Access to sewage—Nine random settlements (%, 2000–2013)
100
90
80 Milota
70 Gyöngyöstarján
Kishartyán
60
Zalaszentgrót
50
Domaszék
40 Pócspetri
30 Felsőzsolca
Sásd
20
Tab
10
0
20002001200220032004200520062007200820092010201120122013
Source: World Bank staff calculations.
Figure A2.2. Long-term unemployment—Nine random settlements (%, 2000–2013)
500,
450,
400, Milota
350, Gyöngyöstarján
Kishartyán
300,
Zalaszentgrót
250,
Domaszék
200, Pócspetri
150, Felsőzsolca
Sásd
100,
Tab
50,
,
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Source: World Bank staff calculations.
67
Figure A2.3. Access to sewage—Settlements with the lowest coefficient of variation
110,
100,
Nemesvámos
Almásfüzitő
90,
Balatonszárszó
Szekszárd
80, Tatabánya
Komló
Keszthely
70,
Siklós
Sárhida
60, Kincsesbánya
50,
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Source: World Bank staff calculations.
Figure A2.4. Long-term unemployment—Settlements with the highest coefficient of
variation
250,
200, Vasegerszeg
Iván
Árpás
150,
Salköveskút
Pusztavacs
100, Kup
Örkény
Kőszegdoroszló
50,
Kéleshalom
Vereb
,
Source: World Bank staff calculations.
68
Annex III. Mapping for results in Hungary
Leveraging the approach introduced in Section 5.2 of this report, this Annex demonstrates
an example of how to use subregional information on human development with a project
data overlay. The example uses long-term unemployment data available at the
microregional level, combined with project data from EU–co-funded interventions aimed at
increasing employability and implemented in the 2007–2013 period. 35
As the starting point of the analysis, our first map (Figure 1) demonstrates the levels of long-
term unemployment in 2007, as well as the locations and relative size of relevant
employability interventions funded from TÁMOP 5 between 2007 and 2013.
Figure A3.1. Microregional long-term unemployment rate (2007) and employability
projects (2007–2013)
Long-term unemployment rates have been calculated as the number of jobseekers registered for longer than
180 days, for every 1,000 working-age (18–59) individuals in each microregion. Employability projects are
relevant TÁMOP 5 projects discussed in Table 7 of Section 5.2 (TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3,
TÁMOP 5.1.1-09/6-7, TÁMOP 5.3.1-08/1 and 2, TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1).
Projects were geocoded and subsequently mapped. Bubble sizes correspond to the amount of project funds
disbursed during the project cycle. Color scale gradients start at 16 (lightest) and extend to 97 (darkest).
35
The GIS tool used for this exercise is an ArcGIS Online platform (http://www.arcgis.com/online). This
platform is designed to showcase multiple map layers for data visualization and analytical purposes (also used
for mapping World Bank projects, available at http://maps.worldbank.org). Unemployment and population
data for map layers has been obtained from TEIR, and joined to microregional shapefiles using ArcMap 10.3
software. Project information has been provided by the Prime Minister’s Office: the project data has
subsequently been cleaned, geocoded, and uploaded to ArcGIS Online platform.
69
The map indicates a mixed territorial targeting accuracy of TÁMOP 5 projects aiming to
promote employability between 2007 and 2013. Findings include:
• employability interventions were focused on many microregions with high levels of
long-term unemployment, particularly those in northeastern and southwestern
Hungary—partly as a result of territorially targeting funds to the most disadvantaged
microregions (TÁMOP 5.1.1) that are also the microregions with high levels of long-
term unemployment;
• some microregions with levels of unemployment in 2007 comparable to those
receiving large volumes of projects have not benefited from EU funding at all,
including Pétervásár (northern Hungary), Püspökladány (Northern Great Plain), and
Nagyatád (South Transdanubia). It is important to note that none of these were
among the most disadvantaged microregions—that is, areas targeted through
TÁMOP 5.1.1;
• many microregions with the country’s lowest levels of long-term unemployment
(northwest and central Hungary) have also benefited from employability
interventions; 36
• many microregions where long-term unemployment levels are not the highest, but
are still high (in the northeast, east or southwest) have not received any EU funding
at all.
In the second step of the analysis, we look at the “final picture”—that is, long-term
unemployment data in 2013, keeping the same project overlay.
36
This demonstration ignores the possibility that interventions may have also been driven by higher levels of
unemployment among certain groups, such as youth, without affecting long-term unemployment rate overall.
These issues could of course be revealed by further analysis of unemployment data disaggregated by
geography, ethnicity, gender, and age.
70
Figure A3.2. Microregional long-term unemployment rate (2013) and employability
projects (2007–2013)
Long-term unemployment rates have been calculated as the number of jobseekers registered for longer than
180 days, for every 1,000 working-age (18–59) individuals for each microregion. Employability projects are
relevant TÁMOP 5 projects discussed in Table 7 of Section 5.2 (TÁMOP 5.1.1-09/1-2, TÁMOP 5.1.1-09/3,
TÁMOP 5.1.1-09/6-7, TÁMOP-5.3.1-08/1 and 2, TÁMOP 5.3.1-C-09/2, TÁMOP 5.3.1-08/2, TÁMOP 5.3.9-11/1).
Projects were geocoded and mapped. Bubble sizes correspond to the amount of project funds disbursed during
the project cycle. Color gradients start at 16 (lightest) and extend to 97 (darkest), equivalent to 2007 levels.
The first noticeable change is how much “lighter” the colors have become all around the
country: the fact that the color scale gradients of the 2007 and 2013 maps are normalized
suggests a significant improvement in terms of long-term unemployment since 2007. Indeed,
the average number of long-term unemployed per 1,000 working-age individuals in each
microregion has considerably decreased overall, from 54 in 2007 to 41 in 2013. The map also
demonstrates that territorial disparities have decreased somewhat—that is, the difference
between leading and lagging microregions is considerably smaller overall. Moreover, there
have been cases of “isolated” developments—that is, there are a few freestanding
microregions in overall lagging areas where long-term unemployment has improved more
than in neighboring microregions (for example, the Sátoraljaújhely microregion in the
northeast or the Szigetvár microregion in the southwest).
The GIS tool also provides an opportunity to showcase detailed project information about
employability projects. By clicking on a project location bubble, a pop-up window can share
key project information (location, title, target groups, amount, date, and so on) to external
stakeholders. This feature enhances transparency, accountability, and also provides a
71
platform on which to disseminate and exchange knowledge about EU-funded employability
projects.
Figure A3.3. The platform offers detailed project information
While the tool offers an approach that is easy to comprehend for policy audiences and
clients alike, it is limited in terms of interpretability. Most importantly, the tool is unable to
identify the extent of change that is in fact attributable to EU-funded employability
interventions between 2007 and 2013. The gradually increasing size of the public works
program since 2009 (in 2014, public works contributed approximately 1.5 percentage points
toward the decrease in the Hungarian unemployment rate) has likely had a considerably
higher impact on employment levels in many microregions. Also, private-sector investments
in some lagging microregions may also have contributed to improving employment figures at
the local level, which the tool is unable to capture. These and other dilemmas of the
proposed approach are elaborated in Section 5 of this report. Nevertheless, the data
visualization feature offers inputs to targeting and monitoring EU funds. It also offers a solid
basis upon which to introduce a results-based approach to OP implementation in the 2014–
2020 period, as well as accompanying features that enhance transparency, accountability,
and citizen feedback.
72