___ WORLD BANK TECHNICAL PAPER IN O. 3tAS Work in progress for public discussion k\ 1n11, Combiningy the Quantitative anid Qualitative Approaches to Poverty Nleasurement anid Analsis IX/ hw( ,fi , tid t/1(/t/Z Pott olfal I I_~~~~~~~~~~~W l s~~~~~~~~~~hl RECENT WORLD BANK TECHNICAL PAPERS No. 284 Valdes and Schaeffer in collaboration with Roldos and Chiara, Sutrveillance of Agricultutral Price and Trade Policies: A Handbook for Urugutay No. 285 Brehm and Castro, The Marketfor Water Rights in Chile: Major Issutes No. 286 Tavoulareas and Charpentier, Clean Coal Technologiesfor Developing Couintries No. 287 Gillham, Bell, Arin, Matthews, Rumeur, and Heam, Cotton Production Prospectsfor the Next Decade No. 288 Biggs, Shaw, and Srivastiva, Technological Capabilities and Learning in African Enterprises No. 289 Dinar, Seidl, Olem, Jorden, Duda, and Johnson, Restoring and Protecting the World's Lakes and Reservoirs No. 290 Weijenberg, Dagg, Kampen, Kalunda, Mailu, Ketema, Navarro, and Abdi Noor, Strengthening National Agricultial Research Systems in Eastern and Central Africa: A Frameworkfor Action No. 291 Valdes and Schaeffer in collaboration with Errazuriz and Francisco, Surveillance of Agricultural Price and Trade Policies: A Handbookfor Chile No. 292 Gorriz, Subramanian, and Simas, Irrigation Management Transfer in Mexico: Process and Progress No. 293 Preker and Feachem, Market Mechanisms and the Health Sector in Central and Eastern Europe No. 294 Valdes and Schaeffer in collaboration with Sturzenegger and Bebczuk, Surveillance of Agriculttural Price and Trade Policies: A Handbookfor Argentina No. 295 Pohl, Jedrzejczak, and Anderson, Creating Capital Markets in Central and Eastern Europe No. 296 Stassen, Small-Scale Biomass Gasifiersfor Heat and Power: A Global Review No. 297 Bulatao, Key Indicatorsfor Family Planning Projects No. 298 Odaga and Heneveld, Girls and Schools in Sub-Saharan Africa: From Analysis to Action No. 299 Tamale, Jones, and Pswarayi-Riddihough, Technologies Related to Participatory Forestry in Tropical and Sutbtropical Coutntries No. 300 Oram and de Haan, Technologiesfor Rainfed Agricuiltuire in Mediterranean Climates: A Review of World Bank Experiences No. 301 Mohan, editor, Bibliography of Publications: Technical Department, Africa Region, July 1987 to April 1995 No. 302 Baldry, Calamari, and Yameogo, Environmental Impact Assessment of Settlement and Development in the Upper L6raba Basin No. 303 Heneveld and Craig, Schools Count: World Bank Project Designs and the Quality of Primary Edtcation in Sub-Saharan Africa No. 304 Foley, Photovoltaic Applications in Rutral Areas of the Developing World No. 305 Johnson, Edtucation and Training of Accountants in Sub-Saharan Anglophone Africa No. 306 Muir and Saba, Improving State Enterprise Performance: The Role of Internal and External Incentives No. 307 Narayan, Toward Participatory Research No. 308 Adamson, Bates, Laslett, and Pototschnig, Energy Use, Air Pollution, and Environmental Policy in Krakow: Can Economic Incentives Really Help? No. 309 The World Bank/FOA/UNIDO/Industry Fertilizer Working Group, World and Regional Supply and Demand Balances for Nitrogen, Phosphate, and Potash, 1993/94-1999/2000 No. 310 Elder and Cooley, editors, Stustainable Settlement and Development of the Onchocerciasis Control Programme Area: Proceedings of a Ministerial Meeting No. 311 Webster, Riopelle, and Chidzero, World Bank Lendingfor Small Enterprises 1989-1993 No. 312 Benoit, Project Finance at the World Bank: An Overview of Policies and Instrtments No. 313 Kapur, Airport Infrastructure: The Emerging Role of the Private Sector No. 314 Valdes and Schaeffer in collaboration with Ramos, Surveillance of Agriclultural Price and Trade Policies: A HandbookforEcuador No. 316 Schware and Kimberley, Information Technology and National Trade Facilitation: Making the Most of Global Trade No. 317 Schware and Kimberley, Information Technology and National Trade Facilitation: Guide to Best Practice (List continues on the inside back cover) WORLD BANK TECHNICAL PAPER NO. 366 Combining the Quantitative and Qualitative Approaches to Poverty Measurement and Analysis The Practice and the Potential Soniya Carvalho Howard White The World Bank Washington, D.C. Copyright © 1997 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in the United States of America First printing May 1997 Technical Papers are published to communicate the results of the Bank's work to the development community with the least possible delay. The typescript of this paper therefore has not been prepared in accordance with the procedures appropriate to formal printed texts, and the World Bank accepts no responsibility for errors. Some sources cited in this paper may be informal documents that are not readily available. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. Requests for permission to reproduce portions of it should be sent to the Office of the Publisher at the address shown in the copyright notice above. The World Bank encourages dissemination of its work and will normally give permission promptly and, when the reproduction is for noncommercial purposes, without asking a fee. Permission to copy portions for classroom use is granted through the Copyright Clearance Center, Inc., Suite 910, 222 Rosewood Drive, Danvers, Massachusetts 01923, U.S.A. ISBN 0-8213-3955-9 ISSN: 0253-7494 The cover design, by Beni Chibber-Rao, incorporates a diagram of seasonality and changes in food availability, farm work, income, and disease, prepared by the women of Komaka, Ghana, as a visual for the Participatory Poverty Assessment conducted by the World Bank. Source: Andy Norton, Ellen Bortei-Doku Aryeetey, David Korboe, D.K. Tony Dogbe, "Poverty Assessment in Ghana Using Qualitative and Participatory Research Methods," PSP Discussion Paper 83 (World Bank, Poverty and Social Policy Department, Washington, D.C., 1995). Soniya Carvalho is in the World Banks' Poverty Anchor, PREM. Howard White is in the Institute of Social Studies, The Hague. Library of Congress Cataloging-in-Publication Data Carvalho, Soniya. Combining the quantitative and qualitative approaches to poverty measurement and analysis : the practice and the potential / Soniya Carvalho and Howard White. p. cm. - (World Bank technical paper ; no. 366) "March 1997." ISBN 0-8213-3955-9 1. Poverty-Statistical methods. 2. Poverty-Mathematical models. I. White, Howard, 1960- . II. Title. III. Series. HC79.P6C35 1997 97-16992 362.5'072-dc2l CIP Table of Contents Foreword ............... iv Abstract .............. v Acknowledgements ............ vi I. Introduction ............ 1 II. Characteristics of the Quantitative and Qualitative Approaches .1 III. Strengths and Weaknesses of the Quantitative and Qualitative Approaches .10 IV. Combining the Quantitative and Qualitative Approaches in Poverty Measurement and Analysis .16 V. Conclusion .26 Bibliography .27 -iii- Foreword The quantitative and qualitative approaches to poverty measurement and analysis have often been treated by practitioners as two distinct -- even opposing -- approaches. This paper argues that the two approaches are complementary and there are significant gains to be had from combining them. The different ways in which the quantitative and qualitative approaches can be combined in the measurement and analysis of poverty are highlighted in this paper together with some examples of how this has been done in World Bank Poverty Assessments. While the practice lags behind the potential, a good beginning has already been made in tapping the strengths of the two approaches in analytical work on poverty. I hope this paper -- a precursor to a new series from the Poverty Reduction and Economic Management Network -- will contribute to furthering these efforts. Network Head Poverty Reduction and Economic Management -iv- Abstract This paper highlights the key characteristics of the quantitative and qualitative approaches to poverty measurement and analysis, examines the strengths and weaknesses of each approach, and analyzes the potential for combining the two approaches in analytical work on poverty. The main conclusion of this paper is that sole reliance on either only the quantitative approach or only the qualitative approach in measuring and analyzing poverty is often likely to be less desirable than combining the two approaches. This is because there are limits to a purely quantitative approach as well as a purely qualitative approach to poverty measurement and analysis. Each approach has an appropriate time and place, but in most cases both approaches will generally be required to address different aspects of a problem and to answer questions which the other approach cannot answer as well or cannot answer at all. The need to combine the two approaches in analytical work on poverty cannot be overemphasized. There are three key ways to combine the quantitative and qualitative approaches: (i) integrating methodologies; (ii) confirming, refuting, enriching, and explaining the findings of one approach with those of the other; and (iii) merging the findings of the two approaches into one set of policy recommendations. Some ways in which the integration of methodologies can be achieved are: using quantitative survey data to determine the individuals/communities to be studied through the qualitative approach; using the quantitative survey to design the interview guide of the qualitative survey; using qualitative work to determine stratification of the quantitative sample; using qualitative work to determine the design of the quantitative survey questionnaire; using qualitative work to pre- test the quantitative survey questionnaire; and/or using qualitative analyses to refine the poverty index. "Confirming" or "refuting" are achieved by verifying quantitative results through the qualitative approach. "Enriching" is achieved by using qualitative work to identify issues or obtain information on variables not obtained by quantitative surveys. "Examining" refers to generating hypothesis from qualitative work for testing through the quantitative approach. "Explaining" involves using qualitative work to understand unanticipated results from quantitative data. In principle, each of these mechanisms may operate in either direction -- from qualitative to quantitative approaches or vice versa. "Merging" involves analyzing the information provided both by the quantitative approach as well as the qualitative approach to derive one set of policy recommendations. The quantitative and qualitative approaches are being increasingly combined in analytical work on poverty, but there remains scope for further strengthening the links between them. -v_ Acknowledgements This paper has benefitted from discussions with or comments from Mark Blackden, Adriana De Leva, Lionel Demery, Ann Duncan, Vajeera Dorabawila, Paul Glewwe, Christiaan Grootaert, Margaret Grosh, Jesko Hentschel, Narpat Jodha, Rajiv Kochar, Alexandre Marc, Caroline Moser, Josette Murphy, Andrew Norton, Ian Patrick, Caroline Robb, Lawrence Salmen, and Maurizia Tovo. -vi- Combining the Quantitative and Qualitative Approaches to Poverty Measurement and Analysis: The Practice and The Potential I. Introduction 1. The quantitative approach to poverty measurement and analysis is defined here as one that typically uses random sample surveys and structured interviews to collect the data -- mainly, quantifiable data -- and analyzes it using statistical techniques. By contrast, the qualitative approach is defined as one that typically uses purposive sampling and semi-structured or interactive interviews to collect the data -- mainly, data relating to people's judgements, attitudes, preferences, priorities, and/or perceptions about a subject -- and analyzes it usually through sociological or anthropological research techniques. 2. This paper highlights the key characteristics of the quantitative and qualitative approaches to poverty measurement and analysis, examines the strengths and weaknesses of each approach, and analyzes the potential for combining the two approaches in analytical work on poverty. World Bank Poverty Assessments are used to contextualize the discussion since they represent an important piece of analytical work on poverty at the country level.' 3. The paper is primarily aimed at policymakers and staff from donor agencies involved in poverty measurement and analysis. The discussion of the relative merits and demerits of the quantitative and qualitative approaches and ways of combining the two approaches should also have broader appeal to anyone interested in methodological issues in development research. II. Characteristics of the Quantitative and Qualitative Approaches 4. A number of characteristics differentiate the quantitative approach from the qualitative approach to poverty measurement and analysis. The main differences between the two approaches are discussed below and sununarized in Table 1. Defining Poverty 5. The quantitative approach typically defines poverty in terms of income or consumption although other measures (e.g., access to basic social services, nutritional status, literacy rates) are also often included. Income and consumption are used because they are either seen as "ends" in themselves or because they are considered to be sufficiently well-correlated with other welfare indicators (e.g., literacy, nutritional status) to suffice by themselves.2 '/ Poverty Assessments refer to country-specific analyses of poverty undertaken by the World Bank. They typically include a poverty profile; the examination of the incentive and regulatory framework, public expenditures, and safety nets; and policy recommendations with respect to each of these issues. 2/ Once more importance is given to the ordinal measurement of poverty rather than the cardinal one, consumption has been found to be a good indicator of welfare (Hentschel and Lanjouw, LSMS paper 124). -1- Table 1: Characteristics of The Quantitative and Oualitative Approaches Characteristics Quantitative Approach Qualitative Approach Defmition of Poverty People considered poor if their Poor people defme what poverty means, standard of living falls below the broader definition of deprivation resulting poverty line, i.e., the amount of from a range of factors (not simply lack of income (or consumption) associated income/consumption) adopted with the minimum acceptable level of nutrition and other necessities of everyday life Philosophical underpinning Positivist paradigm: existence of one Rejection of the positivist paradigm: there reality (Chung 1996) are multiple forms of reality and, therefore, it is senseless to try to identify ____________________________ only one (Chung 1996) Determination of poverty Determination by external surveyors Determination through an interactive internal-external process involving facilitator and participants Nature of variables for which Quantifiable, e.g., household Perception variables reflecting attitudes, data is collected expenditures on food, unemployment preferences, and priorities (see Moser, rate 1996); the number of similar responses with respect to each variable can be numerically added-up, but the variables themselves cannot be quantified Interview format Structured, formal, pre-designed Open-ended, semi-structured, interactive questionnaire Sampling Probability sampling Purposive sampling Sampling error Less sampling error but prone to more More sampling error but tends to reduce non-sampling error non-sampling error Sample size 2000-8000 households 1-1000 individuals or communities (Living Standards Measurement Survey, LSMS: 2000-5000 households) Geographic Coverage Wide: typically, national Small: typically, a few regions, or selected communities -2- Average time LSMS: Roughly 2 and a half years for Six to nine months for average-sized PRA the highest quality survey in a country component of Poverty Assessment; where year round coverage is desired roughly four months for average-sized (one year for planning; one year for Beneficiary Assessment. In some field work; six months for initial situations, the qualitative approach can be analytic phase of producing an abstract, time-consuming because of the lead time documenting the data, and setting up required for training interviewers and the other analyses). The planning process lengthy process of classifying qualitative can be abridged if: (i) capacity is very findings to analytical categories (Salmen high; and/or (ii) there is willingness to 1995). compromise on quality. Similarly, the interviewing period can be reduced from one year to something like three montis if the ability to cover the whole year witi analytic questions is sacrificed. The majority of the surveys compromise on one aspect or the other, so the actual time is almost always lower. Priority Survey: 7 months Average cost LSMS: Roughly $500,000 to Roughly US $50,000-150,000 for (average $1,000,000 on average; some part of sized) qualitative component of Poverty this is often provided in-kind by Assessment. The Focused Area Study government and international agency Technique (FAST) which was used to staff so the actual cost to the LSMS review usage of health and education conducting agency may be lower. facilities in Tanzania as an input to the social sector review was conducted at a Priority Survey: $200,000-400,000 (if cost of about $52,000 and took about nationally representative) fifteen weeks. Statistical Analysis Statistical analysis forms an important Statistical analysis makes little or no use of part of approach it. Triangulation is employed i.e., simultaneous use of several different sources and means of gathering and interpreting information. The expectation is that bits and pieces of information gathered from different sources will yield a pattern of responses. Systematic content analysis and gradual aggregation of data based on themes from the household, group, village, district, and national levels may also be used 6. This definition determines the type of poverty data that is collected under the quantitative approach as well as how it is analyzed. A number of conceptual issues are relevant.3 First, consumption is generally preferred over income under the quantitative approach. This is because: (i) consumption is more accurately recorded for households that may have diverse sources of income and for which the net income from several activities may not be known; and (ii) income may fluctuate 3/ A similar list to that given here is given by Paul Streeten in Rolph Van der Hoeven and Richard Anker 1994. -3- widely from period to period whereas consumption will be smoothed. Second, the best unit of analysis is the "individual" -- rather than the "household" which is typically used in quantitative surveys. Households differ in size and composition, and so a simple comparison of aggregate household consumption could be quite misleading about the well-being of individual members of a given household. Measures should, therefore, be stated in terms of per adult equivalent, i.e., for a household of any given size and demographic composition (such as one male adult, one female adult, and two children), an equivalence scale measures the number of adult males (typically) which that household is deemed to be equivalent to (Martin Ravallion 1992). Third, there are non-income consumption items that have to be taken into account, the major items being consumption of own produce and the use of social services. The former is routinely included (a pecuniary value is assigned by the enumerator, but is often thought to be undervalued) although the latter is not. Fourth, the quantitative approach excludes the consumption of leisure inspite of the fact that the quantity and quality of leisure can be a major benefit of socio-economic advance.4 A participatory study in a Pakistani village found that "more time at home" was consistently ranked as being more important than "more income" (Robert Chambers 1995). 7. The poverty line is a key concept in the quantitative approach. There are two important issues with respect to drawing the poverty line under the quantitative approach: the choice between an absolute and relative poverty criterion; and choosing the location of the line. First, the absolute poverty line is commonly drawn based on the cost of meeting some calorie requirement and perhaps adding an allowance for other essentials. Relative poverty refers to the position of an individual or household compared with the average income in the country, such as a poverty line set at one-half of the mean income, or at the 40th percentile of the distribution. Relative poverty lines will vary with the level of average income. Notions of absolute poverty whereby the poverty line does not vary with overall living standards -- appear to be relevant to low income countries while relative poverty is of more relevance to high income countries (Martin Ravallion 1992). Second, the extent of poverty is clearly influenced by the choice of the poverty line. It is common practice to use two lines in World Bank Poverty Assessments; twenty of the thirty Poverty Assessments reviewed by Dayton et al used more than one poverty line (Julia Dayton et al, 1993). To the extent that one is concerned simply with whether poverty is increasing or decreasing, one may use the dominance approach which allows us to determine the direction of poverty over time or with/without some policy change independently of the particular poverty line or poverty measure selected; it is based on a comparison of cumulative income distributions at two times (Martin Ravallion 1992). 8. On the other hand, the qualitative approach adopts a considerably different approach to defining poverty. In fact, the definition of poverty marks a central conceptual difference between the quantitative and qualitative approaches (Bob Baulch 1996). The qualitative approach holds that "income poverty" does not capture aspects of well-being that are considered important by the poor themselves. The indices of living standards based on a range of non-income indicators (such as housing quality, nutritional status, access to services, and asset holdings) for Zambia and Cameroon turned out to be only imperfectly correlated with the headcount (Zambia Poverty Assessment 1994, Cameroon Poverty Assessment 1995). In the qualitative approach, the community being studied itself defines what poverty means: 4/ While economic analysis of quantitative data can incorporate leisure, this is complicated and requires making assumptions that may not be realistic so this is usually not done. -4- The qualitative approach elicits local people's own conceptions of poverty/deprivation and harnesses their own priorities in the complex and heterogeneous societies in which they live (Robert Chambers 1992, 1995). 9;, The definition of poverty typically adopted under the qualitative approach involves a broader conception of poverty and deprivation than does the definition typically adopted under the quantitative approach. The qualitative approach defines poverty so as to capture the processes and interactions between social, cultural, political, and economic factors. It includes a wider range of factors such as vulnerability, isolation, powerlessness, survival, personal dignity, security, self-respect, basic needs, and ownership of assets than does the definition of poverty under the quantitative approach. There is no inherent reason why quantitative surveys cannot include questions pertairning to many of the variables (for example, asset ownership variables) identified as important by qualitative methods. In fact, this has already been happening in recent years, and as discussed later, has been done, for example, in the World Bank's Living Standards Measurement Surveys. Samplig 10. The quantitative approach relies on probability sampling -- i.e., selecting a sample in a way that every unit in the population has some probability of selection and that probability is known -- and typically uses random household sample surveys. Poverty estimates for the population are usually based on such random household sample surveys. Table 2 presents the different ways of selecting a probability sample. Table 2: Probability SamDling Types of Deriitions Probability Sampling Random Sampling A sampling technique in which each and every unit of the population has an equal chance of being selected in the sample. , Stratified Random A sampling technique that divides the population into different groups or classes called strata Sampling and draws a sample from each stratum at random. Systematic Random A sampling technique that selects one unit at random and then selects additional units at evenly- Sampling spaced intervals until the desired sample size has been reached. ource: Statistical Methods, S. P. Gupta, Sultan Chand and Sons, New Delhi, 1982. 11. On the other hand, qualitative research generally uses very different forms of sampling than used under the quantitative approach. The philosophy underpinning the definition of poverty in the qualitative approach typically leads it to focus in-depth on relatively small samples, even single cases, selected purposefully on the assumption that a great deal can be learnt about issues of concern from the detailed study of a few information-rich cases. In purposive (or non-probability) sampling, the expertise of key informants and specialists is used to select poor people or field sites for in-depth study. Wealth ranking (i.e., ranking individuals or households by well-being or wealth), social mapping (i.e., constructing maps of communities or community facilities), or rapid appraisal -5- indicators such as housing condition may also be used to identify the poor. Table 3 presents some techniques that are used in purposive sampling. Table 3: Non- robabilitv Sampling Types of Non-Probability Sampling Definitions Accidental Sampling Sampling is accidental when a person is sampled by accident because she or he happens to be available, or because she or he arrives at your doorstep and wants to talk. In a study in Indonesia, two teenagers came to the house of a study team and asked permission to speak. They proceeded to tell the team a bizarre story about why the hydraulic ram at a particular spring did not work. On checking this story with others in the community, the researchers found that, after initial denials, people acknowledged its accuracy and further elaborated upon the situation. Snowballing A sampling technique that involves asking a key informant to name other people who should be contacted by the investigator in order to understand some aspects of a situation under study. Common sense sampling A sampling technique that includes a range of people or a variety of different situations in the study sample with the basic aim of avoiding error through a bias in the sample by ensuring sufficient diversity. Quota or proportionate sampling A quota or proportionate sample is chosen to reflect the distribution of different socio- economic groups based on the relative distribution of these groups in the population. ource: Deepa Narayan, "Towards Participatory Research, World Bank Technical Paper 307, World Bank, 1996. 12. While the rationale for random sampling under the quantitative approach is that selecting a truly random, statistically representative sample will permit generalizations from a sample to a larger population, the qualitative approach does not aim at statistical accuracy, but rather aims at portraying the perspectives of crucial concentrations of people (Lawrence Salmen 1995). The qualitative researcher generally focuses on the particular research community being studied without attempting to generalize for the country as a whole. -6- Data Collection Methods 13. The varying definitions of poverty under the quantitative and the qualitative approach have implications for how poverty data is collected and analyzed under the two approaches. Under the quantitative approach, the enumerator is always invariably an outsider whose main role is to "extract" information. Quantitative surveys are administered using pre-designed questionnaires with well- defined (rather than open-ended) questions. The answers of each respondent are recorded by the external enumerator. Once the survey has been administered on all households in the sample, the resulting data are encoded into data files and then processed to produce the final statistical product as, for example, in the case of agricultural production figures for the national accounts. 14. On the other hand, in the qualitative approach, an interactive internal-external process involving both facilitator and participant is adopted to gather information about who the poor are instead of the third-party determination supported under the quantitative approach. Participants play an active role in generating analyses and directions for action, rather than simply being informants providing information for outsiders to analyze. Facilitators stimulate examination of issues by local participants but strive to avoid adopting a dominant mode in behavior or attitudes. When the participant-observer technique of qualitative research is adopted, the facilitator actually becomes an "insider" by himself participating in local activities. 15. Pre-designed questionnaires are an exception rather than the rule in qualitative research. Even when they are used in qualitative research, they are most likely to be informal and open rather than fixed or closed; questions can be added or dropped according to the situation. The questions are tackled in semi-structured interviewing. Some methods of data collection under the qualitative approach are presented in Table 4. Table 5 lists the various techniques for identifying the purposive sample or organizing the information emerging from qualitative work. Table 4: Oualitative Data Collection Methods Methods Description Key informant interviews Interviews with persons selected on the basis of their special knowledge and experience in area of interest. Number of informants usually varies between 10- 25. Focus groups Discussion among 8-12 homogenous participants; moderator plays non-intrusive, guiding role. Community meetings Meetings open to all where participants self-select themselves. Answers are elicited to pre-designed list of questions. Special effort has to be made to ensure participation of women and marginalized groups. Structured direct Observation by an individual or team over a period of days or weeks to collect observation information on pre-determined questions. Participant Observation Observation by an individual over a prolonged period by becoming a part of the (PO) community. Differs from structured direct observation in three ways: PO requires more time; focuses on social & cultural phenomena rather than on the physical environment; and requires empathy with the people being studied to get an insider's perspective. Informal Surveys Non-probability sampling using non-structured (conversational) interviews. Source: Kumar (1993), Norton & Stephens (1995) -7- Table 5: Techniques used in Qualitative AnalWsis Technique Description Social mapping and modelling Communities draw a map of their community identifying structures, institutions, or community facilities that are important in their lives. The drawings may be on the ground or on paper. Seasonality maps Communities make a month-by-month picture of how a particular phenomenon (e.g., sickness or food supply) varies over the course of a year. Oral histories and ethno biographies Usually personal but can extend to crops, animals etc. Daily time use analysis Indicating time spent and effort/drudgery on different activities, perhaps witi seasonal variations. Participatory linkage diagramming Diagram of chains of causality. Venn (or chapati) diagrams Showing the relative importance of different individuals and/or institutions in the community. Wealth or well-being ranking Individuals or households rank themselves by well-benig or wealth as defined by themselves. ource: Robert Chambers "The origins and practice of participatory rural appraisal", World Development vol. 22 pp. 959- 961, 1994. Instruments 16. The main types of household survey instruments used in recent years by the Bank for poverty measurement and analysis under the quantitative approach are: the Living Standards Measurement Survey (LSMS); and the Priority Survey (PS).5 The first LSMS surveys were implemented in Cote d'Ivoire in 1985 and in Peru in 1985/86. Since then over 40 LSMS surveys have been conducted in 19 countries and as of 1996 new LSMS surveys were in the field or were being planned in 9 additional countries. The prototypical LSMS has three core modules: the household roster including, for example, age and gender of members; the expenditure module; and the income and assets module. The sectoral modules are: education, health, employment, agriculture, household non- agricultural enterprises, fertility, credit and savings, housing, migration, and anthropometrics. These modules may be modified or dropped according to the requirements and context of the survey. The survey is generally administered in two visits to each household, a visit lasting 3-4 hours (but no more than one hour per person). Data from the first visit are checked for consistency and questions repeated in the second visit as needed. 17. LSMS may be differentiated from more traditional surveys in the following ways: rather than merely measuring welfare levels, LSMS collects data to allow the analysis of welfare determinants; LSMS has smaller sample sizes (i.e., 2000-5000 households) than many other quantitative surveys: the result could be a larger sampling error but the benefit is richer information; and LSMS is flexible with the modular design allowing customization to country and requirement specific purposes (Grosh 1991). In the proto-type design, at the end of the first visit the enumerator draws a sketch of the dwelling with measurements (allowing calculation of living area per person). The enumerator also 5/ The Integrated Survey (IS) is very similar to LSMS and is not referred to separately here. -8- fills in details on the materials used to construct outside walls, floors, main roof and materials with which windows are fitted. An inventory of durable goods including items such as sewing machines, fans, radio and TV, bicycles and cars (as well as the age and estimated value of the goods) are also recorded -- variables that are typically part of a qualitative investigation of living conditions. (For an annotated LSMS questionnaire see Christian Grootaert 1986). 18. Priority Surveys (PSs) are another instrument used for poverty measurement under the quantitative approach. The first PS was conducted in Guinea Bissau in 1991 (The Social Dimensions of Adjustment Priority Survey, 1991). The PS was developed as a means of rapid collection of priority information during adjustment. The main difference between the LSMS and PS surveys lies in the population coverage and the breadth of issues covered; LSMS has long questionnaires administered on a relatively small sample of households (approx. 2000-5000) while PS has a short questionnaire administered on a relatively large sample of households (approx. 8000). The PS is concerned with the detection of trends so it must be administered regularly, probably annually. Data are to be collected on a single visit to each household; they are to be collected by interview with a single respondent; they are to be collected and processed quickly with a target of about three months from the date of collection to the preparation of basic data files and tabulations. 19. On the other hand, the main methodologies used under the qualitative approach are: Participatory Rural Appraisal (PRA), in the 1970s called Rapid Rural Appraisal (RRA); PRA's urban equivalent, Participatory Urban Appraisal (PUA)6; and Beneficiary Assessment (BA).7 PRA/PUA place considerable emphasis on community-level interviewing while Beneficiary Assessments have tended to concentrate on household-level or individual interviews. (Norton and Stephens, 1995). Both methods share many core techniques including conversational and semi-structured interviews, focus group interviews, and participant observation. But, PRA/PUA supplement the core techniques with thematic mapping, wealth preference, and problem ranking, seasonal calendar building, trend analysis, and institutional diagramming and other techniques by which participants generate their own analyses of key elements of their livelihoods.8 Another difference between the two techniques seems to be that Beneficiary Assessments focus primarily (though not necessarily exclusively) on the beneficiary while PRA/PUA lay considerable importance on the views of other stakeholders in addition to the beneficiaries themselves. However, in recent years BAs are increasingly extending their focus to service providers, managers, and donors as well. Moreover, while PRAs are "empowering" for the poor, BAs are directed more towards sensitizing managers. Furthermore, unlike PRAs that did little, if any, sampling, in Beneficiary Assessments, one-on-one conversational interviews are meant to be quantified -- and hence must be numerous enough to be considered 6/ PRA is a label given to a growing family of participatory approaches and methods that emphasize local knowledge and enable local people to do their own appraisal, analysis, and planning. PRA uses group animation and exercises to facilitate information-sharing, analysis, and action among stakeholders. A recent term used to refer to PRA and PUA simultaneously is "Participatory Leaming and Action (PLA)". PLA is not only about the production of information; PLA promotes integrative leaming, shared knowledge and flexible analysis. Advocates of PLA argue that the production of knowledge and the generation of potential solutions should be devolved to those whose livelihood strategies form the subject of research. 71 Poverty Assessments incorporating these or other qualitative techniques have been traditionally called Participatory Poverty Assessments; we do not use the term here because this paper does not address the issue of "participation" in Poverty Assessments which is an important aspect of Participatory Poverty Assessments. I/ Some proponents of PRA distinguish it from RRA, arguing that the former is 'empowering" for the poor whereas the latter was .extractive". -9- significant by decision-makers. Focus group interviews and participant observation are done primarily for illustration and contextual background and need not conform to the same standards of representativity as do the household or individual interviews (Lawrence Salmen 1995). m. Strengths and Weaknesses of the Quantitative and Qualitative Approaches Strengths of the Quantitative Approach Making Aggregation Possible 20. The most powerful strength of the quantitative approach and one that accounts for its wide use in Poverty Assessments is the ability to test hypotheses about behavioral relationships in a way that is relevant at the national level and in a representative manner. The quantitative approach makes it possible to systematically measure the response of many individuals or groups of individuals to a set of questions without necessarily surveying the entire population. The quantitative approach then allows statistical aggregation of the data based on the sample. Providing Results whose Reliability is Measurable 21. Another important strength of the quantitative approach is that it provides results whose precision is measurable. The quantitative approach can indicate the reliability of the findings and the significance to be attached to them. Allowing Simulation of the effects of different Policy Options 22. Quantitative work allows the determination of the impact of different sets of policies on target populations or target variables. This strength of the quantitative approach accounts for its wide use in policy formulation. Weaknesses of the Quantitative Approach Sampling and Non-Sampling errors 23. Data collection under the quantitative approach is subject to both sampling error and non- sampling error. Sampling error refers to the error inherent in making inferences for a whole population from observing only a subset of its members. Sampling error is a function of the absolute size of the sample (not the sample size relative to population) and the variance of the item being measured (if it has zero variance, then a sample of one will suffice). The acceptable degree of sampling error can be specified and a survey designed accordingly. For example, the LSMS survey uses smaller samples than is usual for surveys of its type. This procedure has been defended as follows: In Cote d'Ivoire, the confidence interval around the estimate of the school enrollment ratio is 2 percentage points, which is wider than accepted for some surveys. The issue, however, is not whether one is able to distinguish between enrollment rates of 53 percent or 55 -10- percent. In either case, it is unacceptably low, and the government will wish to raise it (Grosh 1991). 24. Non-sampling error can arise from coverage (data units either accidentally omitted or failing to respond) or content (falsification, misunderstanding or incompetence on the part of either respondent or enumerator). It is, of course, likely that the larger this undertaking then the greater these non-sampling errors will be: partly as cost will dictate a slimmer questionnaire, allowing for fewer consistency checks on the data. Hence, there is a trade-off between sampling error and non- sampling error; a trade-off that will favor sample surveys over censuses for virtually all purposes other than the decennial population census. 25. While the quantitative approach is less prone to sampling errors than the qualitative approach because of the importance attached to sampling under the quantitative approach, the quantitative approach is generally more prone to non-sampling errors compared with the qualitative approach. The values of total household consumption obtained from quantitative methods may underestimate true household consumption as household members forget the details of their past expenditures. This is a major concern as consumption data are the most fundamental measure of household welfare obtained by quantitative methods.9 Also, respondents may not necessarily say what they feel in formal, structured interviews; true feelings may be better revealed by qualitative methods. Dennis Casley puts it thus: The non-sampling error measured in a structured survey may reveal the incidence of incorrect replies, but not the incidence of correct but superficial replies (Introduction by Dennis Casley in Ed Krishna Kumar 1993). Missing what is not easily quantifiable 26. Another limitation of the quantitative approach is to measure what is easily measurable at the expense of more important items that are less readily amenable to quantification. Quantitative approaches tend to focus primarily on quantifiable information and what is not measurable becomes discarded as unimportant: What is measurable and measured then becomes what is real and what matters, standardizing the diverse and excluding the divergent and different (Robert Chambers 1995, page 8). Jodha made an entry into the debate on poverty measurement with his finding: Households that have become poorer by conventional measures of income in fact appear better-off when seen through different qualitative indicators of their well-being (Narpat Jodha 1988, page 2421). 9/ Said with reference to LSMS in Margaret Grosh and Paul Giewwe 1994. -11- 27. Jodha compiled survey results for a sub-sample of 95 farm households in two villages in Rajasthan. Of these 95 households, 35 had experienced a real decline in per capita income of more than 5 percent between 1963-66 and 1982-84. A range of qualitative indicators were then reported for the same households, these indicators being based on vw'riables villagers themselves believed to be important aspects of well-being. These variables included the state of client/patron relations, dependence on inferior jobs, mobility and liquidity position, and "asset position" (including ownership of durables such as a bicycle and radio). Out of the 39 variables covered, the position of these households had declined in only one instance. As shown in Table 6, some of the improvements were dramatic. Table 6: Indicators of Chanainz Well-being for Households with DecliningIncome Variable Percent of Percent of households households 1963-66 1982-84 Residing on patron's land 31 0 Taking loan from other besides patron 13 47 Withdrawing children from school during crop season 17 6 Having ready cash at home of Rs 200 or more during slack 0 26 season Relying on day-to-day petty purchases of key provisions 100 51 Occasionally consuming green vegetables in non-crop season 0 100 Consuming rice on non-festive occasions 0 14 Women and children regularly wear shoes 0 86 House with separate quarters for human and animals 6 52 Possessing radio 0 7 ource: N.S. Jodha, "Poverty Debate in India; A Minority View", econominc and Political Weekli, November 1988, Tables 3-7. 28. Jodha suggests three methodological deficiencies in the quantitative approach that may explain this discrepancy: (i) differences in concepts and categories considered important; (ii) the yardsticks and norms used for assessment; and (iii) a communication gap between researcher and respondent. An example of the first problem is that household income may exclude low-value self-provisioning activities such as those based on common property resources; access to common property resources may also be ignored in the assessment of a household's asset position. Many important activities excluded in the conventional measures of the social science researcher may not be readily measurable, and so are by-passed by the yardsticks selected. As mentioned above, Jodha's attempts to incorporate these aspects revealed vast shortfalls. This problem is further compounded by the communication gap which exists: for example, the units of conversion used by the farmer (such as cartloads) into the -12- scientific units of the researcher, or the precision required by the researcher (hours spent on a specific activity) compared to the relative vagueness of the farmer (ten to twelve days). Failure to Capture the Intra-household Dynamic 29. Quantitative methods typically use the household as the unit of analysis."0 Using the household as the unit in the quantitative approach can restrict the analysis of intra-household allocation if the household head is responsible either for answering questions or indicating which household members are best qualified to answer certain questions." The qualitative approach uses the community as the unit of analysis and focuses on the links between households. It is better able to capture the relations within a household. Qualitative work in Zambia found that men rank assets highly whereas women focus more on basic needs. These differences reflect the division of tasks in the household -- in rural Zambia women can spend upto five hours a day collecting water during the dry season (Zambia Poverty Assessment 1994). Depending on the coping strategy (selling assets versus taking children out of school), men and women will be differently affected. Strengths of the Qualitative Approach Richer Definition of Poverty 30. The qualitative approach is particularly suited for "identification", i.e., to address the question "who are the poor?" It can more readily measure the wider dimensions of poverty, including for example, vulnerability, lack of dignity, and autonomy. Most importantly, the qualitative approach allows the community itself to analyze its own poverty and determine what the most important manifestations and solutions to poverty are for them. At the project or local level such information may be critical for the design of projects and policies aimed at reaching the poor. Explaining Causal Processes 31. The qualitative approach enables causality to be introduced between variables. Some of the variables used in the qualitative approach tell about the processes through which people became poor and about their vulnerability in the future (Bob Baulch 1996). Moreover, qualitative work is particularly suitable for uncovering the reasons for the observed levels of supply response or service up-take by beneficiaries. For example, the qualitative work for the Zambia and Kenya Poverty Assessment highlighted the rudeness of health staff and problems in fee exemption schemes as an important factor which discouraged the use of health facilities. l'/ The use of individuals or communities as the units of analysis is less common in quantitative work than in qualitative work although not non-existent. Increasingly, quantitative analyses are expanding their focus from household heads to other members of the households as well. "/ See comnments by Bridget 0 Laughlin on LSMS in Gender Dimensions in Household Survey Research ISS/NIDI Population and Development Teaching Text, February, 1996. -13- Accuracy and Depth of Information 32. The qualitative approach can elicit accurate responses to certain questions. People are more apt to talk candidly in a conversational setting than to an enumerator conducting a pre-worded questionnaire. The information obtained through qualitative work may be more accurate because the interviewer develops rapport and trust with the interviewer (Salmen 1995). The researcher can ask follow-up questions, and probe for responses. Non-verbal responses can be recorded which may supplement (or even contradict) verbal responses. Moreover, some things that cannot be asked in preset questionnaires may emerge automatically in conversations during qualitative research. In many instances, especially when an interpretive understanding of a phenomenon or process is required, qualitative methods are more successful in obtaining relevant data, ideas, or recommendations. In addition, the open format of the interview under the qualitative approach allows adding or dropping questions according to the context. The qualitative approach provides flexibility to the investigator to explore new ideas and issues that may not have been anticipated in planning the study. Weaknesses of the Qualitative Approach Inability to Generalize Beyond the Research Area 33. The small size of the population covered, the use of non-probability sampling to select that population, the changing content and focus of each of the interviews depending on circumstance, and the open-ended nature of responses often make aggregation and summarization of qualitative data difficult. Qualitative data can give a relatively accurate picture of the prevalence of a phenomenon but not necessarily of its extent or pervasiveness. 34. The South Africa qualitative poverty study discusses the difficulties faced in generalizing the results of qualitative work: Qualitative research can run the risk of being excessively specific to local conditions and needs. As a result, while the recommendations which are made may be appropriate to the particular circumstances in a community or to a particular issue, they are not readily generalizable to the national or cross-sectoral level. Special efforts may be required to analyze the key findings of the Participatory Poverty Assessment and interpret these into issues to be brought into the process of policy formulation (Julian May et al, 1996). Difficulties in Verifying Information 35. In view of the subjectivity involved in collecting and analyzing the data under the qualitative approach, the data cannot be easily verified. The subjectivity results from the nature of the information sought, non-structured interview formats, and the flexible response-recording and response-analysis methods. The chances that the moderator may bias results by providing cues about the desirable types of responses and answers are higher under the qualitative approach than under the quantitative method. 36. In sum, the merits and demerits of the quantitative as well as the qualitative approach underline the importance of not relying solely on any one approach. For example, while the -14- quantitative approach may minimize sampling errors, non-sampling errors may be more of a concern under the quantitative approach than in the qualitative approach. Moreover, while it may be possible to expand quantitative surveys like LSMS to obtain data on variables that are typically included in qualitative work (see paragraph 17) and vice versa, to the extent that the different data collection methods and the different ways of analyzing the information can contribute to greater accuracy and additional insights, reliance on only one of the two methods will generally be less desirable than reliance on both methods. Different reasons have been given in the World Bank's Poverty Assessments for combining the quantitative and qualitative approaches. The proposed Poverty Assessment for Cote D'Ivoire states: Some [qualitative] methodologies will be used to correct for weaknesses in existing quantitative data sets. Others will be used to obtain different kinds of data and to amplify the analysis that can be done on existing and new quantitative information .... These community-based studies will allow us to substantially increase knowledge about those social categories which are often not captured by household surveys -- especially those who may not be found in household samples including the homeless, itinerant poor petty traders, prostitutes, and undocumented migrant workers. These are likely to be among the poorest and most vulnerable groups about whom more must be known so that informed decisions can be made about targeting assistance towards them (Poverty Assessment, 1996). The qualitative study undertaken for Kenya's Poverty Assessment quotes a Bank Task Manager: I do my statistical analysis and go back to what the poor say as a check. In many cases, the Participatory Poverty Assessment provided important information which I am convinced cannot be obtained from surveys -- for example, the information about the poor's perceptions of poverty, distribution of property upon divorce, the detailed information on school dropouts, and how people cope. In other cases, the Participatory Poverty Assessment refuted or confirmed the numbers we had from the statistical survey in which I did not have full confidence because of the many problems experience in doing the survey. For example, the Participatory Poverty Assessment pointed out the greater poverty among female-headed households and also provided very convincing information about the extent of the water problem, neither of which were captured by the survey (Deepa Narayan and David Nyamwaya 1996). 37. The Bank's Operational Directive OD 4.15 on Poverty Reduction is also explicit in recognizing poverty as a multidimensional phenomenon requiring quantitative and qualitative treatment: The economic framework [of a country] not withstanding, poverty reflects the results and complementarities among cultural, sociological, and political factors. Analyses of the cultural constraints, sociological context, and/or political dynamics in which poverty persists contribute -15- to understanding the process of poverty in a particular country and to evaluating the full costs and likely benefits of alternative measures to reduce poverty. Such analyses also contribute to the development of the overall country institutional framework for poverty reduction and strategies to build institutional capacity to analyze poverty and to design interventions to reduce it. These analyses add to the cost of interventions, but they improve policy, project, and program design and sustainability, and reduce failure risks. (OD 4.15, World Bank, 1991, Para 8). 38. Overall, the need for complementing each approach with the additional perspectives offered by the other as well as the need to build links between the two approaches cannot be overemphasized. IV. Combining the Quantitative and Qualitative Approaches in Poverty Measurement and Analysis 39. There are a number of ways in which the quantitative and qualitative approaches may be combined in the measurement and analysis of poverty. Three major ways of doing so are: * integrating the quantitative and qualitative methodologies; * exanining, explaining, confirming, refuting, and/or enriching information from one approach with that from the other; and * merging the findings from the two approaches into one set of policy recommendations. 40. The key is to tap the breadth of the quantitative approach and the depth of the qualitative approach. The exact combination of qualitative and quantitative work will depend on the purpose of the study and the available time, skills, and resources. In general, integrating methodologies can result in better measurement; confirming, refuting, enriching and explaining can result in better analysis; and merging the quantitative and qualitative findings into one set of policy recommendations can lead to better action. 41. Of the three ways of combining quantitative and qualitative approaches (i.e., integrating methodologies; confirming, refuting, enriching, and explaining findings of one approach with those of the other; and merging the findings of both into one set of policy recommendations), the first has been less common in Poverty Assessments, but the second and third have been supported in almost every Poverty Assessment reviewed for this paper -- albeit to varying degrees."2 I2/ Of 11 Poverty Assessments that use both quantitative and qualitative approaches and were reviewed here, only 5 incorporated an integration of the quantitative and qualitative methodologies, whereas all 11 incorporated the other two ways of combining quantitative and qualitative approaches. Overall, beginning in 1993 with the Poverty Assessment for Uganda when qualitative analyses were first used in World Bank Poverty Assessments, there has been an increasing trend in the number of Poverty Assessments that incorporate qualitative analyses: a quarter of the Poverty Assessments in fiscal 1994 used the qualitative approach in addition to the quantitative approach while in fiscal 1995 this figure had risen to half. -16- Integrating Methodologies 42. The sequencing of the quantitative and qualitative work has implications for how the two methodologies can be integrated. Depending on which is undertaken first, a number of options are available. Some options are discussed below. Using Ouantitative Survey Data to determine the Individuals/Communities to be Studied through the Oualitative Approach 43. National or regional poverty data from a household survey can be used to narrow-down the groups and regions for in-depth study under the qualitative approach. For example, based on the quantitative household survey in South Africa, the qualitative poverty study selected three provinces accounting for the largest share of the poor below the poverty line as well as having the greatest depth of poverty for detailed analysis. Box 1 presents further examples of World Bank Poverty Assessments where quantitative work has been used to inform the choice of geographic regions or population groups for in-depth analysis under the qualitative approach. The use of quantitative surveys to select the qualitative sample has been most common way of integrating methodologies in World Bank Poverty Assessments. Using the Quantitative Survey to design the Interview Guide of the Qualitative Survey 44. The quantitative survey can highlight issues that may be covered through qualitative work and the particular ways in which they may best be addressed. The Interview Guide for the qualitative work can designed accordingly. In the Armenia Poverty Assessment, the quantitative survey questionnaire influenced the issues that were examined in the qualitative work especially issues relating to the informal sector. Using Qualitative Work to Determine Stratification of the Quantitative Smple 45. Qualitative work can help determine the appropriate stratification of the quantitative sample (e.g., along ethnic or gender lines). Using Qualitative Work to Determine the Design of the Quantitative Survey Questionnaire 46. Qualitative analysis can be used to gain an understanding of the types of questions that should be asked in the more formally structured interviews. Qualitative work can help to identify variables of importance to respondents which should be included in the quantitative questionnaire. Any omissions would constitute a form of specification error, such that the relations among the variables that are measured would differ if this variable were taken into account instead of ignored. Although this class of error has no effect on simple descriptive statistics, it can be a severe problem in multivariate analyses. Often, omissions are not random given the unavoidable class and occupational differences between survey researchers and many of their respondents. Qualitative work can indicate how important respondents consider specific issues. This will facilitate the design of questionnaires, survey instruments, or other research tools that may be used in quantitative research. Qualitative techniques can initially help set the agenda, which helps determine a larger questionnaire to obtain richer information. -17- Box 1: Using quantitative surveys to select qualitative samples To ensure the relevance of the issues examined for an area wider than a single community, the qualitative work in the Poverty Assessment for Kenya was linked to the quantitative National Welfare Monitoring Survey (WMS) which used a sample of 12,000 people from the national cluster sampling frame. Based on findings from the just completed WMS, five of the poorest districts across Kenya were chosen for qualitative analysis. Within each district, five "clusters" were chosen randomly using maps available at the Central Bureau of Statistics (Deepa Narayan and David Nyamwaya 1996). In Cote d'Ivoire, a small quantitative poverty survey was conducted with 1200 randomly selected households. In order to refine understanding of intra-household differences and especially of gender differences and poverty, a sub-sample of 50 of the poorest households from the 1200 household survey will be interviewed intensively, using an open-ended interview protocol. These interviews will focus on refining information on incomes and consumption patterns, coping strategies, and perceptions of poverty, both of self and of others. They will also increase understanding of perceptions of accessibility and quality of social services, including health and primary education -- as well as the evolution of constraints to access as people move in and out of poverty. This should, for example, yield important insights into reasons for declining female school enrollments and increasing drop-out rates, probable declining purchasing power for pharmaceuticals, changes in rates of attendance at public and private sector medical facilities and trade-offs made among these expenditure categories by people in different poverty groups. Background work on which the Ecuador Poverty Assessment is based comprises a good example of the integration of the quantitative and qualitative approaches. The Cisne Dos study used a random sample of households to identify its sub-sample of women who were to be qualitatively studied. Of the random sample of 200 households, a sub-sample of 30-40 women was selected purposively for in-depth interviewing guided by criteria such as household structure, household size, housing type, and female headedness. A qualitative community level survey of basic services and infrastructure complemented the information obtained from individual interviews; the communities were selected for their location "in a known marginal low-income are, inhabited by squatters, both rural and urban and involved during the past decade in city-level consolidation" as well as having previously collected household survey data from which longitudinal data could be derived. Together, these made it possible to examine the way in which changes in household headship, structure, and composition influenced household level poverty and vulnerability (Moser 1996). In Ghana, the sites for the Participatory Rural Appraisal were selected on the basis of available quantitative and secondary material. This was supplemented by extensive experience of the research teams to create a purposive sample which would be illustrative of the living conditions of the poor in Ghana in terms of: rural-urban balance, modes of livelihood (farming, fishing, pastoralism etc), agroecological zones, major ethnic/cultural groupings, level of access to services and infrastructure, and level of integration with markets (Andy Norton, David Kroboe, Ellen Bortei-Dorku, DK Tony Dogbe 1995). -18- Using Qualitative Work to Pre-test the Ouantitative Survey Questionnaire 47. The qualitative approach is also useful beyond the preliminary phases of developing the quantitative survey questionnaire. In particular, it can augment the pre-testing that is necessary to review the survey instrument. An advantage of qualitative methods is that it is much easier to detect if participants fail to understand a question as the researcher intended it. Pretesting with qualitative techniques would not only locate such problems but allow an immediate correction. Using Oualitative Analyses to Refine the Poverty Index 48. Qualitative work can help to determine the variables that should be included in the poverty index. The ranking for each of the variables emerging from the qualitative work can be used to weight the different variables that are incorporated in the quantitative poverty index. Examining, Explaiining, Confirming, Refuting, and Enriching Data from one Approach with the Other Examining: Generating Hypothesis from Qualitative Work for Testing through Quantitative Approach 49. The qualitative approach can generate research hypothesis that can be submitted to further research and testing through the quantitative approach. The richness of qualitative analysis may also help in the generation of a more elaborate research agenda than can be obtained from quantitative analysis alone. By the same token, it may also be possible to follow-up quantitative work by in-depth qualitative analysis of specific issues. In Ecuador, the qualitative work undertaken for the Poverty Assessment highlighted the importance of land, land use, and land productivity issues in understanding poverty -- issues that were then picked-up for particular examination in the quantitative work. Similarly, a new round of LSMS is examining in more depth the linkages between poverty and tenureship -- an issue first raised by the urban qualitative work. Explaining: tJsing Qualitative Work to Explain Unanticipated Results from Quantitative Data 50. At the later stages of a survey, when the data are in and the analysis begins, the qualitative approach can serve as a follow-up data collection instrument, pursuing "exploratory" aspects of the analysis. This is especially important when the results are puzzling to the researchers. Qualitative work can explain unanticipated survey results. Did the respondents understand and respond to the questions in the way the researchers intended? Did they consider factors that the researchers failed to question them about? Can they give a basis for their responses that can be tested as a hypothesis? Qualitative work can help to interpret previously obtained quantitative results. Qualitative techniques can then help obtain a richer understanding of questions raised by the larger survey -- and may sometimes uncover spurious results. For example, in the Mali Poverty Assessment, information from the qualitative study explained the perceived anomalies from the quantitative survey, for example, the disproportionate amount of money spent on clothing was explained by the fact that in the past there was a strong emphasis on clothing/cloth and jewelry being items for investment in addition to being recognized as status symbols (Caroline Robb 1996). Box 2 presents further examples of Bank Poverty Assessments in which quantitative results were explained by the qualitative work. -19- Box 2: Explaining the results of one approach with the other In Togo and Benin, the qualitative work explained the high expenditures on traditional ceremonies; this was not an irrational waste but was, in fact, an investment to decrease vulnerability by gaining prestige and strengthening social ties. In Ecuador's Poverty Assessment, the LSMS analysis found that about 15 percent of the population live in extended households, of which half are poor, while only about one third of the nuclear and mildly extended households are poor. The reason was provided by the qualitative study (Cisne Dos); it found that the expansion of households was one of the main methods of shielding the elderly or young relatives with children from falling into extreme poverty. Confirming or Refuting: Verification of Ouantitative results through Oualitative Approach 51. Quantitative research in a given site can be followed-up with rapid appraisal validation in other communities to check that the priority issues in terms of poverty that have been identified in the quantitative study site also apply in other similar communities (Norton and Stephens 1995). Participatory research can contribute to validating the results of more conventional analysis based on household survey data and government statistics. When verification of an issue through a quantitative study has occurred sufficiently frequently, then lower costing rapid qualitative methods may substitute completely for larger-scale formal surveys. In Kenya, qualitative work yielded very similar poverty estimates to those obtained from the quantitative survey thus confirming the latter as shown in Table 7. Box 3 provides other examples of confirmation in World Bank Poverty Assessments. 52. Sometimes, cross-checking may result in refuting the findings of one approach with those of the other as exemplified by Jodha's work. Other examples are found in the Ghana and Togo Poverty Assessments. In Ghana, the GLSS found that there is little serious poverty problem in Accra (lowest incidence) and in urban areas generally. But, the qualitative research suggested that according to the criteria of local people, urban poverty is widespread and the numbers of people who are self-defined as poor in urban areas is greater than some rural areas especially those in the south. The reason for this discrepancy is explained thus: Household expenditure-based measures (used by GLSS) niay only give a partial picture. Many aspects of rural livelihoods are not monetized. While expenditure surveys routinely allow for self-provisioning of foodstuffs by imputing values, there are very large other elements of livelihoods which would appear as "expenditure" for an urban household but may be "self-provisioned" for a rural household. For example, fuelwood, medicines, and food can be large items in urban household expenditures but these may be gathered from the forest or communal land in rural areas . Also, some items of urban expenditures are simply unnecessary to rural people (e.g., transport expenditures) (Norton, Kroboe, Bortei-Dorku, Dogbe 1995, page xvi). -20- Table 7: Comvarine Poverty Measured by the Ouantitative and Oualitative ADDroaches District Absolute Poverty* Line Qualitative Work for Poverty Adult Equivalence Assessment (%) (Social Mapping, wealth ranking) Bomet 64.7 64.0 Busia 67.6 68.0 Nyamira 54.0 54.0 Kisumu** 39.1 58.0 Kitui 58.0 76.0 Kwale 50.0 62.0 3ource: Kenya Poverty Assessment, Report 13152-KE, World Bank, 1995; * GOK: Kenya: Po very Profiles, 192, o hen si Mukui, 1993; ** Cluster sampling procedure not followed precisely. Note: (i) The qualitative poverty esdimates are based on wealth ranking. The sampling frame for the qualitative work was based on the national cluster sampling used for the quantitative Welfare Monitoring Survey (WMS). The Central Bureau of Statistics (CBS) assisted in the random selection of five clusters, which often coincided with village boundaries from five of the poorer districts. As the table above shows, for three of the five districts in which the cluster sampling approach was followed closely, the results from the two approaches are almost identical. The two districts where the qualitative method gives much higher levels of poverty are from semi-arid districts, Kitui and Kwale, both severely affected by the drought in the last two years (the WMS was conducted prior to this time). Final qualitative sample comprised 3,500 people; (ii) The method for establishing the levels of poverty under the qualitative work involved a four step process carried out in large groups which met in public. In the first stage, a map of the village was created on the ground with sticks and stones used to mark the key features of the village and households. The second stage involved a discussion -- which became quite heated -- centered on the characteristics of people who are very poor, poor, average, and rich. In the third stage, once consensus was reached, people were asked to categorize each household in the commnunity as rich, average, poor, or very poor (the process is called wealth-ranking). In the final stage, the female-headed households in the village were identified. 53. In Togo and Cameroon, the qualitative work revealed a food security problem more s *ious than one would expect on the basis of the available quantitative data. Enriching: Using Oualitative work to identify issues or obtain information on variables not obtained by the Ouantitative surveys 54. Perception variables (reflecting attitudes, preferences, or priorities) are not easily captured by quantitative surveys. Examples of such variables are how the poor "view" their own poverty or how "satisfied" beneficiaries are with a service -- quantitative work can be enriched by adding this perspective. In Ecuador, the qualitative work (Cisne Dos) yielded an additional insight to the quantitative work: more poor people rent their houses than the non-poor in urban areas - one of the major strategies for households to shield themselves from poverty being the use of their house as a shelter for impoverished relatives and for informal sector activities. Box 4 provides other examples of enriching in Bank Poverty Assessments. 55. In principle, each of the above mechanisms, i.e., examining, explaining, confirming, refuting, and enriching may operate in either direction -- from qualitative to quantitative approaches or vice versa. -21- Box 3: Confirming In Tanzania both the quantitative and the qualitative analysis found that the proportion of poor in rural areas was about 50 percent notwithstanding that the qualitative analysis used criteria defined by poor such as powerlessness and coping strategies whereas the quantitative survey used consumption and expenditure data. In Ghana, both the quantitative and qualitative study found that poverty is greatest in the rural north. Both point to a critical problem of access to health care services for the rural poor and to the problem of quality in the provision of education services. The participatory assessment highlighted the concern of the poor for improved water supply; this priority especially in the north of Ghana is amply demonstrated by the findings of the Ghana Living Standards Survey (GLSS). The GLSS data also confirmed the qualitative finding that the burden of water collection is borne mainly by girls and women. In Armenia, unlike in many other countries, smaller households are poorer compared with larger households; this was shown by the qualitative analysis and corroborated by the quantitative analysis. The main reason for this finding is probably that land was re-distributed to households based on household size with bigger households getting more land; given that land is a very important production factor and those with larger plots are able to trade surplus production, bigger households tend to be better-off. Other possible reasons for this frnding were provided by the qualitative analysis which found that larger families were better able to pool resources, received more help from friends and neighbors and enabled additional family members to devote themselves to informal sector activities which could raise total family income significantly. Larger families are more likely to have more relatives, increasing the chance that some relatives may have emigrated to work. This hypothesis is corroborated by quantitative data on the structure of reported income. Both the quantitative and qualitative work found that the very poor are the ones who do not benefit from family transfers and private remittances and do not receive revenue from the informal sector. Both studies found that the lack of extended family support, private remittances, or informal sector earnings are strongly correlated to poverty. Moreover, the qualitative survey confirmed the quantitative survey finding that the earthquake zone and the border regions were also the poorest and that the formal safety net had very little impact on the population. The informal safety net (remittances, informal trade, inter-household transfers) is the only reliable coping mechanism. In the Ecuador Poverty Assessment, the rural qualitative survey pointed out that widows are a vulnerable group which was then confirmed using LSMS. Furthermore, the LSMS and the qualitative study (Cisne Dos) both show that female-headed households do not have a higher poverty incidence than male-headed households. This is because of the tendency of very poor female-headed households to move in with their relatives since they simply cannot afford to live alone. In addition, an attempt was made to determine how the community studied under the qualitative study compared with the larger national picture. Specifically, the Cisne Dos settlement was compared with other urban areas in the Costa and the national level to provide a context for the analysis of urban poverty in Cisne Dos. Eight indicators were chosen to compare Cisne Dos to the urban Costa and the national urban areas: education of the household head, gender of the household head, household size, average number of rooms per household, water source, hygiene facilities, garbage disposal method, and cooking fuel. Urban Costa and national areas are defined in terms of the five different income quintiles derived from the 1990 Urban Employment Survey. The values of the eight variables for Cisne Dos were matched to the respective values for each income quintile of the urban population in order to identify in which urban segment the settlement best fits. Closeness was defined as the smallest variation between the Cisne Dos values and the larger urban area values per indicator because most indicators themselves contain different values, for example, the education variable contains the share of the population having attended primary school, secondary school, and a higher education institution. The eight indicators are then aggregated by assigning rank values according to the sum of variances for each individual indicator. Evaluating the closeness of similarity between all eight variables together, Cisne Dos is closest to the poorest income quintiles of the urban Costa and national urban area. As expected, the Costa community is closer to the poor segments of the Costa urban area tihan for the national urban area. But, Cisne Dos has some specific characteristics which distinguish it from an "average' poor urban area. To sum up, the characteristics of Cisne Dos fit best the lowest income quintile of larger urban areas differing from an "average" lower income urban community only with respect to some socio-economic characteristics. -22- Box 4: Enriching A recent study of poverty in Africa found that qualitative work enriched the poverty profile by encapsulating local perceptions of poverty and vulnerability, increasing the understanding of the impact of public expenditures, increasing knowledge about the constraints experienced by the poor, and supporting policy analysis of safety nets and of coping strategies used by the poor (Taking Action for Poverty Reduction in Sub-Saharan Africa: Report of An Africa Region Task Force, World Bank 1996). The South Africa qualitative poverty study, highlights the dynamics of decision-making, coping strategies, influence of seasonality on household behavior, intra-household gender relations, and constraints to access of services -- aspects of poverty that will not be captured by the quantitative analysis. In Ghana, the qualitative work reviewed the attitudes of the poor towards their poverty, and their views about how best to resolve their problems. The qualitative assessment contrasted the conception of poverty in the rural north where the problem was thought to affect whole communities and to threaten food security (in the hungry season) with that in urban areas where poverty is viewed more as an individual condition and where well-being is defined in terms of access to stable employment rather than food security. Incidence analysis quantified the distribution of government spending on health and education while the qualitative study provided valuable insights into the factors underlying the use or non- use of government services by the poor (Ghana/World Bank 1995). The qualitative work in a number of countries, for example, in Uganda and Armenia analyzed coping strategies adopted by the poor -- something the quantitative work does not address. In Armenia, four major categories of coping strategies were identified: reductive strategies, i.e., reducing consumption and substituting cheaper alternatives; depletive strategies, i.e., using up household resources and savings and selling assets; maintaining strategies, i.e., short-term survival strategies; and regenerative strategies, i.e., increasing a household's resource base and expanding its sources of regular income. Merging the findings from the Two Approaches into One Set of Policy Recommendations 56. The information provided both by the quantitative approach as well as the qualitative approach can be analyzed to derive one set of policy recommendations. Policy recommendations based on both quantitative and qualitative information as opposed to those based on only one of these two types of information will have obvious merits. While quantitative data has been used traditionally to draw policy conclusions, drawing macro-policy implications from micro-qualitative data may require special attention. This is discussed in the Zambia Poverty Assessment: A rapid assessment exercise can indicate what people see as the main trends in their lives -- and can assess how they themselves perceive issues such as stress points in the annual cycle in terms of household expenditures, vulnerability to disease, food security etc. It cannot, however, quantify these in any detail. It presents an "agenda" in terms of poverty and poverty reduction that reflects concerns in poor rural and urban communities. To situate the significance of this -23- agenda in terms of poverty reduction as a whole requires inevitably a broader process of policy formulation (Zambia Poverty Assessment 1994, Vol V page 6). 57. Do the World Bank's Poverty Assessments use both the qualitative and quantitative findings to arrive at a set of policy recommendations? Since quantitative work is invariably heavily drawn upon to derive policy recommendations in Poverty Assessments, we focus here instead on the extent of the influence of qualitative analysis on policy recommendations. In cases where the qualitative work mainly "confirms" the findings of the quantitative analysis and thus addresses many of the same questions as the quantitative analysis, it is difficult to assess the impact of the qualitative work per se on the policy recommendations. The links (or lack thereof) between the qualitative work and the policy recommendations of the Poverty Assessment are far more explicit in cases where the qualitative work provides additional insights and "enriches" or "refutes" rather than simply confirms the quantitative findings. 58. While, in general, the qualitative work undertaken in the context of the Bank's Poverty Assessments has influenced policy recommendations, overall there is variation among the Poverty Assessments with respect to the extent of such influence. It also appears that the imract of qualitative work on policy recommendations is greater in cases where the qualitative work is more comprehensive (i.e., the time spent on the qualitative work is more than a couple of weeks, the geographic coverage of the qualitative work is more than a few cities/villages, the range of techniques used for data collection and analysis are wide and varied, and the scope of the inquiry is not restricted simply to "who are the poor" questions but also includes "why poor" and "what can be done" questions); of course, the comprehensiveness of the qualitative analysis is not a sufficient condition for the analysis to have a strong impact on policy recommendations. The Ghana, Cameroon, Zambia, and Ecuador Poverty Assessments incorporate comprehensive qualitative analyses whose findings feed noticeably into the policy recommnendations. On the other hand, the limited qualitative work in Uganda and Mali has a limited influence on the policy recommendations contained in the Poverty Assessments for these countries. Box 5 provides details of how the qualitative work has influenced the policy recommendations in each of these cointries. -24- Box 5: Merging the findingsfrom the TWo Approaches into One Set of Policy Recommendations Ghana. The qualitative work in Ghana spanned five mondhs, covered fifteen communities (five urban and ten rural) representing nine of Ghana's ten regions, used a range of PRA research techniques, and included "who are poor", "why poor", and "what can be done" questions. The Poverty Assessment based its policy recommendations quite heavily and explicidy on the policy messages emerging from the qualitative work. In the agriculture sector, the qualitative work in Ghana highlighted the need for improved women technologies to lessen their labor burden. The policy conclusions in the Poverty Assessment picked up these recommendations and emphasized the need for investments in grinding mills and improved transport and energy-supply technologies. In the health sector, the qualitative work found that poor households had difficulty in finding the needed cash for medical costs especially during the hungry season. It also found that exemptions for fees based on means testing were not as effective as exemptions based on age or type of disease from which the poor suffer disproportionately. Based on these findings, the Poverty Assessment recommended changes in the methods of collecting fees and in exemption provision. Furthermore, bodh the qualitative work and GLSS found that the rural poor are being asked to pay more for health services than the urban poor perhaps because of extra-legal fees and the fact that service providers are unaware of the correct scale of fees. Accordingly, the Poverty Assessment recommended that awareness be increased among rural service providers and users about the scale of health charges. In the education sector, while primary and junior secondary education is free in Ghana, GLSS data suggested that households pay tuition for these levels of education. Qualitative work "explained" the reasons for the discrepancy: the charges that were paid were probably for private tuition by teachers. Based on the quantitative and qualitative findings, the Poverty Assessment emphasized the serious need for greater discipline among teachers in the public sector and recommended further investigation into the fee issue. Cameroon. The Cameroon Poverty Assessment also appears to have internalized the findings of the qualitative analysis quite well in its policy recommendations. For example, the qualitative study found a hiatus between the government and the governed. The overall consensus was that there was a lack of avenues that allow for people's participation in making decisions that direcdy affect their lives. Accordingly, the policy recommendations in the Poverty Assessment emphasized the need to address institutional performance and management issues through performance-based rewards and sanctions aimed at re-establishing confidence and trust in both public and private institutions at all levels. They also emphasized the need to give voice to the poor and strengthen local initiative and participation. Furthermore, the focus given in the policy recommendations to improving food security also derives direcdy from the emphasis given by the poor to problems associated widh hunger, dietary inadequacy, and high food expenditures as revealed in the qualitative work. Ecuador. The policy recommendations of the Poverty Assessments in Ecuador and Zambia draw quite extensively from the wide-ranging qualitative studies that preceded them. In Ecuador, the qualitative work "enriched" what was known about the poor by highlighting the fact that street crime and violence restrict women's ability to get to work away from home. This fmding is addressed in the Poverty Assessment which identifies the provision of functioning street lighting and guarded public buses in the evening as simple but effective solution. Moreover, the qualitative work found that in urban areas childcare facilities were important to enable women to go to work and the policy recommendation, therefore, included reopening the childcare system which has been closed. Zambia. In Zambia, the finding that seasonality influences living conditions which emerged from the qualitative analysis feeds directly into policy recommendations, for example, that the payment of school fees be spread over the academic year so as not to coincide widh peak expenditure periods for poor urban and rural households. The qualitative work also emphasized the need for reducing barriers to health care access imposed by cost, for example, allowing deferred payment (in other words, providing healdt care on credit). The Poverty Assessment recommends that cost recovery issues be addressed in the context of a healdh financing framework and supports the "revised healdt financing scheme" which will "promote equity in access to health services"; it is not clear, however, if this includes deferred payments or any of dhe odher payment medhods recommended by the qualitative study. Uganda and Mali. The Poverty Assessments for Uganda and Mali are early examples of qualitative analyses in Poverty Assessments. Each supported only a few weeks of qualitative work -- one week for Uganda and six for Mali. The geographic coverage of the qualitative studies in both countries was limited; in Uganda, the rapid rural appraisals and pictorial dmwings were confmed exclusively to three regions which had not been covered by the household budget survey because of the civil war, and in Mali BA-type work was conducted only in the capital city of Bamako. Only "who are the poor" and a few "why poor" questions were examined through the qualitative work with little attention to the question "what may be done". While the poverty profiles have been "enriched" by the qualitative work and there are some links between the qualitative work and the Poverty Assessment recommendations, on the whole it appears that more comprehensive qualitative work -- which would, undoubtedly, have had budgetary and time implications -- might have led to the qualitative work influencing the policy recommendations more noticeably. It may be noted that a highly participatory project for Mali currently under preparation that will finance community-requested interventions in disadvantaged areas identified dhrough a qualitative approach similar to the one used for the PA -- a case of demonstration effect. -25- V. Cq,mdusion * There are limits to a purely quantitative approach as well as a purely qualitative approach to poverty measurement and analysis. Each approach has an appropriate time and place, but in most cases both approaches will generally be required to address different aspects of a problem and to answer questions which the other approach cannot answer as well or cannot answer at all. The need to build links between the two approaches cannot be overemphasized. * There are three key ways to combine the quantitative and qualitative approaches: (i) integrating methodologies; (ii) confirming, refuting, enriching, and explaining the findings of one approach with those of the other; and (iii) merging the findings of the two approaches into one set of policy recommendations. * Some ways in which the integration of methodologies can be achieved are: using quantitative survey data to determine the individuals/communities to be studied through the qualitative approach; using the quantitative survey to design the interview guide of the qualitative survey; using qualitative work to determine stratification of the quantitative sample; using qualitative work to determine the design of the quantitative survey questionnaire; using qualitative work to pre-test the quantitative survey questionnaire; and/or using qualitative analyses to refine the poverty index. * "Confirming" or "refuting" are achieved by verifying quantitative results through the qualitative approach. "Enriching" is achieved by using qualitative work to identify issues or obtain information on variables not obtained by quantitative surveys. "Examining" refers to generating hypothesis from qualitative work for testing through the quantitative approach. "Explaining" involves using qualitative work to understand unanticipated results from quantitative data. In principle, each of these mechanisms may operate in either direction -- from qualitative to quantitative approaches or vice versa. "Merging" involves analyzing the information provided both by the quantitative approach as well as the qualitative approach to derive one set of policy recommendations. * Of the three ways of combining quantitative and qualitative approaches, integrating methodologies has been the least common in World Bank Poverty Assessments while enrichment, validation, and explanation of the findings of one approach with the other, and the merging of the quantitative and qualitative approaches into one set of policy recommendations have been commonly supported -- although to varying degrees. * The comprehensiveness of the qualitative analysis (defined in terms of the time spent on the qualitative work, its geographic coverage, the range of techniques used for data collection and analysis, and the scope of the inquiry) seems to be a necessary condition -- but may not be sufficient -- for the analysis to have a strong impact on policy recommendations. * Quantitative and qualitative approaches are being increasingly combined in analytical work on poverty, but there remains scope for further strengthening the links between them. -26- Bibliography Baulch, Bob. 1996. "Neglected Trade-offs in Poverty Measurement". IDS. Bulletin Vol. 27. No. 1. Casley, D. J. and D. A. Lury. 1987. "Data Collection in Developing Countries". Second edition. Oxford: Clarendon Press. Casley, Dennis. 1993. Introduction in "Rapid Appraisal Methods." Editor Krishna Kumar. World Bank Regional and Sectoral Studies. Chambers, Robert. 1995. "Poverty and Livelihoods: Whose Reality Counts". IDS Discussion Paper No. 347. Brighton: IDS. . 1992. "Rural Appraisal: Rapid, Relaxed, and Participatory". IDS Discussion Paper No. 311. Brighton: IDS. _ 1983. "Rural Development: Putting the First Last". Longman. London & New York. . 1994. "The Origins and Practice of Participatory Rural Appraisal". World Development. Vol. 22, pp. 959-961. Chung, Kimberly. 1996. "Combining Quantitative and Qualitative Techniques For Imnproved Policy Analysis". Policy Research Department. Washington, D.C.: World Bank. Dayton, Julia, Aziz Khan, Helena Ribe, Marc Schneider. 1993. "Country Policies for Poverty Reduction: A Review of Poverty Assessments". ESP Discussion Paper No. 15. Washington, D.C.: World Bank. Dudwick, Nora. 1995. "A Qualitative Assessment of the Living Standards of the Armenian Population". (Draft). George Washington University, Washington, D.C. Glewwe, Paul. 1990. "Improving Data on Poverty in the Third World: The World Bank's Living Standards Measurement Survey". Policy Research Working Paper 416. Washington, D.C.: World Bank. Griffith, G. 1987. "Missing the Poorest: Participant Identification by a Rural Development Agency in Gujarat". IDS Discussion Paper 230. Grootaert, Christian. 1986. "Measuring and Analyzing Levels of Living in Developing Countries; An Annotated Questionnaire". LSMS Working Paper 24. Washington, D.C.: World Bank. Grosh, Margaret and Juan Munoz. 1996. "A Manual for Planning and Implementing the Living Standards Measurement Study Survey". LSMS Paper No 126. Washington, D. C.: Wprld Bank. Grosh, Margaret and Paul Glewwe. 1994. "Improving the Quality and Policy Relevance of LSMS -27- Household Surveys: A Research Proposal". PRD. Washington, D.C.: World Bank. Grosh, Margaret. 1991. "The Household Survey as a Tool for Policy Change: Lessons from the Jamaican Survey of Living Conditions". LSMS Working Paper No. 80. Washington, D.C.: World Bank. Hentschel, Jesko and Peter Lanjouw. 1996. "Constructing an Indicator of Consumption for the Analysis of Poverty: Principles and Illustrations with Reference to Ecuador". LSMS Paper No. 124. Washington, D.C.: World Bank. Hill, Polly. 1986. "Development Economics on Trial". Cambridge University Press. Jodha, Narpat S. November 1998. "Poverty Debate in India: A Minority View". Economic and Political Weekly. Laughlin, Bridget O., 1996. "On LSMS in Gender Dimensions in Household Survey Research ISS/NIDI Population and Development Teaching Text". Kumar, Krishna (ed). 1993. "Rapid Appraisal Methods". World Bank Regional and Sectoral Studies. Washington, D.C.: World Bank. May, Julian, 1996. et al. "The South African Participatory Poverty Assessment - Synthesis Report". Draft for Discussion. Washington, D.C.: World Bank. Moser, Caroline, Michael Gatehouse, Helen Garcia. 1996. "Urban Poverty Research Sourcebook: Modules I and II". Washington, D.C.: World Bank. Narayan, Deepa and David Nyamwaya. 1996. "Kenya Learning from the Poor: A Participatory Poverty Assessment". Environment Department Papers No 034. Washington, D.C.: World Bank. Narayan, Deepa. 1996. "Toward Participatory Research". World Bank Technical Paper 307. Washington DC: World Bank. Norton, Andrew, David Kroboe, Ellen Bortei-Dorku, DK Tony Dogbe, November 1995. "Poverty Assessment in Ghana Using Qualitative and Participatory Research Methods". PSP Discussion Paper No. 83. Washington, D.C.: World Bank. Norton, Andrew and Thomas Stephens. 1995. "Participation in Poverty Assessments". Environment Department. Washington, D.C.: World Bank. Panse, V. G. 1958. "Some Comments on the Objectives and Methods of the 1960 World Census of Agriculture". Cited in Chambers 1983. Patton, Michael. 1990. "Qualitative Evaluation And Research Methods". Sage Publications. Ravallion, Martin. 1996. "How Well can Method Substitute for Data? Five Experiments in Poverty Analysis". World Bank Research Observer, Vol. 11 No. 2, pages 199-221. -28- _ 1992. "Poverty Comparisons: A Guide to Concepts and Methods". LSMS Working Paper No. 88. Washington, D.C.: World Bank. _ 1991. Martin, Gaurav Datt, and Dominique van de Walle. "Quantifying Absolute Poverty in the Developing World". Review of Income and Wealth. Robb, Caroline. 1996. "Participatory Poverty Assessments: A Preliminary Review". PSP/ENV (Draft). Washington, D.C.: World Bank. 1996. "Participatory Poverty Assessments: A Review of the World Bank's Experience". PSP (Draft). Washington, D.C.: World Bank. Salmen, Lawrence. 1995. "Beneficiary Assessment: An Approach Described". Environment Department Paper 023. Washington, D.C.: World Bank. _ 1987. "Listen to the People: Participant-Observer Evaluation of Development Projects" Oxford University Press. . 1995. "Participatory Poverty Assessment". Environment Department Paper No. 024, World Bank. Streeten, Paul. 1994. "Poverty Concepts and Measurement" in Rolph Van der Hoeven and Richard Anker, "Poverty Monitoring: An International Concern". St Martin's Press. World Bank. 1996. "Armenia: Confronting Poverty Issues". Report 15693-AM. Washington, D.C.: World Bank. . 1995. "Cameroon: Diversity, Growth, and Poverty Reduction". Report No. 13167- CM. Washington, D.C.: World Bank. . 1996. "Poverty in Cote D'Ivoire: "A Framework for Action". Washington, D.C.: World Bank, (Draft). _ 1995. "Ecuador Poverty Report". Report No. 14533-EC. Washington, D.C.: World Bank. . 1995. "Ghana: Poverty Past, Present, and Future". Report No. 14504-GH. Washington, D C.: World Bank. . 1996. "Taking Action for Poverty Reduction in Sub-Saharan Africa: Report of An Africa Region Task Force", Report 15575. Washington, D.C.: World Bank. a 1996. "The Social Dimensions of Adjustment Priority Survey: An Instrument For The Rapid Identification and Monitoring of Policy Target Groups". Washington, D.C.: -29- . 1991. "The World, Bank Participation Sourcebook". Washington, D.C.: World Bank. * 1996. "The World Bank Participation Sourcebook". Washington, D.C.: World Bank. . 1993. Uganda: Growing Out of Poverty. Report No. 12029. Washington, D.C.: World Bank. . 1994. "Zambia: Poverty Assessment. Report 12985-Za. Washington, D.C.: World Bank. -30- Distributors of COLOMBIA HAM ITALY NEWZEALAND ROMANIA SWrTZERLAND Inioenlace Ltda Cuatue Dfuajon Licosa Commissionaria Sansooi SPA EBSCO NZ Ltd Compani De Ubrarni Bucuresti S.A. Libraide Payot Service Instlubionnel W orld Bank Carrera 6 No. 51-21 5, Rue CapoLs Via Duca Di Calabha, 1/t Private Mail Bag 99914 Str Lipscani no. 26, sector 3 Cites-de-Monthenon 30 D 1*1- s- ~~~~~~~~Apartado Aereo34270 C.P.257 CaselbaPostaleS552 New Markel Bucharest 1002 Lauspanne Publications Santoad de BogotAlD.C. Por-au-Prince 50125 Firenze Auckland Tel: (401) 613 9645 Tel: (41 21) 341-3229 Prices and credit terrous ar fror Tel:(571)295-2798 Tel: (509 1)39260 Tel: (55)645-415 Tel: (64 9) 524-8119 Fax (40 1) 3124000 Fax: (41 21)341-3235 country to cntrtry. Cons',t yoor Fex (571) 285-2796 Fax: (s5) 641-257 Fax. (64 9) 524-N67 local dtstribstor before placirng an HONG KONG, MACAO E-mail: lcosa9llbcc 6 RUSSIAN FEDERATION ADECO Van Diemen EditsoTeschniques order. COTE DIlVOIRE Asia 2000 Ltd US: ht9ir/, -tlcdictoosa NIGERIA Isdatelsvo