E! ~R ERG IO NA L AN D SECT OR AL ST9U D ES .1~~~~~~~~~( Rapid 6ILI&ksr Appi aisal Methods s~~~~~~~~l-'~A Zi- ,-~~~ :,,' .. A..... ,.-i,2 EDITED BY KRISHNA KUMAR ; Rapid Appraisal Methods WORLD BANK REGIONAL AND SECTORAL STUDIES Rapid Appraisal Methods EDITED BY KRISHNA KUMAR The World Bank :-Wsigbton, D.C. O 1993 The International Bank for Reconstruction and Development/ The World Bank 1818 H Street, N.W., Washington, D.C. 20433 All rights reserved Manufactured in the TiTnitcd Sttes of America Frst printing August 1993 The World Bank Regional and Sectoral Studies series provides an outlet for work that is relatively limited in its subject matter or geographical covcrage but that contributs to the intellectual foundations of development operations and policy formulatioziL Thesc studies have not necessarily been edited with the same rigor as Bank publications that cory the imprint of a university press. The findings, interprctations, and condusions cxprcssed in this publication arc those ofthe authors and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to the membcrs of its Board of Executivc Dircctors or the countries they reprcsent. The material in this publication is copyrighted. Bequests for pemission to repioduce portions of it should be sent to the Office of the Publisher at the addrcss showni in the copyright notice abovc. The World Bank encourages dissemiination ofits work and will normaly give perniissiorn promptly and, when the reproduction is for noncommercial purposes, without asking a fcc. Pennission to copy portions for classroom use is ganted through the Copyright Clearance Center, 27 Congress SStret, Salem, Massachusetts 01970, U.SA. The complete backlistofpublications from theWorld Bank is shown in the annual IndexofPubicatioxs,which contains analphabetical titlc listandindcxcs ofsubjccts, authors, and countrics and rcgions. Thc latest edition is availablc flce of charge from Distribution Unit, Office ofthe Publisher, The World Bank, 1818 H Strect, - * N.W., Washington, D.C. 20433, U.S.A, or fom . Publications, The World Bank, 66, avcnue d'I6na, 75116 Paris, Frnce. The maps that accomnpany the text have been prepared solely for the conveniencc of the reader, the designations and presentaiion of material in them do not imply the expression of any opinion whatsoever on the part ofthc World Bank, its affiliates, or its Board or member countries concerning the lcgal status of any country, territory, ciy, or area, or of the authorities thereof, or concerniing the dAelimitation of its boundarics or its national affiliation. Cover desgn by Sam FPmo library u/Congr Catalogin.g-in-Publicafion Data :Rapid appraisal metiods / editcedby Krishna Kumar. p. cm. - (World Bank rcgional and sectoral studies) Includes bibliographical references. ISBN 0-8213-2523-X 1. Economic development projects-Evaluation. I. Kmar, Ksishna. II. Series. HD75.9.R36 1993- 338.9'0068'4-dc2O 93-21584 CIP Contents Foreword xi Acknowledgments xlii Introduction I Dennis r. Casley 1. An Overview of Rapid Appraisal Methods in Development Settings 8 Krishna Kumar Methodological Paradigm 10 Formal and Ihformal Modes of Data Collection 11 Core Rapid Appraisal Methods 13 Limitations of Rapid Appraisal 16 Rationale for Using Rapid Appraisal 18 Appropriate Role for Rapid Appraisal 20 2. Monitoring a Large-Scale Resettlement Program with Repeated Household Interviews 23 Thayer Scudder Country and Program Setting 25 Methodology 28 Data Collection 35 Analysis 40 Problems Arising from Methodology 41 Fmdings and Recommendations 43 Utility of the Methodology for Other Uses 46 References 48 Appendix: Annual Evaluation of the SettIement Component of the Accelerated Mahaweli Programme 50 v vi Contents 3. Use of Group Interviews in Evaluating an Area Development Project 56 M Krishna Kumar Research Methodology8 Planning of Group interviews 59 Conducting Group Interies61 Major Findings and Conclusions 65 Lessons in Conducting Group Inteviews 68 4. A Commodity Systems Assessment Methodology Workshop: Imnproving Agricultural Production and Marketing 72 R.J. Haggerty and I.E Armstrong Country and Progra'm Setting 73 Research Design and Rationale 74 Nepal CSAM Ginger Workshop: Planning and Questionnaire P'reparation 78 Workshop Implementation and Problems Encountered 80 Findings 86 Accomplishments 87 Suggestions for Potential Users of this Methodology 88 References 9 Appendix: Block Descriptions 92 5. Using Focus Groups to Develop and Promote an Improved Weaning Food Product 94 Cecilia Cabaiiero-Verzosa, Cecile M. Johnston, and Olabode .Kayode Prc4ect Overview 95 Focus Group Planning 98 Methodolcgy 100 Analysis of Results 103 Findings 107 Notes1 6. Rapid Appraisal Methods in a Diagnostic Assessment of Vegetable Seed Marketing 112 John S. Hotvtman Background: PA in Agricultural Marketing Research 113 Purpose and Scope of Marketing RA Exercise 113 contents vii Description of Field-Based RA Methods 115 Problems and Possibilities for Improvenent 125 Key Findings and Resulting Measures 127 Suggestions for Users of RA in Agricultural Marketing Research 128 Conclusion 133 References 134 Notes 135 7. Systematic Observation in the Analysis of Primary Health Care Services 136 Stewart N. Blumenfed, Manuel Roxas, and Mauicor de los Santos Systems Analysis Approach to Problern Identification in Service Programs 137 The PRICOR Project 139 The Philippines Country Study 141 Results 152 Direct Observation Summary of Lessons Learned 155 Notes 155 8. Using Urban Commercial Counts and Marketplace Censuses to Appraise Agricultural Development Projects 157 G-ordon Appleby Country and Program Settings 159 Project Results 160 Desciption of Methodology and Rationale 161 Data Collection and Analysis Activities 163 Findings and Conclusions 166 Guidelines and Suggestions 172 References 173 Notes 173 9. Participatory Rural Appraisal: A Case Study from Kenya 176 Charity Kabutha, Barbara P. Thomas-Slayter, and Richard Ford Site Selectiot 179 Data Collection 180 Data Synthesis and Analysis 180 Ranking Problems 180 Ranking Opportunities 181 Adopting a Village Resource Management Plan (VRMP9 182 vii'i Contents Implementation 182 Findings and Conclusions 184 Costs 187 PRA in Per-spective 188 References 188 Appendix: Information Gathered 190 Selected Bibliography 212 Contributors 214 Boxes Box 3.1 Excerpts from the Interview Guide for Dowa West Farmers' Club Members 60 Box 3.2 Encouraging Women to Participate in Group Interviews 64 Box 4.1 Sample CSAM Questionnaire for Marketing Ginger, Rapti Region 81 Box 5.1 TIhe Use of Focus Groups in Developing the Product and the Communications Effort 99 Box 5.2 Focus Group Screening Guide 102 Box 5.3 Focus Group Discussion Guide for Mothers: Pre-In-Home Product Test 104 Box 5.4 Focus Group Report Guide: Mothers' Groups 108 Box 6.1 Informal Inte-rview Guidelines: Vegetable Seed Traders 117 Box 6.2 Informal Interview Guidelines: Vegetable Seed Producer-s 120 Box 7.1 Sample Pages from the PRICOR Thesaurus 142 B3ox 7.2 DOH/PRICOR Systems Analysis 145 Box 7.3 DOH/PRICOR Systems Analysis 146 Box 7.4 DOH.PRICOR Systems Analysis 149 Box A9.1 Information about the Vilage Sketch Map 190 Box A9.2 Vilage Traansect 192 Box A9.3 Information about the Farm S-ketch 195 Box A9A Information about the Time Line 197 Box A9.5 Informnation about Trend Lines 200 Box A9.6 Infornation about the Seasonal Calendar 203 Box A9.7 Information about Farm Interviews 205 Box A9.8 Information about Vllage Institutions2 Box A9.9 Information about the Tetnical Survey 210 contents ix Figures Figure 2.1 Accelerated Mahaweli Program 27 Figure 4.1 Principal Components for a Commnodity Systems Assessment 76 Figure 6.1 Schematic Overview of Rapid Appraisal and Applied Research Linkages 115 Figure 7.1 Oral Rehydration Therapy Service Delivery Model 138 Figure 9.1 Village Resources Management Plan (extract) 183 Figure A9.1 Village Sketch Map 191 Figure A9.2 Composite Transect 194 Figure A9.3 Far-m Sketch 196 Figure A9.4 Time Line 199 Figure A9.5 Trend Lines 202 Figure A9.6 Seasonal Calendar 204 Figure A9.7 Farm Interview (excerpt) 206 Figure A9.8 Institutional Diagrams 209 Figure A9.9 Technical Survey 211 Tables Table 21. Accelerated Mahaweli Programm7e Household Sample 30 Table 2.2. Wealth Index 34 Table 2.3 Household xpenditures for Basic Needs, 1988-89 35 Table 2.4 Loans to Mahaweli Settlers and Repayment Rates 40 Table 3.1 Cumulative Numbers and Membership of Farmers' Clubs, Dowa West Area, 1981-84 58 Table 5.1 Composition and Location of the Focus Groups 101 Table 8.1 Counts of Commercial Shops and Transport in the PNS Area, by Level of Center, 1987 167 Table 5.2 Counts of Other Retail and Service Establishments in the PNS Area, by Level of Center, 1987 167 Table 8.3 Vendor Counts, by Commodity Array, in Marketplaces of PNS Area, by Level of Market, January 1987 168 Foreword SOCIAL RESEARCH METHODS, though powerful, are not often :sed to meet thie pressing informiation needs of decision-makers in develop- mnent. This has sparked a growing interest in an arry of less structured data collection methods called 'rapid appraisal," which aim to supply needed information in a timely and cost-effective manner. This volume outlines five rapid data collection methods-key infor- mant interviews, focus group discussions, group interviews, structured observation, and infonnal surveys-that have been used by the expo- nents of rapid appraisal. Eight case studies illuminate the use of thao methods in development settings, particularly for the appraisal arid evaluation of development projects and programs. The authors take a reflective, if not critical, look at the data collection enterprise and draw practical lessons for the development commnunity. 'They desaibe the nature and types of data generated and the problems, encountered with each method. They also discuss the strengths, limita- tions, and applicability of the methods in different settings. The implicit message of the case studies is that formal social science research and rapid appraisal are complementary. For example, rapid appraisal methods have often been used to design complex socioeco- noiruc surveys as well as to interpret their findings and conclusions. In manty instances, both methods are used sequentially. In situationts where a choice is made, it is dictated by such factors as the nature, purpose, -and ultimate use of the information; available technical and monetary resources; and time constraints. For the development community, this may turn out to be an indis- pensable guide to monitoring and evaluating development projects. Robert Picciatto Director Gener-al Operations Evaluation Department The World Bank June 1993 ; :~~~~~~~~~~~~x Acknowledgments SEVERAL COLLEAGUES AND FRIENDS have helped me in the prepa- ration of this volume. Annette Binnendijk, director of the Office of Evalu- ation, at USAID's Center of Development Information and Evaluation ,' t (CDIE), enabled me to organize a three-day workshop during which some of the papers included in this volume were presented. Janet Ballantyne, the forms director of CDIE, encouraged me to edit the workshop papers for a wider audience. John E. Eriksson, the associate assistant administrator of USAID and jock Conly, chief of CDIE's Pro- gram Operations Assessment Division, took a personal interest in this endeavor. Dennis Casley, forner chief of the World Bank's Operations Monitoring Unit, not only reviewed all the chapters, but also wrote a thoughtful introduction. Graham Donaldson, chief of the Agculture and Human Development Division in the World Bank's Operations Evaluation Department, took upon himself the responsibility of getting this volume published. I wish to record my deep gratitude to these colleagues and friends. I dare not thank individual contributors who authored different chap- ters and were most gracious in making suitable revisions. After all, this volume is as much theirs as mine. I would also like to acknowledge my debt to the six anonymous readers who reviewed the first draft of the nmanuscript and made valuable comments. This book is undoubtedly improved because of their suggestions. Last but not least, I am grateful to publisher James Feather and managing editor Kenneth Hale at the World Bank for the publication of this volume. Krishna Kumar U.S. Agency for International Development June 21, 1993 :X;~~~~~~~~~~~~~~~xi Introduction Dennis J. Casley THE GROWTH IN THE ACADEMIC RESPECTABILITY of so-called rapid appraisal (RA) methods in the 1980s has demonstrated the frustra- tion of many working in the development field (particularly rural devel- opment) with the contribution made by surveys designed by statisticians using sampling theory. In the early days, there -was a ten- dency by some proponents of such methods to make a virtue of damn- ing statistics and sampling theory, so that there was a danger of claiming that in almost any development context, purposive sampling of an arbitrary, limited number of sample units was a defensible alterna- tive, to the application of formal sample size determination and selec- tion. Fortunately, this danger is diminishing. More and more, It Is being realized that rapid surveys involving open-ended interviews of purposively selected or self-selecting cases provide a complement to the use of random sampling and structured questionnaires. Krishna Kumar and I emphasize this complementarity in our two books (Casley and Kurnar 1987, 1988) on monitoring and evaluation of agr;sulttue development projects-particularly in the second volume, in which we describe in some depth the use of such techniques as key informants, focus groups, and the like. It is giving away no professional secrets to say that it was Kumar who provided the expertise in this area, for my own experience was based largely on formal surveys and an adherence to probability sampling methods. I have always argued that such surveys need not be large, cumbersome, slow, or excessively ex- pensive, so there is no need to introduce other methods to overcome these disadvantages. This is not to say that, misguidedly, many survey practitioners (often ill-trained in sampling theory) have not produced such monstrously useless and expensive surveys. However, this is an mdictment of those individuals, not of the discipline of statistics. No, the advantage of the methods that Kumar describes in his over- view is that they fill a gap in the data collection spectrum, 7here is a 1 2 In troduction pLace for enumeraors following a detailed structure in interviewing ran- domly selected respondents, and there is a place for a skiled team un- dertakidng open-ended probing conversations with purposively selected groups. If I were a project manager of a major development project today, I would demand that information flow using both methodolo- gies, If I needed a regular update on the number of farmers responding to project ini'tiatives, I would cali for the statistician to design the adop- lion rate survey and require that precision levels set by me are meL If I needed to know why farmers in Zone C are relatively slow in embracing my services, I would call for one of the authors in this volume to put together a skilled team to visit the area and exercise their craft. And if all I had at hand was a young, inexperienced monitoring officer, I would recommnend that he or she consult closely his or her well-thumbed set of Casley and KCumar, and -also pay attention to the experiences of the authors represented in this present volume. This brings me to the reason I am happy to be assocAated in a mrinor capacity with this book One of the problems with- rapid appraisal is Jiat it has been difficult to lay one's. hands easily on a set of experiences hin which the practitioners set out what they did in some detail, warts and all, with a frank assessment of what thiey achieved, including the' limitations. There have been growing exchanges among those using rapid appraisal methods, but for the uninitiated or the survey designer trained in for-mal statistical methods, accelss to such examnples has been limidted. Nearly all the contributions in this book provide examples of infor- mation gathering in which-given the aims of the study in each instance-the non-random selection of respondents is clearly appropri- ate. Some indeed illustrate cases in which no sample as such is in- volved. In these instances, the appraisa process was applied to the actual midcro-area in which a community-assisted development activity was occurring or was being planned. One of the contributions does provide an ex-ample of purposive sample selection in a situation in Which most survey practitioners would consider orthodox sampling procedures appropriate. I return to their individual cases a little later. One of the underlying themes in nearly al the case studies is the value of seeking out and soliciting in an unprejudiced manner the views, ideas, and aspirations of those who are the participants in and, to some extent, the sub~jects of the development process. There is perhaps a tendency by some current practitioners of rapid appraisal studies to assume that this was not a commnon practice hitherto. In my experience. this is not so. Development banks, development agencies, individual government departments, and smaller, private funders of development Deimis Casky 3 proiects have engaged in a vast amount of project preparation and ap- praial work which, at least in a substantial number of cases, involved travelling within the areas concemed and taldng to local people. It is a myth that international staff stay in the main cities and do not muddy their expensive footwear in the 'bush.' In my 30 years' involvement with government, Food and Agriculture Orgpnization of the United Na- tions (FAO), and World Bank undertkings, I have witnessed a vast amount of diligent effort to get to and talk to small farmers, rural trad- ers, women attending clinics, and so on. As long ago as the 1950s, I was encourged by my then-colonial superr to spend three months with a tent, living various parts of rural Uganda in order to get a feel for the pattem of small famning systems and economics. No, what the contnbutors to this volume and oters are doing is to codify these informatior-seeking processes and tD improve the manner in which they are conducted. For myself, I would have benefited a great deal in my earlier years in Africa if I had known how to organize, for example, a focus group meeting. Kumar and I have descbed the monitormg function as having three components: physical and fimancial monitoring, beneficary contact monitoring, and diagnostic studies. Our second component we set in the context of monitoring penetration and adoption rates, which re- quires probability sampIes, aIthough tiere is room wituin th;S category for selective appliation of the techniques described in this volume. It is for the third component of monitoring-namely the diagnostic study-that rapid appraisal techniques come into their own- Wlhen the reactions of the papants or a subset of thiem, are unexpected or exhibit changes from an earlier trend, the project manager needs a diag- nosis of the likdy cause or causes, and such diagnosis is needed fast. A skMed team spending weeks rather than months in seeling out the opinions of the particpants thrugh open-ended discussions is likely to be the best way of providing such a diagnosis. In the opening chapter, Kunar provides an overview of the metlods most Commonly used by rapid appraisal practitioners. H.e includes very important sections on both the limitations and advantages of these imethods, as viewed from the perspective of a rapid appraisal practitio- ner. It is indicative of the improved understanding between such prac- titioners and statisticians that the limitations receive such equal coverage. The claim that RA tedmiques can provide more accurate information when used in appropriate circumstances hinges on the broad definition of accuracy. Statisticians tend to equate accuracy with quantifiable measures of sampling and non-sampling errors. Kumar, for his part, is concemed with accuracy, in terns of achieving a true insight 4 introduction into a respondenes feelings through a communication process which is more revealing than that achieved in a structured interview. One may put it thus: the non-sampling error measured in a structured survey may reveal the incidence of incorrect replies, but not the incidence of corret but superfical replies. It is the latter that Kumar believes can be minimized using the techniques he describes. Following the overview, this volume contains eight examples of re- cent experiences in applying these methods in practice. Of them all, I consider the first by Scudder to be both one of the most important and one of the most controversial. It is important in that such an effort to monitor and evaluate the Mahaweli Program is a rare example of a sustaied, consistent input over a number of years despite many prob- lems, induding the common one of needing to survive the danger of being the messenger who bears bad tidings. Many such efforts have been cut off from funding and access when early results embarrassed the executing and funding agencies. It is controversial, in that the small sample was selected in about as non-random a way as would be possible to devise and was then main- tained almost without rotation over a number of years. The author pre- sents very dearly the dangers of such biased samples, but on balance believes other advantages accrued. Nor does the breaking of sample survey orthodoxy end here. Some of the sample households were actively assisted with capital or lobbying of goverment agencies to overcome bureaucratic delays or advice, but were sil retained in the sample. The most worying confession by Scudder, however, is that despite this emphasis on convenience and continuity, the data base has not been maintained in an easily retrievable format This seems to obvi- ate the advantages sought of such a methodology for which such a price in unknown biases was paid. Nevertheless, for those who employ a similar method, there is much to be learned from this example, inlud- ing Scudder's own warings. Kumar gives a good example of an assessment based on group inter- views in a classic context in which the groups-farmer's clubs-are wel defined. Once again, as the author notes, there is the danger that the selection of the dubs was rbiased toward those performing well and those more readily accessible geographically. This must surely be the most common problem with rapid appraisal methods and should not become one that practitioners too readily accept, for the limitations then imposed on data interpretation are extreme. The Kumar example does provide a very dlear and useful guide as to how to conduct group inter- views, parficularly in the care needed to obtain balanced participation and control pressures that build up within groups. His conclusions, too, Dennis Casley 5 laugely avoid unjustified generlization of the findings-a good example of interpreting the information within the limitations imposed by the method. Haggerty and Armstrong describe an.example that combines the methods of intervews and focus groups. There is an intrguing use of a coordinating commnittee in the role of moder-ator, and a very detailed splitting of the main issue into over 30 component parts, with a ques- * ~tionnaire to be completed by the group for each part Reading this, I wondered at times whether this was a description of a rapid appraisal method or of how to run a workshop. But this, perhaps, is merely a semnantic issue, although the selection process for participants in the workshop is cdearly of major significanoe when interpreting the findings. The use of focus groups to assess a weaning food in Nligeria is very well described by Cabaiiero-Verzosa, Johnston, and Kayode. In this case study, one gets closest to classic market research techniques. Again, some rapid apprmaa practitioners (and I do not refer to the authors of this study) are not aware of how much they owe to such earlier develop- nents. Cerptanly, focus group interviews have been used by market esearchers for many years. the novelty in this instance is not the tech- nique, but its use in an underdeveloped, rural setting The condusions of this study provide a particulta ly good example of the non-random nature of the partiipant selection being of little concer, in terms of tie accuracy of the results. A blend of methods different from that used by Haggerty and Armstrong is described by Holtzman in the description of his study of vegetable seed marketing in NepaL In this case, a combination of key informant interviews and direct observation were the methods of choice. In this contribution can be found a well-argued case for seeking out progressive farmers and marketers when agricultual market research is being -undertaken. It is almost refreshing given the current emphasis of development iteratu, to find a researcher who is prepared to defend not seeking out the vulnerable groups in certain cicumstances when the otential for change is being assessed. Once again, ease of access played a role in site selection and, not for the first time in the case studies in this volume, one is concerined at the slowness of analysis compared to the speed of data collection.. Biumenfeld, Roxas, and de los Santos ilustrate the direct observation technique in a very appropriate context; namely, the -diagnosis of prob- lems in health care delivery in the Philippines. The authors justification for the method adopted is very well argued and the description of the survey details should be of considerable value to oter practitioners. Here again we have sampling controversy. The selection of the only 6 Infroduction province induded in the study was influenced by "practical consider- :- ations of geographc accessibility, interest and cooperativeness of re- gional and provincial staffs." Health units were selected systematicaly from a ranldng of all 54 according to the quality of care deemed to be provided-an interesting exampIe of selecting from a list ranked by a qualitative attribute; But here, too, geographic accessibility and security : - influenced the fnal selections. Also important here is the point made by -the authors; that the health units studied were likely- to -be better than average in the quality of their facilities and services, so that problems detected in these cases were likely to be at least as prevalent in others. Important, too (in a methodological sense), is how easily the direct ob- - servers were accepted and soon merged into the geneal background without apparently altering the behavior of those being observed. In the next-to-last case study, Appleby avoids the purposive sam- pling problem by adopting a complete coverage approach and going to some trouble to reach even remote centers, which in Zaire can mean some trouble indeed. Simple direct observation, counts, and listings of commercial activities in urban centers and marketplaces were the main data collection tools. This study also provides an example of a suitable use of key informants to provide background information on the local- ity. Another interesting aspect of this case is the application of such data when monitored over time to assess the impact of rural development within the context of central-place theory. The author recognizes the theoretical basis and designed his study accordingly. In the final case study, Kabutha, Thomas-Slayter, and Ford present a * very detailed Kenyan example of the preparation of a development project, which closely involves the intended participants in the appraisal process. No sampling controversy is involved here, because the- data collection and analysis are conducted in the community in which activi- -ties are to be identified and pursued. The use of simple tools such as sketch maps, village transects, and local preparation of historical time series are well described. The most interesting aspect of this example is how the data are used to prepare options for discussion and decision- makling by the local community. There are, I believe, many lessons to be obtained from a close reading of the examples of rapid appraisal methods, as applied in practice by seasoned practitioners. From my own experience, I noted two of the most common occurrences that arise singly or together hi many of the contributions-excessive emphasis on convenience in selecting respon- dents and sites, and insufficient planning of the analysis relative to the planning of the data collection. I stated earlier in this introduction that most of these studies were inappropnate for probability sampling methods. But there is room for Dennis Casley 7 concern in some cases at the kieglect of even minimum safeguards re- quired to reduce the danger of misleading conclusions being drawn. Speed and cost constraints are insufficient justification for the accep- tance of "geographical convenience' or "proximityr to roads" that af- fected the selection of respondents in these cases. And there is little to be said for speeding up the- data collection process if the analysis is to take as long as some of the examples of formal surveys often quoted by rapid appraisal prctitioners in justification of the need for new approaches. The value of these contributions is not least in the frankness with which the authors reveal these points of potential criticism.L This, with the value-added effect of bringing together in one volume such varied experiences, all with something to commend them is, I believe, sufficient jus tification for the effort that Krishna Kumnar and all the contributors have made. References Casley, Dennis J., and Krishna Kunmar. 1987. Project Monitoring and Evaluation in AgTiculture Baltimore~, Marylanct- John Hopkins University Press. 1988. The Collection, Analysis, and Use of Monitoring and Evaluation Data Baltimore, Maryland: John Hopkins University Press. An Overview of Rapid Appraisal Methods in Development Settings Krishna Kumar In this chapter, Krih Kumar provides an overview of rapid ap- praisal methods, focusing on their nature, types, limitations, the ratinalefr fheir use, and apropriate roles. According to Kumar, rapid appraisa methods fJil within a con- tinuum of informal and formal modes of data collection used to provide decision-related infonnration in devliopment settings. At onie extreme are the highly infomal methods that rely on intudtion, experience, and common sense, and which do not generate informa- lion that can be verified. At the other extreme are highly formal methods developed and refined by social and economic researchers. Such methods generate quantitative data that can be statistically analyzed to draw conclusions. The authwr suggests that the rapid appraisal methods lie between these two ends of the continuum. The are neither highly injbrmal nor fidlyformal ized, and require more than common sense on the part of the investigator. He identifies fie mefhods-key informant intkeracks, focus group discussions, community/group interviews, structured direct observation, and informal surveys-that consti- tute the core of rapid appraisal methodology. Several limitations of rapid appraisl methods are mentioned by the author. First, the reliability and validity of the infomnation gen- erated can be questionable in many instances, due tofactors such as the use of informal samnpling, individual biases of the investigatorl interviewer, and the difficulty in recording, coding, and analyzng the qualitative data Second, rapid appraisal methods do not gener- ate quantitative data from which generalizations can be made for a whole population. And finally, the general credibility of these methods is low compared to formal survey methods. These limita- tions shoud be weighd against the obvious strengths of rapid ap- 8 Krishna Kumar 9 praisal metiods. Such methods can rapidly gerate relevant infor- mation with relatively low investment of resources. Moreover, xpe- rience shows that they can provide in-depth understanding and information in the project or program setting. The author suggests that rapid appraisal methods should be used rather selectively, depending on the purpose of the study, availabil- ity of resources and, above all, the nature of the information re- quired. He concludes this chapter by identifying situations in whidt their application is particularly appropriate. DURING THE PAST DECADE, development practitioners have devel- oped, tested, refined, and applied many rapid appraisal methods to gather information and ideas for the design, implementation, monitor- ing, and evaluation of proqects and programs. They have published papers, articles, and monographs articulating the nature, underlying .iises, strengths, and limitations of rapid appraisal methodology. They have also held many regional and international conferences on the subject, bringing together experts from different disciplines and back- grounds. As a result, the phrase "rapid appraisal" and its various syn- onyms (rural reconnaissance; rapid, low-cost methods; intermediate methods) have entered the vocabulary of the development community. The methodology has acquired an inteUectual legitimacy never dreamed of by its early proponents, who invented it while grappling with the problem of gathering relevant information for agricultural and rural de- velopment initiatives with lmited time and resources. We can indeed paraphrase Victor Hugo by saying, "Rapid appraisal is an idea whose time has come." This chapter explains the nature of the rapid appraisal methodology and the various methods that are generally subsumed under it. It also describes the methodology's strengths, limitations, and general applica- bility. The purpose here is not to resolve considerable confusion and controversies still surrounding fiis methodology, but to state our posi- tion in order to provide a framework for the case studies included in this volume. Rapid appraisal methodology is discussed here in the context of the goal of applied research; that is, to provide tinely, relevant information to decision-makers on pressing issues they face in the project and pro- gram setting. The aim of applied research is not to solve theoretical puzzles contributing to the generation and verification of social or eco- nomic theory, but to facilitate a more rational decision-making process in real-life circumstances. This distinction is important because the success of applied research is to be judged, not only by the scientific criteria of validity and reliability, but also with reference to the rel- 70 An Overview ofRapid Appmisa Methods in Deueopment Settings evance of the research to the problem and its timely delivery in a cost- effective fashion. Methodological Paradigm The exponents of rapid appraisal methodology have come from two entirely different intellectual traditions with differing perspectives on the nature and style of social and economic research. The first paradigm, which has been developed by social phenom- enologibts or symbolic interactionists (although significant differences exist between them, we have coupled them for the sake of simplicity), questions the premise that objective reality can be captured by social science methodology. Its proponents view social or economic phenom- ena as constituting not one, but a set of over-arching multiple realities. The usual simile given is that of peeling an onion: as one layer is peeled, another comes to surface. In the same fashion, various investigators re searching a phenomenon encounter multiple layers of realities, which are largely, though not totally, exclusive. According to this paradigm, the premises, preferences, and interests of various investigators largely condition their construction of the real- ity. For example, a development intenrention is likely to be construed differenly by its various stakeholders-the donor agency, the host gov- ermnent, the bureaucracy that manages it, the clients for whom it is designed and implemented, and finally, the other actors who constitute its wider environment Each of these stakeholders, the exponents of this paradigm argue, is likely to perceive the role of the intervention differ- ently, to emphasize different sets of issues, to highlight different achievements and failures, and invariably, to make different recommen- dations. Thus, there is no single reality that can be obtained by an investigator. Within the context of this paradigm, many experts view rapid ap praisal as a tool to articulate the opinions, concerns, judgments, and perspectives of those who are often ignored by the social scentists re- searching development interventions. Thus, they stress the use of rapid appraisal methods for discovering such "indigenous knowledge" as the views of small farmers-particularly women and the landless-and the perspectives of deprived groups, and not as a purs lit of objective reality by an objective investigator. In sharp contrast, the logical positivist paradigm is based on the premise that a social or economic phenomenon exists not only in the minds of individuals, but also as an objective social reality. The fact that -it may be viewed differently by individuals does not negate its exist- ence, nor does it imply that it cannot be objectively described by investi- KCrishrna KumarZ 21 gators. Consider, for example, the case of an initiative designed to re- form the macroeconomic policies of a developing country. The various actors involved may have varying conceptions of it and may stress dif- ferent effects. In fact, often the people who may be most affected may not understand the initiative or its impacts on their lives. However, this does not mean that economists cannot examine the initiative and its intended and unintended effects on the concerned people. Logical positivists believe that the primary, task of social and eco- nomidc research is to go beyond the superficiai, partial reconstructions * and arrive at the objective reality as physical scientists do by using es- tablished procedures for gathering and analyzing infor-mation. Its pur- * pose is to discover and describe social phenomena and processes, systematically explain their causes, and identify contingencies; under which they occur. Rapid appraisal is construed here as one of many sets of data gathering methods to achieve this objective. Formal and Informal Modes of Data Collection For us, rather than debating the above alternative views, rapid appraisal methodology can be better explained as falling withiin a continuum of various informal and formal modes of data collection that are used to provide decision-related information in development settings. On one extreme are highly informal modes of data collection for which precise procedures are not established. These informal ap- proaches rely on intuition, experience, and common sense, and do not generate systematic information that can be verified. Examples of such modes are conversations with concerned individuals, general perusals of official records, and short visits to existing or planned project sites or institutions. For example, a project manager who wants to know whether small entrepreneurs are satisfied with the technical assistance provided by the project might talk with some of them or have a simple discussion with a fe-w field staff. If enough time is available, the project manager may even visit a few firms that have received technical assis- tance. On the basis of such meetings and visits, the manager may be able to reach a conclusion- The majority of decisions made in develop- nient bureaucracies are usually based on the data gained through such informal methods. The strentgths of informal approaches to data collection are that they are quick and inexpensive, and usually do not require outside assis- tance. Often the, decision-maaker alone is abl-e to gather relevant data in a readily usable form. However, the problem with informal approaches is the uncertainty concering the quality of the information gathered. Per- sadoe biases and prejudices can affect the reliability and validity of the 12 An Ovrvuiew of Rapid Appraisal Medwds in Development Settings information collected. Often the investigator finds what he or she wants to find and overlooks what does not support his or her thinldng or intuition. As a result, the credibility of such info rnnation, and con- sequently, of the decisions based on it, tend to be low in the eyes of otrs. On the other end of the continuum are the highly fornal data collec- tion methods that researchers have developed and refined over the years. These methods, which include cross-sectional and longitudinal sample surveys, censuses, experiments, and non-reactive data collection, have contributed to significant advances in sociaL economic, and be- havioral sciences. Their procedures are largely specified, and the in- vestigator is expected to scrupulously follow them in theory if not in practice. Such methods generate quantitative data that can be sta- tistically analyzed to draw conclusions within specified confidence mzargmIs The most important strength of formal methods is the relative accu- racy of the data generated by them. Although the problems of indi- vidual bias and erroneous inferences are not completely solved, they are greatly reduced. At least, other investigators can easily exane the .studies based on formal methods for such limitations. Consequently, the study findings have credibility in the eyes of decision-mnakers. Despite their accuracy and wide popularity, formal methods have their own limitations in many development settings. Often, they require greater time and resources than are avaiIable to the managers of the development interventions. For example, a medium-sized sample sur- vey may take four to six months to complete, and a larger survey may take even longer. Moreover, large sample surveys cost thousands of dollars, especially when an' expatriate specialist is involved. Above all, there is the question of whether managers really need the type of precise and often extensive information these studies produce. Between these two extremes lie rapid appraisal methods, which are neither highly informal nor fully formalized, and which require more than robust common sense and understanding on the part of the investi- gator. The investigator using rapid appraisal methods must have suffi- cient grounding in formal data collection methods to use them effectively. Although the investigators generally have considerable flex- ibility, they should make every attempt to systematically report procedures so that others can scrutinize them for accuracy and rel- evance. In most cases, rapid appraisal methods do not generate quanti- tative information from which generalizations can be made in a statistical sense to populations larger than those in the immediate cases examined. Krishna Kumar 13 Core Rapid Appraisal Methods The following five methods constitute the core of rapid appraisal and have been used in various permutations by experts in the project or program setting. Key Informt Intervews Perhaps one of the most important methods is the key infonnant interview, which is widely used by development practitioners. In fact, a majority of evaluations and policy-oriented studies conducted by inter- national development organizations rely largely on key informant inter- views. As the name indicates, such interviews involve interviewing a select group of individuals who are in a position to provide the needed information, ideas, and insights. Two special features of key informant r'; | ' ' interviews can be mentioned here. First, key informant interviews are essentially qualitative interviews, and are carried out with interview guides that list topics and issues to be covered in a sessiorL The interviewer frames the questions in the course of interviews and subtly probes the infonnants to elicit more informa- : tion. The atmosphere is informal, resembling a conversation among acquaintances. The interviewer takes extensive notes that are developed later. It is the unstructured nature of the interviews that invests them with special meaning and relevance in the context of rapid appraisal Second, only a small number of informants are interviewed who are selected on the basis of their specialized knowledge and experience on the subject under investigation. Depending on the nature and scope of an inquiry, the investigator identifies appropriate groups from which the key informants may be drawn, and then selects a few from each group. Thus, for example, if the researcher is interested in leaning about the functioning of agricultural credit institutions, he/she would first identify groups most likely to indlude people who can shed light on the subject, such as traders, moneylenders, village chiefs, farmers, local government officials, and other experts. The investigator would then select a few informants from each category so that diverse viewpoints and concerns are fully represented. The number of informants usually ranges from 10 to 25. The accuracy and depth of information obtained from key informant interviews depends primarily on the care that is exercised in selecting the informants, developing suitable interview guides, training inter- viewers, conducting interviews, probing informants, and recording responses. 14 An Ovemviewu ofRapid Appraisal Methods in Development Settings * Focus Group Interuiews Another method, recently added to the repository of rapid appraisal, * is the -focus group interview. Focus groups have long been used by marketing researchers to gauge the r-eactions of potential consumers to new products and services. In recent years, development experts have started using focus groups to designi, ixnplement, and evaluate health and family planning projects. There is no reason, however, why they cannot be used in other sectors as well. * ~~Focus group interviews are conducted to discuLss a specific topic in grup sessions. Participants discuss ideas, issues, insights, and experi- ences among themselves, and each member is free to commaent, criticize, or elaborate on the views expressed by other-s. The premise underlying the focus group method is that free discussions generate fresh ideas and insights because the participants stimulate each other. Focus groups are limited in size to 8 to 12 carefully-selected partici- pants. Suc-h small groups tend to facilitate the free flow of discussions. As much as possible, groups are homogenous in composition, with * members sharing sfimilar background and experience. A session gener- ally lasts one to two hours, although in some cases it can be longer. The moderator introduces the subject, keeps the discussion going using subtle probing techniques, and tries to prevent a few participants from dominating the discussions. Generally, several sessions with different participants are held on a specific topic. The composition may vary among the groups. X Community Interziews Unlike in focus group discussions, in-which participants discuss a subject among themselves, in comnmunity interviews, die investigator(s) asks questions, raises issues, and seeks responses from the participants. The primary interactions are between the intenriewer(s) and the partici- pants rather than among participants. Community interviews take the form of public meetings open to all communumty members. The date and location of the meeting are an- nounced in advance. The number of participants tends to be large * more than 15 persons), although past experience shows that, unless concerted steps are taken, certain groups-especially women and people- of lower socioeconomic strata-are often underrepresented be- cause of social and cultural barriers. Community interviews are ideally conducted on the basis of a carefully prepared questionnaire that lists al important questions to be asked. The advantage of having a well- -Krshna Kumar 15 designed questionnaire is that questions can be phrased in language that the participants can understand. Every effort is made in community interviews to make the discussion interesting and to ensure the participation of all those presenL Al- though community interviews can be conducted by one interviewer, a team of two or more is preferable, because it is difficult for a single individual to preside over the meeting, ask relevant questions, and record the answers. Structured Direct Observation Structured direct observation is yet another rapid appraisal method that has been used with considerable success in developing countries. It involves careful gathering of data based on well-designed observation formns, which are designed to take into consideration thp nature of the coject to be observed. In most instances, direct observation also in- volves individual or group interviews, or both. Structured direct observation should not be confused with the ethno- graphic method of participant observation. Three major differences be- tween the two may be noted here. First, the participant observation method is a long-tern process; a researcher observes a phenomenon or process for months, even years. In contrast, studies based on structured direct observation can be completed within days or weeks. Second, while participant observation focuses primarily on social and cultural phenomena, direct observation can deal as well with physical objects, such as roads, dams, or agricultural production. Finally, in participant observation, the observer tries to empathize with the people being stud- ied to gain an insider's perspective. This is not always the case in direct observation. Direct observation is better conducted by a team of experts than by a single individual. A team approach contributes to more comprehensive data collection and helps to prevent individual biases. Informal Surveys Finally, informal surveys have emerged as an inportant tool in rapid appraisal studies of agricultural and rural development interventions. Such surveys are usually conducted on the basis of an open-ended ques- tionnaire that permits respondents to answer questions in their own words. The sample size for infonnal surveys usually ranges from 25 to 50 people, who are selected on the basis of noni-probability sampling techniques. One popular technique is convenience sampling, in which 16 An Overview of Rapid Appraisal Methods in Development Settings respondents are interviewed in markets, shops, public meetings, organi- zations, and other places selected on the basis of easy accessibility. Fi- nally, the interviewers enjoy considerable flexibility in asking questions and are not constrained by the given questionnaire. Although informal surveys are in many ways similar to key infor- mant interviews, important differences exist between them. First, while the key informants provide information about others, respondents in infonnal surveys answer questions about themselves. Thus, to under- stand the problems faced by the owners of micro-enterprises, in a study based on key informant interviews, the investigator will not only inter- view the enterprise owners, but also others who might be knowledge- able about them, such as project staff, concerned government officials, traders, executives of ownerd organizations, and other experts. An in- formal survey, on the other hand, will primarily target the owners. Sec- ond, while similar if not the same sets of questions are asked of each respondent in informal surveys, this is not the case in key informant interviews. In fact, different sets of questions are often put to key infor- mants of different backgrounds and experience to obtain a more bal- anced and comprehensive understanding of the subject. Third, the number of respondents in key informant interviews is typically smaller than in iformal surveys. LTimitations of Rapid Appraisal Before examining the strengths and potential of rapid appraisal medt- ods, it is important to recognize their limitations. Because of the initial success of many rapid appraisal studies in generating relevant informa- tion and enlarging understandin& dtere exists considerable euphoria about tiem. As a result, practitioners tend to minimize or ignore the following shortcomings. First, the reliability and validity of the information generated by rapid appraisal methods may be questionable in many cases. In the language of social research, validity refers to the soundness of the research find- ings. If the answer to the question is sound, the research is supposed to be vald, and vice versa. Reliability is one of the constituent elements of validity and refers to the extent of random variation in the results of the study. Three factors may contibute to the low reliability and validity of the findings of rapid appraisal studies. Probability sampling is not used in the selection of individuals or groups for such studies. Investigators rely largely on their convenience or on expert judgment to select the people to be interviewed, the sites to be visited, or the activities to be observed. As a result, it is quite pos- sible that the sample is not representative of the whole population. For Krishna Kumar 17 example, when communities are selected using non-probability sam- pling& outside investigators are more likely to select communities that are easily accessible by transportation. Moreover, poor and deprived groups are likely to be underrepresented in community interviews, un- less efforts have been made to ensure their parficipation. During the data collection stage, individual preferences, judgments, and views of the interviewer/observer may significantly affect the con- duct of inquiy. This happens, not because of any deliberate attempt on the part of the researchers, but primarily because the research instru- ments used in rapid appraisal investigations are usually open-ended. The interviewer/observer enjoys considerable flexibility in fming and asking questions, observing phenomena, and recording answers and ob- servations. While such flexibility contributes to more in-depth discus- sion, it also leads to increased probability of introduction of bises and distortions. For example, one persistent problem with key informant interviews, focus groups, and group interviews has been that unless investigators are extremely careful, they hear what they want to hear and ignore what they do not want to hear. Still worse, they may not even be aware of this problem- For instance, in many focus. groups, we find that modera- tors unknowingly introduce biases by reacting with enthusiasm to com- ments that confirm their preconceived views, while showing indifference to opposing views; by being patient with those who seem to be supporting their positions, while demonstrating impatience with oth- ers; or by not probing those who articulate a different viewpoint or volunteer information that contradicts their preconceptions. In any case, the result is that data, opinions, and arguments that do not support the pre-existing framework of the investigator are not always fully pre- sented in the groups. Qualitative data, by their nature, are difficult to record, code, and analyze objectively. This problem is not unique to rapid appraisal meth- ods, but is found in ethnographic investigations as welL Ordy recendy have anthropologists and sociologists started focusing on developing and refining techniques for recording and analyzing qualitative data and for improving the data's reliability and validity. A caveat is necessary here. While there is little doubt that validity and reliability can be a major problem in many rapid appraisal studies, this is certiy not a universal limitation. In fact, most investigator; using rapid appraisal methodology usually take several steps to im- prove the accuracy of their findings. First, they use a well-articulated conceptual framework, which is often-though not always-empirically grounded. Thus, they try to solve the problem of validity by focusing on what researchers call "external validity!' Second, they employ many 18 An Overiew ofRapid Appruis Methods in Development Setfings strategies used in ethnographic investigations to minmize the element of bias during the collection and analysis of data. Third, they try to use more than one rapid appraisal method so that data generated by one source can be cross-checked with those produced from the other sources. Fourth, niany practitioners are becoming increasingly aware of the problem of subjective bias, and threfore critically scrutinize their own procedures. Second, rapid appraisal methods do not generate quantitative data from which generalizations can be made for a whole population. They can give a relatively accurate picture of the prevalence of a phenom- enon, attitude, perception, or behavior pattern, but not of its extent or pervasiveness. For example, an investigator may leam that farmers are not availing themselves of the short-term agricultural credit offered by public sector banks because of the cumbersome delay in processing loan applications. However, the investigator could never know what percentage of farmers in the project area are being deterred by this factor. Even when some quantitative data are generated by some of the rapid appraisal metbods, they cannot be the basis for making generali- zations for the entire uriverse. Fimaly, there is some problem with the credibility of the findings reached by using rapid appraisal methodology. Most decision-makers are more impressed with precise figures than descriptive statements. For example, the finding of a sample survey that 83 percent of the local entrepreneurs were satisfied with the techical assistance provided by a project is lily to carry more weight than the conclusion based on key informant interviews, that most of the entrepreneurs interviewed seemed satisfied with the technical assistance. Rationale for Using Rapid Appraisal Why should investigators use the rapid appraisal methodology if it has these shortcomings? The smple answer is that it has several advantages in the context of development projects and programs that must be weighed against its limitaticns. Fmist, the nsst compelling reason for using rapid appraisal methods is that the cost of such studies is usually much less tian others that use more rigorous methods. For exampIe, one sample survey conducted by a US. firm is likely to cost UTS$100,000 to US$200,000, an amount that can support three to four rapid appraisal studies. The costs of these studies is low partly because of the smaller sample size and more fo- cused nature of the investigation. It should be recognized that under the conditions of scarce resources prevailing in developing countries, the opportunity cost of resources Krishna Kunar 19 * 0 spent on information-gathering is high because these resources can be * - - easily expended on activides that would more directly contribute to increased production and incomes. Thus, the US$200,000 needed for conducting a socoeconomic survey of fanmers could instead be used to provide farmers with agricultural inputs that would have a tangible, positive impact on agricultural production and incomes of the con- cemed households. By- reducing the overall cost of studies, rapid ap- praisal can therefore help project and program managers opfimize the use of available resources. Second, rapid appraisal studies can be completed quicldy, thus ensur- ing that the findings and recommendations are available to decision- makers when needed. For example, through community interviews, an investigator(s) can reach eight to ten communities or villages within a week, thereby soliciting the views of hundreds of people. Such a course is not possible in the case of a carefully conducted sample survey. Like- wise, focus group interviews enable the investigator to benefit from the judgment of concerned people within a matter of days, rather than months. This is indeed a very inportant advantage, because in project and program settings, administrative deadlines and not the require- maents of field research determine the time span available for conducting studies. Seldom do managers have the option of postponing crucial decisions in anticipation of infornation. In most cases, managers must - xake important decisions at a given time-with or without information. The timely availability of data and findings is important to them, even if these are not as precise or elaborate as might be wished. The widespread interest in rapid appraisal methodology was initially heightened by the realization among development practitioners that studies based on formal methods are often too time-consuming. There- fore, even when such studies generated valid and reliable findings, they were of little use if these results were not available at the time a decision had to be made. (It should be recognized, however, that with careful planning and the use of micro-computers, the time required for sampIe surveys, and even censuses, can be reduced.) Third, in many instances, especially when an interpretive understanding of a phenomenon or process is required, rapid appraisal methods are succesful in obtaining relevant data, ideas, or recommen- dations. A major limitation of many formal metiods is that they tend to focus primarily on quantifiable information, and much information is lost in *f - the process of "operationalizing" social and economic phenomena. Thus, they are often of limited value in studying complex socioeconomic - -changes, highly interactive social situations, or people's underlying mo- tivations, bdliefs, and value systems in prcject and program settings. 20 An Qveroiw ofRazpid Appnda Methods in Deepmenxt Settings Much of this kind of ifraoncan be captured by rapid appraisal mnethods. Finally, rapid appraisal methods provide flexibiiy to the investigator to explore new ideas and issues that may not have been anticipated in planning the study, but that are relevant to its purpose' For example, suppose in the course of a study, a key informant indicates that one of the main reasns entrepreneurs are not takng out loans in the micxo- entrprsedevelopment project is the complex and cumbersome loan application procedure. The investigator can pursue this issue with the other informants, even though it was not hincuded in the interview guide. Such a change is not possible in sample surveys or censuses once the questionnaire is designed and the investigation is underway. Appropriate Role for Rapid Appraisal Perhaps the most important question that can be asked about rapid appraisal methods is, fWhen should they be used in development set- tifgsr In antswering this question, it should be recognized that the choice of appropriate methodology-fonral or rapid appraisal-should be dictated primarily by the io equire t of the decision- maker and the resources and time available for conducting an inquiry. It should not be deterined by some abstract notion of validitr and reliabilty, which, though essential for basic research and theory formu- lation, is only one of the many considerations in designing, implement- ing, and evaluating development interventions. Three factors should be examidned in choosin3 appropriate methods in a given situation The first obviously, is the purpose of the study. If the primary purpose of ar investigation is to make major policy choices or programmatic decisions, the need for employing formal methods, which are more lily to generate precise and valid information, is ap- parent For example, if the objective of an evaluation is to assess the effects of structurl adjustment programs on lower socioeconomic strata or to test the effectiveness of a highly innovative agiculd t develop- ment program introduced in a country, comprehensive longitudinal and cross-sectional surveys will undoubtedly be necessary. In such cases, the stakes are ioo high to rely on less reliable measures. On the other hand, if the objective of a study is to make a simple assessment of how a project has been doin, what services it has been successofy providing, and what implementation problems it has been encounterin, rapid ap- praisal methods may better serve the purpose. This is, in part, because very preciseinformation is not really needed and the risks involved in arriving at less accurate results are likely to be limited-if present at all-nd may be small compared with the resources and time saved. Krishna Kwnar 21 The second factor, which logically follows from the first, is the nature of the information required. One should not assume that formal meth- ods are always efficacious in answering all kinds of questions. In fact, many questions that require in-depth, inside information can be better answered through rapid appraisal methods. Such methods are particu- larly appropriate for the following situations. * W desctive information is sufficientfor decision-making. Such infor- mation may pertain to assessing organizations and institutions, socio- economic conditions of an area (village and communities, for example), or characteristics of the relevant populations, including cul- tural patterns, behavior patterns, and values and beliefs. In fact, rapid appraisal studies have been most promising in stimulating in- sightful, extensive discussions of smallholders, the economic and so- cial environment in which they live, the institutional constraints they face in adopting recomnmended innovations, their views and percep- tions about development interventions, and their suggestions and recommendations. It is hardly surprising, then, that fanning systems speciaIists have been the most outstanding exponents of these meth- ods. * Whm an understanding is required of the motivations and attitudes that may affect people's behavior, in particular the behavior of target populations or stakeholders in an intervention. Rapid appraisal methods are quite successful in answering thre 'why' and "how" questions. For ex- ample, key informant interviews or focus group discussions are more hkely than samnple surveys to provide insightful answers to such questions as, "Why are farmers not adopting the recommended vari- ety of seeds?" or 'How is the internal politics in an agricultural uni- versity affecting its capability to do strategic planning?" or "What institutional barriers are coming in the way of promised policy changes in the health sector?" or "How are mnacoecononmic policies being implemented in the country?" a WMen available quantitative data must be interpreted. Usually, donor agencies, hosts governments, and project and program managers have access to routinely-generated quantitative data (for example, data about financial outlays, targets reached, volume of inputs and services provided to the participating populations, or beneficiaries contacted) or data gathered for other purposes (for example, data collected by donor agencies, projects, or host govenmuents). Many of the rapid appraisal methods are extremely useful in interpreting such data, resolving inconsistencies, and deriving meaningful conclusions. Suppose, for instance, project records show that female farmers are not using the technical package recommended by the agricultural development project Interviews with selected key informants and 22 An Overiew of Rapi'd Appraisal Methzods in Development Setfiugs one or two focus groups can shed light on the factors that explain this behavior. M Wen the primary purpose of the study is to generate suggestions and rec- ommendations. In many cases, the prime reason for an investigation is to solve a problem facing a project or progrm.- What is needed is a set of practical recommendations. For example, the manager of a contraceptive social marketing project may be more concerned with finding out what can be done to, augment bontraceptive sales than with conducting an in-depth, quantitative study of the subject The *managers needs can be better served through interviews with the concerned doctors, pharmacists, medical workers, traders, and cur- rent or potential users to elicit their suggestions. An alternative may be conducting a few focus groups on the subject. * Whe the nieed is to develop questions, hypotheses, and propositions for more elaborate, comprehensive fornnal studies. Key inforniant and group inter- views, along with the literature review, are widely used for this pur- pose. The third and final factor that should 'be considered in choosing be- tween formal and rapid appraisal methods is the available resources. These include money, technical expertise, time, and the institutional support for research. All must be considered in any given situation. Monitoring a Large-Scale Resettlement Program with Repeated Household Interviews mfayer Scuddler In this dwpter, Thayer Scudder describes the mefthdology of the repeated informal surveys he and Kapila 'Vimaladharama conducted to assess the -impact of the AcceleratEd Mahaweli Programme in Sri Lanka. Thiis major irrigation and resettlement project was desgned to bring under irrigation 50,000 hectares of new land and to im- prove irrigation facilities of another 10,000 hectares of already irri- gated Land. To understand the project's effecQ: on the lives of the settlers and the problems thy faced, Scudder and Vimzaladharama repeatedly interviewed the same households over a ten-year period. The num- ber of households interviewoed r-anged from 29 to 45. Over time, they established rapport with thes households, and the interviews were conducted in an informal atmnosphere. In addition to giving information about theonelves, the respondents were asked to give information about other households who were in the same irrigation turnout or came from the same community of origin. They supple- mented intermiew data with those -obtained from community inter- views. The author mentions that their methodology generated usefut information, insights, and understandings that could have helped the project management. For example, as early as in 1983, their data indicated that the full potential of the project was not likely to be realized, as the net income of the settlers was not sufficient to geneate major multip$ier effects in terms of enterprise development and employment generation. The findings of the 2984 survey fur- titer confirmed this condusion by showing that the living standards of the sampled households deteriorated or remained stagnant The report of the 1985 survey also noted that Tamil-speaking settlers 23 24 Monitoring a Large-Scle ResettLement Program were not beingfuilly incorporated into the program. All these find- ings and concdusions proved to be correct with hindsight. -Three limitations of this methodology should not be overlooked, however. First, since the sample is both small and opportunistic, there exists a high probability of bias in thefindings. In our judg- ment, the sample seems more appropriate for highlighting problem areas to be further explored than for making generalizations about an entire population. Second, the researchers'familiarity zwith the respondents-and the latter's expectations of help from then- might have influenced their responses. For example, it is not un- -liley that the respondents over-stressed their problems and under- stated the benefits they obtained in anticipation that the author would exercise his influencE to obtain greater assistance for them. Third, a time interval of one or two years is too long for accurate recall. Despite these limitations (which can be easily overcome with more careful plannin& and by using a larger sample), this method- ology has great potential for monitoring and evaluation. NOW IN lTS 12TH YEAR, an ongoing review (Scudder 1979, 1980, and 1981a, and Scudder and Vimaladharma 1984, 1985,- 1986, and 1989) of the settlement component of Sri Lanka's Accelerated Mahawel Programme (AMP) has dutee major purposes. The first is to evaluate, in very general terms, the impact of the AMP on over 50,000 households settled on irrigable lands in the Kala Oya, Mahaweli, and Madura Oya basins since the mid-1970s. The second is to assess the implications of those impacts for realization of the AMPs major goals as they relate to employment generation, increases in production and productivity, ris- ing living standads, and regional development The third purpose is to 'report findings to the relevant Sri Lankan govenunent agencies, to the United States Agency for International Development (USAID), and to other interested donors within a fortnight after completion of each field survey. Throughout, USAID has funded the study-through its Office of Evaluation up to January 1979, and thereafter, through the Institute for Development Anthropology, or through the Clark UriversitylInsti- tbte for Development Anthropology Cooperative Agreement on Human Settlements and Natural Resource Systems Analysis (SARSA) with USAID's Bureau for Science and Technology. - : To date, eight project-related surveys have been made, inluding two - in 1979, and one each in 1980, 1981, 1983, 1984, 1985, and 1989. Brief visits were also made in 1988. Further assessment is to be undertalken during 1990 and 1991. In-country evaluations last four to seven weeks, with a report submitted during the final week Costs, involving two senior investigators, international travel for one, and in-country travel :myer Scudder 25 and per diem for both, range between $20,000 and $40,000 per evaluation. Even though most USAID funds went to one irrigation command area (System B), it was agreed from the start that the study should include the three major settlement areas of the AMY (Systems H, C, and B). It was also agreed that the investigator-s would have the flexibility, to follow up on unexpected findings. The settlement of large numbers of people in problem-prone environ- ments can be expected to have a wide range of impacts, most of which have policy implications. included are impacts on the natural resource base, human healfth community organizationt, political relationships be- tween different categories of settlers, and living standards at the house- hold, community, and regional levels. This case study's scope covers all of these, along with the effectiveness of the planning and imnplementing institutions. Throughout, however, the main emphasis nas been on household living standards. The key aspect of the methodology for surveying these living standards has been repeated interviews with a small, stratified opportunity sample of households. A fundamental assumption of the investigators is that it is the rising disposable income of hundreds of thousands of small-scale producers that drives development forward during the early stages of industrial- ization (Mellor 1986). As disposable incomes rise, low-income produc- ers not only diversify their household economides in predictable ways, but also buy a remarkably similar r-ange of locally- and nationally-made goods and services-the demand for which increases the generation of a widening range of non-farm enterprises and employme-nt UJohnston arid Kirby 1975). Such behavior is especially characteritic of households colonizing new lands (Scudder 1985). Conversely, if settler households are unable to move beyond subsistence, or if incomes fall, goals relating to in- creased production for the market, to employment, and to regional de- velopment suffer. Either way, concentrating on -changes and continuities in settler living standards enables evaluators to assess the extent to which project goals are being met. Counizy and Program Setting The Accelerated Mahawdi Programme Sri Lanka's first major attempt at regional development, the Acceler- ated Mahaweli Programme, is one of the largest development projects currently under way in the tropics and sub-tropics. In 1977, the Sri Lanlkan Goverrnment decided to accelerate the development of the 26 Monitoring a Large-Scale Reseftement Program country's dry zone that had been outlined in a number of LTNDP/FAO- assisted feasibility studies. Four mainsteam dams on the Mahaweli, Sri Lanka's largest river, were the key intervention. Increasing the national supply of electricity by about 50 percent during the AMP years, these dams-in conjunction with hundreds of kilometers of main canals, and other dams in the Kala Oya and Madura Oya Basins-would also store sufficient water to intensify production an approximately 10,000 hect- ares of previously irrigated land, and also to bring under irrigation ap- proximately 50,000 hectares of new land in north central and northeastern Sri Lanka. System H was settled first, with ongoing settle- ment still underway in Systems C and B during 1990 (Figure 21). With donor assistance totaling over US$1.0 billion dollars, project costs to date approximate US$2.0 billiort dolars. The Settlement Component of the AMP As planned by a small interdisciplinary task force in the early 1970s, the implementation of the settlement component of the Ah?P involved a comprehensive program of land reform whereby each head of house- hold, including married sons over 18 years of age, was to receive one hectare of irrigable land and a 0.2 hectare homelot. This size was based, not on income calculations, but rather on the amount of irrigated land that experimentation had shown could be double-cropped by family labor alone. After the 1977 election, a parastatal organization, the Mahaweli Authority of Sri Lanka (SL), was created to accelerate implementation. Though intended to catalyze a process of integrated area develop- ment, the AMP was planned and implemented primarily as a paddy production scheme. Four kinds of settlers were involved. Priority was given to the host population already resident in Systems H, C, and B, and to at least 30,000 evacuees who had to give up their lands and/or homes to maake way for dams, reservoirs, canals, and other infrastruc- ture The numbers in both categories were seriously underestimated. This reduced the number of outsiders from different electorates throughout the country that could volunteer for settlement and who made up the third category of settlers. Most of these government-spon- sored settlers were selected on the basis of landIessness and family type (young couples with small dhildren were preferred). As for the fourth and smallest category of settlers, this consisted of spontaneous settlers who had moved into a given Mahaweli command area prior to a cutoff date, after which more recent spontaneous settlers were ineligible for selection. f~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~~~hy Scude !27 '0- S~~FS . Aceeae Maawl Progra :~~~ ~afi Pon ,P.d.r. . , .~ ~ ~ ~ ~ ~ auny : ,~~~~~~~~~abrr ; : he L1 Accelerated M;ihaweliA Prarpatb ColomboPe ir ' X tS W~~~~~~~~~~~~~PRUI *>~~~~d Walaw :: o vJ ~~~~~~~~RsrvL Ga e *28 MonitorMg a Large-Saule Resettlement Progm Methodolog The repeated interviews with the same households over an extended time period were the most important and time-consuming component of the survey methodology. These interviews provided the basic informa- tion for assessing project impacts and institutional effectiveness by con- centratig on project-related problems, on how different members of each household attempted to cope with those problems, and on the new circumstances in which settlers found themselves. In the field, house- hold information was checed aginst, and complemented by, informa- tion obtained from larger community interviews and intemviews with government officials and other knowledgeable people. Back in Colombo, household data were furthe complemted by discussions with headquarters staff in government agencies and with academic re- searchers, and by a review of governuent, donor, and academic reports on the AMP. Draft Qral reports were then written during the final days of each visit and submitted to the Goven-ment of Sri Lanka and AID. Although comments were requested, few were received, with the result that the final report-submitbed after a waiting period of approximately two months-differed little from ithe draft final report. The two-person survey team proposed in 1980 to AID and the gov- emment that an ongoing evaluation of the AN? based on repeated in- terviews with a relatively small number of households be initiated. AID concurred, building annual evaluations for the 1981-86 period into their Januaxy 1901 Proect Paper on Mahaweli Basin Development Phase II. Since then, the methodology for such interviews has continued to evolve, the most recent addition being greater emphasis on studying the net works to which the 45 1989 households belong. As used in the Mahaweli case, the methodology that has evolved has some major weaknesses. Some of these are inherent in reliance on re- peated interviews with a relatively small number of households. These can -be mitigated, but not eliminated. Others are due to the way in which the methodology evolved and these can be eliminated. While there is no evidence that these weaknesses have led to erroneous conclu- sions, valuable lessons have been leamed, which should benefit other users. These are discussed in the last secdon of this chapter. There are several justifications for this methodology. Emphasis on household living standards makes sense because it is rising productivity and disposable income that will determine the extent to which projects like the AMP meet their goals Repeated interviews are important, since goal achievement can not be expected to occur within a five. to ten-year period in a project in which the main producers are thousands of house- holds pioneering a new habitat with a problem-prone technology. hayer Scudder 29 Keeping the sample size small is essential, since large-scale surveys based on random sampling are seldom repeated because of their high costs, in termns of money, personnel, and data management. Even where * ~benchmark surveys are completed, their costs often delay the publica- * ~tion of results by years, as opposed to the months involved in producing reports based on repeated interviews-as in this case-with a small number of households. Repeated Intemiews Repeated interviews also provide more reliable information, since re- asearchers are better able to evaluate the accuracy of the information given, and household members are more likely to provide thoughtful answers to interviewers who are knowledgeable about their affairs. The methodological weaknesses during the early years of the study notwith- standing another justification of the Mahaweli study is that none of the major conclusions has subsequently been shown to be erroneous. A major purpose of repeated interviews with 30 to 50 households is to identify problems affecting the realization of project goals as they arise. If one-third of a small sample are having problems with the preparation of, or water delivery to, their one- hectare holdings, and if over half are having problems with malaria and the repayment of their seasonal agri- cultural loans, it is reasonable to assume that malaria is a prsoblem and that land preparation, water delivery, and provision of credit are defec- tive. Siuilarly; if only a small mrinority of households have moved be- yond subsistence over a five- to ten-year period, one can assumne fewer multiplier effects, in terms of employment generation and non-farm en- terprise development. The purpose of such evaluations is not to produce statisticaly signifi- cant correlations for academic publications, but rather, timely feorma- lion for policy-makers, planners, and implementing officials. Repeated interviews with a relatively smatl number of households produce just that type of information. While complementing them with large-scale surveys will provide more "reliable" information, there is apt to be a major cost, in terns of the prompt initiation of corrective action associ- ated witho its delayed receipt Such delays suggest that a more cost- effective approach would be to improve methodologies based on repeated -interviews with small samples. The same two-person research team has been involved in the study since its commencement. One member (KC P. Vimaladharrna) is a senior Sri Lankan civil servant with a special interest in irrigated land settlement schemes as a development intervention. The other member * . (T. Scudder) complements his colleague's Sri Lankan expertise with 30 Monitoring aLarge-Scale Resettlemnent Program knowledge of land settlement elsewhere in Asia, Africa, and the Middle East. This combination has proved exceptionally fruitful in developing the survey methodology. Sample Selection and Size The basic unit for sampling is the household. In large-scale agricul- tural development projects, households selected should be stratified ac- cording to location, background, and wealth criteria. Table 2.1 shows the number and characteristics of the selected survey households in 1981, 1985, and 1989, with all of the 1981 households incorporated within the 1985 sample, and all the 1985 ones incorporated within the 1989 sample. In the Mahaweli case, location criteria meant selecting households within the thiree Mahaweli systems and, to an extent, from different areas within each system. Table 2.1. Accelerated Mahaweli Programme Household Sample Categoq 2982 1985 1989 Sample size 19 33 45 -Type of settler household Host a 5 9 13 Evacuee 8 12 14 Selectee 3 7 12 Spontaneous 3 5 6 Location System H 13 is 15 System C 5 8 10 System B 1 10 20 Language and religion Singala (Buddhist) 19 33 40 Singala (Muslim) 0 0 1 Tanmil (Hindu) 0 0 3 Tamil (Muslim) 0 0 1 a. Although hosts are mainly residents of long-established purave (villages), in four cases, they are settlers who were recuted in the 1970s in connection with older settement schemes or land development programs. Thayer Scuddrr 31 Background crteria should vary according to type of project and par- ticipant households. In the AMP-type of household (hosts, evacuees, govenunent-selected, and spontaneous settlers), language Singala and Tamil) and religion (Buddhist, Hindu, and Muslim) were emphasized, although Tamil-speaking Hindu and Muslim households were not se- lected until 1989, because over 95 percent of project househoIds through 1985 were Singala-spealdng Buddhists. Two simple wealth categories were used: those within the project with sufficient capital to invest in off-farm enterprises and improved housing from the start, and those (the large majority) who were initially dependent on government and kin-based assistance, on their Mahaweli 02-hectare homelot and one-hectare field allotments, and on income from wage labor and ardsanal and other skills. A disproportionate number of households were selected in the first category to increase the proportion of "key informants" in the sample. Tending to be entrepre- neurs and local leaders, anthropologists have found such informants to be an excellent source of information on development constraints and strategies and on community affairs They were also disproportionately represented in the AMP on the assumption that development con- straints adversely affecting such informants would have worse effects for households with fewer resources. Analysis of interviews suggests that 30 carefully selected households are sufficient for identifying major problems if a disproportionate num- ber of "key informants" is included; if supplemental information is gath- ered on a reference group about which each household is knowledgeable; and if household interviews are combined with inter- views within the community with relatives and neighbors.. In the Mahaweli case, the number of households has been increased to 45 (see Table 21) as the AMP brought new areas under cultivation, including the northern portion of System B, where a majority of the host popula- tion for the first tine were Tamil-spealing Hindu and Musli. villagers. Selection of these 45 households was done on an opportunistic basis, meaning that households were incorporated that were available at the time and that met criteria for selection. For example, during the 1979-81 period, we actively sought out evacuees, whereas in 1989, we added two Muslim and two Hindu households. Most were selected because they happened to be available at home when their homelots were visited. Respondents in the sample were also asked to give infornation about other households (reference groups) who were in the same irrigation turnout or came from the same community of origin (village in the case of hosts, evacuees, and spontaneous settlers; and electorate in the case of govenment-sponsored settlers). Thus, respondents provided informa- tion not only about their own households, but also about others as well. 32 Monitornng a Large-Scale Resettement Progrnm In addition, community interviews were conducted when neighbors and relatives came to the sample household during or following an inter- view. The sample also included seven key inforniants. Such an opportunistic sample is clearly biased. As Table 2. demon- strates, a disproportionate number of households are evacuees. Because of specific inberest in how evacuees cope with compulsory relocation in connection with large-scale development projects, several such house- holds were seleted before the study became longitudinal. Two of the three households that had been interviewed prior to 1981 were therefore included in the larger 1981 sample in order to provide some continuity back to 1979. Another bias relates to the disproportionate -number of wealthier settlers, a bias that is apt to cause problems to be underesti- mnated rather than overestimated. En the AMP case, however, there is no evidence that such biases have led to faulty conclusions. Where the evaluator is looling for evidence of major problems and major impacts, it makes little difference whether the percentage who have been unable to move beyond subsistence, or more specifically, have been unable to pay back their bank loans, is 60 percent or 75 percent. Substantive Issnes wad Research Instruments From the very beginning an attempt was made to question both adult male and female members of each household about a wide range of topics. The selection of topics was largely based on a series of formal questionnaires that the Institute for Development Anthropology had formulated for a multi-year, AID-funded global evaluation that began in 1979 concerning tie experience with land settlement in the tropics and sub-tropics (Scudder, 1981 and 1984). Selected specifically for the study of raided and irrigated settlement projects, these topics dealt with tfhe broad issue of land settlement "success"-success being defined in terms of regional development resulting from increased productivity, increased disposable income and living standards,and associated multi- plier effects. Other than getting more detail (including increasingly quantitative data) little change has been made over the years in topics selected for the Mahaweli study, presumably because those topics were derived from previous analysis of a world-wide sample of over 100 settlement projects. Without jeopardizing their utility for comparative international analysis, the wording of questions was adapted to Sri Lankan condi- tions. During the first interview, fairly detailed information on house- hold history prior to incorporation within the AMP was collected, including the agricultural and employment experiences of the house- hold head since the completion of schooling. Other information was T hayer Scudder 33 gathered on the nature of the AMP resource base and the various comn- -ponents of the household production system (homelot, field allotment: and water conveyance system, livestock, and such offfarm activiie as wage labor, carpentry and masonry, and busines enterprises) and asso- dated problems; costs of production; nature of bousin& househod fur- nishings, and such equipment as sewing machines, plows, and carts; local organizations, mdluding water user associations; social services; health, and relationships with government officials. A one-page wealth index was also created (Table V2) dealing with five major topics relating to living standards, as demonstrated primarily by a visual inspection of each household's homelot Supplemented by a few questis, thi enabled a rapid assessment of changes in living stan- dards to be made as they related to housing, household furishings, production gear, fuel and lighting; and domestic water supplies and waste disposaL In the absence of detailed quantitative infornation such a wealth index can provide, within a five- to ten-minute period, raidngs on a scale ofI to 5 for each of the five variabes. While questiors were taken from formal questiorazakes, answers -were not written down on a schedule in the 1979-85 period. Rather, the evaluators carried with them a list of questions to be asked, the arswers to whih were written down in notebooks during the interview. Each interview was typed up that evening or within the next few days if possible. This procedure is not recommended, since there was a ter- dency to skip various questions, especialy after lengthy digressions to follow up on interesting issues mentioned by household members. While such open-ended information is invaluable, it should not be col- lected at the expense of a core of key questions on which comparative and longitudinal analysis is to be based. Fortunately, one of the binefits of repeated interviews is that many oxissions can be subsequently cor- rected. In 1989, a detaled interview form was used for three reasons. The first was to correct for the deficiencies of not using such a form in the pastL The second was because of the need to gather information for a four-year period rather than a single year, since AID funding was un- available for the 198688 period. The third reason was the desire of the evaluators to gather more quantitative data on income and expeditures to make possmble the type of analysis presented in Table 2.3. Looling to the future, a more efficient mechanism is needed for re- cording information in the field. While the complexity of the data gath- ered and the importance of open-ended questioning predude coding material in the field, for 1991, the evaluators planned to use a laptop personal computer during each interview for entering new data directly on files that contain time-series data on each household. 34 Monitoring a Large-Scaf Resettlment Progrmn Table 22. Wealth Index Category Ranking A. Nature of housing 1. Tile or other improved roofing and over five rooms 5 2. Tile, etc. roofing, 4-5 rooms 4 3. Tile, etc. roofing, 1-3 rooms 3 4. Thatch or cadjan roofing, over 2 rooms 2 5. Thatcb, etc. roofing 2 rooms or fewer 1 B. Nature of domestic water supplies and sanitation facilities 1. Improved (concrete) well/piped water supply and water-ealed toilet 5 2. Improved well etc. or water-sealed toilet, etc. 3 3. Imprved wel and unimproved sanitation I C Home furnishings 1. Five major items of furniture such as glassed-in cupboard, large wall dock, televison, settee set, spring bed, radio/ cette player 5 2 Three or four major fumishings 4 3. One or two major furnishings 3 4. No major furnishings, but cane or simple wooden furnituhre and a small radio 2 5. Small radio or care/wooden chairs, etc. only 1 D. Farm equipment/transport/boutique/mill 1. Tractor/truck, boutique, and/or grinding mill 5 2 Motorcycle/water pump plus 2 trained buffalo/oxen 4 3. Motorcycle/water pump or 2 trained buffalo/oxen 3 4. Bicycle only or dairy cows, commercial pigs, etc. 2 5. No bicycle, hand tols only 1 E. Appliances/lighting 1 Kerosene or other type refridgerator, stove, and one other major appliance 5 2 One or two majr apphances 4 3. No major appliances, but working petmrmax (pressure) lamp or electric lighting 3 4- Parafin wick lamp only 2 5. Candles only 1 Note: Items used should be adapted to each project situation. Thayer Scudder 35 Table 2.3 Household Expenditues for Basic Needs, 1988-89 (average for a 12-month period in rupees) Need Expenditure Food (33 households) 20,449 Production for paddy (17 househoIds) 18,558 Clothing (family of five) (37 households) 4,400 Travel (37 households) 946 Social and ritual (38 households) 943 Medical (39 households) 800 Minor household (pots, pans, etc.) (35 households) 470 Total 46,566 Note: October 1989 exchange rate SL Rs 39.69 - US$1.Q0 Nature of the Interviews While the basis of each interview at the household level was a series of pre-determined questions for comparative and longitudinal analysis, the answers invariably raised interesting issues that were then explored trough open-ended questioning. As an example, questions about health indicated a relatively high level of stress associated with AMP settlement, as revealed by suspicions and accusations of sorcery with regard to relatives and neighbors, and by high rates of suicide within the reference groups of sample households. Through open-ended ques- tioning, more detail was elicited on both subjects. Fears associated with the communal strife that has stricken Sri Lanka during the 1980s were also explored in det4il along with actions based on those fears. Al- though no pre-determmined balance between structured and unstruc- tured questioning was set, perhaps 25 percent of the information gathered came from unstructured interviewing. Data Collection Selecting Households Selection was always opportunistic In the field, the survey team traveled by car. Opportunity sampling was used not orly to select households, but also to check out any activities of interest In the latter case, unannounced side trips into offices and projects sign-posted along the way were made in order to learn about their activities from partci- pants and managers. In one or more towns in each system, censuses 36 Mdonito ring a Large-Scale Resettlement Program were taken of the number and type of business enterprises, with conver- sations often leading to re-visits to the same business in later years Simiarly, fr-equent stops were made to query individuals and groups about their activities as seen fr-om the road. In some cases, the households of individuals interviewed during roadside stops were incorporated in the sample. For example, two households of goverznment-sponsored settlers were selected after inter- views were carried out with the worker/settler group to which each household head belonged. (En the AMP, once household heads were selected as settlers, they preceded their families and worked in govern- ment-constituted groups to assist with the construction of the water conveyance system. They also cooperated in building temporary hous- ing on their homelots.) In another case, a household of spontaneous settlers-first inter- viewed because they had a small roadside boutique-were added in order to see if subsequently they were incorporated within the project (they were and have been interv-iewed during each visit since 1979). Conversations with a Hindu family in 1979 led to their subsequent mn- corporation into the sample in 1989. In other cases, govenrunent officials were asked where settlers meeting certain criteria might be found. Theni, in the village mentioned, a household would be picked in which members wer-e present and who met those criteria. Interudwi ntHrousehold Members To inteiew household members, the reseach team sinply arrived unamounced. In most cases, the interview then proceededl If it was dear that the time of arrival was inconvenient, then a specific time to return was arranged. To date, interviews have never been refused- another benefit of sequential visits. While the evaluators may be per- ceived at times as a nuisance, especialy by wealthier settlers involved in wany activities, after a number of years their presence was taken for granted. Indeed, in 1989, after a three-year absence, the research team * was warm ly welcomed in a majority of cases, not so much as friends but as influential outsiders who were familiar withethe difficult lives of set- tler households. Though the questions are written out, in English, interviewing is al- eways done in the vernacular. Flexiblrity is needed in working out the particulars of the interviewing process. Where research assistants do most of the. interviewing, a formal quesionnaire should be used with questions tranlated into the vernacular on the form. In the Mahaweli case, practicaly al inteiewing has been done in the presence of the * two senior investigatomrs Two procedures dominate. In one, the team Thayer Soider 37 splits up. While the Sri Larkan works alone, his expatriate colleague works with a research asistant In the other, the two work together, with one asking the questions in the vemacular and the other writing the answers in English. Although the second procedure takes longer, owing to the need for translation, it actively involves both team mem- bers in the research. This partnership has proveed to be synergistic, resulting in a more probing interview than would otherwise be the case. On occasion, the research team has also invited a colleague with special skills such as agricultural economics or regional planning, or with Tamil-language competence, to join them for a few days to conduct interviews. The time required for interviewing each household has gradually in- creased from about an hour to between two to three hours. Given the intensity of each interview, when travel time is included along with community interviews and interviews with officials, two interviews per day tend to be the maximum that is possible. In collecting data, the presence of a number of household members is preferred, but not visi- tors from other households. In cases where it is not tactful to ask visi- tors to leave, a household interview is changed to a community interew. While 60-90 minutes appear to be the optimal length for an interview, in only two cases did greater length require spliting the interview be- tween two or more visits. Both related to wealthier settlers, the com- plexity of whose affairs in relationship to other households required both more tact (due to an understandable unwillingness to discuss in- come) and more time, including evening visits. Over the years, to increase familiarity with household activities and conditions, an effort has been made to visit members of each household at their home, in their one-hectare allotment, and at any non-farm enter- prises. Visits to fields are especially important to help assess frequent complaints of settlers about the inadequacy of land preparation (espe- cially leveling) and delivery of irrigation wates Such visits also pro- vide an opportunity to discuss water management with a number of cultivators in the irrigation turnouts to which each household belongs. Starting in 1989, a decision was made to begin interviewing more widely in the kinship networks to which household members belong Previously, this had been done on an ad hoc basis. When a household head has died between visits, he or she is replaced with the deceased's nominated heir. Should that person be the head of a new household, information continues to be gathered about the whereabouts and activi- ties of other members of the original parental househoId. In other in- stances, where a household happened to be absent at the time of a visit, relatives livng nearby are interviewed. While this broadening of net- 38 hMoniring a Large-ScaleResettlemen t Program works has increased the number of people associated with each house- hold, prior to 1989, such interviewing was done more to maintain conti- nuity than to provide new types of information. That approach changed in 1989.. All Mahaweli households maintain ties of various sorts with people who live outside the project Especially in the case of evacuees and government-sponsored settlers who did not live in the Mahaweli sys- tems prior to the AMP, th-ese ties involve economic transfers of wealth to the various AMP systems and from those systems to outside persons and investments. The research team had been aware of such transfers for some time. However, the decision to expand interviewing in all cases -in which major sources of capital came from non-Mahaweli rela- tives (in the form of loans, ard remittances and other types of gifts), or where AMP settlers made major investments outside of the Mahaweli scheme, was made when one evacuee household moved back to the sending area in order to concentrate on business operations there. While the nearby household of his mother was substituted during 1989, in the future, the original household head (who has been interviewed since 1979) will also be interviewed in his new home. Such "network" interviewing is important, not just to better under- stand the production system of sample households, but also to better understand the flow of resources into and out of the Mahaweli systems. For, example, as households try to move beyond subsistence during the early years of settlement, resource flows from without-in terms of la- bor, loans, gifts, plow animals, and agricultural equipment-tend to pre- dominate. Such assistance may be essential for survival as a Mahaweli settler. In one case, a government-selected household experiencing fam- ily health problems that required them to mortgage their land would have lost that land without a major gift from a non-Mahaweli relative. On the other hand, as settler households improve their living stan- dards, they diversify their economic activities, not just witiin, but with- out the -settlement areas. This is especially the case with evacuees and selectees who may diversify into agricultural and non-farm activities in the area of origin where parents, brothers, sons, or other relatives still live. To better understand the nature of multiplier effects associated with such projects as the AMP, these resource flows began to be as- sessed for the first time in 1990 and 1991. Assisting Household Members During the first ten years of the study, visits brought no direct ben- efits to sample households other than prints of color photographs, which it was hoped would reduce the risk of informant "burnout." 7Thuyer Sacdder 39 Though benefits should be kept to the minimum to avoid introducing yet another bias, during 1989, the research team decided to help six families solve some immediate probIems. In three cases of extreme poverty and misfortune, households have been given enough capital to purchase the necessary farm inputs for the next cropping season or to cover the expenses of a child about to sit for the secondary school gradu- ation examinations. In three other instances, special problems (such as totally inadequate land preparation on the part of the government, or foot-dragging in providing homelots to married sons of settlers who want loans for widening the production base of their extended family) have been brought to the attention of senior MASL officials. Confidentiality Except in the few cases in which household members request that information be given to Mahaweli officials, the contents of all interviews are kept confidential While it probably is not possible to hide the identi- ties of households involved in repeated interviews, in this case, there is no evidence that AMP officials have tried to learn their identity, or if they have, that they have paid special attention to them. Other Types of Data By the time the formal interview has ended and the wealth index completed, more often than not neighbors and other visitors have ar- rived so that most interviews end with a "community" discussion of various issues of interest to the people and to the research team. "Com- munity" discussions are also held in irrigation tracts when household members are visited in their fields. Such interviews provide an oppor- tunity to check reference group information provided by sample house- holds, as well as to obtain other types of data, such as the proportion of households unable to repay seasonal loans and problems with land preparation, irrigation water delivery, schools, and health facilities. Table 2.4 provides an example of the data that could be interpreted through interviews. While in the field, household and community interviews are mixed with other data-collection activities. As with interviews with settler households, the approach used in such data collection is repeated inter- views with the same officials, the same business people, and the same researchers. Generally speaking the purpose of these other data is to enable the research team to elicit a wider range of opinion on settler perception of problems, to assess opportunities available to the entire settler population, and to assess constraints to the realization of those 40 Monitoring a Large-Sade Resettlement Program Table 2.4 loans to Mahaweli Settlers and Repayment Rates (SLRs) Bank of CW/on Hatton's Bank System B area System Hl- area (approximately (approximately 7,500families) 5,000families) Season Loans given Repayment Loans given Repayment Yala 1985 623 558 (90%) 861 91% Maha 1985/86 1,322 1,051 (80%) 2,040 92% Yala 1986 1,273 930 (73%) 798 91% Maha 1986/87 Z502 1,860 (74%) 1,130 95% Yala 1987 2,305 1350 (59%) 418 99% Maha 1987/88 Z249 1,044 (46%) 1,124 79% Yala 1988 1,076 647 (60%) 434 97% Maha 1988/89 943 628 (60%) 446 97% YaIa 1989 990 Collection still No under way cultivation Meaa 1989/90 No dkta yet No data yet 867 Source: Scudder and Vimaladharmna (1989, Appendix 4). opportunities. Under opportunities, the research team assesses the im- plications of special MASL and donor projects that are designed to di- versify income-earning possibilities for settlers and the children of settlers, and to increase-through water user associations, dairy societ- ies, and other producer organizations-the involvement of settlers in project management Under constraints, the researchers are particularly interested in insti- tutional weaknesses of the Mahaweli agencies, including poor relation- ships between settlers and officials, and in settler indebtedness, health, conflict, and a wide range of other factors (such as poor land prepara- tion, inadequate irrigation, increased weed growth in paddy fields, and disease in chillies) that interfere with the efforts of settlers to improve their living standards. Analysis Analysis of the impacts of the Accelerated Mahaweli Programme on the lives and livelihoods of several hundred thousand settlers is based in large part on the fime series data collected from sample hotseholds. Analysis of other project-related impacts and problems is based on complementary information gathered during sequential interviews with T7haer Scudder 42 govenmuent officials, with private sector bankers, equipment providers and other business people, and with researchers. Analysis of tLhe wealth index, for example, shows the extent to which living standards have improved, stayed the same, or deteriorated in regard to the criteria as- sessed. The formal interview provides a much wider range of informa- tion for comparative, longitudinal analysis. Repeated visits with Hatton's National Bank in H System, and with the Ford dealer in the same system, provide time series data on the proportion of settlers tak- ing out credit, on repayment rates (see Table 24), and on purchase of equipment Sequential visits to the same towns make it easier to assess the impact of settler purchasing power on the number and kinds of business enterpises, and on changes in their inventories. Because the same research team has been collaborating for over ten years, their fa- miliarity with the data has enabled them to draft a final report toward the end of each three- to seven-week survey within a iree-day period. Problems Arising from Methodology Skepticism among Government/Donors When evaluations in 1983, 1984, and 1985 conduded that the poten- tial of the AMP settlement component was not being realized due to a number of planning, implementation, management, and maintenance deficiencies, the initial reaction of government officials was to dispute their accuracy because of the nature of the methodology. At that time, the general impression in government and donor circles was that the AMP was achieving its goals; hence, any studies that reached opposite condlusions were received with skepticismL The fact that none of the condusions reached has subsequently been shown to be wrong has in- creased the acceptability of the two investigators' evaluations, although discomfort over the nature of the methodology remains. Ways of deal- ing with this problem are discussd in the last section of this case study. Intemcptions While repeated interviews are the most important component of the methodology, they are also the most difficult to ensure because of the risk of interruptions in, or tennination of, a long-tenr study. Interrup- tion or termination can oocur for a number of reasons, such as loss of interest by the investigators, outbreak of hostilities, or withdrawal of support by either the government or the funding agency. Though the Mahaweli research has benefited from the collaboration of the same investigators over a ten-year period, their withdrawal at 42 M;DnitoringaLarge-ScaleResefttentPrgram some point needs to be anticipated. One way to ensure a degree of continuity is to institutionalize the study within a local research institu- tion. To this end, another researcher has been periodically invited to joint the team from the governmnent-affiliated Agrarian Research and Training Institute (ARTI), a research institution that has the capacity to continue the study. Serious communal strife in Sri Lanka since the mid-1980s has been a constraint. For example, it kept Vimaladharma from re-establishing contact with a -majority of sample households for nearly a year following AI's agreement to renew fuding in August 1988. The researchers were able to re-interview mbembers of al 33 previously selected house- holds (or, in three cases, close kin or neighbors). Another 12 households were added in October-November 1989. As with communal stife, there is no way that the investigators can guard against withdrawal of government and donor support. In the case of the AMP, while individuals in the Mahaweli agencies disagreed at times with study conclusions, and while some officials resented any criticism of what was the country's Lagest and most important develop- ment progra, tern is no indication that govelrmaent officials requested AID to stop funding. When funds were stopped between 1986 and the end of 1988, this was due more to an in-house memo on the effective- ness of AID's role mtan to objections to the 1985 Mahawelh report. Time Constraints In planning evaluations to be based on repeated interviewing of the same households, it is coidal that enough time (and hence, enough funding) be allocated for data collection, management, and analysis. In Mahaweli, time constraints have been due largely to the busy schedules of the research team, resulting in insufficient time for data corection, and virtually no time for data management and analysis folowing completion of fieldwork and report writin, or preceding the next field trip. As a result both in the field and during report-writing more often than not they have had to work a 12-hour day, 7 days a week, and then, having submitted their report, de-emphasize any further data analysis. Dazta Management and Analysis Long-term studies presenrt serious data management and analysis problems as the amount of time series data piles up. In Mahaweli, the research team initially did not pay enough attention to such problems, with the result that their evaluations have relied too much on their Tha yr Scudder 43 memories of the histories of the sample households, complemented by checking those memories against past notes, interviews, and wealth cri- terna, as opposed to detailed analysis of the time series data itself. Though most household interviews through 1985 have been typed up, because of time constraints, the information is only now being organized to facilitate more systematic analysis and further data collec- tion, management, and analysis. Two efforts are under way. First, files are being prepared on each household that presents the data in a histori- cal format that covers, for each interview, information obtained on ten topics. During future interviews, laptop computers will be used to enter new information directly on the file. The ten topics are: (1) household history before incorporation within the AMP and updated details on household members; (2) activities, and income and expenditures relating to the one-hectare allotment, and to livestock, since the last interview; (3) water management, including the performance of water user associations at the turnout level of the irrigation system; (4) community-based organizations such as social welfare, religious, and more economically-oriented institutions; (5) description of the homelot, including housing and household fumishings; (6) other sources of in- come; (7) community dynamics (including information on such topics as numbers of community members who have draft and dairy animals and tractors; who have sought work in the Middle East, Singapore, or else- where outside Sri Lanka; who have corin,itted suicide; and so on); (8) problems; (9) plans; and (10) experiences with govenment institutions (especially Mahaweli agencies, schools, and medical I cilities). Second, data are being coded on 68 variables, by interview year, for computer analysis. Findings and Recommendations Findings The most im-. - att finding of the surv, iys, and the one that initially created the most a.ntroversy within the l4ahaweli agencies, was the 1983 conclusion tiat the potential of the AMP-in terms of major goals for the settlement component-was "not b 4ing realized" (Scudder and Vimaladharma 1983, p. 38). Although more detailed micro-studies deaI- ing with a portion of System H had po!...ed up specific problems at an earlier date (Lund 1978 am.. Siriwarc.ena 1981), the 1983 report was ap- parently the first to suggest serŽ.Lr deficiencies throughout the project. The major problem was that net income was insufficient to generate major multiplier effects in terms of enterprise development and employ- - -i44 Mining a Large-Scale Resettlement Progrnm meat generation. While production goals were being met in terms of paddy yields, for many settlers the income received was insufficent or barely sufficient to cover costs of production. The 1984 report emphasized ongoing poverty as the major AMP con- strainL That conclusion was based primarily on interviews in System H (the oldest Mahaweli settlement area) that "living standards of seven of our 16 sample households (44 percent) appear to have deteriorated, while those of another three famlies have stagnated at a poverty level" (Scudder and Vimaladharma 1984, p. 5). FPurther interviewing in all three AMP systems during 1985 led to the condusion that "the double- cropping of paddy of a 25-acre holding will not advance the large ma- jority of Mahaweli settler households beyond a subsistence level, even after the correction of land preparation and irrigation system defects." (Scudder and Vimaladharna 1985, p. 8). The failure of a paddy-dominated farming system to replace wage income lost when contractors moved on after completion of physical infrastructure was a major reason behind drops in living standards, as were governunent miscalculations concering the rapidity with which the "second-generation problems" would arise. This has hastened the illegal subdivision of holdings to accommodate children as they mar- ried, with those cultivating subdivided holdings more apt to emphasize paddy production for food security, as opposed to the cultivation of higher-value crops (Wanigaratne 1984). The 1985 report also empha- sized deteriorating relationships between settlers and AMP officials, due in part to increasing politicalization of the project lnis indluded ap pointment of lower-level Mahaweli officials as the presidents of what were supposed to be separate, settler-run water user and community development ab 7.ociations. As noted in the 1985 report, another finding dealt with what ap peared to be an exceptionally high suicide rate among evacuees and government-sponsored settlers. The research team suggested that this was due to insufficient attention paid to settler welfare, including insuf- ficient awareness of the inevitable stress that accompanies the coloniza- tion of new lands (suicide rates being lower among host-population settlers). As with others, this finding received support in a later, more systematic analysis in which the condusion stated, "The early phase of new Mahaweli settlemen'ts is characterized by an exceptionally high sui- cdde rate." (Silva and Kumara, no date, but incornrating data through 1987). -:- The-1985 report also noted that Tamil-speaking Hindu settlers were -- not being incorporated within the AMP according to their proportion in the national population, and that a significant proportion of host-popu- - lation, Tamil-speaking Hindus were being passed over in System B. 7Thayer Scudder 45 Contrary to stated govremment policy, this trend reinforced the belief among many Tamil-speakers and their supporters that the AMP was being used as a mechanism both to increase the proportion of Singala- speakers in an area in which Tamil-spealcers were in the majority and to displace some of the latter from their lands The research team stated a concern that a continuation of such actions would lead to increased communal strife at the expense of both Singala-speaking selectees and Tamil-speaking hosts. This indeed was the case in May 1987 and Febru- ary 1989. Recommendations Derived from the comments of settlers and individual Mahaweli offi- cials, as well as from the analysis of the research team, each report has included a broad of range suggestions and recommendations. Some focused more specifically on the production system, such as recommen- dations dealing with correction of land preparation and irrigation sys- tem defects, with weed growth, and with diseases of chilies. Others dealt more broadly with the settler population, including its social wel- fare, and with the composition of the settler population. Still others dealt with institutional and regional planning issues. During the 1983-89 period, report recommendations emphasized the need for diversification at the household and community level (with special emphasis on higher-value crops, livestock production, fisheries, and a range of non-farm activities) as the "best" way to raise living standards, to increase demand for locally-provided goods and services, and to increase non-farm enterprises and employment opportunities. A gradual handing over of management responsibilities to appropriate set- tler organizations was also emphasized. Such recommendations have contributed to the formulation and implementation of a number of ma- jor policy changes. AID's Scope of Work for the 1989 evaluation noted that the series of Mahaweli reports 'have been extensively used by MASL, USAID, and other donors in revising policies and projects re- lated to the AMP." Earlier reports emphasized the importance of upgrading existing towns rather than developin& at great financial cost, new towns mn the * center of the irrigation command areas that had been planned as if they existed in a vacuum. As for the balance between Singala- and Hindu- speaking settlers, the 1989 report induded two recommendations. The first was that "The Mahaweli family of agencies should stop settling poor people in zones where they are at risk because of conflicting land claims" between ethnic communities. The second was that the Mahaweli agencies and the donors should proceed at the earliest pos- 46 Monitoning a Large-Scale Resettlemnent Program sible time with the development of the right bank of Systemn B. Right bankc development was considered to be an important part of a national reconcilation effort cur-rently under way, since that was the area re- sewved for the settlement of Tamil-speakers. The research team empha- sized its development and its settlement by Tan'uil-speakers, since both the government and the donors appeared to be de-emphasizing their earlier commitment to proceed. Utility of the Methodology for Other Uses During the ten-year period during which this interview methodology has evolved, lessons have been learned that should improve its utility, for other users. Sampling While the investigators believe that a methodology based on repeated interviews with a relatively small number of households has proven merit as a major, low-cost evaluation technique, how the sample iLs stratified and proportioned between different strata should be done with more care than in the AMP case. Provided it is car-ried out in time, a pre-project benchmark study could provide valuable inform-ation for sample selection. Furthermore, use of key informants, reference groups, ?-nd community interviews should be mare systematically incorpor-ated within the methodology from the start. The key component of such a methodology, however, should continue to be repeated interviews with a small number of carefully-selected households, complemented by in- formation gathered from- and on the local leaders, reference groups, and commrunities with which these households identify. Data Collection Where repeated intterviews are carried out on relatively small num- bers of households, -the amount of data. collected can easily become ex- plosive. Though flexibility is crucial in order to add questions as new issues are identified and to drop -questions that were misconceived or are no longer relevant, researchers should attempt at the start to identify- "core" data to be collected. They should also decide which questions need to be asked of every household during every interview versus questions that need be answered by only a proportion of the house- holds. Quantitative data cannot be collected from every household simnply because of the time required. For example, to record in detail expenses -Thaycr Scudder 47 for double cropping in one year and income received requires well over an hour, leaving insufficient ti-me to deal with other questions. The AMP research team has tried to solve this problem by relying on a small number of informants (say 5-6) to provide detailed information on agri- cultural expenditlres, while other informants are asked to give a de- tailed accounting of expenditures on foodstuffs, clothes, school and household expenditures, and so on. If the information given is quite similar (as it was for costs of production) or can be easily fitted into two or three settler categories (as was the case with food consumption), then composite costs (and incomes) can be suggested, as in the 1989 report (Table 2.3). Data Management Managing time series data collected on small samples of 30 to 50 households is nonetheless a daunting task, the importance and duration of which was seriously underestimated by the research team. More than any other factor, poor data management has threatened the extraction of ful value from the research completed to date. Regardless of where their interests lie, evaluators who plan to utilize methodologies based on sequential interviews should work out thir data management systems before initiating the first set of interviews. If this had been done in the Mahaweli case, not only would analysis have improved, butmore com- prehensive data on a somewhat smaller set of variables would have been collected from the start to the benefit of both policy and academic analysis. * . Complementary Means for Checkdng Results An ongoing evaluation utilizing repeated interviews with a relatively small number of households should preferably be institutionalized out- side of the planning and implementing agencies. However, those agen- cies should also have their own monitoring capability, induding the *. ability to assess the conclusions and recommendations arising from re- peated household interviews (and complementary data collection), and to carry out special surveys as needed. Greater encouragement also should be given to universities, and especially to masters and doctoral candidates, to carry out low-cost, intensive research in prcject areas. If properly integrated (with degree candidates given financial incentives to research certain critical issues in more depth), these two types of investigation can complement each other in ways that offset each others' * :: X weaknesses. With the exception of an initial benchmark survey, large surveys based on random sampling are not recommended. This is be- 48 Moanl taring a Large-Scale ResettLemen t Progrmm cause repeat surveys are rarely, if ever, undertaken because of the time and money required, while several years usually pass before the results of the initial survey becomne available. References Johnston,4 Bruce, and Peter Kilby. 1975. Agricultural and Structural Transforma- tion: Economic Strategies in Late-Deveoping Countries. New York: Oxford University Press. Lund, R. 1978. "A Survey on Women's Working and Living Conditions in a Mahkweli Settlement Area With Specia Emphasis on H-ousehold Budgets and Household Surplus." Study Papers. Research Department, Peoples Bank Colombo, Sri Lanka. Mellor, John W. 1986. In J.P. Lewis and V. Kallab, eds., Development Strategies Reconsidered. New Brunswick, New Jersey:- Transaction Books for the Over- seas Development Council Scudder, Thayer. 1979. "Evaluatory Report on Mission to Sri Larnkan Settlement Areas: A Discussion of Some Basic Issues.' Evaluation report. USAID/ Colombox (photocopy) 1980. "The Accelerated Mlahaweli Programnme (AMP) and Dry Zone De- velopment: Some Aspects of Settlement"' Evaluation report USAID/ Colombo. (photocopy) _- 1981a. 'The Accelemrted Mahaweli Programme (AMP) and Dry Zone De- velopment. Some Aspects of Settlement Report Number Three.' IDA Working Paper No. 14. Binghamton., New York. Institute for Development * ~~Anthropology for USAID. - cause e 198 su. 'The Development Potential of New Lands Settlement in the Tropis and Subtropics: A Global State-of-the-Art Evaluation with Special Emphasis on Policy Implications." Binghamton, New York Institute for De- velopnient Anthropology for USAID. 1984. "The Development Potential of New Lands Settlement in the Trap- ics and Subtropics: A Global State-of-the-Art Evaluation with Special oem- phasis on Policy Implications. (executive summary). US/AID Program Evaluation Discussion Paper No. 21. Washingtona, DC.: U.S. Agency for ntemrational Developmernt Ruddr,1985. "A Sociological Framework for the Analysis of Land Settlementsf In Michael L Cernea, ed., Puffing People Firs Sodiological Variables in Rural Development. New Yorkc Oxford Univer-sity Press for the World Bank, pp.. 121-15. Scudder, Thayer, and KCapila P. Vimaladharma. 1984. "The Accelerated Mlahaweli Programmene (AM) and Dry Zone Development: Report Number Four-" MA Working Paper No. 23. Binghamton, New Yorkc Institute for Development Anthropology for USAID. 1- 985. "The Accelerated Mahaweli Programme (AMP) and Dry Zone De- velopment:S Report Number Five.. WA Working Paper No. 24. Binghamton, New York: Institute for Development Anthropology for USAID. Th ayer Scudder 49 * 1986. "The Accelerated Mahaweli Programme (AMP) and Dry Zone De- velopment: Report Number Six.' IDA Working Paper No. 25. Binghamton, New Yorlc Institute for Development Anthropology for ISAID. .1989. "The Accelerated Mahaweli Programme (AMP) and Dry Zone De- velopment: Report Number Seven!' Binghamton, New York: Institute for Development Anthropology for USAID. Silva, K Tudor, and W.D.N.R. Pushpa Kumara. No date. 'Suicide and Sexual Anomie in a New Settlement in Sri Lank In Padmasiri de Silva, ed., Suidde in Sri Lanka. Proceedings.of a workshop held at the Institute of Fundamental Studies, ICandy, Sri Lanka, pp. 5164. Wanigaratne, Rangit Dissanayake. 1984. 'Subsistence Maintenance and Agricul- tural Transformation on the Frontier in, Sri Lanka: The Kaltota Irrigated Settlement Project." Unpublished PhD. dissertation. University of Wiscon- sin, Madison, WiLsconsin. 50 Monitoring a Large-Scale Resettlenent Program Appendix: Annual Evaluation of the Settlement Component of the Accelerated Mahaweli Programme 1989 INTERVIEW FORM FOR OLD SETILER HOUSEHOLDS Name of Interviewer Date of Interview Name of Settler Location/Village Name Block System Adult Members of Household Present at Interview SOCIAL CHANGE AND CON1INUITY 1. Changes in household structure since 1985 interview (births, including those of married children; deaths; marriages; divorces; and in- and out- movement of people-marrying out of children, for example, or marrying in of childrens' spouses, or arrival or departure of other relatives. Note. where new sons-in-law have moved in, note background, in terms of place of origin and land status). LAND AND YIELDS 2. Changes in access to land since 1985 interview (including learing of new land; leasing in or out of land, and reasons involved, sharing of land with children or son-in-laws; involvement in bethna, etc.) 3. Changes in character of allotment since 1985 (including further land preparatior, changes for the better or the worse in the water conveyance system; weed growth; inadequacy of water supplies; waterlogging; etc.) 4. Yields since 1984/85 Maha- Other Crops Paddy in bushels Chillies (Bombay onions, etc.) - 1985 Yala 1985/86 Maha 1986 Yala 1986/87 Maha 1987 Yala 1987/88 Mha- 1988 Yala 5. Changes in the character of the homelot (coconuts now fruiting; status of other perennial and annual crops) 7uayer Scudder 5.1 6. Current ownership of livestock and type of ownership Origin (purchase; ande; birth ifpurd.aes, Number date and price) Buffalo. Draft cattle Dairy cattle 7. Credit dependence Date and amount of last bank loan Amount and type of indebtedness (bank; relative or friend; mudalali; etc) INCOME AND EXPENSES FOR AGRICULTURE & Costs of production for main allotment (inputs; hired labor, hired equip- mentr etc) 1988 Ya2a (or 1987 Yala 1987/88 Maha if 2988 harvest not complete) Seed Land preparadon Transplanting Weeding Harvesting Transport Fertlizer Herbicides Insecticides Other expenses (-3 Total - Note: Type of "equipment" should be noted under land preparatiorn buffalo; reat cattle; two-wheel tractor; etc. Labor costs should note number of man-days involved; wages (induding cash, food, or other compensation); and whether type of labor is daily, seasonal, or permanent 9. Costs of production for crops grown for sale on homelot 10. Income from main allotment .: D (bushels/kilos sold; price per kg; and incomes from sales) 52 Monitoring a LarauScle Rsettlment P?ogrwn 11. Net mcome from main allotnent 12. Income (if any) from the homelot during last 12 months 13. Sale of livestoc during last 12 months 14. Income fram livestock (hiring out dairy; etc.) during last 12 mwnths (if recollection poor try at least to record estimated income during an average week or month) 15. Income from wage libor during last 12 montkv (inwluding wage labor of resident children and in-laws) 16. Income from special skills such as carpentry, masonry, etc. during last 12 months (if recollection poor, tby at least to record esimated income durig an average week or month) 17. Income from boutiques and other comiercial ventures such as dairying or fish ponds during last 12 months (agair, try at least to obtain estimate for an average week or month) 18. Other souzces of income OTHER EXPENSES' 19. If paddy supplies exhausted, when did the family begin purchasing paddy? Costs per week? 20. Costs per week for fish (# of purciases per week and price) and other protein 21. Costs per week for bread and other carbohydrates (other than paddy) 22. Costs per week for Vegetables/fruits/nuts Mlk/cheese/yoghurt/curd Tea, sugar, etc. Cuny ingredients Cooking oil Cigarettes, betel liquor Soft drinks Soap and detergents Fuel 23. Estimated expenses forclothes during pastl2months Thayer Scudder 53 24. Estimated expenses for minor household purchases during the past month (major purchases listed separately under PROPERTY category) 25. Travel expenses during past 12 months, including home visitationt, travel for health needs, and pilgrimage 26. Medical expenses during past 12 monthis (or last major medical expenses if canet sunarize) 27. Recreational expenses (cinema, etc.) during the past month 28. Special occasion expenses (weddings, funerals, etc.) over past 12 months 29. Other (gits, un-repaid loans, etc.) PROPERTY AND RESOURCES 30. Major property obtained since 1985 interview (tractor, trailer, juamp, TV, sewing machine, and other major household furnishings; etc.) Type of Item Date Purchased P-rice 31. Status of major property acquired before 1985 (especially as to wlhether tractors, pumps, TVs, etc., are still operational) 32. Changes in housing since 1985 interview (note type of improvem-ents or deterioration) HEALTH AND SANITATION 33. Changes in toilet facilities since 1985 interview (note type of improvement or deterioration) 34. Changes in domestic water supply since 1985 intervew (note type of improvement or deterioration) 35. Incidence of malaria among household members during past 12 months 36. Hospitalization among household members during past 12 months 37. Conflicts with fam-ily members, relatives, and neighbors since 1985 interview and status of previous conflicts (including suspicions of sorcery) 38. Other health problems 54. Monitoring a Large-Scale Resettlement Program TRAVEL 39. Date of last "home visit 40. Number of "home visits" during the last 12 months 41. Date and location of last pilgrimage TURNOUT AND OTHER COMMUNITY ORGANIZATIONS 42. Status of turnut group (including its functions and effectiveness; whether or not turnout leader is a farmer or unit manager; extent of conLflicts over water delivery; maintenance; and turnout heterogeneity, especially where members come from different electorates, etc.) 43. Status of hamnlet association 44.' Status of temple, welfare, and other religious arid social organizations MBA AND MASL RELATIONSMIPS 45. Participation in MEA arid other Mahaweli courses 46. Relationtship with and assessment of unit manager and othier Mahaweli officials OPEN-ENDED QUESTIONS 47. How has your situation changed since 1985 interviews? 48. How has your household benefited from Mahaweli? 49. What are your major problems at this time? 50. What are your plans for the future? 51. In what ways has Mahaweli met your expectations at the time you first became a settler? 52. In what ways has Mahaweli failed to meet those expectations? COMMUNiTY CHARACTERISTICS (State here what the-nature of the community is. For example, 59 Bowatanne people, 100 Kotmale evacuees, 120 selecbees from Kcandy, 12 turnout miembers, etcj.- 53. Number of community households with two-wheel tractors -zayer Scudder 55 54. Number of conmunity households with boutiques 55. Number of community households benefiting from special Mahaweli programs (here specify number from dairying, fish ponds, special projects, etc.) 56. Number (proportion) of households that have mortgaged/leased out (a) part of their land (and reasons for such mortgaging/leasing) (b) all of their liand (and reasons for such mortgaging/leasing) 57. Number of households that have returned to area of origin 58. Number of households from which family members have gone. (a) to the Middle East (b) taken jobs that require them to reside outside the communuity 59. Number of suicides within the community since 1985 interview 60. Respondent's assessment of the proportion of community mnembers who have raised their living standards sinoe 1985 interview; who have 'stood still"; and who have "gone downhill.' Use of Group Interviews in Evaluating an Area Development Project Krishna Kmnar In this dia pter,. Krishna KCumar illustrates the use of communityl group in terviews for the evaluation of an area development project in Mulazvi. The case study outlines the planning and conduct of group interoiews and gives examples of the type of info rmation and data that were generated by them. Finally, it presents afew lessons derived from this effort about the conduct of group interviews in developmtent settings. In this project, farners' clubs were being used as the sole me- dium to provide short-term loans to farmnir for purchasing recom- mended agricultural inputs. Under the leadership of the author, the * evaluation team undertook group interviews with the members of * eight farners' clubs. The selection of these clubs took into consider- ation their size, age, accessibility, and djffrent ecological zone. The interviews took the formn of public meetings in which the team miemibers askedf questions on the basis of a carefullyprepared inter- view guide. The team made every effort to ensure the partidipation of al those present in the meeting. As the chapter indicates, the team was able to gather considerable information through. group interviews, which, with the data generated by the review of project records and key informant interviews, formed the basis for the evaluation. Kumar also identifies'a few problems zwith his methodology that might have undermined the validity and reliability of the findings. First, although every effort was made to solicit advice from a num- ber of experts about the selection of the dlubs for group interviews, it was quite possi'ble that the selection process was biased, in that the project management and government officials recommended dubs that had performed well. Second, the club members were generally reluctant to reveal any information tha seemed critical of the government and project authorities. Finally, there was a ten- 56 Krishna Kumar 57 dency among club leaders to dominate discussions. On the basis of his experience in Malawi, the author suggests several measures that can help future investigators in generating relevant and objective data through community/group inteknews. THE DOWA WEST AREA DEVELOPMENT PROJECT was one of the earliest initiatives implemented under a national rural development pro- gram introduced by the Malawi Government. It was designed and implemented :-,s an integrated rural development project with two key objectives: (1) to increase the incomes of 9,300 smallholder fanners through the sale of an increased surplus of cash crops and an incremen- tal increase in livestock production; and (2) to improve the living condi- tions of these farners and their families through expanded and better water supplies, health facilities, community development services, and energy conservation. The project was funded by the International Fund for Agricultural Development and supervised by the World Bank, and began function- ing in late 1981 under the auspices of the MaIawi Ministry of Agricul- ture. It focused on provision of agricultural credit, extension services, research on crop storage and milling, storage capacity, roads, drinldng water, and health services The project established precise targets in each of these areas, to be accomplished over a six-year period. A midterm evaluation of the project was conducted in 1984 by a three-member team led by the author. The team remained in the field for three weeks of field research and spent another two weeks preparing the draft evaluation. One of the several substantive areas that the evaluation team studied was the growth and contribution of fanners' clubs to agricultural devel- opment. The farmers' dub movement started in Malawi in 1974, five years before the Dowa West project was formulated (1979). The country's extension service pmmoted farmers' clubs to encourage cooperation in agriculral operations, particularly in strip cropping and the purchase of agricultural inputs in bulk quantities. At least 36 clubs were function- ing in the project area by the end of 1979. However, the real momentum for their growth came from the project itself, as is evident from Table 3.1. A dub usually encompassed one or more villages, depending on the size of the local population. Although the club sizes ranged from 10 to 100 members, most had a membership of between 20 to 40. Nearly a quarter of their members were women. The Dowa West project relied exclusively on the cIubs for the deliv- ery of seasonal agricultural credit for the purchase of seeds, fertilizers, 58 Use of Group Inferujews in Evaluating an Area Development Project Table 3.1. Cumulative Numbers and Membership of Farmers' Clubs, Dowa West Arel 19814 = NO.of No.of No. of Year armers' dubs male members fetmae members 1981-82 61 1,770 501 1982-83 265 8,090 904 19834-8 403 11,279 2,268 anLd other inputs. Project records indicated that they were especially * ~Succsflin this regard; the amount of credit channeled through them * ~to the target populations inareased seven-fold within two years. This exceeded the optimistic projections included in the project appraisal re- port In addition, the clubs also provided midterm credit for farm equipment suich as ridgers, plows, ox-carts, and oxen. At the time of the midterm evaluation, extension work was also ularely organized around the dubs. Club members regulairly attended extension meetings, demonstrations, and training courses, and they also cooperated in agricultural operations. Research Methodology The basic research method used by the evaluation team was group inter- views with farmers' dub members, invoiving the use of direct probing techniques to gather information from several individuals in a group session. TIhe evaluation team had only one week to examine the workings of the clubs and assess their project contribution. This time constraint was further compounded by logistical problems; the dlubs were scattered all over the project area and were not always accessible by automobile. Also, more time was needed than was available to alert dubs about the evaluation team's pimpending visit, so that their memberts would be available to be interviewed. After a careful review of the available documentatione and in-depth inerviews with key project staff, govemrment officials, and ther ex- perts, the team decided to conduct some initial group intewrews with the members of a few clubs to gain first-hand information about their activities, achievements, and problems. In this particular case, grup interviews took the form of a meeting of the dub members in which -questions on relevant topics were posed and answers received. Ktihna Kumar 59 Planning of Group Interviews The evaluation team recognized the need for careful planning at the outset, and developed an interview guide, a protocol for team members, and a list of the clubs to be visited initially. The Interview Guide The evaluation team decided that the interviews should focus on a limited range of topics that could easily be answered in a group setting. .- - With fewer questions, the interviewers would have more time to pursue leads in-depth before moving on to other items. Moreover, a shorter agenda would enable the participants to express themselves freely and in a relaxed atmosphere. While there was a general agreement on this issue, the evaluation team members differed on what subject areas the guide shou;d cover. After considerable discussion, agreement was eventually reached on the topics to be explored. The evaluation team also debated whether the interview guide should list questions or just the topics. The advantage of merely listing topics is that interviewers are able to frame questions spontaneously, thereby "tailoring' them by following the flow of discussion and consid- ering the general background of the participants. On the other hand, predetennined questions keep the discussions focused. They also re- duce the risk that the participants will not understand what is asked, which can occur when the questions are framed on the spur of the moment. After considemble discussion, the team decided to list the questions in the interview guide, but not to restrict team members to their use. They agreed that the interviewers should feel free to add questions that arose naturally during the discussion or to change the sequence of questions. For example, if a dub official made an interest- - -hing point while introducing the evaluation team, the teamr. member could use the comment as a starting point for the interview. Some excerpts from the interview guide are shown in Box 3.1. Interview Protocol The evaluation team also developed a.n interview protocol by fonnu- lating the following three ground rales to gde its members during the interview. These ground rules proved extremely useful in ensuring the interviews proceeded smoothly. First, the team leader was to introduce all members, explain the objec- tives of the evaluation, and ask questions about the origin and organiza- tion of the dub. He was allotted 20 to 25 minutes for this purpose. The 60 Use of Group Interviews in Evaluating an Area Developmenzt Project Box 3.1 'Excerpts from the Interview Guide for Dowa West Farmers' Club Memkbers 1. Origin and Initiative * Why did you join the dub? * Who organized the first meeting? *Did you know other members well before joining the dlub? *Did you receive any help or assi-stance from extentsion workers or other government officials in organizing it? S How did you frame the rules and regulations? 2. Organization and Activites * What is the toya number of members in the dlub? * Do you have women members who are the heads of households? If so, how m-any? * Do you still welcome new members? * Have members dropped fiiast of the dlub? If so how many and why? * Some people tell us that if credit is not available, many members will lose interest in the club. What is your assessment? * How many times does your club meet in a month? Z How many persons attend these meetings? 3. Credit and Supply of Inputs * Do you get the amount of credit you generaly need from the club each year? If not, why not? * How many of you present would like to have a larger amount of seasonal credit? * Do you have problems in obtaining credit vouchers on time? If so, what are they? a If you do not get loans from the cub, what other sources of credit are available to you? * Did the ub haoe difficulty in getting repayments from the mem- bers in the pnst? If so why? a What do you do to persuade all members to make the prompt repayments? .L Benefit Assessment * How many of you also purchikse fertlizers and other inputs for cash? How many of you were using fertizers before joiing the dlub? * Do you think that the, availability of inputs through the dlub has contributed to an increase in your uaize production? *Do you think that your income has increased as a result of partici- paton in the dub? Krishna Kumar 61 other two members were given 15 to 20 minutes each to ask questions on credit and input supply and impact assessment. After all the team member-s had covered the items listed in the interview guide, they had 20 minutes to pursue unlisted issues. Second, all members were to take elaborate notes to be compared and developed the same evening. Third, team members were not to interrupt one another and were to resist the temptation to help out a colleague by hying to interpret a participant's response. Such an action was permifssible only when a team member misunderstood the respondent. Club Selection Process The evaluation team orighinally planned to hold group interviews with ten clubs. However, because of logistical and time constraints, it completed only eight interviews. To select these clubs, the team consulted several well-informed per- sons, including the project staff, and requested that each of them list 15 dubs for group interviews. These persons suggested about 30 names, from Which. 10 were selected. This took into consideration fac- tars such as size, ecological zones in which clubs were located, year of origin, and accessibility. On the whole, the selection process was not entirely satisfactory be- cause of inherent bias. It is likely that the project management and gov- ernmuent officials suggested the dlubs that had good performance records. In addition, as mentioned above, the clubs located in remote areas were generally excluded because of transportation problems. Also, the team had to be content with a very small sample. Conducting Group Interviews The duration of an intervew ranged between 1 1/2 and 2 hours. Partici- pation levels were very good. An overwhelming number of members attended each of the meetings, anid in two cases every member of the dlub 'was present. In no instance were more than three members abs-ent out of memberships ranging in size from 25 to 30. (Obviously, since all members of a dlub resided either in the same village or adjoining vil- lages, it was not inconvenient for them to attend.) Initial Introductions Because the dub members were not accustomed to interacting with -people from different cultures, the. evaluation team members tried to 62 Use of Group Interiews in EvaIuating Area Development Project - : put them at ease by initiating individual conversations prior to the start of the interview. The team members also introduced themselves as - interested individuals, and not as representatives of an intematonal organization. They gave their country of origin and family back- grounds, and mentioned their earlier visit, to Malawi Where possible, the group sat in a semi-circe to keep the atmosphere informal. At the beginning of a meeting the dub chairman/secretary wel- comned the team. The team leader then introduced his colleagues and stressed that the purpose of their visit was to lean and to know the views, opinions, and suggestions of the cdub members. He stated that they would like to hear from each person present, and encouraged any- one with a divergent view or thought to express it freely. He also mentioned that team members would be delighted to answer any ques- tions. Probing Techniques tsed A question-and-answer session began after the team leader made his introductory zemarks. hn most instances, the evaluation team started with questions about the size of the dub, the year of its inception, and its present activities. Because of their simplicity and non-controversial nature, such questions helped to establish an initial rapport between the evaluation team and the club members. The members easily answered them and felt a bit more confident and better prepared to answer more difficult questions. The evaluation team generally asked most of the questions listed in the interview guide. However, the sequence of the questions changed depending on the flow of the conversation. Moreover, team members asked many additional questions not listed in the guide. In the interests of obtaining more useful information, the team mem- bers tried to adopt, though not always successfully, a posture of "so- phisticated naivetV' by conveying tie impression that while they knew about famers' clubs, they lacked an in-depth knowledge and under- standing of them. Using this approach, team memnbers were often able to obtain more details through remarks such as "You know that I am not verr familiar with farminsg practices in this part of the country, so you will have to tell me the agricultural activitics in which you have helped each other." For probing purposes. queries based on 'what, "when,' "where, "which," and "how" were used frequently. The evaluation team tried to seek as many specific details as possible. For example, one issue of prime interest to the evaluation team was the 100-percent repayment rate for seasonal credit the team members usually asked several ques- Krishna Kumar i'3 dions on that topic, for example: "Do you know any mnember who was in financial difficulty and therefore could not make the repayment in time?' "What will people do if a member is unable to repay the loan?"lo "Was there any time. when, in your dlub or elsewhere, people had to exert pressure on a member to repay?" However, team members were careful not to make the participants feel that they were being cross- examn ed. To this end, they asked probing questions in a highly infor- mal way. Obtaining Balanced Participation One problem that the evaluation team constantly faced was that a few dub mnembers often tried to dominate the session. Such persons tended to be club officers, vilage or community leaders, and in some cases, party functionLaries. ConstrainLing them required considerable tact and interpersonal skills on the part of the evaluation team. The team used two strategies to ensure that participants contributed equally to the discussions. First, team nmenmbers addressed questions to individual participants. For example, a team member would turn to a reticent participant and say, 'I would very m-uch like to hear what you have to say about this Issue." Such a remark often succeeded in evokcing some response from the participant. In many instances, the same ques- tion was asked of several persons. For example, the question, "Why did you join. the club?' was typically addressed to many participants in a meeting. Second, tlhe evaluation team took polls on selected questions. For example patcpants were asked, 'Those of you who did-receive the amount of. seasonal credit you really needed this year, please raise hands." This strategy enabled the team to get an indication of the opin- ions or experiences of participants, even when they were reluctant to speakc However, the evaluation team was not entirely successful in obtain- ing the participation of women in all of the group interviews. Because of their low social status and prevailing socio-cultural barriers, many women pre-ferred to remain silent spectators. One strategy that partially ~worked in some situations was a judicious sense of humor, samples of which are show-n in Box 3.2 ContWalling Group Pressure The evaluation team was also concerned about group pressures that might inhibit dissenting participants from expressing their views on a particular subjecL There were severa\ reasons for such a concer. In Malawi society, great value is attached to consensus. People are not 64 ULse of Group Interviewus in Evaluating an Area Development Project Box 3.2 Encouraging Women to Participate in Group Interviews To encourage the participation of women in the group interview, the team leader included the following in his introductory remarks: When I was comidng here, my boss called me and told me that he was interested in knowing the views of all the people in a meeting; he would not make his decisions on the basis,of the opinions of a few individuals. InL fact, to tell you the truth, he promised me -a raise if I succeeded; othierwise he might evenx fire me. So.please pr-omise me that all of you will participate in discussions. If you. don't, you will have to give me a piece of land so that I can join you. (Loud laughter) The team leader then turned to women participants, who usually sat sepa- rately, and adc.d: But this is not the ortly problem thatlIhave. My wife has heard a lot about you and your participation in farmers' clubs. She wants to know more about what you have been doing, what your experi- ences have been, and what can be done to improve your participa- foiL If my boss getsangy,hecanonlylre meaut ifmy wife gets upset, I might be ikn greater trouble.. These remiarks gave the team leader an excuse to humromusly probe the participants. Whenever some people were not participating, he would sinply say, "Oh, my hiends, you seem to be forgetfing my problemna Club members would laugh ant respond to his questions. supposed to differ, at least in public, with their elders, leaders, gover- ment officials, or others holding higher status. Moreover, dub members were sib'sistence farmers who were not ve articulate. Thus, it was reasonable to anticipate that they would agree to a view expressed by another person simply to extricate themselves from the uncomfortable position of answering a question in public. To minimize sucgh group pressure, the evaluation team took the fol- lowing thiree steps. First, whenever the team members saw that an idea was being gener- ally adopted without any discussion or disagreement, they would spe- ifically ask for other ideas, explanations, or recommendations. For example, in several cases, the initial responise of the participants to the question, 'Why did you join the club?" was that they wanted to cooper- ate with one another. Confronted with this reply, the team members pressed-for additional reasons, and oflten the participants then said that they also wanted credit to purchase agriculturl inputs. Pressed further, same of them said that extension workers had explained the advantages Kris)mhlKumar 65 of forming a dlub and indlicated that, as dub members, farmers could easily gain access to extension services. Second, the team members would pursue one response with mention of another idea or explanation. For example, in response to the question concer:ning why allcdub members regularly repaid thieir loans, many of the respondents would say that it was in their own interest to do so; otherwise, they would not get credit in the future. The team members typical would then ask, "Do you think that the defauIters would be concerned about their relationship with other members?" (In the project area, a dub is responsible for all the credit advanced to its members, and if one mnember defaults, the entire dlub is barred from further aredit.) In many cases, respondents would answer afArmatively and then go on to explain the social ostracism that a defaulting member would encounter in the local community, Finally, team members tried to observe the non-verbal behavior of the participants. If a member felt that some persons looked skeptical or doubtful, he would look at them, and say something like, "What about you? You might have a different view.' Such remarks often encour- aged participants to express dissenting views. Recor-ding Group Interviews All team members took notes during the interviews, which they then compared at the end of the day. This helped them to fill in the hiforma- tion gaps and avoid potential sources of error. The evaluation team then prepared a short summary of each group interview, outlining the information obtained, the team's impressions, and implications of the information for the evaluation. Such summaries prved very useful during the preparation of the evaluation report One un-itended advantage of the preparation of the interview sum- maries was that this exercise provided useful insights and ideas that could be explored in future inteviews. For example, in the first group interview, the evaluation team learned that the extension staff had helped in drafting the constitution of the dub. The team pursued this lead in the remaining interviews and found that this was the accepted practice. MajoIr Findings and Conclusions As indicated earlier, group interviews were not the only sources of data for the evaluation. The team members also reviewed the available itera- ture and conducted in-depth interviews with several key infomnants. Moreover, Mtey had access to the quarterly reports of clubs, which were often prepared with considerable precision and care It is therefore 66 Use of Grvup Intervews in Evaluating an Anna Development Project difficult to isolate the information and insights gained through group intervews from those from other sources of data. However, there is little doubt that the group interviews were extrem ely valuable in gener- ating tlhe following information. Profiles of alub Membership The group intervews revealed that practically all of the dlub mem- ben were smaliholders. An overwhelm-ing majority of them owned and cultivated between 2 to 3 hectare of land. Rarely did the team fminda member who owned and cultivated more than 4 hectares. An earlier study had indicated that very tiny smallholders, that is, those cultivating less than 1 hectare, were underrepresented in the dubs. Although suc-h snmaflholders contstituted 36.1 percent of the farm- in oulation, they represented only 11.2 percent of dlub memberships. Group interviews, however, provided a plausible explanation for suc-h low representation, revealing that a high proportion of these smafiholders worked on estates or took employment in nearby towns, and were therefore only part-time farmers. Moreover, some of them were economically or physically too handicapped to. work effectively in the dlubs. In either case, they often did not meet the membership criteria. Club Leadership The group interviews revealed that dlub leadership was generally provided by the firmers who had been exposed to outside influences. These included farmers who bad previously worked in mines or on estates, literate individuals, and party functionaries. (Only one legal political party existed in the country at the time this project was under- taken.) The village headman was invariably a prominent member. On the whole, dub leadership represented a blending of traditional as well as modern authority structures, which contributed to its legitimnacy in public eyes. Al the senior officals of the dubs were men. Distribution of Credit TIhe most important function of the clubs was to provide credit to their memkbers for agricultural inputs. Even the non-funded dlubs'stir- vived in the hope that they, too, would ultimately receive credit (Many dubs had remainedl unfunded because of the shortage of funds at the - natieonal level. The evaluation team found that the curnent momentum Krishna Kumar 67 for the growth of dlubs miight be'arrested unless additional funds were made available for seasonal loans.) Credit was provided in kind and not cash. Seasonal credit was meant for the purchase of seeds, ferilfizers, and other inputs, which were sup- plied by a- parastatal organization. Medium-term credit was meant for fann imnplemients such as rid gers, ploughs, ox-carts, and oxen. - Available documentation suggested that the clubs had a policy of distributing seasonal credit equally among their mnembers, inrespective of the size of their holdings. However, interviews indicated that thiis policy was not uniformldy implemented and that members could sometimes mLake informal arrangements with one another to get aroumd it. Group interviews also rbevealed that the amount of credit advanced to dlubs was not sufficient to meet members' requirements. on average, members received enough credit to buy only a single package for maize, even though they needed two. AtricultuT4l Cooperation The goverument viewed agcultural cooperation as the prime justifi- cation for establishing the dubs. Al the members of a club were sup- posed to engage in group farming, or at least cooperate with one another in agricultural operations. However, group interviews indi- cated that despite the stated policy, only limnted cooperation existed among the dubs' members. The examples of cooperation most often cited incuded bringing inputs from the parastatal organization, assist- ing in soil preparation, and helping iembers during ilness or other emergencies. A few dubs owned common plots that were jointly culti- vated by their members. Role ofAgricultuna Extension The role of extension staff has been most crucial in the formation and functioning of the dubs. The field assistants (grassroots extension work- ers) have worked very closely with farmers to establish new clubs and assise2d them in fra uing appropriate rules and procedures. Once a lub is formed, field assistants have provided assistance to maintain accounts and obtain credit They have also supervised the supply and use of inputs by the members. The field assistants have tried to make the dlubs a focal point for extension activities by establishing 'demonstration garden' and orgai- nizing meetings and discussions. Attendance at such meetings was vir- tually mandatory. Group interviews revealed that dub members were 68 Use of Group Interviews in Evaluating an Area Development Project not very enthusiastic about extension services, in part because the exten- sion staff focused primarily on maize. Once the members had used the recommended technical package or improvised on it, they had nothing to gain from attending more meetings or visiting the demonstration farms. Repayment of Credit The most remarkable achievement of the cdubs was the 100-percent loan recovery rate. An obvious reason for this was that loan recovery was the collective responsibility of the dlub; czedit was given to a dub and not to individual members. Since a defaulting dub was not eligible for further credit, all members had a vested interest in assuring that the loans were repaid an time. The interviews revealed the intense pressure brought to bear by the clubs on potential defaulters. In one case, a member who delayed his repayment was visited by dub memlers and virtually coerced into mak- ing instant payment. Such group pressure undoubtedly works because the members belong to closely-knit communities. OveraIf Adchievment The group interviews lent some credence to the project nmanagements assertion that the clubs had contributed to increased agri- cultural production. In all the clubs, participants stated that because of their involvement their maize production had increased and their stan- dard of living 1. .d improved. Three reasons were typically given for this. First, there was a universal shortage of credit in the project area, and without the dubs it would not hAve been possible for most of the members to raise resources for the purchase of needed inputs. Second, as a result of their participation in the clubs, members also received significant extension advice. However, as previously noted, several members indicated that the extension workers focused primarily on maize, and in many cases had noffiing to add to what farmers already had learned. Third, club membership contributed to a continual pres- sure to improve agricultural production and productivity. Lessons in Conducting Group EIterviews The project team learned several lessons about planning and conducting group intenriews, and these are described below. Krishna Kumar 69 Preparation of a Structured Interview Guide As mentioned earlier, the evaluation team discussed whether the in- te-rview guide should simply list issues and topics aloine or provide pre- cise questions, and eventually opted for the latter. However, the team members argued that interviewers should enjoy. considerable freedom to explore issues not listed in the gide. Following this course during the interviews proved eminently practi- caL One advantage of this was that posing the same questions in all the interviews generated comparable data, fa&ilitating the eventual analy- sis. Another important advantage was that the structured interview guide saved time, because the interviewers did not have to construct questions on the spur of the moment. After one or two group inter- views, teamn members had almost nemorized the questionnaire, and their questioning looked quite spontaneous. The flexibility the interview guide provided enabled the team members to explore new issues and ideas that had not come to mind at the time of its preparationL Thus, the list of questions was constantly being revised. For example, the team had not initially realized the im- portant role of field assistanis in framng the dubs' constitutions and providing overall supervision and gulidance. Once this fact was re- vealed, the team members began posing questions about it in subse- quent interviews. To sumn up, it is suggested that the best interview guide design lists specific questions for group interviews, but allows the interviewers con- siderable flexibility. Determining Optimum Group Size In starting out, the evaluation team paid little attention to the issue of group size, and assumed without making inquiries that no more than 60 to 70 percent of dub members would attend the interview sessions. As indicated earlier, this arsi!mption proved to be wrong. Thus, the size of group meeting ranged between 25 to 31 persons, although one meeting * ~ was composed of more thin 50 people. However, the evaluation team - : found that it was difficult to effectively manage a group interview with more than 30 individuals. : : . It is recommended that investigators carefully consider the probable size of groups to be interviewed and have contingency plans. For ex- ample, if the number of participants is over 30, the group should be divided into two or more subgroups. 70 Use of Group Interoiewos in Evaluating an Area Development Project Geeting Quantitative Data The evaluation team also found that group interviews can also gener- ate quantitative data or intformation. Although such data, being purely nominal, have obN laus limitations in ternis of possible analysis, they can still be of value for rational debcision-making. On several ispues-such as the number of seasonal loans provided by the club, the effects of club participation on agricultural productio, -and the problems facing the members-the evaluation team took polls and thus gathered some quantitative data. The data generated by such polls can be aggregated and analyzed either by treating individual respondents as separ-ate cases, or treating each group as a separate case: Effects of Group Inhibitions on Veralcity of InfoAration it is very important that interviewers not take al respondents' com- ments in group meetings at face value. Many people, particularly in totalitarian societies, feel constraifned in expressing themselves freely particularly in goups in which individuals in positions of authority are present. The information thus obtained on sensitive topics must be cross-checked with other sources of data. As an example, the evaluation team often felt that club members were extremely reluctant to reveal any information that was critical or might be construed as critical of the gover:nent or the project authorities. Some participants appeared to go out of their way to make positive comments about the extension stal, local leaders, and the govemrment As another example, most of the club members indicated that there was no discriminration against women, though considerable anecdotal evidence exists that women smamholders were universaly discouraged from joining the clubs. Also, the members did not reveal that many farmers who had initialy used the recommended variety of maize were discarding it at the time of the interviews, and that production remained * stagnant on many fanns. Post-Meeting Conversations Usualy, post-meeting conversations are extremely useful for obtain- ingfinformation that some participants might not like to share in public, and it is recommended that such conversations be an integral part of the group interview process. Krishna Kumar 7! Because of time constraints, the evaluation team could not arrange for such conversations in most instances. H-owever, when this was possible, many issues were clarified and insightful information obtained. For example, in one such post-meeting conversation, a dlub member men- tioned that many farmers did not find the recommended technical pack- age entirely satisfactory. This was later confirmed by many key informants. Another participant complained that 'demonstration gar- dens' did not serve any useful purpose, as most of the farmers knew about the recommended technical package and its strengths and limitations. A Commnodity Systems Assessment Methodology Workshop: Improving Agricultural Production and Marketing R.J. Haggerty and J.E. Armstrong In. this chapter, Hag gertyj and Armnstrong describe a variant of the group interoews technique, in which a workshop format is used to generate. information and recomnsendatlions. Haggerty and his asso- diates were asked to determnine ways to improve the production and marketing system for ginger in the Rapti Zone of Nepal. Practi- cally no published material uws available on the subject, and the project did not have time and resources to mount a comprehensive study. Under these conditions, they employed what is known as "Commodity Systems Assessment Methodology' (CSAM),in which a series of questionnaires arc used to generate informnation in a workshop setting. .E-aggerty and his associates organized a five-day workshop of 40 participants, including ginger farmers, merchants, agricultural ex- perts, project staff, and concerned government officials.'* The par- ticipan ts were divided into four groups and held 3 6 group sessions which focused on djffe!rent difmensions of the system. At the end of each session, group leadens filled in structured questionnaires drawn from the data, information, and ideas generated during the discussions. Thiefindings of various groups were synthesized in the findl report. By tapping the experiences and opinions of the major actors involved in the production and marketing of ginger, the team was not only able to generate a reasonable body of knowledge that could befiurther tested and refined,bUt also able to produce a set of recommendations that were used by the program managers and the Nepalese government. The methodology used by Haggerty aznd Armstrong is essentially qualitative in nature and is suitable for analyzing a -broad range of 72 R.J. Haggerty an d J. Armsfrong 73 problems in commodity system. However, it can only be used to assess aspecific comnmodity in a.limited geographic area. Extreme ca- is also needed in selecting participants, refining question- naires, moderating discussions andfill;ing out the questionnaires. Otherwise, the discussions may be misleading. THIE.PURPOSE OF THIS STUDY was to determine ways to improve the production and marketing system for ginger in the Rapti region of west- ern Nepal. USAID/Nepal contracted with a US. institution to collabo- rate with a Nepalese consulting firm to conduct the study. The U.S. and N4epalese.contractors were charged with conducting a workshop in thie Rapti Zone to analyze the production, post-harvest handling, marleting, infrastructure, and govenmuent/private sector in- - puts and outputs for ginger, using the Commodity Systems Assessment Methodology (CSAM, La Gra, 1990). The workshop, held April 24-28, 1989, had the following objectives: - provide a rapid, low cost analysis of production and marketing of ginger in the Rapti Zone; *: i identify problem areas in the system, and propose interventions to - alleviate them; * * improve communication and coordination among the members of the giger commodity system; and * train Nepalese technical assistance personnel in the use of the CSAM. Two U.S. consultants each spent a total of 18 days in Nepal to complete the assigment. Country and Program Settino The Rapti Zone, located in westem Nepal, has very few roads in the region, and most movement of people and goods is via backcountry footpaths The workshop took place in Dang District, which lies partly in the Gangetic plain. The zone's four other districts (Pyuthan, Sal,yan, Rolpa, and Rukum) are situated in mountainous regions. The USAID/FHis Majesty's Government (HMG)-funded Rapti Devel- opment Pojiect was initiated in 1980 for the economic and social devel- opment of the people in the Rapti Zone. The first phase of the project, ending in 1987, achieved significant progress in infrastructure develop- ment, in availability of services to the rural people, in natural resources management, and institutional development. The project continues efforts to improve crop, livestock, and forestry production through a decentralized and highly participatory approach involving local governent entities, farmers, and private entrepreneurs. * - Programs are aimed at the fulfillment of the basic needs of the people in 74 A Commodity Systems Assessment MethodoIogy Workshop the region. USAID/Nepal seeks to assist HMG in improving crop pro- duction, post-harvest handling and processing, and marketing of a vari- ety of agricultural products in the Rapti region. USAID/Nepal determined that there was a serious lack of information available about the production and marketing of ginger, an important cash crop in the region. To facilitate improvements in the commodity systemw the appli- cation of participatory and qualitative assessment techniques was needed. Research Design and Rationale The Commodity Systems Assessment Methodology (CSAM) is a rapid, Iow-cost data collection method involving elements of group interview and focus group discussion methodologies. The CSAM was developed by the Postharvest Institute for Perishables (PIP) at the Uniiversity of Idaho in the United States under the leadership of JerTy La Cra, a mar- keting spe6ialist worldng for the Inter-American Institute for Coopera- tion on Agriculture (La Gra, 1990). It was designed to provide scientists and decision-makers concerned with post-harvest problems a method- ological tool for collecting field data on magnitudes and causes of post- harvest losses, primarily with perishable commodities. The information obtained with CSAM is used ta design economically viable solutions or to recommend priority areas in which additional information is required to analyze and resolve post-harvest problems. CSAM was also designed to develop a local capacity in the host country for applying the method- ology without expatriate assistance. CSAM is based on the premise that exaniination of perceived prob- lems cannot take place without considering the commodity system in which the problems exist, including the participants in the system. Be- cause a commodity system is influenced by the agronomic and physi- ological features of the partcular commodity, by weather conditions, other geographical variables, and the socio-political-economic environ- ment, the system of necessity encompasses all the participants involved in the planning, financing, production, processing marketing, and sup- port services of a particular commodity, including a diversity of public and private institutions and marketing intermediaries. Figure 41 ilIus- trates the generic categories of activity ir. a commodity system used in CSAM. An assessment of problems in a commodity system requires input from an interdisciplinary group of people fmiliar with different aspects of the system in question. The CSAM assumes, in general, that the various participants in a comunodity system familiar with different aspects of the system know how well it is functioniing. These people have insights about what the R.J. Haggerty and .E. Armstrong 75 ;-:.problems are and their root causes, and can generally propose solutions. Further, CSAM assumes that a group of system participants can estab- lish priorities for their problems, recognize commonalities, and propose specific solutions or improvements that will benefit the whole system. The CSAM approach is therefore interdisciplinary in nature and draws upon the knowledge of local people. Each application of CSAM must be conmmodity-specific and geographically limited. CSAM is applied in a workshop setting in order to achieve objectives that are clear to all participants. The methodology requires discussion groups to complete a set of about 30 questionnaires representing func- tional components of the commodity system. The components, shown in Figure 4.1, are presented in a circle format. The center part of the circle is divided in half, identifying those components that fall into pre- harvest versus post-harvest stages. Each half-circle is further sub-di- vided to indicate whether the components deal with pre-production (planning, policies, and institutions), production, post-harvest handling, or marketing The components are addressed in chronological order, from pre-production planning to marketing. Each one of the components is potendally important, because the decisions or actions occurring at that point may affect production, pro- ductivity, quality, or cost of the product. However, not all the compo- nents are relevant for every coLmmodity system. In such cases, the irrelevant components are elminated from discussion. The list of ques- tions that guide the discussion on each component should be modified im accordance with the commodity and the circumstances of the com- modity systemn CSAM is conducted in three phases: collection of background infor- mation, problem idendfication, and solution identification. To complete all three phases, CSAM utilizes-as previously stated-a group discus- sion method in a workshop setting. Depending on the situation, for example, if the size of the group is large, there may be more than one discussion group operating simultaneously. CSAM discussions are not completely open-ended. In all tiree phases, discussions are directed toward objectives of which the participants are made fully aware. The main pv"ose of the background information collection phase of the workshop is to generate information that describes the commodity system in question. This is accomplished by having the discussion group answer about 30 structured questionnaires. If there are several groups in the workshop, each discussion group answers the same series of questions from each structured questiormaire. Because not all group members are knowledgeable about all aspects of the commodity system, answering the questions requires input from group members who are familiar with these different aspects. To be able to fill out the question- .76 A Commxodity Systems Assessment Methodology Workshop Figure 4.1 Principal Components for a Commodity Systems Assesment D-;~~~~~~~~~2 26: 19~~~~~~~~~~0 Sorc LGra-e a. (97 . 9) modi76 y systmltt whsom rigtnt otherwise havetheoppotntove 0 FiOnce the Prcmoity systoemX horas beedityn dsrbdi hs mAnner,sten 17 t: ~~~~~~~~~~~~~~~~~~~~~~~~~u Sources La Gra et al. (1987, p. 9). nare as a group, consensus withiin the group needs to be reached. The resulting interchange of ideas and perspectives highlights thie secondary - ~purpose of this phase of the CSAM workshop. That is, the group dis- ;cussions enac commuriication among diverse members of a com- -modity system who msight not otherwise have the opportmity to viewf the sytstem. from someone else~'s perspective. : 0 ~Once te conunodity syrstem has been described in thils ma-er the discussions move into their second phase, which is to identify perceived problems in the commodity system and to assign priorities to the probe RP4. Haggerf-y and .E. Armstr(mg 77 : 0 len.-s identified, in terms of their importance. Producing such a priority list of problems usually requires extensive discussion among members of the group. The third phase of the CSAM workshop builds on the outcome of the second. During this phase, the groups must identify possible solutions .t the problems previously identified. The solutions must be realistic and serve as the basis for recommendations to authorities for interven- tion or further research. Each discussion group requires a moderator to guide tie discussion ---- and ensure that the questionnaires are answered and the group com- - pletes its task in each session. At least one observer is present to assess the non-verbal reactions of the group(s), and to observe group dynamics for discussions with the moderator(s) and other workshop leaders after each sessiorn In discussions by workshop leaders after each session, the observer(s) and moderator(s) compare their views concering what they heard and observed during the group sessions and determdne the most appropriate actions for subsequent sessions. These monitoring discus- sions between sessions also are a validity and reliability check of the process. - - In a workshop in which sevenal discussion groups answer the same questionnaires, it is necessary to pool the results and summarize them into one set of responses. A report is also written at the conclusion of the workshop. The report summarizes the results of the group sessions (that is, description of the system: production, post-harvest handlin& and marketing practices; constraints identified in the system; and sug- ygested solutions to those constraints), incorporating the moderator's in- terpretation of group findings. The report serves as a document to help stimulate the recommended next action by the appropriate organization. - - - - In contrast to the traditional group interview method, CSAM uses a group cf moderators who are referred to as the "coordinating commit- - - tee." Ideally, this committee consists of host country specialists who are knowledgeable about or familiar wit policies and planrin& finance, production, post-harvest, marketin& and support services aspects of the commodity and geographical location in question. The coordinating committee usually indludes a US. consultant whose role doubles as overal CSAM workshop moderator and trainer of CSAM. It is a goal of - : -- the developers of CSAM that, following a CSAM workshop, other mein- bers of the coordinating committee should be able to conduct similar -: workshops for data gatherng without outside assistance. The coordinating committee works together to plan the CSAM workshop, prepynres all logistical arrangements for the workshop, modifies the questionnaires in advance to ensure their appropriateness : --to the situation, moderates group discussions, conducts monitoring -78. ACmodity Systems Mssessment Metowdology Worksho discussions after each group session with a group -observer, pools and sununanizes responses to questionnaires, and prepares the final report. Nepal CSAM Ginger Workshop: Plauiung and Questionnaire Preparatioin Prior to the Nepal CSAM ginger workshop, the Nepalese contractor, mn coordination with the US. contractor, identified key people to serve as . the Coordinating Committee and over 30 individuals as workshop par- ticipants. The Nepalese conLtractor had several years of experience working -in the Rapti region and had become acquainted with many farmers, merchants, and extenion personnel in the area, in the context of previous Rapti Development Project activities. Many of these indi- viduals were-invited to participate in the workshop. The Coordinating C ommkitt-Cc included: * a planning and policy specialist (USAID/Nepal) * a planring and policy specialist (NepaIese contractor) * a marketing specialist (Nepalese contractor) : a production specialist (National Agricultural Research and Services *Center) * a production and post-harvest specialist (Nepalese contractor) * a marketing specialist (U.S. contractor) * a post-harvest specialist (U.S. contractor) Upon arrival of the U.S. team, the final technical plan for the work- shop was developed by the Coordinating Conunittee (CC). In lengthy sessions of the CC in Kathmandu, and with secondary information on gmger gathered by the Nepalese contractor, the C(AM questionnaires were modified to address the known characteristics of the ginger pro- duction/marketing system. 'This workshop was designed Lb include as participants small-scale farmers and local marketing agents with fimited formal education. (In previous: applications of CSAM all workshop participants were edu- cated to at least the high school level, and most held bachelor or gradu- ate degrees.) It was therefore anticipated that illiteracy or language .-*::f differences would exist. In the interests of saving time and expense, it was decided to prepare the questionnaires in English, with the language gap being bridged by English/Nepali-spealing moderators, who would translate the questions orally in Nepali and complete the questionnaires in English. . The preparation of the questionnaires among the CC resulted in the elimination of the "cooling' component, one of the aforementioned 30 components, which was felt to be irrelevant since precooling is an ml- P4 Haggerty and I.E. Annstrong 79 necessary step in ginger processing-and is not carried out in the Rapti Zone. There were also numerous changes in wording to tailor the ques- tions to the ginger commodity system. CSAM- questionnaires are designed to foster discussion, as well as to obtain valid, useful, and objective information. The questionnaires are structured to generate descTiptive information and do not focus much on the gathering of quantitative information. Questions probe for infor- mation by asking "who, wihat, when, where, why, and how." Some of these are open-ended questions intended to elicit comments and obser- -vations from the participanrts in the discussion groups. Samples of all CSAM questionnaires are found in La Gra's 1990 document, and can be obtained at the address noted in the references. A sample of the CSAM questionnaire designed for the preser' study is shown in Box 4.1. la previous CSAM workshops, the questionnaires were developed to tdeal wit the commodity sysbem in the order of a discussion of pre- production components, and continuing through production, post-har- vest handling, and marketing. However, because this workshop involved farmers who might be uncomfortable and reluctant to speak out, it was decided to start with the production component with which they would feel most famiLiar. The CC felt this would ensure continued -: - active participation. Furthermore, to ensure there was sufficient time available for discus- sion1 on. all components and a sequence that would maximize active * ;- participation, the component questionnaires were clustered into 9 blocks contanining a total of 29 components. Each of these blocks then served as the basis for one discussion session. The CC felt this approach would be less likely to overwhelm the participants w,i. thick bundles of papers and questions. The block descriptions and daily schedule are shown in the appendix to this case study. As a matter of protocol, once the questionnaires had been revised and the tentative schedule for the workshop developed, the material was presented to appropriate government officials for review and comment Planned. workshop activities were also discussed with representatives from these agencies. Each of these individuals was invited to attend a : ::;. final seminar after the worlkshop to be presented in Kathmandu. The final workshop planning took place in Tulsipur. The Nepalese *',' ':contractor produced stenciled Nepali translations of the workshop schedule, opening statements, and block descriptions. Meetings were held with the representatives of U3AID/Nepal and the Rapti Project Coordinator's Office (PCO) to discuss details of workshop implemen- tatiorL. Participant certificates were prep,;,ared. The Rapti Project Coordinator's Office volunteered the participation of some of its person- 8A Commodity SystemsA AsesmMthodology Workshop nel, all of whom spoke and read English, but who knew little about ginger production and marketing. The CC intended to take advantage of the volunteers'language skills to enhance communications by includ- ithem in gup discussions. It was deternnined that one member of each discussion group would be a volunteer from the Rapti Project Coordinators Office with English -'language skis who would act as discussion &;-nup moderator and re- corder. The moderators translated the questions orally in Nepal for the discussion group and fied out the questionnaires in English. In a departure from previous CSAM applications, it was decided that o Itwepalese member of the C would join each discussion group as a - CSAM trainee" to observe the process, rather than function as modera- tor. Each trainee would function as an observer in order to learn about the application of CSAM for fuiture use on other commodity systems. The trainees were madetresponsible for assessing the process of infor- mation collection, groupn comunication, and evaluating the appropri- ateness of the questions. The trainees were to have minimial participation in the discussions and were to rotate to a different discus- sian group each session. The CC planned to hold monitoring discussions between sessions and at day's end to determine what changes, if any, were to be made for the next session and/or day. In this way, the trainees would have their understanding of CSAM reinforced and would develop their ability to apply the methodology. Workshop Implementation and Problems Encountered Approxinmately 40 participants attended the workshop, which was held at the conference haRl at the Rapti Project Coordinator's Office complex in Tulsipur. Among these were 15 merchants and 12 harmers including one woman faner. The mcix of participating farmers and merchants represented enterprises ranging from small to above-average size, and frm below-average to above'-average prsperity. The remainder of the participants were govermxnent agents and staff from the Rapti Project Coordinator's Offie.eAll workshop partiipants were acquainted with * one or mnore of the NepaIese people working for the Nepalese contrac- ton a The workshop was completed in five days. Opening Activities After an opening ceremony was held, four discussion groups were formed. Each partiipant was assigned to adiscussion group by one NepaIese member of the CC. Each discussion group was formed so as PLI. Haggert y and J.E Annst-rong 8!' Box 4.1 Sample CSAM Questionnaire for Marketing Ginger, Rapti Region STORAGE NAME OFDATA COLLECIUORh TEL. TITLTh nqSTIT:i:ON: 1. Identify the points in the post-harvest system where storage takes Place. Duration of Storage Yes Nbo (Days) *cmnthe farm()) * rural collecturigpoint() (j 0, regional collectling point() () *1packinghouse C * '~~9retailnmarkcet() () * wholesaleniarket() () * agro-industry C * superniarket()) * export warehouss ) ( * container temrmina (export) () () *gvrnmcn marketing board () ) * import warehouses() () *otlier()) 2. For each instance of storage identified, provide the following informa- tiorn a) Type of storage (from 1 above): b) Who is responsible for the storage? farmer (,middleman(, govlt ( ,wholesaler (,retailer ),processr () ther ( c) What is the purpose of storage? await shipping (,await better market price ),maintain qut"ity (,assemble larger volumes C ,other() d) Describe the storage facilities and equipment e) How long after harvest does the product norxmaly go into storage (hours and days)? contiRneed on nxt page '.82 A Commodity Systems Assesment Mdetiwdology Worksho Box 4.2 (continued) * F) How long is the holding period? g) At what degree of ripeness/imaturity is the product whent it is normally placed into storage? * It~~) How does the quality of produce change during the storage period? Ls temperature controlled in the storage enviroment? yes no ExpLaizu j)What is the range of air temperature in the storage environment? Ic) Is humidity controlled inL the storage environment? yes ()no ( Explalir 1) What is the range of relative humidity in the storage environ- nx nt? . .n) Is the atmosphere in the storage facility modified lor contred 9?Describe- n) Is the product nornmaly stored by itself ( ,or with other produce ()? If with other produce, spec-ify what kidnd: o) DesAibe the type of container in which the product is packaged during storage. p) Who owns the storage facilities? cq) Who operates the storage facilities? 4 What is the cost of holding the produce? s) Is the storage facility operated efficiently? yes rs( )ro If no, explain ) Summarize storage problems that may affect post-harvest losses. 3. 4. Observations: I'I. Haggerty and J.E. Annstrong 83 to contain about ten members, representing a mix of farmers, merchants, extension agents, financiers, and researchers. In the end, the generl makeup of the discussion groups was typically two farmers, two mer- chants, two bankers, one moderator (recording results in English), and one observer (a member of the CC learning how to use CSAM). One CC member (from the U.S. contractor) acted as overall observer of all four discussion groups, without participating in any group discussions. A group solidarity exercise was conducted by all the discussion groups. Each of the four discussion groups created a slogan for their individual group. This exercise was initiated and led by the Trining Coordinator (a Nepalese) from the Rapti Project Coordinator's Office. Each discussion group's slogan was presented to all other discussion :*. groups and the members were introduced. This exercise seemed to reduce nervousness among the participants and helped prepare them for the interactive nature of the workshop. The final workshop schedule was negotiated with the whole group and settled quicldy. Collecting Bacground Information Over the cmurse of the first three days, the discussion group partici- pants conducted the background information collection phase of the workcshop. During this phase, all four groups held nine simultaneous discussion sessions, corresponding to the nine blocks of component questionnaires, and answered the structured questionnaires. In this s way, a total of 36 group discussion sessions were held during this initial : phase. V phaseBloc I-The Importance of Ginger" was conducted as the initial task It served as an introduction to the CSAM approach of information- gathering through active discussion and the completion of the question- - aire. The discussion groups were led through this block by the Nepali-speaking workshop moderator. A printed explanation, in Nepali, of the block objectives was distributed to all the particpants at the beginning of each session. The Nepali-speaking workshop facilitator introduced all four groups to each block by reading out Ioud a definition of the blocks components and the objectives of the session's discus- sions. :All four discussion groups then simultaneously discussed the topic and completed the component questionnaire for this initial block. Members of each discussion group sat in a circle to facilitate open dis- cussion. Introduction and functioning of all subsequent discussion sessions during the first phase followed the same procedure as Block 1. One set of questionaires was given to each group moderator, who filled out the questionnaires with input from the discustion group. 84 A Commodity Systems Assessmenit IMehodalogy Workshop * ~~~A the enLd of each discussion session, each discussion group reported to U heothers, callinig for questionLs and comments. At the end of the first workshop session (Black 1) eaich discussion group moderator re- ported his group's results to all the other groups. In the monitoring discussion held after this initial session, the CC concluded that this mode of operation limited the involvement of farmers and merchants. Thereafter, the moderators recorded results and another group member did the reporting. The introductory session's discussion went well. At the conclusion of each discussionsessionk, one U.S. CC member summarized the results of all four group questionnaires into one set of responses and entered these into a portable computer. The other U.S. CC member documented observations made durng all sessions for in- lusion in description and analysis of the CSAM workshop process for the final report. Problemn Identification On the fourth day, the problem-identification phase of the workshop began. The taskcs were organized as follows: each individual pat. tipant identified as many problems as possible; each discussion group col- lected these problems and presented 10 to 20 of them to the other dis- : ccussion goups; and in rotation, each group presented its top prriorityr problem to be tackled in the Phase Three exercise (solutions to prob- lem). Exluding duplications, the discussion groups presented prob- lems umtil 15 had been selected. The discussion groups were then briefed by the CC on approaches to solving problems and were encouraged to develop solutions they them- : ousselves could accomplish as wel as ones requing outside help. Judging by the energetic, boisterous activity of the second and third workshop phases, the creative task of problem identification was a welcome change from the first phase task of completing questionnaires. The dif- - ference in language was not a constraint to satisfactory completion of the tasks. Solution Identification Continuing on into the fourth and fifth days, the groups developed solutions to tfhe problems identified-the third phase of the CSAM workshop. As stated, the participants were encouaged to identify solu- tions that they themselves could accomplish rather than just solutions that someone else nmght provide for them. Each group's solutions to the problems were presented to all workshop participants on poster paper -* - ^ 0 - -P4.ffaggertyandJ.E.Annstrvng 85 and the other groups contributed comments on ways to improve the solutions. The dsion on ese was very vigorous. Problems Encountered "Block 2-Producticn took a long time to complete. The nmain prob- lem arose in trying to balance completion of the questionnaire with open, full discussion among the focus group members. There was lengthy discussion during this session, and it progressed slowly. Con- cem arose that all the blocks would not be finished in time for the workshop to proceed on schedule. Consideration was given to eliminat- ing another component or two. However, the CC instead encouraged the moderators, under the supervision of the observers, to be slightly :more pro-active in guiding the discussion group to complete its task. The CC determined from the first session that the sessions covering problem identification and solution proposals would require all of days 4 and 5. As it turned out, the groups moved much faster on day 2. None of the components had to be eliminated, and they were rapidly s.ummarized and entered into the computer. The discussion groups, in general, seemed to work quite effectively. Part of the problem with one group which lacked vitality was that the moderator assigned to that- group had other commitments and waset * - diliget in attendance. 'The group that inluded the one female participant made efforts to *; enhance her active involvement in the discussions. Her opinion was sought by the moderator in a non-threatening -wray. The efforts had little success Although the woman attended every session diligently a-nd paid attention, she made very little verbal contribution to the discus- sions. i The exchange of views among the individual discussion group mem- bers appeared to be very good. This was especially true between mer- chants and farmers. Though it was anticipated they iight be mutually antagonistic, they had consistently active, amicable interaction in the X*f discussion groups. Caution was taken by the moderators to ensure that the merchants did not feel. proprietary information was being sought from them. (Mhe information provided by the merchants in fact may not . have been the most accurate, in the interets of protecting their busi- * t 2 nesses.) In all cases, the group moderators made conscious efforts to balance participation and involve all members to the maximum extent possible. - - - The information gene.-ad was comprehensive, even if it was not absolutely consistent among, the groups in every detail. Enough geo- 86 A Commodity -Systems Mssessrnex t Methodology WoTrkshop graphic diversity exists, even in the r-elatively small Rapti regioni, to account for some differences in practices from area to area. In general, however, there was remarkable consistency among groups in the intfor- mation generated. By the end of day 3 when the questionnaires were completed, discus-. sion group participants appeared tired. Th-ree days of continuous ques- tions and discussions about production and marketing had become tiresome. It was evident that all participants had a broader and deeper understanding of the production and marketing system under study. However, one member of the CC pointed out that, without reinforce- ment in the near future, Cite participants would have lost what they had gained. Group members were confused about a format to be used for presen- tation of solutions and treated the first round of proposal development as an experiment for fashionidng a workable format. Tlhe CC encour-aged the groups not to take time in developing a formnat, but to concentrate on the following items: problems to be solved, actionts needed, identifica- tion of who would act, and description of how the solution would be accomplished. With that information, the CC was later able to formu- late more specific project profiles. The sessions on solution develop- ment were a more tedious process for the participants than problem identification had been. However, when individual groups prescnted their solutions, the discussionLs became very animated. Findings Phase I-Database Generation on Ginger Production and M-arketing With the active participation of more than 40 individuals, the CSAM workshop developed a new, larger informiation base on ginger produce tion and marketing in the Rapti region. The information generated pro- vided more specific details about ginger production, processing, and marketing in the Rapti region than existed in any zecondary source in Kathmandu at that time. Post-harvest losses were found to be snmal in quantitative terms, but significant in terms of lower prices earned for the lower-uality ginger produced. Phase Il-Signqlcant Probliems Identified Using the background information, the participants identified 15 sig- nificant problems they felt needed to be addressed. Tfhese problems included: P4 .Haggerfly an d J. E. A ms trong $7 * monopsony power of hIdia and Indian merchants * lack of price information * lack of plant protection materias and techniques *0 no unproved varieties available * use of traditional practices due to lack of new production technology available * ~* no knowledge of alternative ginger products * lack of coordination among support services and organizations * no or-ganization or- institutional support for private sector develop- ment * *~ no organization or institutional support for improving production/ marketing skills * lack of drying technology * lack of knowledge about relationships between soil, ginger varieties, and fiber development a no knowledge of consumer or market needs * lack of stor-age and packing materials and facilities * lack of commiercial ginger industry (processing and transformation)' lack of credit'faclities Phase III-Identification of Solutions to Ginger System Problems To resolve the identified poroblems, it was suggested that the follQw- ing activities be undertaken to intervene in the system: creation of a ginger research site in Salyan; development of imnproved ginger market assembly points;,international m-arket and consumer research; process- ing improvements and industrl development; and packaging techol- ogy improvement Accomplishments This application of CSAM produced good-quality descriptive informa- tion in a short time and at relatively low cost. Enhancement of Communication The workshop also achieved the other significant objectives of in- creasing dialogue between producers and traders, and developing and improving the capacity of local personnel to conduct actitvities lke CSAM. It was suggested that additional CSAM workshops be con- ducted in the Rapti region for otheir important crops such as apples and vegetable seeds'. 88 A Commodity Systems Assessment Methodology Workshop Development of CSAM Skills The U.S. team felt strongly that the workshop was a success and that the CSAM, as modified to study ginger in Nepal, was an appropriate and effective tooL However, as stated above, some further modification could be made in the format for completing the questionnaires that would result in a less tedious operation for the participants and would stimulate more discussion. The main strength of the workshop was that it effectively provided a forum to improve communications among farmers, merchants, and facilitating services personnel, especially be- tween farmers and merchants. The information gathered was very comprehensive, although not as detailed as it might have been, since not all the questions were an- swered. However, with the information gathered by the Nepalese con- tractor before the workshop, a very complete picure resulted. The problems identified and the solutions developed by the partici- pants were well-focused and structured. Certanly, all the participants broadened and deepened their understanding of all aspects of ginger production and marketing in the Rapti Zone. Four of the farmers in the group said they now considered forming an assocation to strengthen their marketing position. Whether this kind of enthusiasm will endure depends on adequate follow-up to the work- shop. There is certainly ari interest in continuing application and modi- fication of CSAM, thus, the U.S. team felt that the training objective of the workshop was clearly met. The members of the CC from the Nepalese contractor, the National Agriculbtral Research and Services Center, and USAID/Nepal demon- strated a firm command of how to apply the CSAM and recognize its strengths and weaknesses. One shortcoming, however, was that the recorders/moderators working in the discussion groups had received no prior exposure or training in CSAM Their role in the functiornng of the groups was critical, and they did a good job. However, their effec- tiveness would have been improved if they had studied the question- naires in advance or had some orientation to CSAM. This would have deared up some of the ambiguities and vagueness experienced on the firt day, and better information might have been tle result Suggestions for Potential Users of this Methodology Guidelines A portable computer and printer were essential to the timely prepara- tion and modification of the questionnaires. A small Toshiba 1100 Plus was brought by the U.S. contractor, since it was known to be compatible PRJ. Haggerty and JE. Armstrong 89 with the equipment of the Nepalese contractor. The report editing would have been almost impossible without this type of on-site word processing capability. Fundamental to the success of the workshop was the effective man- agement provided by one of the Nepalese members of the CC working for the Nepalese contractor. Not only was he an excellent overall mod- erator and facilitator, he also made all logistical arrangements flow smoothly. Perhaps his most critical contribution was to effectively select and encourage participation by the farmers and merchants. It is clear that such an effective local organizer will be critical to success in future work with the CSAM. The workshop was also a success because the partcipants took it seriously and were fully committed to it. The PCO staff was veiy sup- portive and worked enthusiastically. Limitations Some caution about the limitations of the CSAM should be noted. First, the CSAM is most appropriately used to assess a specific commod- ity in a limited geographical area. Covering a very large geographical area makes it more difficult to accurately descnbe a commodity system. The process becomes more cumbersome and the information less reliable. Second, the CSAM is probably better-suited to identification of prTO- lems internal to the functioning of a commodity system than it is for identification of problems in goverrment policy and institutionaI inputs. Farmers are not likely to be well-informed about detailed aspects of national policies. It is also likely that the dynamics of group discussions would be impaired if high-ranking government officials joined groups with local farmers and merchants. The person of lower social status would probably feel inhibited and not contribute effectively to the group discussion. The result would be unbalanced group discussions and, perhaps, less reliable infbrmation. Third, it is critical that questionnaires be tailored for each application of the CSAM. If the right questions are not asked, then the information generated has limited utility. However, modifications in the component discussions are needed to stimulate more discussion and break the te- dium of answering questionnaires. Appropriately refined question- naires are essential to the successful application of CSAM. It would also be prudent to consider limiting the number of questions and/or ques- tionnaires for groups of individuals who may be less-educated. Users of CSAM should not be overly concerned with quantifying post-harvest losses in a commodity system. The CSAM is really in- tended for analyzing a broad scope of problems in a commodity sys- 90 A Commodify Sysfems Assessment Methodology Workshop tem. Even if post-harvest losses are specifically being addressed, the CSAM is not designed for quantifying these losses in detail, but rather, for establishing their magnitude, the causes, and the economic and tech- nological possibility of reducing them. CSAM discussion group moderators should have adequate prevous exposure to or training in its application and its questionnaires to eqfec- tively guide and stimulate discussions. The application of the CSAM requires an interdisciplinary or team approach. it is highly unlikely that one person will have all the knowl- edge to properly identify the problems related to pre-production, pro- duction, harvest, post-harvest, and marketing that make up any commodity system. Furthermore, application of the CSAM is not limited to perishable crop commodity systems. The CSAM could conceivably be applied to grains, seeds, and marine or freshwater fish. It would be useful to test the CSAM in conducting rapid, low-cost assessments of these types of commodity systems. Applications The CSAM will prove useful to short-term consultants and decision- makers interested in rapid appraisals and development from a commod- ity system perspective. The CSAM manual (La Gra, 1990) can be used in a workshop envi- ronmnent to train professionals in the commodity systems approach, ei- ther from a theoretical point of view, or as an applied, in-service case study (specific commodity) form of trainin& In the first instance, the trainees may be of the same or of different disciplines. When the case study approach is used, the trainees should include persons with exper- tise in economics, agronomy, social sciences, food technology, post-har- vest activities, and marketing. The CSAM manual will also prove useful to ministries of agriculture, marketing boards, corporations, research institutes, and other national institutions interested in the systematic improvement of production, post-harvest handling> and marketing within existing commodity sys- tems. At the regional or national leveL the methodology will be valu- able in the identification of agricultual development projects. It will be of particular value in the execution of rapid appraisal exercises, using interdisciplinary teams of national specialists. References Creenbaumx, Tl. 1988. The Practical Handbook and Guide to Focus Group Research. (ISBN No. 0-669-14775-3). LTxnngton, MAc D.C Heath Company. RJ. Haggerty and J.E. Armnstrong 91 Honadale, G. 1982. 'Rapid Reconnaissance for Development Adninistratiorn Mapping and Moulding Organizational Landscapes. World Development 10(8):633-649. La Gra, J. 1990. A Cmmodity Systems Assessnt Methodology for Problem and Project Identificztion. Moscow, Idaho: Posthiarest Institute for Perishables, University of Idaho. La Cra, Jerry, Leong Poo Chow, and Robert f. Haggerty. 1987. A Postharvest Methodology: Commodity Systems Approachfor 7the Identification of Inefficiencies in Food Systems. Moscow, Idaho: Postharvest Institute for Perishables, Uni- versity of Idaho. McCullough, J. and R. Haggerty. 1989. An Applicatioz of a Commodity System Assessment Methodology to Improve the Functioning of the Production and Mar- keting System for Ginger in the Rapti Region of Nepal. GT;S Report No. 104. Moscow, Idaho: Postharvest Institute for Perishables, University of Idaho. McCullough, J., and a- Murray. 1988. Approaches for Improving Mae-eting and Reducing Postharvest Loss of Agricultural Products fmm the Rapti District of NepaL. G0S Report No. 101. Moscow, Idaho: Postharvest lstitute for Perish- ables, University of Idaho. 92 A Commndity Systems Assessment Methodology WorkShop Appendib Block Descriptions The blocks discussed in the workshop included the following: Block 1-Commodity Importance (Day 1) Relative Importance of Crop to be Studied Component 01 Block 2-Production (Day 1) Farming Systems Component 02 Characteristics of the Commodity Component 03 Natural Resource Constraints Component)04 Availability of Seeds and Planting Materials Component 05 Facilitating Services Component 06 Farmers' Cultural Practices Component 07 Theoretical and Actual Costs of Production Component 08 Block 3-Harvesting and Pest Controls (Day 1) Pre-harvest Treatments Component 09 Pests and Diseases that Affect Product Quality Component 10 Harvest of Commodity Component 11 Block 4-Post-Harvest System (Day 2) Identification of Points in Post-Harvest System Component 12 Block 5-Handling and Processing (Day 2) Selection, Grading, Classification, and hIspection of Horticultural Produce Component 13 Packaging of Horticultural Produce - Component 14 Post-Harvest Chemical and Thysical Treatments Component 15 Other Operations Component 16 Ago-Processing of Horticultural Crops Component 17 Block 6-Post-Harvest Losses (Day 2) Magnitude of Post-Harvest Losses Component 18 R.J. Hazggerty and JE. Armstrong 93 Block 7-Transport and Storage (Day 3) Transport of Commodity Component 19 *Storage of Horticultural Produce Component 20 *Delays or Waiting Component 21 Block 8-Marketing (Day 3) Intermediaries in Marketing Component 22 Mdarket Price Information Component 23 Consumers Component 24 Eixportation of Horticultural Pr-oduce Component 25 Mdarketing Costs of Horticultural Crops Component 26 Bllock 9-Policy and Institutions (Day 3) Institutions Relevant to the Commuodity System Component 27 Policies Component 28 Farmere Organizations Component 29 5 Using Focus Groups to Develop and Promote an Improved Weaning Food Product Cecilia Cabafero-Verzosa, Cecile M. Johnston, and Olabode Kayode As discussed in Chapter 1, the focus group discussion has emerged as one o fthe innovative information-gathering methods being used in development settings. In this chapter, Cabaniero-Verzosa, Johnston, and Kayode illustrate the use of this method to assess mo thers' reactions to an improved variety of weaningfood designed to improve infant nutrition and develop effective communication strategies to market the weaning,food as part of a health interven- tion effort in Nigera. This chapter descnbes the various steps-the composition offo- cus groups, locations of discussions, selection of participants, dis- cussion guides, group discussions, and data analysis-involved in the implementation of this component of a larger research project. More inportant, it also discusses a fw changes that the irnesti- gators made in the standard focus group methodology to make it more suitable for a developing country setting. For example, per- mission of the village chief was sought to interviewfemale partici- pantsforfocus groups. "Pre-" and "post-"focus groups were also conducted. First, mothers were taught an improved recipe for a commonly-used weaning food in a focus group context Later, after the mothers had tried the recipe at homefor some time, their prefer- ences and suggestions were discussed in another set offocus groups. Thnefocus groups generated useful information about both prod- uct-related and communication-related questions, which was ulti- mately used in the final decision-making related to the more widespread introduction of the nutritionally-improved weaning food. For example, the groups indicated that roasted cowpea flour was most acceptable to mothers as a fortifying ingredient; that mothers coutd easily prepare the improved food; and that an addi- 94 Gecdlia Cabafero-Verzosa, Cedle hv Johnston, and Okabode Iayode 95 tional preparation step-the addition of malt flour-was acceptable to mothers. The focus groups also suggested that the improved wean ing food could be better marketed as a foodfor a healthy ch ild, rather than as a "medidnalt"foodfor a sick one, and that the addi- tional ingredfents to te added in the old variety should be sold separately so that mothers couldfortify the food themselves. IN NMCERIA, as in other developing countries, poor nutrition often pre- disposes young children to diarrhea and other childhood illnesses. When the Dietary Management of Diarrhea (DMD) Project' got under- -::- way in Nigeria, the team uncovered a pattern of feeding during the weaning period that seemed to exacerbate infant susceptibility to dis- eases, including diarrhea. Traditionally, Nigerian infants beg the weanin process at four to six months. Their first weaning food, called EKO, is a very watery concoction made from a maize or. guinea corn pap called ogi. Infants are given EKO daily and are introduced to more nutritious solid foods only at a much later date. In fact, the research data indicate that at 12 monffis of age only half the children are consuming solid foods as part of their regular diets. Families also take ogi as a daily breakfast and snack food. The nmix- ture often contains as much as 70 percent water and, needless to say, is not very nutritious. It can also be a carrier of the water bacteria that cause diarrhea. Project Overview This paper describes how focus groups were used in conjunction with quantitative research techniques to develop and promote a new1 en- * riched weaning food to Nigeran mothers as a means of improving both the nutritional and health status of their babies. The chapter provides background on the use and mechanics of focus groups as a rapid ap praisal method, and explains how the group discussions for the present project were planned. The chapter then discusses the specific focus group methodology used in this project and the analysis of results, and finally highlights howv the study results were used. The Interuention A DMD Project team of medical and public health professionals and social scientists reasoned that EKO could be fortified to create a new weaning food that would help to improve the nutritional status of these children. Cowpea flour, red palm oil, and sugar were eventually chosen * 96 Uising Focus Groups * as fortifying ingredients-items that are all readily available in the com- munity and provide the required nutrients for weaning-age children. The new product came to be known as E3KO ILERA, or "1EKO for Health"'. The fortified EICO ILERA is much more nutritionally sound. than the traditional watery E3KO2 * ~The improved weaning food was introduced in two local goverrnment areas, Asa and Oyun, in the KCwara state of Nigeria. This area offers a good mix of urban and rural communities and is comprised mostly of the Yoruba~ ethnic group., This envirornment provided a homogeneous audience far the project's communication comnponent and for its re- search compon'ent as well. During the last stage of the intervention, health care workers at each of 12 sites trained 10 mother leaders who in turn were each responsible for training 10 neighbors in the preparation and feedin~g of the new EICO ILERA. Training materials included flipcharts, flyers, and product samples. Mothers were taught how to prepare the new EKO ILERA during cookdin demonstrations held in the community and in public places like the markets. In all, approximnately 1,200 mothers from Kwara State were trained during the intervention. The Research Issues This project was based on research related to two basic areas of deci- sion-making. The first were product decisions dealing with the specific composition and mixing of the EKG ILERA.. The second dealt with the nature of communication support needed to effectively enLcourage the acceptance of ERO ILERA among mothers. The four sets of product-related questions that had to be answered * Is fortification of the traditional ogi acceptable to mothers and E3KO sellers? What fortifying ingredients are acceptable? Why? * ~ oCan mothers learn the recipe? Can they teach others? * What product char-acteristics are important to mothers? Is the addi- tion of _-cialt flour to maintain the desired liquid consistency an addi'- tional step in the cooking process that would be tolerated? a Will mothers prepare the new EKO( ILERA at home? The communications-related questions indluded such issues as: * What communication strategy is appropriate? Is there an audience other than mothers? What benefit can be identified for the targt audien ce-the mother-and for the ultimate beneficiary-the child? * What materials wfi be effective in teaching the mothers to use EKG ILERA? Focus groups were the research vehicle used to answer these ques- tions. The qualitative, exploratory nature of focus groups seemed ideal Cecalia Cabaffero-Verzos%, Cedle M. Johnston, and Olabode Kayode 97 for examining reactions to a new product that deviated from traditional methods. Because group discussions lend themselves to probing and uncovering perceptions, attitudes, and feelings, it was believed useful for gauging mothers' impressions about a new food. Because EKCO ILERA deviated from conventional practices, focus groups were an ideal format to informally explore possible resistance and to learn what ap- peals might prove persuasive to the Nigerian mothers. However, the focus groups were only part of an overall research program of the DMD Project that included in-depth interviews, ethno- graphic assessments, observational studies, cost monitoring, clinical studies, and surveys. The focus groups, with their opportunities for group dynamics and consensus-building, functioned very effectively as a complement to these other types of research. While the more quantita- tive studies were oljective, definitive, descriptive, and measurement- oriented, the focus groups were subjective, exploratory, and interpretive. To summarize, in addition to the focus groups, the following research studies were undertaken in the effort to effectively introduce the new weaning food to Nigerian families: Ethnographic Studies. Key informants provided information on infant feeding, diarrhea taxonomy, and household feeding and treatment pattems during diarrheal episodes. At the time that the project team was trying to reach a decision on whether to fortify an existing watery pap used as weaning food, or to introduce solid foods earlier, ethnographic interviews were also con- ducted to assess which option mothers would find more acceptable. * Surveys. A representative sample of 2,655 mothers of children less than 3 years of age provided information on child feeding practices and provided anthropometric assessments of targeted children. -Food Price Monitoring. Quarterly market surveys were carried out in both urban and rural markets to establish the cheapest sources of energy and protein. * Longitudinal Household Treatment Studies. Laduba, a rural village near the city of Ilorin, was chosen as the site for conducting dietary intake studies and diarrheal epidemiology among 45 children aged 5 to 30 months. * Reipe Trials. Recipe trials provided mothers with first-hand experi- ence in the preparation of the new food. A list of possible ingredients for fortification was compiled. Motiers were invited to focus group discussions and cooking demonstrations to determine the acceptabil- ity of these fortifying options and the food preparation procedures. A second set of recipe trials occurred at the individual homes of moth- ers who volunteered to try two recipes 'hat affected the liquid consis- tency of EKO. 98 Using Focus Groups * Clinical Trials. Clinical trials assessed the acceptability, safety, and * nutritional quality of the maize-cowpea weaning diet in children with acute diarrhea. A total of 60 children aged 6 to 24 months were randomly treated with either the DMD candidate diet or a commer- cial soy protein isolate, lactose-free infant formula immediately fol- lowing rehydration therapy. Focus Group Planning Focus group interviews offer a means of obtaining in-depth information on a specific topic through a discussion group. The underlying premise is that people who share common experiences, problems, or concerns are willing to reveal them in a group atmosphere. Focus group inter- views are not simplv individual interviews conducted in a group set- ting; the moderator tbes not ask the same question of all respondents. Rather, focus group interviews represent a situation in which the partici- pants are stimulated to talk with each other on the chosen topic under the guidance of a moderator. The primary role of the moderator is to promote group discussion. Focus groups can be carried out in developing country settings.. but because developing country conditions often present constraints, re- .searchers must take steps to ensure the quality of the research data. These quality assurance steps may include ensuring that the recruit- ment process brings qualified participants into the discussions; the moderator functions as a facilitator rather than as an authority figure on the issue under discussion; and the results of group discussions are adequately recorded and analyzed by someone who has a clear under- standing of the goals of the research. There are other, indirect advantages to using focus groups as a re- search tool in developing countries. They provide a mechanism through which the researcher and the community cooperate in the solution of the community's problem. Focus groups also provide a means for research- ers to work with the project beneficiaq-the community itself-in all phases of prcect work from identfying issues, to developing and test- ing solutions, to preparing communication materials, and finally, to in- troducing an intervention. To explore the questions concerning product and communication strategies vis-k-vis the new weaning food, four sets of focus groups were conducted as indicated in Box 5.1. The first set of focus groups was held with two different audiences: mothers, and women who were active in the cottage industry of making and selling EKO. These initial groups were exploratory in scope. With mothers, the discussions turned to feeding practices with EKO and reac- Cecilia Cabaiero-Verzosa, Cci l M. pohnston, and Olabode Kayode 99 Box 5.1 The Use of Focus Groups in Developing the Product and the Communications Effort Question Area Focus Groups To Respond Product Questions Is fortification of Initial groups-mothers EKO 4ccepLAtble? Initial groups-EKO sellers Can mothers learn Recipe trial groups recipe, then teach others? Is tlhe use of malt Pre! and post-groups with in- flour acceptable? home product use test Will mothers prepare Post-groups with in-home test new EKO at bome? Communications Questions What communication Initial groups-mothers strategy will work? Initial groups-EKO sellers Materials pre-testing groups What materials will be Materials pre-testing groups effective teaching aids? tions to the addition of fortifying ingredients. Among EKO sellers, the groups examined cooking procedures and selling practices. With regard to the seller groups, there was keen interest as to whether these women could serve as agents for making a fortified version of their product or participate in the intervention in some other way. As a second goal, these initial groups examined possible messages and communication strategies for the interventiorL The second set offocus groups was conducted with mothers in conjunc- tion with recipe trials. Several fortifying options and cooking proce- dures were illustrated during the recipe demornr-rations. The focus groups provided a format for testing mothers' reactions to the new in- gredients and the resulting products. A third set of pre- and post-focus groups wrapped, like bookends, around an in-home product use test The test took place in four conimu- 200 Using Focus Groups nities and determined preference for two recipes that affected the liquid consistency of EKCO. One recipe included malt; the second had no malt. The discussions were held at a central location within each community, and mothers discussed their perceptions of the two pap products. Finally, Jbcus groups w.llh mothers were used in pre-testing graphic ma- terials. A flipchart describing the food, the ingredients, and the cooking process was developed, along with a scaled-down version in a flyer that mothers could take home. Focus groups gauged reactions to the print materials and also proved useful for finalizing the product name and the final positioning of the new, fortified EKO as a weaning food. Methodology Group Composition and Size In general, each group discussion included six tt ten individuals. Group members were homogeneous with respect to two different char- acteristics: nursing mothers who currently gave their children tradi- tional EKO, or EKO sellers who made and marketed the product in their communities. Group Locations All group discussions were held in the community. Often the village leader offered his residence as a venue for the group discussions I-is home was often centrally located for participants and usually had a quiet, inside room for the discussionr Because of the wide discrepancy in maternal behavior pattems between urban and rural areas, it was important to hold groups in both venues. Table 5.1 summarizes the composition, size, and locations for the four sets of focus groups. Participant Recruitment Recruiting mothers in Nigeria for focus groups posed some unique challenges. The interviewing staff usually arrived in the village a week or so in advance of the focus groups. Nigerian women are often dis- couraged from talling to strangers, so permission to interview them was first obtained from the village leader. Only then could interviewers visit the mothers to invite them to participate in a group discussion. To deternine whether a woman qualified, potential candidates were contacted in person and then led through a structured sequence of ques- tions. As previously stated, depending on the specific focus group, the Cadia Cabafero-Vnrosa, Cence AL Jlomston, and Olabod Kayode 101 Table 51 Composition and Location of the Focus Groups No. of No. of Rurl Urban Groups Member groups menbers locations Loaions () Inital Nursing 7 88 Alapa Alanamu mothers Ballah Baboko aged 20-40 Otte Erin-Ile Offa (1) Initial EKO sellers 6 51 Ballalh Alanamu aged 30-50 Oke-Oye Baboko Otte Erin-Ile (1) Recipe trials Mothers 8 apx 60 Alapa Alanamu Ballah Baboko Oke-Oye Erin-Ile Otte Offa (I) Pe-in-home Mothers 4 apx 40 Oke-Oye Alanam aged 15-40 Otte Baboko Post-in-home Mothers 4 apx 40 Oke-Oye Alananuu aged 15-40 Otte Baboko (IV) Material Mothers apx 12 apxt100 Oke-Oye Alanamu pre-test Otte Baboko women had to meet crtain criteria, such as currently nursing a child and feeding the child the traditional pap, or being in the busiess of maldng and selling EKO. An example of such a screening questionnaire appears in Box 52 The Focus Group Team The focus group team consisted of three individuals: a lead modera- tor, a moderator's assistant, and a marketing specialist The moderator (and observers and note-akers who also attended each focus group) were recruited largely from the corps of field researchers and supevi- sors who had worked on the baseline DMD research projects. With few exceptions, none had previous experience with the focus group research tecnique. Training for the staff induded organized sessions and role- playing :The lead moderator was a woman who had previously conducted individual interviews for the quantitative surveys and the ethnographic studies for the DMD prect She knew the Yoruba language and cul- ture and had the interpersonal skills of a good moderator. She could put people at ease, offer unconditional positive regard, withhold her own opinions, and encourage discussion. This person received on-the- 102 UTsing Focus Groups Box 5.2 Focus Group Scrdening Guide Good momiingfgood'eveniing. We are from the Uniiversity of forim We are in your vdige to mneet with some mothers to discuss child cart. We met your village chief and he has agreed to our bikidng with you. May we ask you a few questionLs? DAT: VILLAGE -Urbank__ Rural__ HOUSEHOLDNMAMR CHiILD'S NAME: AGE OF YOUNGEST CHiLD: __less than 3 years of age inore than 3 years of age IS CHILD CURRENTLY BEING FED OGI/EKO? Note to intrwier. OGIsis a paste made from fermented and sieved maize or guinea corn. A pap called EKO is prepared by addinig some of the ogi paste to boiling water until it thickens. If the mother hs a child less than three years of age who is currently fed ogleko1 please invte the mothmer to a meeting to be held: Date______ Time_______ Place_________ Otherwise, thakR her for talking with you today. job training that incuded organized sessions with role-playing and spe- cific advice on topic sequencing and probing. She also received written guidelines on moderator techniques and had a chance to ry out her techniques in pilot groups held as a pre-test for the topic guide. The lead moderator was assisted by a profIessor of health education from the University of Ilorin. He was the field manager for the conmu- Ce7ia Cubaffero-Verzosa, Cee aeM Johnston, and C'labodeKayode 103 nicatons component of the DMD project and was well-versed in the rationale for the focus groups. He attended the groups as an observer and prepared summary reports following each one. The marketing specialist, a US.-based consultant, also joined the local team during the focus groups This person worked with the U.S.-based multi-disciplinary DMD am that prepared the discussion guide prior to the site visit. Discussion Guides Discussion guides for the four sets of focus groups were drafted in the UJnited States by the marketing consultant, with input from the multi-disciplinar group. The guide relied on input from baseline data and other ongoing research. This discussion guide was then pre-tested and revised in Nigeria prior to conducting the actual focus groups. (Ihe moderator was also trained during this pre-test.) The first two focus groups in each set were used as a pilot. If it proved that major changes were needed in the discussion guide, the research team was prepared to delete these first two pilot groups from the overall analysis. Excerpts from one of the focus group guides are shown in Box 5.3. Conducting Focus Groups In general, a focus group moderator leads participants through a se- quence of topics that reflects an inverted pyramid. Very geneal behav- ioral and attitudinal issues are discussed first These are followed by topics of ever-increasing specificity, from child-rearing practices, to reac- tions, to concept statements, and preferences among product options. For the DMD Project, in the first set of groups, mothers began by dis- cussing the food and methods of feeding for children under three. The discussion moved on to sources of ogi and reasons for use. Eventually, the conversation was guided to reactions to a list of possible additives. Mothers completed the session by talking about credible sources of new information. Wlhenever possible, the focus groups were held indoors, with partici- pants seated in chairs in a circle. Although the home of the village leader was often pressed into service for this purpose, on other occa- sions, the group discussions were held outdoors in some communal living space. All sessions were audio-taped. Analysis of Results When focus groups are conducted in the United States, the moderator usually prepares the final report. This approach poses a problem in 104 Using Focus Groups Boc 53 Focus Group Discussion Guide for Mohes: Pta-In-Home Prod- uctTest PART ONE: RESEARCH PLAN Objectives To determine overall consumer preferece among three recipe varia- lions: Ingredientslcdpes A. Ogi cooked with palm oil and roasbed cowpea flour B. Above withmltflour added before serving C. Ogi cooked with palm oil and roasted cowpea flour, with malt flour mixtur re-boiled before serving Metiwdology The products will be tested in the home by motfeis who have children between the ages of six and eighteen months who are fed ogi Four sites, two urban and two rural, will be chosen and up to ten mothess will partici- pate in each group, for a total sampe of 4 mothers. Moers will be taught the recpes in focus group dscssions, to be held in a central location within their viage or urban neighborhood. For the recipes containing malt, mothers will be permitted to choose the method of prepartion tey prefer; that is, whether to add the cowpea flour while cooking the pap, or to add the cowpea flour along with the malt after the cooked pap has cooled somewhat Mothers will be given sufficent supplies of roasted cowpea flor and malt to last for ten days. They will test malted foklified ogi for five days and unmalted fortified ogi for five days. The order of tsting will be vaied between locations as fbllows: Alted unmalted FlOgi FlOgi Urbanl lst5days 2nd5days Urban2 2nd5days lst5days Rural 1 Ist 5 days 2nd 5 days Rural2 2nd5 days IstS days Mothers will each be given a cup and spoon at the start of the testing period for this product in order to encourage the desired behavior of spoon-feeding. continued on next page GaIa Cbaro-Verzos, Cecil o A Jhnston, and Oliadt wfayode 105 Box5.3 (Continued) Observers will be assigned to each test site in order to record data about in-home preparation and feeding practices of the test products, and to answer any questions the mothers may have. At the end of the testing period, the olbvers wilI complete an individual questionnaire with each mother before the final focus group discussion is held. A final locus group discussion (FGD) will be held with all participating mother in each test location to determine overall product prerences method of preparaion and feeding, quantity and frequency of feeding, and intent on the part of the mothers to adopt the new recipe. Below is an example of a focus group discussion guide used for both the recipe-teaching and materials-testing FGDs. PART TWO: MODERATOR'S GUIDE (sample) L Introduction A, Introduce team, purpose of visit B. Positioning-We want to know what you thtink about a new way of malcing ogi to help make your baby strong to cope with diar- rhea and other diseases. We are woridng on several ingredients and we want you to try the recipes in your home and tell us about your experience with them. IL Presentlngredients (Rotate order) A. Present roasted cowpea flour and explain how it is prepared. Asic * Have you ever seen it in this form? * Have you ever used it? * What might it be used for? * is it available in the market? * What do you think about adding it to ogi? B. Present malt flour and explain-how it is prepared. Ask the same questions as above. IIL DemonstratE Recipes (Rotate order) A. Get reactions to preparation steps/time/ingredients B. Reactions to appearance/consistency of finished ogi C Taste of -the finished ogi ciued on next page -:106 UsingFoaus Groups Box 53 (confinued) D. Overall impressions: * Would they give it to their child/family? * Would they add/omit anytfiing? *I this recipe better/worse/about the same as previous ones tried? Prepare a large enough quantity so that enough remains after tast- ing in order to make a comparison of the three recipes. IV. Volunteer Recpe Demonstration A. Ask for a volunteer to choose one of the recipes and prepare iL B. Why did she choose that recipe? C Reaction to preparation steps/time. D. Reaction of volunteer and group to finished producL E How can sle teach anothr mother to prepare the recpe. F. Problems expectedc V. Overall Preferences A. Rank overall preferences. Reasons. B. Of the first preferences: * Is it liked a lot/a little/not much? * Isit foraby/family? * Problems expected. VL Product Test Inst rucfiz A. Introduce observer who will come to their homes. B. Instruct which recipe to try first. C. Distribuie ingredients, cups, and spoons. D. Thank mothers for participating Cecilia Cubaiero -Verwso, CedileM Johnsfon, and Olabode Kyde 107 developing countries, however, since few trained moderators are avail- able who know both the language and culture and who are sufficiently conversant with social marketing principles to understand how the fo- cus groups can affect the program. Often, moderators are trained on- site from among health workers or interviewers. Consequently, the task of preparing the analysis gets divided among several persons. In this Nigerian project, a data plan was drafted prior to each set of focus groups. It dearly delineated what types of information were needed and how they would affect the program. As soon as possible after each focus group, the staff of moderator, observers, and note-takers met to discuss and concur on the key findings. A short summary report was prepared by a professor of health education after each group dis- cussion. A report gude for this purpose is shown in Box 5.4. Labor-intensive tape tanscriptions, often completed by two indepen- dent listeners, were not undertaken for these groups due to cost and time considerations. Although the group discussions were taped, the team referred to the tapes mainly to clarify points discussed. Findings Product/Product Use Features Fortification. The first round of focus groups revealed some key points on how to go about fortifying the traditional EKO. The EKO sell- ers were reluctant to tamper with their successful recipe formulas unless there was a large-scale mass media campaign to support the intro- duction of the new food. EKO sellers were therefore eliminated as pos- sible agents of change during the DMD research phase. Mothers, on the other hand, were already quite used to fortifying the pap themselves after purchase to sweeten it or add variety. For EKO ILERA, then, it was concluded that mothers accepted the concept of fortification and should be responsible for forifying the ogi themselves. Preferrred ingredients were uncovered during focus groups held at the recipe Wials. Of the four possible fortfying ingredients, roasted cowpea flour emerged the winner for several reasons. Cowpea, a com- mon household item, was readily available at the market and was af- fordable to villagers. The final roasted-owpea EKO looked similar to high-status infant foods like Cerelac, Nan, and Sinmlac. Finally, mothers believed that by drying and roasting the cowpea flour, its shelf life could be extended from two to eight weeks. Lering and Teaching the Recipe. Recipe-teaching trials showed that mothers could definitely leam the recipe and teach this newly-learned skill to other mothers. However, the teaciing of a new recipe meant 108 Using Focus Groups Box 5.4 Focus Group Report Guide; Motheus Groups L Introduction Comment on place/date/group (the mothers)/moderator. Comment on composition of group, e.g., older/younger members, total number, changes during the course of the FGD, and special circumstances that may have affected the group, e.g. outside distractions, etc. H. Current Feeding Habits Summarize mothue description of child feeding practices. Describe the age of weaning; foods given; frequercy of feeding. Probe their reasons for beieving that current feeding practices are desirable. Determine their cocept of the healthy child and the relationship betwiren feeding and the cld's health. IEL Ogi Preferences Summarize their overall preference. Identify the reasons for their choice. Describe in detail food preparation and feeding of the various recipes. Probe their concept of the 'cost" of the new recipe, in terms of monetary cost and other factors, including psychological resistance to change and time needed to prepare and feed the new weaning food. Describe mode of feeding, frequency, food handling, and food storage practices. Identify any negative perceptions about the recipe. IV. Ogi Additves Sunmarize answers and probe whether these additives are also good for children with diarrhea. V. Concept Test (where applicable) Comment on reaction to concept(s) tested. The concepts introduced were: A. This new ogi will make your baby light and active, because it conains cowpea, which makes your baby strong. With this new ogi, your baby wilI be better able to cope with illness. B. This new ogi will make your baby light and active, because instead of takdng too much water, the baby can take more ogv With this new og, your baby will be stronger after being ill with diarrhea. Specifically comment on overall reaction (positive/negative), believability, and what was liked or disliked. continued on next page Gea7ia Cabaflen-Venosa, cele M. Johnston, and Olabode Kcayode 109 Box 5.4 (continued) VI. Solid Food Intiduction Summaize mothers practices regarding the feeding of solids. Probe be- liefs about the feeding of solids during the first year of life. VIL Sourcs of rnmafion Describe sources of information about child care, specially feeding. Who are credible authorities? VIH. Impliations/Forward Action Indicate decisions made by the debriefing team as a result of the PGD regarding need (or no need) for additional FGDs, changes necessary in the moderator's guide, and changes/new concepts to be tested in future FGDs. that mothers would need to remember to add new ingredients or modify the traditional cooking process. The accompanying focus groups provided a chance to clarify a few issues, including the follow- ing that the additional ingredients used in the new EKO ILERA are readily available; that the food is easy to prepare; and that the cooking process entailed adding malt flour to make the EKO thin. Product Characteristics. The ogi of cowpea flour, red palm oil, and sugar had a very thick consistency. The DMD team was concemed that this would make the product unacceptable to mothers, most of whom practiced hand-feeding and force-feeding. The thick consistency would require spoon-feeding Since hand-feeding is a deeply entrenched prac- tice, the DMD team decided that it would be beyond the time and finan- cial resources of the prqect to promote a new fortified food and a new feeding mode at the same time- The nutritionists experimented with a unique solution commonplace in the beer industry, the addition of malt flour, which gave the final product a thin consistency. This product modification meant an additional ingredient in the recipe, and also an additional step in the cooking process. Furthermore, it meant teaching mothers how to sprout, dry, and grind maize or guinea corn to produce malt flour. This product modifitation was intro- duced to mothers in the third round of focus groups. An in-home prod- uct use test was preceded and followed by focus groups, which 110 UsingFocus Groups attempted to determine the acceptability, convenience, and feasibility of this additional step. During the in-home test mothers prepared the cowpea-fortified EKO two ways: with malt and without malt As expressed in the post-focus groups, the malt recipe was well received by mothers. Reports that it produced healthy, strong babies, stopped diarrhea, and helped babies sleep and play well were commonplace. Preparation at Home. During the at-home trials, mothers were given enough malt and cowpea flour to cook the new ogi in their homes for ten days. They were visited daily by DMD staff to observe whether the food was prepared and how it was cooked, as well as to provide assis- tance for any problems they encountered. At the post-focus group, a drawback was identified. Would the addition of malt prior to serving invite contamination? This problem was eventually solved by a nutri- tionist who suggested additional reboiling after adding the malt On this basis, the DMD team felt convinced that mothers could prepare the product effectively in their homes. Focus groups had been useful in moving a new product from concept stage to final fomL Mothers had supplied input on acceptability, pre- ferred ingredients, texture, and ease of preparation. Communications Developing a Communications Strategy. The initial groups developed the communication strategy in three critical ways: they suggested posi- tioring EO ILERA as a weaning food; they recommended mothers rather than EKO sellers as the target audience; and they isolated a mes- sage for the campaign. At the outset, the project faced a dilemma, in terms of how to position EKO ILERA-as a food for diarrhea or as a weaning food. Focus groups with both EKO sellers and mothers supported the weaning food strat- egy. According to sellers, attempts to make EKO 'medicinal' by adding ingredients for a child with diarrhea were old-fashioned and likely to detract from EKIO's use as a family food. Positioning EKO ILERA as a weaning food was judged to be consistent with mothers' beliefs and behaviors, while maintaning the status of EKO as a food for the whole family. A second issue that was unresolved prior to the focus groups was what role the EKO sellers would play in distributing the fortfied pap. Could they, for example, revamp their cooking procedure, add the forti- f3ying ingredient, and then market the product through their usual chan- nels? The focus groups argued agaist this tactic EKO sellers were reluctant to tamper with their successful recipes or to add any ingredi- Cecilia CfbJWTero-VCrzosO, Ceale AL, Johnston, wand Ol4abde Knyde III ents that would detract from EKO's status as a general family food. It was further discovered that mothers already are quite used to fortifying the pap themiselves. Consequently, it made sense to exdlude EKO0 sellers from the intervention and to make fortification the job of the mother. Lastly, the conmnunication strategy required a promised benefit of the new weaning food to encourage full participation of the target auaience. The mothers welcomed a concept statement that promised that fortified EKO0 would strengthent a child to cope better with childhood diseases. The "healthy baby' promise, as portrayed in the name EKO ILERA, became a message of the final intervention. Pre-Testing of Communication Materias. The final contribution of the focus groups was the refinemnent of communication materials. The pro- gram planned to use a flipchart far teaching and a flyer that mothiers could take home with them.L Three versions were tested for comprehen- sion in focus groups with nursing mothers. When mothers were able to enumxerate the ingredients and follow the cooking process, the materials were printed and used in the intervention. Notes 1. The Dietary Management Project was funded by USAID between 1985 and 2989 for the purpose of developing practical methods for either reducing or eliminating the ad-verse nutritonal effects of diarrhea in children. The project waS carried out in Nigeria and Peru. The BEALTHCQM Project, also funded over five years by UTSAID, provided technical assistance to DMD) in the develop- inent and pre-testing of trainLing materials. 2. EKO, a traditional weaningz food, a maize or guinea corn pap, was fortified with toasted cowpea flour, red palm oil, sugar, and malt The energy densityr of dthis recipe was 85 kcal/1OO gram wet weight, a considerable increase over the traditional EKO0 of 25 kcal/100 gram. The protein density supplied by the recipe was 2.2 grams, 100 gram wet weight, compared toonly 028 grams/100 grants for traditional EKO. 6 Rapid Appraisal Methods in a Diagnostic Assessment of Vegetable Seed Marketing John S. Holtzman John S. Holtzman presents the methodology andpfindings of a rapid appraisal study on vegetable seed marketing in Nepal. In this in- vestigation, he and his associates relied heavily on key informnant intemiews with seed growers, farmers' organizations, traders, dis- tnlmtors, and vegele growers. 7hey prepared structured inter- viero guides jor intmriewing different categories of informants. In addtion, t conducted direct obsemation of vegetable seed pro- duction, processing and saes, as well as group intrviemws with farmers and seed dealers. As the dhapter indicates, the investigators succeeded in obtaining a wide range of information, insights, and recommendationsfrom the study. The author discusses several shortcomings of the methodology and makes practical recomnmendationsfor thefuture. One sugges- tion is to temper the enthusiasm of investigators to seek out more progressive farmers, traders, and processors instead of the typical participant in thefood system. Another is to undertake a system- atic analysis of the secondary data to cross-checkfindings and con- dusions of the infeniews. Such an analysis should be done in advance and independently of the rapid appraisal exercise. Still an- other recommendation tihat Holtzman makes concers the interdis- dplinary composition of the research team in which economists work dosely with agricultural or commodity marketing specialists. T1HS CASE STUDY is based on rapid appraisal (RA) field work as ap plied to agricultural marketing research. The field work was carried out in Nepal by the author in collaboration with Nepalese analysts dur- ing June-July 1989. This activity built upon the findings of a collaborat- 112 John S. Holtzman 113 ing group of analysts who carried out an earlier, related rapid appraisal in October-November 1988. The latter paid special attention to a remote production zone not visited in the June-July 1989 effort. Both rapid appraisal exercises were diagnostic assessments of public sector programs and private sector efforts in the production and market- ing of vegetable seed, and were designed to identify opportunities for investment by USAID/Nepal in that country's vegetable seed sub- system. USAID/Nepal was especially interested in the emerging private vegetable seed industry in Nepal and the constraints faced by private actors in a subsystem historically dominated by public entities. Based on the findings of the later (1989) RA exercise and its rec- ommendations for pilot innovations in vegetable seed handling and pro- cessing, USAID/Nepal decided to incorporate a vegetable seed subsystem improvement component into an agro-enterprise project that it designed the following year. Background: RA in Agricultural Marketing Research The first objective of a rapid appraisal in agricultural marketing research is to provide a snapshot of how the current marketing system is orga- nized, how it operates, and how it is performing in accordance with criteria such as technical efficiency, operational efficiency, pricing effi- ciency, progressiveness, equity, and wholesomeness/nutritional quality of the food supplied to consumers. The assessment of performance relative to such qualitative and quantitative norms provides the basis for problem identification and diagnosis. The experience of the analyst comes into play in assigning prorities to problem areas for further in- depth applied research. In addition to providing an accurate snapshot of the current situation, a marketing rapid appraisal should focus on examining forces for change and improved productivity, as well as on identifying successful and creative participants in the food system who are wilLing to take risks and experiment with new technology, production and management methods, and institutional arrangements. This examination of the dy- narnic evolution of marketing systems requires analysts to seek out some participants who are progressive and searching for ways in which they can improve productivity ffirough better organization, manage- ment, information, and technology. Purpose and Scope of Marketing RA Exercise A long-term objective of the govemrunent of Nepal and donors such as USAID is to increase domestic production of vegetable seed in order to 114 Rapid Appraisal Methods in a DiagnosticAssessment substitute for imports from India and other countries, and to export selected vegetable seed varieties (especially open-pollinated temperate types) that cart be produced competitively by Nepalese small farmers. With this objective in mind, from mid-1988 to April 1990, the Agricul- tural Marketing Improvement Strategies Prqoect (AMIS), with funding from the Bureau for Research and Development of AID/Washington and USAID/:Nepal, partcipated in an assessment of the vegetable seed subsystem in Nepal. The assessment focused primarily on the fledgling but vibrant private vegetable seed industry, and on policy, regulatory, institutional, technological, and management constraints on its emergence. The two discrete rapid appraisal studies carried out by AMIS in col- laboration with Nepalese analysts in October-November 1988 and in June-July 1989 were related to two USAID/Nepal-funded activities. One of these was a longer-term applied research program on agricul- tural marketing. The other was an area development project in the Rapti Zone of Midwestern Region (The Rapti Zone Development Project), which began in the early 1980s. The applied research program began with an exploratory trip to Nepal by the RA team leader in August-September 1988. During this penod, the key study objectives were identified, the research program was designed, study implementation responsibilities were assigned, and a timetab!e for completion of different tasks was elaborated. Given resource limitations and the difficulties inheret in managing formal surveys in remote vegetable seed production zones of Nepal, rapid ap- praisal was chosen as the data-gathering method. Figure 6.1 depicts schematically how rapid appraisal can be used in an applied research program, and the linkages among the activities of RA, applied research, policy analysis, and monitoring and evaluation of policy reforn or other interventions in technology, management, institu- tions, and organizations. The Rapti Zone Development Project has focused inueasingly in re- cent years on promoting production of high-value cash crops by small farmers. The high value-to-weight (and volume) ratio of vegetable seed, plus excellent isolated growing conditions in remote rural areas of the Rapti Zone, make vegetable seed an excellent potential source of cash income for smallholders. In addition to the two RAs in Nepal, the AMIS Project carried out two related RA market surveys on behalf of USAID/Nepal and the Nepalese governmsnt in Bangladesh (October-November 1988) and in Thailand -anuary 1990). These surveys were designed to inform Nepalese pri- vate vegetable seed companies, public sector agencies mandated to pro- mote exports (the Nepal Trade Promotion Centre), and IJSAID/Nepal of John S. Holtzman 125 supply and demand conditions for temnperate types of vegetable seed, and potential export opportunities for Nepal in Bangladesh and Thai- land. Description of Field-Based RA Methods Rapid appraisal of the vegetable seed subsystem in Nepal relied heavily on key informant interviews with selected farmers, wholesale traders, retail seed dealers, public officials, extension agents, and knowledgeable observers. The RA team also visited farmers' fields, vegetable seed processing facilities, and seed dealers' shops. An important part of the larger applied research progrm, however, was aembly and tabulation of extensive secondary data on vegetable seed production, marketing, and prices, which had to be compiled from numerous sources and sev- Figure 6.1 Schematic Overview of Rapid Appraisal and Applied Research Linkages Perceved food systm problem or oppxtuity (Rneah sporwor initiative) P Pzoblem identification w - ,* ,, Pupi~~~~~RRdappraid s .. .. , %Xt~(Kowledge eeation and problem diagnosis) Piqr t B| Ma&eigIst= P ueitm I I innoraSons . . _ ; # Ptsentation of RAfindis and feedback from reseawk sponsors MonitoTiO d and Designof apprem / t kwingand evaluation \ evaltion Implementation of AR Program Formal and informal surveys Periodic RA Peid working papers InterAacon with policmakers Study update ([sing HA) Note: Unbroken lines indicate flow of applied research activities. Broken lines indicate.feedback loops. Boxed off items are project outputs. 116 RapidAppraisal Methods in a DiagnosticAssessment eral agencies. Bringing together such a large volume of secondary data into one paper (Holtzman and Munankami, 1990) took several monts. These data were presented primnarily as annexes and used to supple- ment and complement the findings of the field studies. Developing Structured Infornazl Intewiew Guidelines Approximately one week was devoted to developing and refining stuctured informal interview guidelines to be used in the key informant interviews of the field work This was a team effort in which the expa- triate consultant and chief Nepalese investigator drafted the guidelines, wlith were then crtiqued by the Nepalese agro-input specialist who had participated in the October-November 1988 RA effort (Chilton and Shrestha, 1989). This individual had carzed out numerous interviews with producers, public officials, and dealers during this earlier field work, and his technical knowledge of vegetable seed production meth- ods and the NepaIese seed industry enabled him to verify that the tech- nical content of the structured informal interview guidelines was accurate and that the right technical questions were being asked. Actual qutestions from the interview guidelines used in interviewing vegetable seed traders and vegetable seed producers are shown in Boxes 6.1 and 6.2. The interview guidelines were designed to serve as a struc- hired checkist, however, rather than a formal instrument Typically, the questions were prepared in abbreviated form rather than being- fully elaborated, and interviewers were free to depart from the guidelines, particularly when opportunities to probe for unanticipated responses arose-provided that most of the key questions in the guidelines were addressed. This approach allowed for flexibility in uncovermg new knowledge and in probing producers' motivations, opirnions, and per- ceptions, while providing sufficient structure across interviews to en- sure comparability of interview findings. Selecting Key Informants As Kumar has argued (Kumar, 1989), key informants are selected because they possess special knowledge or insights by virtue of their position in the economy or govenmment, or their experience in studying a particular problem or topic In food systems research, key informants can be characterized as participants or knowledgeable observers (Holtzman, 1986). Such participants indlude farmers, first handlers, wholesale traders, processors, storage and transport agents, and dis- tributors who are actively and productively engaged in commodity sub- stms. John S. Holtzmxan 117 Box 6.1 Infonral Interview Guidelines: Vegetable Seed Traders Location 1. Brief histoxy of involvement in trade * Date and place it began * Iitial product mix and change over time 2 Description of business * Colect seed from farmers? * Sell seed to otier traders, exporters? * Sell seed purchased from other traders, finms at retail level? * Itnerant colection/sale? * Collectin/sale from fixed place of business (shop)? 3. Seed purwcases during last two crop seasons -wde and summer) Seed type Quantify Price Seller type lation 4L Use of contacts Seed Location # Farmer Quantity Price Deli Inpufs Type Quanfty Credit? 5. Do you offer premiums/discounts for quality differentials? If so, describe. 6. Do you provide extension/supervison services? Describe number of visits (per crop) and types of extension input 7. Do you hire your own extension/supervision agents to provide super- vision? If so, descnbe their trainig/experience. How long are ffey in your employ? How much do you pay them? Are they effective in work- ing with local farmers? How do you supervise them? continued on next pag 118 RapidAppraisalMethods in a DiagnosticAssessment Box 6.1 (cntinued) & Non-contract purchases: Do you use spot markets or purchase directly from the farm gate? Location # Sellers Seed Type Quantity Price Quality 9. Seed sales during the last crop season Seed Type Quantity Price Buyer Type Location Credit 10. Sales potentiak Could sales be expanded? Do some potential customers go unsatisfied? Are there shortages of particular types of seed (varieties, hybrids)? 11. Describe arnr promotional efforts. 12. Processing: Do you dlean, sort, and grade vegetable seed? Location Type/Technology Cost Observations 13. How do you package seed for wholesale and retail sale? * Wholesale: tpes, costs, and effectiveness of packaging * Retah types, costs, and effectiveness of packaging. * Repackaging? Costs, methods. 14. Storage Seed Type Location Method Period Cost Losses 15. Have you experimented with improved storage methods? If so, de- scribe and disus whether you were satisfied with the results. 16. Transport methods, costs of transporting to different locations, losses. Have you experimented with improved packaging or bagging to reduce losses in transport? If so, describe 17. Relatonships with suppliers: Who, whee, length of relationhiup, vol- ume commitments, price negotiation, mode of shipnmnt and packaging Are you satisfied with the relationships? Have you ever considered alter- native suppliers? continued on next page John S. Holttman 119 Box 6.1 (continued) 18. Buyer types and relationships farmers, other traders, retail seed deal- ers, Agricultural Input Corporation (AIC). Informal contracts, long-stand- ing relationships. Percentage distribution of sales by buyer type, prices received, other services rendered. 19. Feedback, if any, from buyers/growers about seed quality, purity, gSer- mination rates. 20. Marketng costs and margins * Most covered above. Note other costs. * Trade license or export fees. * * Handling costs not induded in above costs. * Infoal fees, payments. 21. Place trader's problems in rank order and ask him to propose solu- tions. 22. Perceptions/opinions of roles of AIC and Vegetable Development Di- vision (VDD) in their areas. How could their services be improved? Does the AIC floor pnce for vegetable seed impose a constraint? Are private traders able to use AIC facilities (cleaning storage)? Does AIC provide extension to growers? 23. Describe export/import operatiors (irnluding cross-border trade with fva): * Buyers/sellers (are they established7). * Costs of exports/imports. * Communications with inporters and knowledge of prices in ex- port markets. * Export/import share in total seed business. * Problems. 24. Note opinions/perceptions of farmer organizations specializing in vegetable seed production and narketng Do traders intend to organize farmers? Are tiere advantages to working through fanner organizations rather than with individual farmers? 120 Rapid AppraisalMethods In aDiagnosticAssessment Box 62 Informal Interview Guidelines Vegetable Seed Produces 1. Basic farm data * Farm size * Laborers' availability (ful-me, seasonal) * Principal crops by season Z Vegetable seed production history * Whm did commerci production begin? * Descrbe how it happened: AIC or trader contract? * What have been the changes in area cultivated and crop mix over timne? 3. Vegetable seed crop mix, 1988-89 and 1987-88 Year Crop Area Production Quantity Sold Sales Price 4. Input use (including hired labor) for 1989 and 1988 seed crop Availability Crop Price Source Quantity Quality T7nely? Application * Observations about problems 5. Production practices and production problems: *Preparing seed bed * Transplaning * Rouging * Irrigatin& controlling pests * Harvesting continued on next page John S. Hol tman 121 Box 6.2 (amntinued) 6. Post-harvest handling practices and problems * Harvestingmethods * Dzying o Transport to storage place * Sorting/grading/cleaning * Storage methods (packaging) 7. Contracts * Inputs provided Crop Input Cost CreditDate DdeieyDate Application * Production supervisionr number and timUing of visits, infomaon conveyed, assessment of value of information conveyed. * Outputsupplied Cop Quantity Price Ddivery Date Place Postliroest Reqs. * Grower observations on advantages/disadvantages of contracts - Quality, timely availability, useulness of inputs - Quality, timely availability, usefulness of extension - Fairess of price - rnely payment Provision of hiputs on credit * Recommended changes for fuhtre 8. Sales not on contract Crop Locafion Dale Buyer Quantity Prices Credit? continued on next page 22RaupidAppmisal Methods in aDiagnostic Assessment Box 6.2 (continued) 9. Intentin for 199 Crop Area Contract Inputs Output Pre 10. Evaluation of Agriculural Input Corporation (AIC) and Vegetable De- velopment Division (VDD) * AM contract price, quality/availability/timeli of inputs * VDD 11. Use of revenues from vegetable seed sales. Projected use of 1990 rev- sues. 12. EIportance of revenue from vegetable seed sales relative to other en- * Gais: quantity sold, sales revenue * Seasonal labor * Lvestock sales * Fruit and vegetable sales 13. Perception/opinion of fanmer organizations (FOs) * Feasible? * Desirable? Perceived advantages. * How best to orgiaize? * Who would manage? * Role of such FOs. * Could FOs effectively manage improved harvesting and process- ing equipment? * Contracts between traders and FOs? Perceived advantages and disadvantages. John S. Holf_mr 123 The Nepal vegetable seed RA team concentrated initay on inter- views with key informants representative of different scales of operation and types of technology and economic organization. These key infor- mants included vegetable seed growers, wholesale seed traders, retail seed dealers, officials of the Vegetable Development Division of the Nepal Ministry of Agriculture (VDD) and of the goverunent-sponsored Agricultural Inputs Corporation (AM9, extension agents, and selected horticultual producers who use Nepalese vegetable seed. In order to gain valuable contextual information, the team also interviewed knowl- edgeable observers, who included expatriate advisors, representatives of donor agencies, and local analysts who were not associated with the government An important secondary task of the RA team was to seek out as key informants selected progressive individuals in the vegetable seed sub- system, who provided assessments of experiments in progress and indi- cations of how the subsystem was evolving in a dynaric sense. An example of such a "progressive informant" is a large-volume wholesaler or processor who is experimenting with formal or informal contracts with first handlers or producer groups. Another example of progressive informants is producers or producer groups who are experi- menting with alternative institutionaI arrangements (such as producing under contract to a wholesaler or processor), new harvesting and post- harvest handling or processing technology, and improved organiza- tional forms (such as farmer marketing groups for achieving scale economies in input supply or commodity storage and transport). Conducting the Interviews and Recording rindings The Nepalese senior analyst took the lead in most of the interviews. He opened with a brief explanation of the puipose of the inquiry, in- duding mention of the auspices under which the study was being con- ducted. No mention was made of explicit project or program assistance to informants. (As a general rule, interviewers should avoid promising projects or credit programs, even if the RA is being conducted as a feasibility study for an anticipated intervention.) Interviews with farmers and most shopkeepers were conducted in Nepali. Govemrment officials and formal sector seed traders were usu- ally fluent enough in English so that these interviews could be con- ducted in English. In interviews carried out in Nepali, the junior Nepalese analyst typically translated questions and responses for the benefit of the expatriate analyst He was able to do this simultaneously in most cases, or after questions had been asked or answered in other 124 Rapid Appraisal Methods in a DiagnosticAssessment cases. Since the team had developed the interview guidelines before the RA field work began, only brief reference needed to be made to the questions posed to the various respondents. The two RA team members other than the senior Nepalese analyst-one a junior Nepalese analyst and the other an expatriate- took shorthand notes during the interview in cases in whic h the respon- dent did not appear to be intimidate by note-taking. When respon- dents appeared uncomfortable, however, or in fields, shops, or marketplaces where note-taldng was cumbersome, the analysts recorded interview findings as shortly as possible after the interview was con- duded. In this way, little of the detail of the interview was lost Write-Up of Results The expatriate analyst drafted much of the report and left the collabo- rating local analysts with a worldng draft of most of the report. This provided an adequate base that the local analysts could expand, modify, and refine. Presenting RA Findings The draft RA report was disseminated three months after field work was completed. Although this was a longer-than-desirable delay, it was acceptable, given the effort required in tabulating and analyzing second- ary data that were used in the body of the report and in most of the 17 annexes. Despite the delay, key findings of the field research were presented to USAIDD/Nepal after the expatriate analyst had drafted much of the report, and before he left Nepal. A two-hour briefing was held within ten days of completion of the field work A very desirable technique of RA studies of agricultural marketing systems is for study sponsors to disseminate the final report widely to interested public and private sector parties and convene a follow-up workshop to discuss its policy and program implications. If an expatri- ate analyst or two participates in the field work, this may require bring- ing her/him back, albeit at considerable expense. The expense may be well worth it, however, if the RA findings are intended to effect policy reform, regulatory streamlining, or interventions in technology, insti- tutional arrangements, or management. Without a final workshop among RA team members and interested local parties, a RA report risks collecting dust, as have so many short-term studies in developing countries. Five months after the field work was completed by the two agricul- tural econonists (Holtzman and Munankarii), the senior Nepalese ana- John S. Holtzmmn 125 lyst (Munankani) and the expatriate vegetable seed specialist (Chilton, who had participated in the filst field study in late 1988) presented the findings of both the first and second rapid appraisals to a broad public and private audience. Both analysts, sewved as subject miatter resource specialists at a national workshop on private agro-enterprise develop- ment held in KCathmandu in November-December 1989. Problems and Possibilities for Improvement Potenial Bias in Informant Sdelcion Rapid appraisal is often criticized as being plagued by bias in infor- mant selection and because of its inability to generate precise, statisti- cally valid estimates. This issue is discussed above, where it is argued that analysts examining agricultural marketing systems need to purpo- sively seek out key inform-ants with special knowledge, insights into system organization and operation, and a willingness to takce risks and experiment in order to improve their productivity and system perfor- mance. The enthusiasm with whic-h rapid appraisers seek out the most progressive and skillful farmers, traders, and processors, however, needs to be tempered with interviews with less progressive yet more typical participants in the food systemt. Knowledge of new technology, management, and marketing methods and institutional anrangements has to be balaniced with an appreciation of constraints facing a broad range of participants. Knowledge of what is possible and new ways of doing things does offer insight into those constraints that need to be dealt with and those facilitating factor-s that must be strnmgthened in order to achieve higher levels of productivity. As a general rule, rapid appraisers need to interview as many key informants as possible at critical stages of commodity subsystems in order to obtain as objective and unbiased an understanding as feasible under time and resource constr-aints. Skillful cross-checking of re- sponses within individual interviews, across interviews with partici- pants at the same stage of the food system, and across interviews with participants at different stages of the food sysem can help to identify misiformtionand atypical behavior, practices, and op-inions. In as- sessmentts of constraints (rank ordering) and in opinionx and perception questions, respondents at different stages of the commodity subsystem cannot be expected to agree. More-over, different interpretations of con- straints may not be readily reconcilable. This reflects parochial perspec- lives and diff-erent perceptions of problems and opportunities. Reports with divergent results, however, can provide fertile ground for design of further applied research. 126 RapidAppraisal Methods in a Diagnosfic Assessment Organizational and Coordination Limitations on Research The difficult terrain and dispersion of vegetable seed production sites in Nepal made national geographic coverage impossible. Under these circumstances, the RA field research was made more manageable by restricting the effort geographically to a given region. (However, cover- age can be expanded by using multiple teams in different vehicles.) The RA exercise in Nepal used one team to cover several areas of the Terai and lower Hils. The earlier field work done in August-September 1988 by the two vegetable seed specialists was focused on one remote pro- duction region in the mid-Hills. However, it would have been better had the field work of both teams of analysts been carried out simulta- neously and had report writing been coordinated so that a single draft report was produced. Furthermore, one or more additional teams could have strengthened the RA and given it a more comprehensive geo- graphical scope. Composition of RA Teams The field work of both the June-July 1989 and October-November 1988 exercises was conducted by teams comprised of analysts with simi- lar training and disciplinary expertise. The first RA was carried out by analysts trained in agronomy and experienced in seed husbandry and trade. The second was conducted by two agricultural economists and an economist. However, it would have been better if at least one seed specialist and one economist had collaborated on each interviewing team. This would have provided better balance on technicaL agricul- turl, and economic issues. Resource, Time, and Geographical Obstacles to Widening the Scope of Study Nepal poses very difficult problems for researchers wishing to pen- etrate deeply int rural areas. The team that performed the June-July 1989 study did not trek much off the road to visit farmers. This was due partly to a geographic focus on the Terai and lower Hills, where more farms are readily accessible to roadways. It also resulted from a focus more on seed marketers Etan producers. However, the earlier (October-November 1988) study fo- cused more on producers and on government officials and extension agents worldng in isolated areas. For this effort, analysts were required to trek long distances to visit isolated villages suitable for vegetable seed production. In this sense, the two studies were complementary. Still, John S. Holtzman 127 more field research in more remote production zones would have strengthened the entire rapid appraisal effort, but time and resource limitations restricted this type of field work. Greater Partcipation by JuniorAnalysts In. the Nepal RA, team size was constrained by the fact that sufficient funds were not available for rental of a larger or a second vehicle. For this reason, only one junior analyst participated in the field work, which was unfortunate, since RA can be professionally broadening for young professionals who need field experience under the supervision of expe- rienced senior analysts. Key Findings and Resulting Measures The results of the two RA exercises became the basis for a reconmmenda- tion by the AMIS Project for the establishment of an agri-business strengthening project, which would also contain a vegetable seed com- ponent. USAID/Nepal acted on this recommendation and designed the pilot innovation during the first half of 1990 (Chilton, 1990). Because the RA exercises had identified vegetable seed handling and processing as key constraints that lowered the quality of Nepalese seed in both do- mestic and foreign markets (particularly Bangladesh), a key feature of this project focuses on experiments with improved seed harvesting, han- cling, processing, and testing technology. In large part, these can be adapted from other South Asian economies such as Thailand and Tai- wan (China). A second constraint identified during the RA studies was the poor organization of the private vegetable seed industry in Nepal during a period when opportunities were emerging for the private sector (at the same time that assistance provided by public agencies is being scaled back). The AMIS Project therefore reconmmended that the recendy formed Nepal Seedsman Association be strengthened through USAID/ Nepal assistance to identify and screen improved technology for veg- etable seed harvesting and processing, represent interests of the private seed industry in public fora and in national seed legislation, and con- duct domestic and foreign market research. Finally, limited NepaIese knowledge of other South Asian markets for vegetable seed was also identified as a key constraint As a result, USAID/Nepal plans to fund marketing studies by private entrepreneurs and selected public officials (principally in the Trade Promotion Centre) in nearby South Asian countries. The AMIS Project has already carried 128 RapidAppraisal Methods in a DIagnosticAssessment out reconnaissance studies of the vegetable seed subsectors in Bangladesh (Zaman, 1989) and Thailand (Welsh and ICayastha, 1990) to begin this process. USAID/Nepal has also funded an Agro-Enterprise and Technology Systems Project to provide support to agro-enterprises in Nepal. This is the contractual mechanism for implementing pilot innovations in veg- etable seed technology, for strengthernng industry organization and rep- resentation, and for conducting foreign market intelligence. Suggestions for Users of RA in Agriculhtual Marketing Re- search Several lessons emerged from the rapid appraisal studies and from the reconnaissance surveys in Bangladesh and Thailand. Some of these are broadly significant for agricultural marketing research in general, while others are quite specific to rapid appraisal methods for field research. Nepalese Agricultural Marketing in an International Trade Context One important lesson was the need to examine vegetable seed mar- keting and trade in a regional (South Asian) context Although Nepal has exported modest quantities of radish seed to Bangladesti, it faces stiff competition from Japanese, Korean, and Tai- wanese competitors in that markeL The competing suppliers export top quality seed having very high gerinination rates and purity to Bangladesh in attrctive packaging (that is, tins). The Nepalese export- ers ship a lower quality, albeit slightly cheaper, product to Bangladesh, but they need to upgrade seed handling, processing, packaging, storage, and shipping practices in order to improve their competitiveness. The Thai market also offers limited possibilities for Nepal in the short term. However, Thailand imports approximately 60 percent of the vege- table seed sold commercially from New Zealand, Australia, the United States, Taiwan (China), China, Japan, and other economies. In addition, a local vegetable seed industry has emerged in the highlands of north- east Thailand. The lessons of the vegetable seed RAs, as well as many otier recent agriculturl marketing studies, are that the international marketplace and world supply and desnand conditions cannot be overlooked, even by small, landlocked, isolated countries such as Nepal. An examination of world production and intermational trade suppliers, flows, prices, and practices needs to be built into many agricultural marketing studies that may seem at first glance to be solely domestic studies. As world mar- kets become increasingly integrated during the 1990s, this will become an even higher priority. John S. Holtzmn= 129 Secondary Data Collection and Analysis Compilation and tabulation of data on the vegetable seed subsystem was carried out independently of the RA field studies. These secondary data were integrated, however, into the final RA and proved to be a valuable complement. Unfortunately, the data were not available to the researchers before the field work was undertaken. Even in those studies where this is the case, it usually makes sense to provide resources for local agencies or firms to compile, tabulate, and analyze available sec- ondary (and perhaps selected primary) data. Building a commodity subsystem data base can prove useful for analysts and policy-maukers in future work. In cases in which secondary data are readily available, this material can be examined before field work begins in order to provide better knowledge about historical patterns and current supply-and- demand conditions Key analytical findings can then be integrated into the RA report Looking at secondary data forces analysts to take a longer-term his- torical perspective. If data cannot be obtained before ariving in- country, allocating somne time for gathering and analyzing available secondary data before beginning the field work is strongly recom- mended. It also pays to make some effort to obtain data tat may appear inaccessible on first try. If necessary, analysts are advised to hire people to copy numbers out of abstracts when they are not available on computer diskettes. Sometimes an incentive payment in the form of a consulting contract to 'ownes" of public data may be necessary.2 Devising Guidelinesfor Structured, Informal Interviews Before beginning field work and during exploratory pre-testing the investment of considerable time in developing workable, structured, in- formal interview gudelines will usually have a high payoff. For the most part, this process will follow the literature review and analysis of initially available secondary data. In addition to basic factual questions, interview guidelines should address issues, knowledge gaps, and themes that emerge from the literature review and data analysis. It is also strongly reccmmended that private sector marketing agents, par- ticularly wholesale traders and importers/exporters, be queried about their perceptions of policy and regulatory barriers, system constraints, and untapped, under-exploited, or emerging opportunities. It is strongly recommended that a written guide be prepared, al- though analysts will not always be able to consult it when interviewing informants, who may fear misuse of the information, or who may only provide accurate information if the interview is conducted confiden- tially (without recording precise responses). Pteparing a written guide 130 RapidAppraisalMethods in a DiagnostcAssessment forces one to think in terms of the best sequence of questions or streams of questions based on the likely aIternative responses to key questions early in the interview. A guide also assists later in recording informa- tion, which should be done as soon as possible after an interview is completed. After interviews are concluded, or at the end of each work- ing day, discussions of interview findings with colleagues participating in the RA are strongly encouraged in order to compare interpretations of what informants reported. The informal interview guidelines may be attached to the RA report as an annex. It should be noted that, in all likelihood, an analyst will only rarely be able to ask any one informant all the questions in the guidelines. What will emerge, however, is a composite understanding of the activities and perceptions of different actors in the marketing system. Conducting interviews with a minimum of 25-30 participants at each major stage of the marketing system is recommended. This sample size is large enough to permit statistical analysis of some variables if deemed necessary. In many studies, however, fewer interviews will likely be carried out per informant group due to time, resource, and logistical constraints. Since the objective of the RA is to sharpen problem identifi- cation and diagnosis, rather than to do statistical analysis of findings from typically non-random samples, smaller sample sizes need not cause undue concern. When precise information is required for certain key variables at a particular stage of the marketing system, the RA team should consider conducting a mini-survey, in which the sample would be a =mnium of 30 respondents (Kumar, 1990). Interviews with key infornants should be conducted in private, pref- erably in quiet settings. When key informants are contacted at market- places or in their fields, interviews may be brief (less than 30 minutes, and as short as 5-10 minutes in some cases). Follow-up interviews away from where the informants are doing business may be scheduled for a later time, though this may prove difficult in RA. Effective interviewing of key informants requires tact and diplomacy, sensitivity to the respondent's perceptions and needs, persistence in ad- hering as closely as possible to the structured informal interview guide- lines, and the abiIity to think on one's feet and to follow up either vague or stimulating unanticipated responses with skillfully sequenced, prob- ing questions. While some analysts have the personality traits, intellect, and predisposition to become excellent key informant interviewers with relatively little training most analysts will have to develop effective informal interviewing skills through training methods such as role-play- ing, exploratoxy practice interviews with informants, and on-the-job in collaboration with an experienced interviewer. fohn S. Holtman 131 In recording key findings after an interview is completed, the team participants in an RA exercise should collectively recall specific re- sponses and, where necessary, disus their implications In the case of some respondents, what they do not say or the way in which they re- spond to particular questions can be as illting as what they actu- ally say. Since the interpretation of subtle nuances is more of an art than a science, it is useful for team members to disuss their interpretations of interview findings shortly after an interview is concluded, rather than to wait until a later date, such as the time when findings are being written up. Visiting Foreign Terminal Markets Agrultural marketing research usually requires site visits to farms and markets, and interviews in production zones, marketplaces at higher levels of the distribution system, and in terminal markets. As donor interest in crop diversification and export promotion inceases, however, the terminal market is often in a second country or in several other countries. In these cases, the analyst is well-advised to go to at least one of those other markets. This is well worth the extra trouble and expense, because one gains a more complete, first-hand picture of the competing export suppliers to this market, recent changes in market share (and reasons underlying changes), and intermediate or end-user (processor, consumer) perceptions of the quality, availability, price, reli- ability, and timeliness of delivery of the product under study relative to the competition. In the June-July 1989 RA study, the participating analysts did not visit foreign markets at the time of the field work or shortly thereafter. How- ever, based on their existing knowledge and on subsequent visits, mar- ket profiles of the Bangladesh and Thai vegetable seed subsystems were later prepared by the expatriate vegetable seed specialist who had par- ticipated in the earlier RA (Chilton and Shrestha, 1989) and by the Nepalese agricultural economist who collaborated with the author. These foreign market profiles were part of the broader research program managed and coordinated by UISAID/Nepal and AMIS. They provided useful and timely information that was incorporated into the design of the USAID/Nepal agri-business strengthening project. Report Writing To retain their crispness and policy relevance, RA findings need to be captured on paper as soon after completion of the field work as possible. If write-up drags on for six months to a year, production and marketing 132 RapidAppraisal Methods in a DiagnosticAssessment conditions, as well as selected policies and regulations, may change. Expatriate analysts are strongly encouraged to write up their sections of the report, or at least detailed notes of key findings before leaving the country. It is preferable for report-writing to be divided up among all of the participating analysts. When analysts have different disciplinary skills, they should of course be asked to write up findings in their area of expertise. In most cases, however, a chief writer and editor will need to be designated. This person will have the authority to edit sections writ- ten by other analysts, as wel as the responsibility for organizing and integrating individual sections into the final report In some instances, an outside consultant (who is typically an expatriate) may be able to play this role most effecdtveIy, since she or he has no other responsibili- ties in that country. In an earlier exercise in Liberia, the author did not play this role and local analysts were left with the task of drafting most of the RA report This led to delays in producing the report, minimal editing, and unevenness in the draft that could have been avoided in large part had the author and other partcipating expatriate analysts played a more active role in report productionL Briefing the Client Briefing the study sponsor soon after completion of RA field work is strongly encouraged for several reasons. Fist and foremost, RA is driven by client needs and timetables. A key advantage of RA is that it generates timely and policy relevant output at relatively low cost Sec- ond, timely discussions of RA findings allow the sponsor an opportu- nity to shape the final report and raise questions for selective follow-up research. The seminars give the sponsor the chance to challenge pze- lmnary findings, which at the least foxce the RA team to consiaer alter- native interpretations and to be careful not to state their preliminary findings too definitively. Tentative findings, depending on their impor- tance as an input into policy-maidng or program planning and monitor- ing& may require further, more focused, applied research. Third, requiring the RA team to present findings early on disciplines them to tecord their findings as shortly as possible after completion of fied work This is pragmatic, in that it discourages procrastination and enables analysts to capture findings when they are fresh. Obviously, the team needs enough time (10 to 14 days) to prepare an adequate draft report or at least a detailed outline for presentation. Shorter deadlines may create more problerns than they solve if they induce a poorly orga- nized, uneven draft After the seminar with the RA sponsor and a week or so of distance from the draft report, the HA team can begin to finalize the report from a better perspective. Johm S. Holfzman 133 Team Composition and the Value of Junior Analysts RA teams can be as large as four or five people and mixed in compo- sition, in terms of age, experience, and disciplinary expertise. However, RA can be an especially good trairnng exercise for jurior analysts, who typically have had little fieId experience. A junior analyst can learn a great deal in a short time by worldng with a skilled senior interviewer and analyst. The former can observe the interviewing techniques, ques- tions asked and their sequence, and methods of probing for further information used by senior analysts. They can also learn from discuss- ing RA findings and interpreting informants' responses with senior ana- lysts. Junior staff may also add a fresh perspective to senior researchers who may have done miiany similar exercises and lack spontaneity or originality. Interview teams of three or four are workable, although some inti- macy is sacrificed when more than two analysts work together. In larger interviewing teams, one or two junior analysts are typically paired with one or two senior investigators. As the RA progresses and junior analysts master interviewing techniques and fuly understand the objectives of the RA exercise, teams can be split into smaller units to maximize geographic and informant coverage, as well as to give junior analyst an opportunity to lead interviews. If junior analysts have not previously participated in formal or infor- mal surveys, they typically gain an appreciation of the potential pitfalls in interviewing private participants in the food system, many of whom are illiterate and not strongly numerate. Younger, less-experienced staff begin to appreciate how difficult it is to gather valid and accurate infor- mation from farmers and traders and the demands this places on inter- viewers This can prove useful when they themselves design a formal survey. Conclusion The rapid apprisal of the Nepal vegetable seed marketing subsystem conducted in June-July 1989 was one of several RA exercses carried out as part of an applied research program on the subsystem and related export opportunities. It used in-depth, key informant interviews with subsystem participants, policy-makers, and knowledgeable observers, as well as site visits to farms, markets, and processing facilities, to identify and diagnose constraints on improved performance of the vegetable seed subsystem. Structured informal interview guidelines were devised and helped to guide the inquiry. A major strength of the vegetable seed RA was that it was an imnpor- tant component of a broader research and development program. The 134 RapidAppraisalMethods in a DiagnosticAssessment RA team visited areas that had not been covered in an earlier RA study; focused more on issues of economic orgazation, markelting channels and costs, and- price policy than that study; and incorporated a lot of diffuse but useful secondary data in its report However, the exerise would have been more effective had economists and vegetable seed specialists been able to participate jointly in the field work; had the team been able to cover a broader geographlic area and interview more farm- ers in remote seed production zones; and had more junior analysts been able to particpate in the RA field work in what is typically a valuable on-the-job training experience. References Chiltor, MichaeL 1990. Proposa fJor a Pilot Innovation to Improve Vegetable Sed Handling and Marketing Methods in Nepal. Agricultural Marketing Improve- ment Strategies Project. Moscow, Idaho: University of Idaho, Post-Harvest institute for Perishables. Chilton, MichaeL and Rajendma P. Shrestha. 1989. Report on a Vegetable Seed Production and Marketing Strategy fir Nepal. Agricultural Marketing Im- provement Strategies Project (USAID). Bethesda, Maryland: Abt Associates, Inc. Holtzmnan, John S. 1986. Rapid Reconnaisnce GuidelinesforAgricultural Market- ing and Food System Research in Developing Countries, MSU International De- velopment Working Paper No. 30. East Lansin& Michigan: Department of Agrcultural Econonics, Michigan State Urniversit3r. Holtzman, John S., and Ramesh B. MunankamnL 1990. Rapid Appraisal of the Vegetable Seed Marketing System in Nepd. Agricultural Marketing Improve- ment Strategis Project (USAID). Bethesda, Maryland, and Kathmandu, Nepal Abt Associates and No Frills Consultants. Kunar, Krishna. 1989. Cnducting Key Infonnant Interviews in Developing Coun- tries. Washington, D.C: Bureau for Program and Policy Coordination, USAID. ____ 1990. ConductingMini-Surveys in Developing Countries. Washington, D.C: Bureau for Program and Policy CoordinatiorL WeLs, rlm, and Jamuna Kayastha. 1990. Market Potential in Taandfor Nepal- Produced Vegetable Seed. Agricultural Marketing Improvement Strategies Project (USAID). Moscow, Idaho: Post-Harvest Institute for Perishables, University of Idaho. World Bank 1990. World Developmenf Report. London and New York: Oxford University Press. Sanmir. 1989. Observations on Vegetable Seed Production, M1arketing, and Imporfation in BangLadesh. Agricultural Marketing Improvement Strategies Project (USA]D). Bethesda, Maryland: Abt Associates, Inc. John S. Holtzman 235 Notes 1. The RA study was conducted shortly after India had closed most of the border crossings and terminated many of the trade and transit privileges accorded Nepal. Fuel was scarce and could only be found on the black market with difficulty and at great cost The vehicle rental budget was limited, making it prohibitively costly to leave vehicles at roadheads for days while the researchers wer trekking to distant villages. 2. Access to infornation is a property right in many developing countries particularly where public analysts and officials are paid poorly or paid late. 7 Systematic Observation in the Analysis of Primary Health Care Services Stewart N. Blumenfeld, Manuel Roxas, and Maricor de los Santos In this dfpter, Blumenfild, Roxas, and de los Santos present what is undoubtedy n excellent exampe of the use of structured direct observation in the development setting. The purpose of this study, an integral part of a larger multinational research project, was to identify deficiencies in the primary health care systemn in the Philip- pines. The authors prepared a set of direct observationforms, tak- ing into consideration the activities, tasks, and subtasks that health care workers must carry out in health clinics to accomplish discrete dlinical obiectives. The Jbnns were dosed-ended, and in most cases observations could simply be checked to save time. The au- tors focused on 18 rural health units and ther satellites using a set of performance criteriafor direct observation. As the chpter indicates, the authors were able to identify many operational problems that required immediate attention by the gov- ernment. For exacmple, health workers were not getting all the information they needed to make correct diagnoses for some condi- tions such as acute respiratory infections. Many were giving some misinformation about children's nutrtional status. Still othes of- ten failed to communicate with the mothers tofolloew through with the requisite treatment at home. The authors learned three importanrt lessonsfrom the experence that are of wider importnce. First, direct observation generates details that cannot be obtained by any other methods, such as field surveys or qualitative interviews. Second, training of workers for direct observation is not as difficult as it seems. Local sff can be trained to observe people without significantly disturbing normal functions Third, dosed-ended observation instruments promote the reliability and consistency of data. 136 Stewart N. Blumen, Manuel RZoxas, andMarcor de tos Santos 137 A gneral obsermation about dtis methodology can be made herm Although the authors in this particular case constructed a relatively large sample to improve the "generalizability" of theirfindings, it does notfollow that the methodology cannot be used on a smaller scale. Even when monitoring and evaluation staffdo not have time and resources to construct a representative random sample, they will obtain more reliable data by using this approach than if they do not use direct observationfonns. FOR MANY OF THE MAJOR HEALTH PROBLEMS that still kill large numbers of children in the developing countries, efficacious technolo- gies alredy exist Imiunization can prevent most deaths and serious complications due to such childhood diseases as polio, tetanus, diphdte- ria, measles, and whooping cough. Oral rehydration therapy can treat or prevent potentially lethal bouts of dehydration due to severe diar- rhea. And a conscientious program of growth monitoring can provide early detection of children who are malnouxished and therefore at much greater risk of contracting infectious diseases and/or suffering physical and intellectual stunting. The problem as seen by many who know health systems in develop- ing countries well is that these tedhnologies are not implemented by basic level health workers according to the protocols that make them effective. The central roIe of the health worker's performance in deter- mining the effectiveness of the service system may be seen when the system is diagrammed as a classic systems model. As an example, Fig- ure 7.1 shows such a model of one important component of a public health care program, the diagnosis of dehydration due to diarrhea and its treatment by means of oral rehydration therapy. The entire process component of the model comprises tasks that must be caried out by the health workers. Systems Analysis Approach to Problem Identification in Service Programs A systemn model makes it clear that the goals and objectives of a service delivery program (which correspond to impacts and outcomes in the system model) cannot be accomplished if the process is poorly imple- mented or if required inputs are absent Thus, when ad hoc evaluations or routine management infonmation show that targeted objectives are not being attained, identifing the underlying reasons requires identifi- cation of missing inputs and analysis of the service delivery process. This chapter concentrates on the process of service delivery; that is, on the activities of the workers who provide health care service. 138 Systematic Observation in the Analysis of Primary Health Ca Services Figure 7.1 Oral Rehydration Therapy Service Delivery Model Inputs Oral rehydration solution Trained staff Mother/child Information Information, education, communication Support Outputs Training system Supervision Logistics system Management information Information, education, system communication niformiation, education, communication system Planning _- _ Process istory |Physical Treatment Counsel Duration 4 Temperature Give oral k mother Frequency $ Skin resil- rehydration 4 Mixdng Blood ience W solution Oresol Vomiting ' Mucous Talke home Continue Fever membranes t oral feeding Decreased Fontanelle 4 irehydration k urinke . solution Thirst Home fluid | Been doriae Proximate Distal Outputs Outcomes Outcome Impact Correctly Rehydrated Appropriate Reduced diagnose child utiliztion mortality child Dehydration Correctly treat averted child Follow-up Mother with performed Oresol Knowledge- able mother Proper record Stewart N. Blumenthal, Manuel Roxas, and Maricor de Los Santos 139 While collection of statistics on the outputs and outcomes of health service systems is routine in most countries, collection of data on the quality of the services provided by health workers is rare. Instead, service system managers rely heavily on a combination of workers' ini- tial training plus supervision to assure that services continue to be car- ried out according to specified norms. Our work on the PRICOR Project has shown this reliance to be misplaced. The PRICOR Project The Philippines-based study reviewed in this chapter is one of a series of 12 country studies that were carried out under the auspices of the Primary Health Care Operations Research (PRICOR) Project in the pe- riod 1986-90.1 PRICOR's purposes were: * to develop methods that identify operational problems in the delivery of primary health care (PHC) services;2 - to help cooperating developing countries apply these methods to identify operational problems in their basic child survival service pro- grams; * to assist these countries to carry out operations research in correcting some of these problems; and * to alert service program managers in the comnunity beyond the spe- cific countries in which PRICOR works to operational problems that seem to exist very widely. An operational problem is defmed as a failure by some component of the health care delivery system-including the service providers themselves-to perform according to prescribed norms. A previous phase of PRICOR had developed operations research methods appropri- -ate for resolving operationaI problems in these service systems. The Role of Observational Data In this phase of the project, an attempt was made to develop a reliable means for quantfying the performance of health workers with regard to how well they carry out the important details of their tasks. However, the PRICOR staff quickly ran into a problem: the specific activities car- ried out by health workers are not part of standard management infor- mation system and are thus not part of a standard record Various options presented themselves. One was to interview the health worker and ask him/her to describe in detail what he/she does under specified circumstances, for example, when presented with a child with diarrhea or when he/she is carrying out growth monitoring of a child. However, this was not considered a way to develop reliable 140 Systematic Observation in the Analysis of Primary Health Care Serices ; information. First, many people do not think in a sufficiently detailed and linear way to assure in exact and detailed account of how they would behave. Thus, they may not articulate the small but often impor- tant details. The other side of the coin is that what people say they will do and what they actually do often are very different matters. Another confounding factor is that in dealing with patients, workers often re- ceive behavior-triggering clues from interacting with the patient. Thus, fictitious written or oral case presentation might or might not elicit a true portrayal of the workerfs behavior. An alternative option considered was a variation on the 'what if' interview approach, role-playin&r in which a live person is presented as having certain symptoms and the health worker is asked to proceed in the presence of observers as if the case were real. This was judged to be a great improvement, because it is more likely to trigger behavior simi- lar to that in a real-life situation. Role-playing has limitations, of course, because certain procedures, such as giving vaccines, cannot be com- pletely carried out. Also, under such contrived circumstances, it is hard to be sure that the worker is not on his/her "best behavior," carrying out every detail of the job in a manner that might not reflect his/her actions under unscrutinized circumstances. Ultimately, the PRICOR staff decided that direct observation of health workers as they actually perform their duties in a real setting is the best way to actuaJly determine performance. This approach is also rater contrived, but we felt that no worker could keep up a facade when observed in multiple cases, especally in the fairly hurried environment that characterizes many primary health facilities in developing coun- tries. Thus, our main approach to assessing the quality of care was a structured observation and recording of worker activity that looks spe- cifically for key actions that comprise the norms set by generally ac- cepted protocols for providing these services. Developing Observational Field Instruments Designers and managers of PHC service systems often believe that delivery of clinical services is a straightforward, relatively simple pro- cess that ought to be carried out with little difficulty. In fact, detailed examination of what really is expected of health workers reveals a com- plex series of actions that afford many opportunities for errors both of omission and commission. In order to observe service delivery operations systematically, the PRICOR staff first produced a comprehensive list of the broad activities healIth workers must carry out for each service program component. Each activity subsequently was broken down into its component taslcs, Stewart N. Blnhal, Manuel Roxas, and Markor de los Sanlos 141 and in many cases, these tasks were further disaggregated into subtasks. All of these activities, tasks, and subtasks were compiled in a volume, which we call a thesaurus. A sample page from the thesaurus is pre- sented as Box 7.1. Note the instructions to the user to obtain informa- tion through actually observing an immunization session. Field instruments are developed from the thesaurus. Since health care systems are somewhat different from one country to another, sys- tems analysis designers are encouraged to customize the instruments by extracting from the thesaurus those tasks that fit the particular health care situation to be evaluated. For example, if immunizations are re- corded in a special log at the health center rather than on a card retained by a child's mother (as specified in the thesaurus), then determination of what vaccinations the child is due is made by examining those records. Computeized Forms Design Carrying out accurate observations in a busy health center with many attention-diverting activities going on requires considerable concentra- tion. It is therefore very important that the data collection forms be easy to follow and mark. We attempted to make our instruments as closed- ended as possible, by minimizing the amount of writing required to fill them in and maximizing the number of observations that could simply be checked. For this purpose, we found computer software specifically designed to create forms very helpful. The program facilitates a neat, uncluttered and unconfusing design and allows easy .tput of lines be- tween logical sections. The ease with which whole sections of the fonm can be deleted, expanded, and moved around encourages experimenta- tion with different looks. Most such software allows for easy insertion of checkboxes and provides special characters, such as arrows, to lead the eye from one part of the paper to the next Boxes 7.2 and 7.3 are samples of such instrments developed for our systems analysis the Philippines, created with FormTool software from BIoc Deve!. - Tr int Corporation. The aew generation of fomis de- sign software allo.. -, forms to be filled in d;rectly on the computer and developed as a database without the additinal step of re-entering the data from a paper form. The Philippines Country Study Background: The Philippines Healt; Se vice System The Philippine health care system is a mix of private and public pro- viders. In general, wealthier people tend to use the private sector, while 142 Systematic Obsation in fhe Analysis of Primary Healfth CareServices Box 7.1 Sample Pages from the PFRCOR Thesaurus VERON 1.1 PRIOR MAY 1,1968 * 1*sW senitre tveyfaciily mauling its immunization taets? *. Number of Inunuziations given lat yeas t chlldrt undr I (and/or othr ap per local policy) pa 2O0 chldren under 2 (and/or other age per local pLicy) in the servke diey facility catchlunt area by type ovwci and by vaccin dose (les #) Senrir DeUvay Facility Document Review L % of bImunIation of tugets acdeved lat year by ype of vaccin and by vacine dos (series #) Service Delivety Facility Document Review I Is delred imunhiaHan prwgrm lnqet being atained is Ike servic dediveayjility kchzt aea? & Number of new cases of vmcdne-peveatable dieaes In chiden under 1 (and/cr) othr age per loCa poUcy) last year per I 000 children under 1 (and/or other age per locac policy) In the seve delivery facillty cathment area by type of disease. Service Delvery Facil Document levewr 2.1 PREPARE VACCINES AND IMMUNIZATON EQUIPMENTAND SUPLIES 211 PREPAREVACCINES (SEEIMUNIZATION: LOGISICSUPPORT) 2.12 STFi=UZE NEEDLES AND SYRINCES w Do health uwres skrile nades mad syinga pers adardprotYures? a. % of ImmunIration sssionk for which needle and syringes e sterized per standard procedures Immunizatfon Sesion Observaffon 2.2 IMMUNI CHILDREN 22.1 EXAMINE CHILDS VACCINATION CARD OR QUESTION MOTHER TO DE- -UMNEE IMMUNIZATIONS REQUIE * Do healt wrkers examine childrwn's vacdnalion cards or questio nu w,m to de- Ermine immunIations required? a. S of children (nimunizadon session attendees) fr whom health workers examine vaccination cards or question motes to detennne Immunzhations required Icamunization Enconer Observatlon * The te immunizatin ssionr refers to skuglepurpose sessions during whih Immu- nizations ondy are povided, to multi-pupose sesions during which service In addition to Immunization ar provided and to homne visits during which iimunization are - provided. An immunization session may indlde single or multiple immunization continued on next page StewartN. Bluvmnthu Manuel Roxas, andMAriwr delos Santos 143 Box 7.i (continued) hat are rtnsJr non-imn,niuation of bnmunuafion sedon altends? b. % of chldren (ImmunzatCon sesion attendees) not Immunlzed by reason for non4mmunIzation Immunization Enounter Observation 22.2 ADMINIWER VACCINES 272.21 ADMINISE RECOMMENDED DOSE FOR ALL VACCINES 22.2 UWE CORRECT ADMNISIRATION TCHNIQUE FOR ALL VACCINES 2.2.23 ADNIMIER ALL VACCNES WITH STERILE NEEDLES * Do health workmradtminstrrall vaccies withstaik needla? a. S of vaccine hujectiom given with sterile needles Immunization Eacounter Observation 2.224 ADMISER ALL VACC:IN13 Wrn STERILE SYRINGES? * Do helmh wrkerrs adnnisntrall vacines with sterilk syringes? a. % of vaceine Inkections given with sterile syringes Immunizraion Encounter Observation 2.2±5 PROIECr GG, POUO AND MEASLES VACCINE FROM HEAT AND LIGHT DURING USE * Do health urkrs pro fet BC polio and rslks vacdnes fi,n heat and LtI during use per stahnirdpnedures? a. of imnunization sesskns in whkh BCC, polio and measles vaccines ae protected orn heat during use per standard procedures Immunization See- sion Observaion b. S of immnuilzaton sessins In which BCG, polio and measles vacdnes are pmeected from light during use per standard procedures Immunization Session Observation 22.3 COUNSEL MOWHR CEE IMMUNIZATION: SERVICE DELJVERY-3.1 PRO- VIDE INDIVIDUAL COUNSELING TO MOIHERS OF CHILDREN ATrENDING IMMUNIZATION ESEZONS) 144 Systematic Observation in theAnaysis of Prmary Health Care SErvices the less wealthy (the bulk of the population) use the public systm. The PRICOR Project dealt with the public system The public system is bierarchical, with the policy-maldng officials and staff and overall program managers for the key cild surviva programs at the top. The job of all these Manila-based officials is to set policies and standards; to develop national goals, objectives and plans; and to monitor overall achievements of the systemL The country is divided geopolitically into 13 regions, each of which has a Regional Medical Office responsible for regional planuing within the national framework but which takes into account allowable local variations, such as special disease problems. A Regional Hospital pro- vides tertary-level care on a reference basis. The 13 regions are divided into 79 provinces. A Provincial Medical Oftice does higly locaIized planning and monitoring and provides, hirough the Provincial Hospital, secondary care and ambulatory pri- mary care. Provinces are divided into districts, each with a District Health Office and a small district hospitaL Attached to each District Hospital are outreach units called Rurl Health Units (RHUs). These number as many as 60 or 70, depending on population and geography. Many RHUs have satellite Banpgay (village) Health Stations (BHSs) to help make service even more accessible. Most RHUs are staffed by a physcian, one or two Public Health Nurses (four years of traing be- yond high school), and one or two Rural Health Midwives (two years of training beyond high school). BHSs are always staffed only by a Rural Hlealth Midwife. Some RHMs are assigned to two BHSs, since the BHS may be open only part-time. The Rural Health Midwives (RHMs) are supervised by a Public Health Nurse (PHN) based at the RHU. PHNs are supervised in turn by the physician directly above theaL The Philippine Systems Analysis PRICORfs mandate from USAID was to focus on the lowest levels in the system, where most primary health care service is provided. In the Philippines, this meant the RHUs and BHSs and their staffs. Discus- sions with staff of the Undersecretariat for Public Health Services re- vealed that a number of service programs were not meeting their targets.3 Four were selected for analysis in four clinical areas: immuni- zation, oral rehydration therapy, growth monitoring, and diagnosis and treatment of acute respiratory infections. In sum, it was agreed that the systems analysis would study performance in these areas of the Public Health Nurses and Rural Health Midwives who staff RHUs and BHSs. The specific activities observed and the tasks that comprise them may be seen in the data collection instruments shown in Boxes 7.2 and 7.3. Stewart N.Blumentha,Manuel Roxas,and Mari curdae los Santos 145 Box 7.2 DOHPRXCORSystemsAnalysis Growth Monltorkrw Observation of Growth Monitoiang Sessio Date (mid) -J- Obmever (Last name, Initia) District P______ I-U _______Es_ _____ Type of scale used 0 OilEd cinIcal 0 AdulItclinical o DUI spring a Tubular spring Oi Bathroom 0 Bar (Espada) Was the scale turned to Oat the beglnulngofthe general session D3Y O N H1W notation of child's age. mc,w O Byasking 0 Fromgirowthdhart How was age deteO rIned? 03 Oeher G there a Lecord showinrg Dcltds date of bizth? 0 N i Y 0 IDOB (m/d/y) .-J. When the chud was weighed.u Was it stripped to pracial limt? 0 Y 0 N Wethekg Was itrelatively still? D8Y a ON Was thewemight read correctly? D y O N What hind of record was used? 0 Gowth chait 0 Log book If lids was fir-time weighing, 0 Y did thedW give mbother growth chaN? 0 N-WhYO PrWoe by whih weight and age 0 NW rote It on chart or in book hansfiteed to record 0 HW catted it out and smeone else wrote it 0 Other_______ Upon recording Was age recadedaccuately? D Y O N Was weight recded accuately? Y O N Did HW intrpret resut for umother? 0 Y 0 N Did HW tell mothe child needs special 0 Nothing 0 Feeding feedinig or other attention? 0 Other__________ Did HW try to verify tat mother undes thi w Instucti lon? 0 Y 0 N Did HW talk about need to mnaintain breasifeeding or good weaning practice? OY 0N Did MW ask mother If she had any questions about childs status? 0 Y 0 N Did HW tel mother when to hbturn fornet weighineg? 0 0 N 146 Systemnatic Observation in theAndtysis ofPrimaryHealth CareServices Box 7.3 DOH/PRICOR Systems Analysis Oral Rehydratlon Obemrflon of Service Deivery Date (m/d) _ bswer atmne, tial) Distrik!______ U4_______BHtS_ _____ B_ HWraine- Fsnd_ P tde :Childsage.an Did HW asic How long child has had darrhea 0 Y 0 N Comments How hequently dcild is psIng stools 0 Y 0 N , ,, If there u blood arucos In stools 0 Y 0 N If vonltng Ls a]S occurlng O Y ON N If d lhas had fever 0 Y O N lfchild has been very thrsty DY ON N I urineoutput i greatly educed D Y O N What has been done athone sofar O Y 0 N Did the HW: Take temperature 0 Y 0 N Comments Examine mucou s embranas of mouth 0 Y 0 N Test sldnusiience aV ONl N Examine fontade 0 Y 0 N Weihcdhild D Y O N Chld was: O Treated at fiity and held for obseraion (.. hours) O Treated at faility and sent hme inmediately O Not treated, sent home O Refeed to Did the HW kest to see that child is able to take fludbynmouth? DY ON if treated at facilit, was the dhild %iven Oesol OY ON Home solution O-Y a N Antibitic OY ON If yeswhat If child was given Orsol at the facily, 0 1-liter container what was used to weasure thewater? 0 DiWerent container, but 1 liter was estimated reasonablywel O Asount of waterwas obviously imodt N Oresol was given, was the entfe packet used? OY ON When the child was sent home, was the mother givem Oresol CY ON Antibiotic O Y O N [fyes,what Presniptioifr anibuuiotic 0 Y O N if yes,what Stewart N. Blumenlhal, Manuel Roxas, and Maicor de los Santos 147 For example, in the observation of diagnosis and treatment of diarrhea, the data collection instrument (Box 7.3) directs the observer to note whether the worker asks a series of questions related to prescribing the correct treatment of the case. Other observed items on the form relate to the way the treatment is provided. By observing the PHNs and RlIhis and comparing their work to the norms set for these activities, we expected to identify those weaknesses that were most likely to account for sub-standard target achievement in the system. Knowing those specific areas in which the workers were not performing to standard would enable Department of Health (DOH) management to take corrective actions, such as additional or better training, improving supervision, better planning, or improving the availability of supplies. Methodology Sampling Approach. Discussions with the Department of Health's Public Health Services staff led to the conclusion that it was feasible to identify one province that would be reasonably representative of the country, in terms of the type and degree of operational problems to be found. A number of selection criteria were established relating to popu- lation health indicators and achievement of service targets in the previ- o-us year. A number of provinces fell in the middle range on these criteria. Among these, practical considerations of geographic accessibil- ity, interest and cooperativeness of regional and provincial staffs, and physical security were used as final selection criteria. Bulacan Province was finally selected. Bulacan is a large province Porth of M1anila that has semi-urban characteristics in the south near Manila, but is decidedly rual further away. Examination of facility distribution and staffing pattes at the district level revealed no signifi- cant deviations from national norms. If anything. in the judgment of the DOH staff, Bulacan might be slightly better off than the national average in terms of the seniority and quality of staff, because it is considered one of the more desirable postings within DOH and thus attracts more ap- plicants. The more objective indices of health, health services, and ser- vice outputs, however, placed Bulacan in the middle of the national range. Therefore, DOH and PRICOR made the assumption that the observed level of perfornance was representative of the middle of the national range. As noted, the major concem of the PRICOR systems analysis was the quality of the care provided to patients by the Public Health Nurses and the Rural Health Midwives who work at the Rural Health Units and the Barangay Health Stations. Bulacan Province has 54 Rual Health Units. 148 Systematic Observation in theAnalysis of Primary Health Care Semices Resources were not sufficient to study all of them, nor was this consid- ered necessary in order to get a picture of the types of operational prob- lems that were most common throughout the province. Since, according to the service delivery systems model, outcomes are a direct product of process (that is, quality of care), a sampling approach was designed to array the 54 RHUs front "bestr to 'worst' These qualitative attributes were quantified in terms of key child survival outcome indicators. Us- ing consensus judgment weights supplied by central DOH and provin- cial health staff, these performance criteria were combined into an index of performance and each of the 54 RHUs was rated on the index. Based on resources available for the systems analysis, it was judged feasible to examine 18 of the Bulacan RHUs and their attendant BHSs. We decided to "bracket" performance by taking six RHUs from near the top, six from near the bottom, and six from approximately the middle of the performance index. Once again, practical considerations of geo- graphic accessibility and physical securty were employed to identify the exact six selected in each category. The 18 RHUs selected had 54 satellite BHSs; these 72 facilities and their staffs comprised our initial sample. However, toward the end of the data collection period, several more RHUs and their BHSs were added in order to obtain more obser- vations of cases of acute respiratory infection and oral rehydration. This brought the total of observed facilities to 90. Data Collection Instrunents. Instruments for data collection were drafted initially on the basis of performance indicators selected from the thesaurus for each of the four interventions. These drafts were pre- sented at workshops to provincial and central managers of these pro- grams, wiho made the modifications required to fit the Filipino system. For example, there was no point in delving into the adequacy of storage of vaccines at the BHSs, since this is not the Filipino approach Instead, for vaccination sessions at the BHS, the question became one of protec- lion of the vaccines as they were transported between the RHU and the BES. The instruments were field-tested in conjunction with the training of the systems analysis staff. It may be noted (Boxes 7.2, 7.3, and 7.4) that the observation instru- ments leave little room for subjective judgment by the observer; most items are answered by a simple yes or no. This design is a deliberate attempt to minimize inter-observer variation and to make the methodol- ogy practical for those managers who have few highly skilled staff at their disposal. The instruments do not require expert judgments about the quality of a health worker's history-taking technique. Observation instruments were produced not only to assess direct ser- vice provision, but also the performance of the PEINs in their role as supervisors of the RHMs. Box 7.4 shows the instument used to assess PHNs in this supervisory capacity. StewartNM Blumenthl,Manuel Roxas,and Markor de los Sanfos 149 Box 7.4 DOIVPRICOR Systems Analysis Cmwth Masdtrlnp: Supervision ObservatIon Date (m/d) . bserve (Last n3fe, Inita]) Superviso rthi/position Obsenrvalin of Supervlboi/Supervluee InteractIon SuperviseeTIeauto site. oflgteration (0 Bgy) (0 DB5) (03 RH-U) Did supervIsorexamine records to 3en If mast -children being weighed anscdtedue? 0 Y 0ON Did supervIsor watkh HW zemo scale? 0CY 0 N Did supervisor watch severa weighing to delermine if weights obtaind corrctl? 0 Y 0N iDid supervso spot check to see If agos were correctly determined? CY ON iDid supervisor spo check to see If weight and age platted/recorded comedy? 0Y VCN Did supervisorlisten toHMWinterpre results to 0 Yes, and HW did Interpret 3 or more mothers? ~~~~~0 Yes, but HW did not interpret O No D)id superviormaukeanoycommnent toDHWabout 0 Yes, praise only techraque or process of weigting? 0 Yes, some cerrective conmments Did supervisor ntake any othe attempt to verify that 0 N HW uses comret Ieduiique anad(or bunsmit nutrition Or4, W-hat______ messages to anothe? Field Staff and Training. Field staff consisted of seven data collectors arnd a field supervisor. All were college graduates. Of the seven data colIectors, four had bac'kgrounds in nutrition, the others in various so- cial sciences. Five of the seven had had some previous experience doing interviews, but none had had experience collecting observation data. None of the seven w-ere famiiliar with the health care service systemi as it functions in the rural parts of the Philippines. The field supervisor, however, was the PRICOR Project Technical/Administrative Advisor in the Philippines, an experienced field person who had worked on other projects dealing with the health care system. Staff training and instrumnent-testing were carried out concurrently. Over a three-day period, the staff were briefed on the purpose and 250 Systematic Observation in the Analysis of Primary Health Care Serices approach of the project, basics of interviewing were reviewed, tech- niques of unobtrusive observation were discussed, and familiarization was gained with the instruments. Then the staff were sent to the field to practice with each instrument while the field supervisor observed them. Their experiences were discussed and problems, either with the instru- merits or their own interviewing or observing technique, dealt withl This training/testing period revealed some interesting problems with the instruments that had not been foreseen. For example, it turned out that the PHNs and RHMs had conceptual difficulties with estimating numeric quantitation in the form of percent- ages. They were dearly at a loss when asked questions that required them to select a percentage-type response such as "less than 25 percent, about 50 percent, imore than 75 percent." Eventually, we were able to work out a verbal continuum with which they seemed comfortable: sel- dom/occasionally/often/almost always. While this permits only an or- dinal-level analysis of the data for these variables, it seemed better to present respondents with a scale they could comprehend rather than force them to use a scale they could not. Data Collection. Data collection took place over a 12-week period Uune to September 1988). This is the rainy season in the Philippines and about one week was lost altogether due to weather problems. A house was rented in the provincial capital to be used as the activity headquar- ters and the team residence. One DOH vehicle was assigned full-time to the systems analysis task. Most of the time the staff were transported from this headquarters to the particular villages in which they would be working for the day. Occasionally, team members stayed in local homes in the area if they were far from the residence. For the most part, each RHJU or BHS was observed for one day. Two or three observers worked at each session, depending on the size of the facility and the expected number of cases to be observed. Each evening, the field supervisor reviewed the instrnuents com- pleted by the staff, discussed any problems, and made any arrange- ments necessary for a revisit. There were very few of these required. One question on everyone's mind was how the RHU staff would react to being observed as they worked. From the combination staff training-field trial exercises, two important lessons were leamed. At first, it was obvious that the RHU staff were somewhat uncomfortable and that it was absolutely vital to reassure them that this was not an individual evaluation, but rather an effort to help the Department of Health find out what kind of problenis workers in general were encoun- tering in trying to deliver service. In order not to bias healIth worker perfornance, however, they were not allowed to see the data collection StewartN. Blumen! ManuelR Ras, and- aricordlsSanos 25 instruments (though staff were instructed not to make a big show of secrecy). The other lesson learned was that it did not take long for EHUJ staff to groDW used to the presence of the observers and settle into its daily routine. This was probably assisted by the relative youth of the oh- sewver teams, their obvious inexperience in service delivery compared to the medical staff being observed, and their overall non-authoritarian, non-th:reatening demeanor. Not unexpectedly, it turned out that some information is very diffi- cult for an observer to obtain with certainty. Ther"e are some actions of a health worker in dealing with a patient that are very subtle. Far ex- ample, in dealing with acute respiratory infections, the worker needs to categorize the case as mild, moder-ate, or severe on the basis of symp- touts displayed. This is critical to providing the appropriate treatmenL Where clinical records are poorly kept, the worker may or mnay not do thiis in his/her head. The only way for an observer to get these data is to ask the worker. Of course, this would remind the worker and bias any further observations. Assessing a child's breathing rate is another criti- cal component for care of respiratory infections, a major determinant of the mild/moderate/severe categorization. Yet a health worker may as- sess the breathing rate very quickly instead of going through an overt timed count, and if the child is very obviously breathing normally or is very obviously breathing too r-apidly, the worker will not be seen to carr-y out a timed count Despite these and ather limitations, we feel nevertheless that observa- tion produced most of the informnation a manager needs to assess the service delivery process. Observers were able to tell unequiyocally whether specific points of medical history were asked9 whether specific actions were taken during the physical examination, and whether spe- cific counseling messages were addressed to mothers by a health worker. .Data Archiving and Analysis. Although the data collection instruments were designed to be as dosed-ended as possible, some open-ended in- terview questions were inevitable. In order not to lose some of the richness that verbatim responses afford, the data were archived in dBase mH Plus to take advantage of the text-storage capability of database man- agement software. Exahiination of verbatim text did, int fact, prove use- ful in providing some insights into why certain operational problems were in evidence. For example, it was through these text responses that we discovered how heavily supervisors rely on the original trafining received by the midwives to ensure that they are doing all the tasks necessary for high-quality care. 152 Sysmaic Obstion in eAnysis ofPriary Health CareSrvis For analytical purposes, the data were transferred to the SPFS statisti- cal package SPSF (or any other statistical software) was not famOir to most staff of the DOH, so data analysis was done mainly-but not entirely-at PRICOR's U.S. offices. To date, data analysis has been re- sticted to very basic procedures: frequency counts and cross-tabula- tions. The results of these procedures, however, leave little doubt about, first, the presence of some major defects in the quality of care being provided, and second, the power of observational techniques to identify the. Results A sizable literature exists on techniques for measuring inputs, outputs, effects, and even impacts of health service systems. This process, how- ever, usually has been examined more along the lines of worker time- motion and facility patient-flow for the purpose of improving efficiency. Although the World Health Organization's training documents on oral rehydration therapy are comparable, in their attention to the specific proces of how this service should be delivered, they are designed for training rather than assessment. Our approach of identifying opera- tional problems by quantifying performance of these subtasks through direct observation of health workers seems to be unique at this time. It is not the purpose of this chapter to carefully delineate the specific op- erational problems identified for each disease service examined. How- ever, to provide a feeling for the ldnd of data that systems analysis affords the healti care system manager, some examples of the types of problems revealed are discussed in the following paragraphs. Examples of Problems Observed in Direct Sevice Ddivery The activities that primary health care workers must carry out to deliver high-quality pediatric services fall into three genel categories: diagnosis, treatment, and counseling the caregiver (usually the mother), on what to do for the child at home and what to expect and how to react For each disease, of course, the specifics of diagnosis and of treatment and the counseling messages are different It is these specifc actions required of the health worker that are listed in the PRICOR thesaurus and that were observed in the systems analysis. The data collection instruments shown in Boxes 73 and 7.4 illustrate what specific tasks were observed for growth monitoring and for oral rehydration therapy (used to prevent dehydration due to diarrhea). Many problems were found in the direct delivery of services. For example, we found that in diagnosing acute respiratory illness patients, Stewart N. Blumenthal,MAm uel Roxas, and Marcor detos Santos 253 both PHNs and RHMs rarely asked if tuberculosis was present in the household-a vital piece of information in determining the treatnent regimen. In more than one-third of observed cases, mothers were not cautioned to continue feeding the sick dIld, even though many mothers tend wrongly to withhold food from sick children. When medicines were prescribed for the child, in 40 percent of the cases the mother was not told the timing and measurement of dosage administration. And in 90 percent of the cases the mother was not counseled to complete the drug regimen, even if the child appeared to be getting better very quickly. Regular weighing of young children is a key component of a program of early detection and treatment of malnutrition. Moreover, studies have shown that it is important to closely involve mothers in the growth monitoring process, because their participation is one of the best ways to ensure that they continue to bring the children regularly into the clinic and that they take appropriate remedial steps if the child's nutrition level begins to decline. To this end, nearly all countries use a growth card on which the child's weight is plotted against his/her age. The plot is overlaid aginst colored bands that show normal weight-for-age ranges and make it easy to see when the child is falling below normal. The card is the key tool for demonstrating to a mother the nutritional status of her child and explaining what needs to be done. It is vital, then, for the health worker to get the plotting right and to point out to the mother how her child is doing. When we observed the weighing and plotting process, many errors were found. For example, in 16 percent of cases the health workers marked either the child's age, or his/her weight, or both inconrectly. In addition, in 90 percent of cases observed, the health worker failed to comment on the results to the mother. Instead of using the weighing and plotting to actively involve the mother in the care of her child, the process was treated very mechanically. Important details such as these cannot be obtained other dtn by observation of how workers deliver service. Yet it is exactly these de- tails that are major determinants of whether the care provided is effec- tive. Armed with this kind of information, the service system manager-or for that matter, an immediate supervisor-can pinpoint and correct significant deficiencies in the system as a whole or in a particular individual. Examples of Problems Observed in a Support System Health workers alone do not constitute a functional health care ser- vice system They require various kinds of support a training system 154 Systematic Observation in fhe Analysis of Primary Health Care Services (both initial and refresher), logistical support, information, plans, finan- cial support, and supervision. Of all of these, supervision is probably the least understood in truly operational terms. Obviously, no service system is deliberately planned with a weak supervision component. But among servL-e system managers, supervision is almost always acknowl- edged as one of the weakest links in the chain that ultimately leads to quality care. Rarely, however, is the supervisor's job defined in specific operational terms, and almost never is the performance of supervisors studied in such tenns. The PRICOR thesaurus displays very specific tasks that supervisors are supposed to carry out, broken out in several categories: knowing the details of the job of the person being supervised in operational terms, actively assessing his/her performance against operational norms, iden- tifying gaps and the reasons for them, taking corrective action, receiving and resolving problems, and imparting new information and skills. The PRICOR systems analysis in the Philippines assessed supervisors' per- formance by observing how they actually carried out the operational tasks that define supemsion. Box 7.4 shows what specific tasks were observed to assess supervision of RHM by Public Health Nurses in the growth monitoring program. While the general result-the wealness of supervision of peripheral service providers-was not unexpected, iden- tification of the specific flaws provided important information that could be used in very practical ways by the DOH. First, the quantification of supervisory performance on specific opera- tional elements added significant impetus to a nascent movement in the Department to take serious stock of the supervision situation and its effect on the quality of care. And second, the data provided useful information for specific corrective actions that might be taken. For ex- ample, as has been noted, the assessment demonstrated that supervisors relied too heavily on RHMs' original training for assurance that these workers would continue to provide all the elements of service that they were taught Non-observational data convinced us that supervisors were not aware of many of the deficiencies in service quality provided by the health care workers they were supervising. But it was the obser- vational data that provided many of the clues as to why. We could document that supervisors do not often watch their "charges' to satisfy themselves that the latter are carrying out all the tasks they are sup- posed to, and they rarely ask them if there are problems for which help is required. As a result of this systems analysis, several operations research stud- ies are being carnied out, both by DOH staff themselves and by outside organizations commissioned by the Department, to develop special StewartN. Blumenthal,ManuelRoxas, andMalricordelosSantos 155 training for supervisors, to devise tools to assist with field supervision, and to develop means for supervision of supervisors. Direct Observation: Summary of Lessons Learned In this case study, direct observation provided important information about qualityofcare issues that may explain in part why some impor- tant objectives of the primary health care system in the Philippines are not being met For example, nutritionists in developing countries agree that involving mothers in monitoring their children's nutrition status is a key to identifying incipient malnutrition or treating a child for malnourishment Yet PRICOR staff observed that 90 percent of workers weighing children did not interpret the result to the mother. And, of mothers who brought a child to a facility for treatment of diarrhea, 55 percent were not reminded to continue feeding the child, even though this is an important and standard component of quality service for this problem. This level of detail probably cannot be reliably obtained any other way. Also, it is possible for observers to do their work without significantly disturbing the system. The key is for the observer to be perceived as non-threatening by those being observed, so this should be a special point of emphasis during training. In some situations, it is even possible for the observers and their activity to be seen as a positive asset, The rapport our young staff established with the health care facilities' staffs was, in many cases, quite warn. Finaly, in order to promote both reliability of the data and inter- and intra-observer consistency, we strongly recommend using instruments designed for maximum objectivity. This has the added advantage of allowing the investigator to employ 'non-expert' observers, because little judgment in the substantive area is required. This may also relate to the point above; presumably under some circumstances, an 'expert' peering over one's shoulder is likely to generate discomfort and an un- natural situation Notes 1. The PRICOR Project is implemented by the Center for Human Services, Inc., Bethesda, Md., and is funded by the United States Agency for Intemational Development, Bureau for Science and Technology, Office of Health, under Co- operative Agreement No. DPE-5920-A-00-5056-00. 2. Services that focus specificaily on child survival are a subset of the services that comprise primary health care in developing countries. However, primary health care consists of a broader concept than just direct service provision. It 156 Systematic Obserion in th Analysis of Primay Health Care Serucs also indudes substantial involvement of communities in managent of the ser- vice system, an emphasis on disase prevention as much as on treatment, and assured access to service in terms of affordability and geographic proximity. Services generally recognized as central to child survival are: imunization, oral rehydration therapy, correct diagnosis and appropriate treatment of acute respi- ratory Infections, and growth monitoring and treatment of malnutrition. De- pending on their own situation, some countries add to the list malaria diagnosis and treatment, as well as maternal health services aimed at preventing low birth weight and neonatal tetanus. 3. The PRICOR Project in the Philippines is a joint undertaking of the Center for Human Services' PRICOR Project and the Republic of the Philippines De- partment of Health (Undersecretariat for Public Health Services and Undersecretariat for Management Services). 8 Using Urban Commercial Counts and Marketplace Censuses to Appraise Agricultural Development Projects Gordon Appleby In this dhapter, Gordon Appleby explans a different the of direct obseraion method, which he used in the north Shaba region in Zaire. This is one in which proxy indicators were used to assess the impact of an agnicultural project covering a large geographical area (15,000 square kilometers). The indicators that Appleby used in this case study were counts f the retail outlets in the region. The essential premise of this methodology is that successful agti- cultuml development raises frm production and increasesfarmers' incomes, which in turn create greater purchasing power and thus foster commercial developmtent In this evaluation, Appleby carded out a systemtic count of retail shops, service establishments, and marketplace goods as a measure of levels of commercial activity. The resulting data showed sign fckant commercial growth, in north Shaba. The data also indicated variations in commerad growth which could be a function of project activities. The area that had received greater assistance in terms of roads, seed suzpplies, and agricultural extension demonstrated greater commercial deveop- ment. A word of caution about the application of fhis methodology is necessary. While agriultural growth generally-though not al- ways-contributes to commerial development, the latter can take place without the former. For example, factors such as the multi- plier effect of project expenditures in the region, improvement in transportation facilities, or the growth of manufacturing can con- trbute to commercial growth. In such conditions, the findings of the study based on this methodology could be mileading. Before using this methodology, investigators should therefore carefuly ex- amine the full range offactors tht could explain changes in levels of commercal activity. 157 158 Using Urban Commercial Counts and Marketplace Censuses HOW CAN THE IMPACTS of a long-term agricultural development project covering a large (15,000 square kilometer) area be assessed? And, how can such an assessment be accomplished in a very short pe- riod of time? Urban commercial counts provide a means, albeit indirectly. By way of illustration, tis case study presents an example of the use of com- mercial counts in the post-project assessment of an agricultural develop- ment project in Shaba Province, Zaire. The underlying logic of this approach is simple. Increased agricul- tural production results in more produce being marketed, which trans- lates into higher farmer incomes and greater commercial activity and development Not all commercial institutions, however, develop equally quicldy. First, marketplaces will offer additional groups of goods (calIed arrays) and attract more vendors. Then, new and different shops and service establishments will be opened. Finally, new centers wMil appear in the region. Conversely, a fall in agricultural production means lower rural incomes, less commercial activity-ftnt in the mar- ketplaces, then in the stores and shops, and in time, in fewer centers. In other words, the expansion and contraction of visible retail trade pr- vides a reliable index of rural economic change. What is required for urban commercial counts is a set of tools for measuring and analyzing commercial development Fortunately, a set of measures derived from central-place theory can be used for this pur- pose. Commercial activity, accordingly, is indexed by the number and types of shops and the diversity of marketplace goods and services By counting the numbers of each type of establishment in each place, the analyst can discen the spatial pattem of regional development. Centers at the same level of development will have similar numbers of each type of retail outlet. Centers at higher levels of development wil have more shops of different types and more complex marketplaces, while centers at lower levels of development will have fewer shops and less com- plex-or diverse-marketplaces. Further, these levels of center devel- opment normally pattern regularly in space. These measures have several advantages for evaluating the inpact of agricultural projects. First, they are quantitative. The counts provide a reliable, replicable measure of commercial development In this regard, commercial counts represent a major advance over much rapid rural reconnaissance, which is highly impressionistic. With commercial counts, the basic data are unassailable; one can query only their inter- pretation of the data. Second, because the counts are replicable, they can be repeated- carried out one year and then redone several years later in order to assess the extent of change. The approach can also be used to analyze differences in project impact within a region, as well as changes over Cordon Appleby 159 time. Third, the method is extremely efficient. Instead of continuous and expensive monitoring of agricultural production-measuring fields, repeatedly inspecting the aops over the growing season, and weighing cuttings to determine yield-one can quickly count enterprises in com- : mercial centers as a proxy measure for the farmer income. In this case study, a description of the Shaba region and of USAID efforts there provides the context for the present evaluation. The article then presents the general theory supporting this applied approach, that is, the geographical theory of central places. The subsequent section discusses how this theory in fact worked out during data collection and analysis. The penultimate section presents the findings and conclusions of the evaluation and the final section draws several lessons that pro- vide guidelines for potential users of the method. Country and Program Settings For a ten-year period (1975-86), the United States Agency for Interna- tional Development (USAID) supported an agricultural development project in the north Shaba region of southern Zaire. Originally designed as an integrated rural development project, Project Nord Shaba (PNS) was revamped in 1982 as a strict agricultural development project. Vari- ous activities such as farmer councils, intermediate technology, and women-in-development efforts were dropped so that project attention could better focus on rebuilding agricultural feeder roads, distributing open-pollinated corn seed, and improving com cultivation practices. A project management unit and a socioeconomnic monitoring cell were also retained throughout the project's life. PNS ended in September 1986, whereupon USAID initiated a kindred activity in the contiguous central Shaba area just to the south. At the time the project started, north Shaba was one of the most isolated and backward areas of Zaire. There had been a vital commerce in the region based on river and rail transport during the colonial pe- riod, but war and rebellion had taken a heavy toll. Bridges had been blown up, isolating large parts of the area. River traffic effectively stopped as river channels silted in, and while the railroad continued to run, its schedule was at best intermittent and undependable. As a con- sequence, many mercantile firms that had shipped bulk produce out of the area and distributed merchandise to farmers had closed long before. By the early 1970s, the regional economy of north Shaba had all but collapsed. Many farmers had of necessity reverted to subsistence agri- culture. This was particularly true in the more populous western half of the project area, centered on Kongolo. There, most Hemba farmers practiced a mixed cropping system, with com as a consumption crop. By contrast, in the eastern area centered on Nyunzu, the patrilineally 160 Using Urban Commercial Counts and Maetp1ace Censuses organized Luba practiced a shifting cultivation, with manioc as the di- etary staple and corn as a cash crop. The Luba had other advantages as well They could open up new forest plots every several years because population densities were much lower than in the Kongolo area (3 ver- sus 15 persons per square kilometer). Also, the Luba had access to Pygmy labor, which was not available in the western part of the project area. Thus, there was more production for domestic export in the Nyunzu area than in the Kongolo area, but wholesale trade everywhere was much reduced. As a result, in 1974 Oust before PNS began), there were only two centers in the entire region, Kongolo and Nyunzu. Each was an admin- istrative center with limited public services, such as schools, religious missions, and dispensaries. Both were on the rail line, which in theory made possible shipments to and from the provincial capital of Lubumbashi and into the neighboring Kasai region. Each also had ac- cess to a productive hinterland, although the economic reach in each instance was limited to the distance that goods could be headloaded or carried by bicycle. Each had a weekly marketplace as well, the only ones in the region at the time. Elsewhere in the region, villagers prac- ticed subsistence agriculture and had few economic ties outside their villages. Project Results Despite these unprom-ising beginnings, PNS succeeded beyond almost everyonet's expectations. By rehabilitating ruril roads and distributing inproved seed, com production increased frm 30,000 metric tons in 1977-78 to over 100,000 metric tons in 1985-86. Moreover, the quantity of corn marketed increased from 10,000 metric tons just before the project to almost 50,000 metric tons by the time the project ended. Al- though it is difficult to impute causality for these increases directly to particular project interventions, it is clear that rebuilt roads, improved com seed, and new agricultural practices-in addition to the strong markets for com in the Kasais and south Shaba that umdergirded a fa- vorable farmgate price-had a decided effect on com production and marketing in the project area. Th.ese improvements were becoming clear as early as 1982, when the project was evaluated and re-designed. Even then, many end-of-project targets were being exceeded. Thus, at the terniination of project activi- ties in September 1986, when the USAID mission in Kinshasa commis- sioned a post-project impact evaluation, there was interest in the evaluation in assessing what other changes in the PNS area were assoc- ated with the sustained ten-year effort. Gordon Appleby 161 As part of this impact evaluation, it was quicidy decided in the field to focus assessment on commercial development The notion was that if FNS had succeeded as an agrcultural project, it would hiave set off a process of regional development that would be reflected in the extent of commerce in rural towns. The use of "urban' commerce to measure rural change may seem counter-intuitive, but it is really quite logical. As an evaluation methodology, it is efficient and cost-effective as well. Towns that serve vilagers in the surrounding areas depend on and develop in conjunction with their hinterlands. Thus, tracking "urban"' commaercial development provides an efficient measure of change in surrounding rural areas, because one need go toonly one town or mar- ketplace to gauge how the 20 or more dependant villages are faring. By extension, one can cover a region by visiting the towns and market- places there, which are much more lim-ited in number than the villages. Description of Methodology and Rationale Central-Place Theory A useful set of field and analytic tools for assessing regional commer- cial development has been devised in anthropological applications of central-place theory (Skinner 1964, 1965 and Smith 1972Z 1976, based on the work of Christaller 1966 and Marshall 1969). Three observations constitute the basis of central-place theory. First, goods, and services that are purchased more frequently can be sold in more market centers. For example, daily necessities, such as bread, eggs, or vegetables will be more widely available than specialized or expensive goods, such as cosmetics, enamnelware, plastics, beds, or ra- dios. Second, and for nmany of the same reasons, goods are added in a predictable manner to any miarket center's inventory. That is, a town that already offers food staples may add more exotic foodstuffs, as well as clothing, plastic goods, and enmelware. And third, centers at the same level of development will be spaced approximately equally apart, with lower-level centers appearing more densely on the landscape than higher-level centers. To study this patterning, geographers inventory the types and num- bers of shops in each town in a region in order to categorize centers into levels according to the typs and numbers of goods and services avail- able in' each. The levels of centers are then mnapped in order to discern the spatial patterning of centers.1 Development studies must include both retail establisSuments and pe- riodic marketplaces, for these are but two parts of a single system. The patterning of shops and markets is usually isomorphic: centers with 162 Using Urban Commercial Counts and Marketplace Cnsuses more stores and services usually also have more diverse marketplaces. Further, the lowest level of marketplace is often found in rural centers that support few or no stores; these markets are effectively an extension of the retail system into the countryside, which cannot support perma- nent stores. Sometimes, however, the two subsystems are structured differently; analysis of these systems can be more complicated (Snith 1985). Centra-Place Field Methods and Analysis Three types of commercial count can be completed for all centers in a region: retail shops, service establishments (including truckers), and marketplaces. Tables 8.1 through 83 in the following section illustrate categories of enterprises that have proved useful in studies in several parts of the world. Several methodological comments are in order. First, while these lists are a reasonable starting point, any additional goods or services avail- able in a region must be added to the list For example, in Shaba here are no wholesale buyers; that is, marketers who come from towns to buy local produce in bulk-staples, or eggs and cheese, or other prod- ucts (for example, tamarind, mushrooms, fish)-that is then taken to the towns for sale. Where this wholesale bulking function occurs, it must be included in the market census. Second, commodities may vary by locale, for example, rice may be the staple in one area, com in another, and cassava in a third. Where the particular commodities in a class (for example, food staples) vary wifhin a region, it is best to code the specific commodities in each marketplace and later collapse those goods into the generic classification (for ex- ample, regional staple) in order to even out the geographic variation. 'itird, marketplace goods, shops, and services must be counted only once and always in the same category. This is relatively easy for shops and services, which usuaIlly sell a single trpe of good or service (for example, groceries, dry goods, cloth, watch repair, tailoring). Market- places, by contrast, can be very confusing, as many vendors assemble a little bit of many very different goods. Probably the most difficult prob- lem a researcher faces is how to code all the sellers whose arrays com- prise a seeming jumble of foodstuffs, goods, or whatever-the mercachzfle of Spanish America and the table of Francophone Africa. These are best coded as "mixed array' under the predominant category of goods handled, for example, foodstuffs or manufactues. This ap- proach will reduce but not eliminate the problem, because some sellers will purvey goods from several (or all) categories of goods. Whatever coding decision is taken, the same rule must always be applied or the results of the counts in different places will be non-comparable.2 Gordon Appleby 163 Finally, not all shops or market arrays are equally important from an analytic point of view. Central-place analysis is relatively insensitive to small errors in the numbers of common establishments and vendors. It matters little whether there are 61, 73, or 92 vendors of local staples in a marketplace if imost markets average 20 such sellers, some average 70 or 80, and a very few have 200 or more. By contrast, the presence of 'higher-order' goods or services that are usually available only in higher-level centers (for example, cosmetics, radios, beds) is critically important to the analysis, for these commodities (or their absence) dif- ferentiate centers by level. Therefore, a special effort must be made to record all of these enterprises. In other words, where precision is r'- quired, commercial and market censuses should be carried out Most times, however, estimates of retail and service establishments by knowl- edgeable local informants and reasonably accurate market counts will stiffice. To analyze these data, the analyst must: (1) rank goods according to the number of centers in which they appear; and, (2) rank centers ac- cording to the number of types of goods they purvey. To do this, goods are ranked within category (that is, retail shops and service establish- ments, including repairmen) and listed down the left-hand side of the page from most common (appearing in most centers) to least common (appearing in the fewest centers). Similarly, it is usually hepful to list the centers across the top of a sheet of graph paper from high-level (most functions) to low-level (fewest functions). This bookkeeping chore usually requires several attempts before the simultaneous order- ing of goods and centers is completed successfully. Tables 8.1 through 8.3 demonstrate common examples of this formatting.3 This analysis can be done for an area at one point of time, for several areas at the same time, for one area over time, or for several areas over time.4 No matter how complex the analysis, the essential measures remain the same-the commercial counts that allow determination of the number of levels and the number of centers in each level, and their distribution over space in relationship to transportation routes. The practical application of these principles and tchrdques is imustrated in the next sectionz Data Collection and Analysis Activities The post-prcject impact evaluation for Project North Shaba lasted four weels-three days in Kinshasa for briefings, ten days in the project zone for field investigations, three days in the regional capital for meetings, and a week in Kinshasa for report wrting and meetings. The first sev- eral days in the project area were spent attempting to assess the extent of social and organizational change. When it became apparent that 164 Using lrban Commercal Counts and Marketplacr Censuses whatever social change had occurred in the project area could not be systematically documented in the time available, it was decided to focus on the possibility of urban' conunercial development The evaluation team postulated two probable findings. First, it was supposed that increased corn marketing would have given rise over time to greater commercial developnment throughout the region. In- creased com marketing itself connotes greater commercial activity- during the limited buying season. The important question was, given that there was undeniably a larger, seasonal trade in corn, was there also an increase in urban commercial development, that is, in the numbers and types of shops and markets that operated year-round? Second, it was supposed that, in view of the project emphasis on corn, those areas that marketed most corn would provide evidence of greater commercial development As luck would have it, the project area divided into two areas: Kongolo, which was more heavily popu- lated and had more rehabilitated roads, but where com was initialy mostly a subsistence crop; and Nyunzu, which was much less popu- lated, had fewer roads, but produced more com for sale. Since the quantities of corn marketed had long been much greater in Nyunzu than Kongolo, the initial premise held that commercial development would be greater in the Nyunzu area than in the Kongolo area. Testing and documenting these expected findings required system- atic counts of retail and service establishments and of marketplace ven- doze inventories of goods ("arrays) in all centers of the project area. The counts were done with questionnaires and market census forms that were relatively simple to construct The questionnaire for urban commercial development was a cheddist of all types of retail and service establishments in the region. The inter- viewer asked informants how many of each type of establishment there were now and how many existed five and ten years before. Of course, the actual conversations were more free-wheeling and open-ended, but the use of a checklist helped ensure that the interviewer did not forget some category or type of shop. It also helped that two evaluatDrs con- ducted the interview, with one taking the lead in the conversation and the other takdng notes and double-checdking the information. Data were collected from several sources. Key informants in each center were asked about the number of shops, retail and service estab- lishments, and truckers in that place over time. The informants, all residents of the place, were merchants or authorities, or even passers-by. The information sought in these interviews was generally known and neither thratening nor overly intrusive, so most any adult could be of help. Even so, it was worthwhile to corroborate the information by interviewin& for example, both an administrative authority and a mer- Gordon Appleby 165 chant. Although their numerical answers might not be identical, there was rarely any great difference in the numbers of establishments re- ported. Moreover, project documents provided important corroborating evidence on these and other matters. By contrast, marketplaces were actually enumerated by the team, be- cause this information cannot be collected through interviews.5 Al- though there are several ways to do a marketplace census, they all amount essentially to counting all the arrays in the market as quicldy and accurately as possible. It is usually useful to have a printed form listing all the market arrays that appear in the region. hi the Shaba assessment, the enumerator carried this form on a clip board. He or she started in one comer of the market, counted all the vendors of each armay in that row or two of sellers, entered the counts on the form, and then moved down to the next tier. of rows. The fact that vendors rent stalls or table space and that vendors sell- ing the same anrays tend to congregate in the same areas of the market helped tremendously in carying out the census. Nonetheless, there were 'problem' areas. For example, women vendors from the same village often sat with their children in a duster, which makes it impos- sible to count the actual number of vendors. Fortunately, as has been mentioned, a reasonable estimate of the number of vendors in these dlustes sufficed, for ffiese women offered mostly very low-order goods. Once the census has been completed, the enumerator talied the counts for each array and verified that all types of arrays had been counted. Marketplace censuses should be done when business is at its peak. In north Shaba, all markets meet daily, although there is variation over the course of the week On Friday, market vendors are required to perform public service, caled solongo, which means that the number of vendors might be fewer than usuaL Saturday morning is the time when other residents perform solongo, so the number of buyers (and h,ence sellers) might be smaller. Conversely, Sunday is a day of rest when people from outlying villages come to market, so Sunday can be a Iarger than aver- age market day. Inclement weather introduces another variable. It would have been ideal to have multiple censuses for each market- place in order to take all this variation into account However, with but oi,ne week for fieldwork and with the work schedules of the other evalu- ation team members to consider, only one marketplace census could be scheduled for each place within the mutually agreed-upon team itiner- ary. Due note was taken of any special conditions that might affect comparability.' Data collection posed other difficulties. Travel in southien Zaire is difficult under the best of ircumstances. One village listed in the scope of work proved to be nearly inaccessible at this time of year-the end of 166. Using Urban Commercial Counfs and Marketplace Censuses the rainy seasorL The team drove for three hours in the rain to reach it, encounterng many difficulties. Happily, however, perseverance paid off, for this village in northern Nyunzu, which once had a thriving com- merce and social infrastructure, was now devoid of any commercial activity outside the corn marketing season, despite the immense amounts of corn produced and marketed there. This visit, and others like it to and beyond the limits of the project area, provided critical information to verify the interpretations made on tht' basis of the com- mercial counts. These problems notwithstanding, all the largest and many smaller centers with shops and marketplaces were censured. As has been de- scribed, the goods enumerated were ranked according to the number of centers in which they appeared, and centers were ranked according to the number of types of establishments and marketplace arrays that they purveyed. The findings presented in Tables 8&. 8-2, and 8.3 interpret these data and rankings. Findings and Conclusions The Commerial System's Evolution in North Shaba, 1976-87 At the outset of PNS, there were only two "higher-lever centers in the region, and to judge from retrospective interviews with key infor- mants, even these supported only a very feeble commerce. Today, by contrast, there are several levels of market center, each with a limited but active commercial life. The two or,jnal centers, Kongolo and Nyunzu, remain paramount Both offer the entire range of retail services available in the region, from kiosks through retail shops to wholesale/retail operations with a trans- port fiunction (Fable 8.1). They also provide many other services, such as pharmacies, hotels, restaurants, and the like (rable 8.2). Further, these two centers have the largest and most diverse marketplaces in the area (Table 83). The only unexpected finding is that, by 1987, Kongolo surpassed Nyunzu on every index as the Major center in the region. Further, an intermediate level of center has newly appeared in the region. The differences between these places-Sola, Mbulua, and Lengwe-and the two high-level centers are clear. The intermediate centers all have several large retail stores, some local transport services, a few other services, and a arketplace. (In the case of Lengwe, the marketplace was successfully established only four years before, after several decades of fitful attempts.) But none of these functions is as complete as in Kongolo and Nyunzu. In the marketplaces, for example, there are many fewer vendors, and even though the proportion of sell- Gordon Appleby 267 ers with manufactures is similar, a much more limited range of goods is available. Sellers in these markets generally have more mixed arrays, combining medicines, hardware and plastics, school supplies, and some doting. There are no sellers of specialized arrays, such as plastic shoes Table 8.1 Counts of Commercial Shops and Transport in the PNS Area, by Level of Center, 1987 Level I Levl 2 Level 3 Categcnr Kangolo Nyunru Mbzlula Sot Lengwe Makutano Butendo Stores Kiosks 50 20 15 0 10 0 0 Retail olgy 25 15 20 10 5 3 0 Wholesale 6. 4 4 1 1 1 0 Transport Pickups 4 0 1 0 0 0 0 Trucks 30 12 3 2 0 0 0 a. The center was visited in the early afternoon, when the market was already disbanding. Therefore, there is no market count for Sola. Table 82 Counts of Other Retail and Service Establishments in the PNS Area, by Level of Center, 1987 Level Leel 2 Level 3 Establishment 1igolo Nyunzu Mbuhdua SolP Lengwe Matano Butendo Service Flour mill 6 7 4 1 1 0 0 Bakery 11 5 1 1 0 0 0 Phannacy 6 4 4 1 1 1 0 Hotel 3 4 2 0 0 0 0 Restaurant 3 2 1 0 0 0 0 Bar 1 3 0 0 0 0 0 a. The center was visited in the early aftemoon, when the market was already disbanding. Iherefore, there is no market count for Sola. 168 Using Urban Commeri Counts and Marketplace Censuses Table 83 Vendor Counts, by Commodity Array, in Maricetplaces of PNS Area, by Level of Market, Januaq 1987 Leol I Level 2 Total no. of Commodwtlservice Kongolo Nyunzu Mbulula Lengwe vendors Localfoodstuffs Tubers/grains 63 23 14 3 103 Greens/spices 43 15 6 2 66 Flour 14 5 11 1 31 Forages 13 10 5 1 29 Vegetables/fruit 23 4 1 0 28 Firewood/charcoal 12 10 4 0 26 PaLm oil 13 3 2 0 18 Tobacco 4 4 0 0 8 Meat- 3 5 1 0 9 Chicken/eggs 1 1 0 0 2 Subtotal 189 80 44 7 320 Importedfoodstuffs Dried fish 17 7 8 1 33 Salt 7 5 4 0 16 Bread 5 3 1 2 11 Soap 2 1 5 1 9 Sugar 7 1 0 0 8 Subtotal 38 17 I8 4 77 Manufactures Boutique/table 14 14 2 4 34 Used clothing 26 4 4 0 34 Hardware 4 10 3 3 20 New cothing 12 1 4 0 17 Plastic shoes 7 4 0 0 11 Pharmacy 4 2 1 3 10 Cosmetics 1 1 0 0 2 Subtotal 68 36 14 10 128 Craft goods (basketry, dcairs, pottery) Subtotad 5 2 2 1 10 Finger foods 10 7 5 2 24 Restaurant 3 2 1 0 6 Watch repair 1 1 0 0 2 Tailor 1 1 0 0 2 Bicycle repair 1 0 0 0 1 Subtoal 16 11 6 2 35 Total 316 146 84 24 570 Gordon Appleby 169 or cosmetics. As feeble as this development may be, it is still ga eater than in any other place in the region, other than Nyunzu and Kongolo. Significantly, all three of these centers lie on the trunk road. More- over, these middle-level centers are all located between major centers, which is a predictable pattern in central-place theory. The third level of center occurs only on the periphery; that is, away from this commercial core. These lower-level centers are few in number and demonstrate very little commercial development In theory and in fact, there are always more lower-level centers than higher-level centers. La the present instance, there are surely more third-level centers than the evaluation team was able to visit Nonetheless, each of the villages visited had only one or two small shops on the order of a small general store. Any other third-level centers would likely have similarly limited commerce. Finally, most villages do not have even a small store. This class of villages, devoid of formal commerce for most of the year, makes up the fourth, and bottom level of the commercial hierarchy. It would prob- ably be the most numerous class had commercial activity been invento- ried in all places in the region. In summary, north Shaba has undergone a commercial renaissance in the last decade. Whereas before there were only two centers along the rail line, several new centers on the tunk road now also perform whole- sale and retail functions. Moreover, whereas before there were only two levels of market center-the two bulldng centers and all other villages- there are now four levels. Besides the two entrepOt centers on the rail line and the three secondary bulking centers on the trunk road, small commercial centers are now appearing in some villages off the main arteries, so that peripheral villages, especially in the Kongolo area, are now differentiating themselves into two levels-those with and those without year-round commerce and trade. The change that has occurred is dramatic, given the backward, iso- lated state of the area at the outset of the project. And the change, small as it may be, could not have been documented without the commercial counts. Spatial Patterning of Commercial Development The second finding is that patterns of comunercial development vary not simply with corn production (income), but also with population c ensity and transport efficiency. Although this outcome runs counter to the team's initial expectation, it accords perfectly with central-place theory predictions. At all levels of the commercial hierarchy, market centers in Kongolo-which has long lagged in com production-are more fuly 170 Using Urban Commercial Counts andAMarketpace Censuses developed than centers in Nyunzu, which has long dominated the corn business. Kcoagolo town has more wholesalers1 more retailers, more transporters, mrore service personnel, and a larger market than Nyurnzu town. Among; die middle-level centers, Sola and Mbulula in KCongolo district have larger and more diverse markets, more stores and shops, and more services than LZen'gwe, wh-ich is in the Nyurnzu area. Of im- portance, there are even transporters in the middle-level centers of KCongolo district, but none in Lengwe. Finally, at the lowest levels, some villages in the KCongolo area boast a shop or two, some of which even purchase local produce at wholesale. By contrast, there are no shops in any of the villages visited in Nyunzu, even though some of these places have a large population and show clear signs of once having been small commnercial centers. This consistent difference between the two areas is readily explained. The project rehabilitated '-ar more woads in Kongolo than in Nyunzu, thus opening up more areas for commercial production. This decision is understandable, inasmuch as popuilation densities in KCongolo ar-e al- mast five times higher than in NyunLzu. Also, even though corn is a consumption crop in Kongolo, farmers in that area practice a mnuch more complex agriculture and typically sell various craps to merchants. Even though the overall value of' these other crops-rice, peanuts, and palm oil-does not rival that of corn, in a particular locality, the total income from agricultural production is much greater than that from corn alone. In other words, the three factors, considered fundamental for com- mercial development in central-place theory-transport efficiency, population density, and income (in this case, from crop diversification), which together determine "demand density"-are all significantly higher in Kongolo than in Nyunzu. As a result commercial develop- ment is greater in the Kongolo area tha the Nyunzu area. Spatial Patterning.of Othier Regional Changes Spatial analyses are powerful because they provide an overall frame- work for interpr-eting other variations within the region. So little work has been done on this topic that the matter is treated here illustratively, with the example of farmer investment patterns. Project reports documented that farmers in Nyunzu cultivated larger areas (3 hectares on average) than farmers in KCongolo (1 hectare on average). Although yields were lower in Nyunzu (2.6 metric tons/hect- are versus 3 metric tons/hectare), fanners there produced and sold much more corn. In fact, three-quarters of all Nyunzu tanners earned more from corn than the top 25 percent of Kongolo producers. Probably 10 percent of the farmers there cultivate 10 or more hectares in corn, Gordon Appleby 1X7 earning as much as US$2000 a year. Nonetheless, patterns of household expenditure have not changed significantly in the region. People basi- caIly meet their daily needs, and if they can afford it, they purchase a radio, bicycle, or sewing machine. Only the largest producers can macke productive investments as well as consumption expenditures. Such in- vestments are essentially limited to building a house in town, opening a store, and buying a pick-up or used truck. Inasmuch as more farmers in Nyunzu are better able to invest, one would expect greater local invest- ment in that area, compared to Kongolo. As the commercial counts document, it is the Iongolo area that ex- hibits greater local investment. Not only is the major center there more developed, but shops are opening in the smaller villages as larger farm- ers open stores in their natal villages and buy trucks. Evidently, the multiplicity of cash crops allows these local entrepreneurs to bulk pro- duce in quantities too small to attract larger merchants. And the pro- longed harvest season for these different crops means that smaller producers have cash available over much of the year, thus supporting local shops. By Lontrast, in Nyunzu, where com remains the maior-if not the only-commercial crop, marketing is compressed into a few months after the harvest, buying is handled by merchants from the ma- jor centers, and consequentiy, there is mnuch less local commercial devel- opment An entrepreneurial famer in the Nyunzu area has but one option-to invest in the major town there. Those who have invested locally have failed, for there is simply not enough cash in the country- side for most of the year to support even a small shop. In a very real sense, diversified cropping, as in Kongolo, supports more widespread regional commercial development than mono-crop export agriculture, whic%, as in Nyuxzu, is leading to the rise of a single primary town with a very seasonal commercial pulse. In summary, urban commercial counts enabled analysis of change within north Shaba over time in wesponse to productive change in the countryside. Fortunately, there were just enough historical descriptions available to piece together a quantitative picture of the pre-project situa- tion that could be compared with the field data collected during the evaluation. Also, north Shaba is sufficiently large that comparison of these counts in the two sub-zones of the project area revealed significant differences in commercial development within the region-differences that will continue urtess corrective action is taken. Together, the tempo- ral and spatial comparisons provide a complex, integrative view of the type and extent of change that occurred over the course of the project, in the countryside, as well as in the towns. This interpretation of the cormmercial counts gave explcit form to the project personrel's intuitive knowledge of developments in the area. Everyone 'knew" that productiort, income, and commerce had in- 172 llsing Urban Conmercial Counts andMarketplace CnsGses creased, but no one understood the extent of the observable change or its spatial patterning. This central-place interpretation provided a fraamework 'or understanding the changes over time that people familiar with the area knew accorded with events. The interpretation also, cinched a major, implicit question in the team's scope of work. That was, apart from the direct consequences of project interventions, had the project sparked a spontaneous process of development? And if it had, what was this process and how could it be documented? It was precisely this question of important, indirect im- pact that could best be anrwered through the rapid rural reconnaissance techrdqueo of urban comnmercial counts. Guidelines and Suggestions It is tseful to conclude with observations about limitations of this ap- proach and a suggestion for possible uses of central-place methods in other contexts. First, the use of urban commercial counts and marketplace censuses for rapid appraisal of agricultural development projects presumes that the project being evaluated is the single or at least the major undertak- ing in the region. In the case of the north Shaba project, no other gov- ernment or donor program existed, so attnbution of the changes in commercial development was straightforward.7 It obviously would be much more difficult, if not impossible, to allocate impact among several agricultural development projects operating simultaneously in the same region. In other conditions, it might be difficui' or impossible to inpute causality; that is, to eliminate factors c-ogenous to the proect (for ex- ample, illegal or black market trade) as the explanation for improvement in regional conditions. Second, the use of commercial counts to evaluate an agrcultural proect presumes that the prcject is worling on crops that are generally important for sale throughout the area. Various ty of agricultural projects do not meet these criteria, and could not be evaluated in this manner. Village garden projects for farily nutrition, for example, do not aim primarily at increased family income. Similarly, projects that target a specific crop in a delimited area would likely not have a wide- spread or regional effect For example, an inigated rice project of, for example, 50 hectares, could, if successful, have only a local-not a re- gional-impact on commerce. These caveats notwitistandin& the field and analytic techniques em- played in the north Shaba evaluation are powerfuL The approach can, in principle, be used in other contexts, either where the project area is Gordon Applkby 173 smaller than a region or where the project focus is more on secondary crops. Commercial counts in these situations could contribute to project monitoring and evaluation through the same spatial and temporal analyses as was done for Shaba on the basis of linmited but rigorous fieldwork The need now is for additional applied studies to test the efficiency of the approach under these varied conditions. References Appleby, Gordon. 1978. fThe Aftermath of Exportation: The Socio-economic Evolution of the Regional Marketing System in Highland Puno, Peru." Ph.D. diss., Stanford University, Department of Anthropology. . 1982 "Price Policy and Peasan't Production in Peru: Regional Disintegra- tion During InflatiorL" Culture and Agrilture No. 15 (Spring): 1-6. -. 1985. "Marketplace Developirent in the Gambia River Basin. In Stuart Plattner, ed., Markefs and Marketing: Monographs in Econonic Anthropology, No. 4 Boston, MA: University Press of America. Christaller, Walter. 1966. Cntral Plaes in Southern Gennany. (Translated by COV. Baskin.) Englewood Cliffs, N.J.: Prentice-HalL Marshall John U. 1969. The Lociiton of Service Towns: An Approacd to te Analysis of Centrl Place Systems. Toronto, Canada: Department of Geography, Udi- versity of Torc.so. Poulin, Roger, Gordon Appleby, and Cao Quan. 1987. "Impact Evaluation of Project Nord Shabaf Project evaluation report for USAID/Zare. (photo- copy) Skinner, G. Willian. 1964. Marketing; and Social Srcture in Rural China.' Part L Journal of Asian Studies 24343. 1965. 'Marketing and Social Structure in Rurl China." Part I. Journal of Asian Sfudies 24t195-228. Smith, Carol A. 1972. 'The Domestic Marketing System in Western Guatentai- An Economic, LocationaL and Cultural Analysis.' Ph.D. dis., Stanford University, Departnent of Anthropology. .1976. RegionalAnalysis. Vol. 1. New Yorlc: Academic Press. .1985. 'Methods for Analyzing Periodic Marketplaces as Elements in Re- gional Trading Systems.! Researwh in EconomicAnthropology 791-337. Notes 1. While geograpner usualy analyze urban centers in more developed regions, anthropologists have adapted the theory and its methods to te study of pi- odic marketplaces; that is, intennittent centers that blip on the landscape once every so many days. Skinner (1964,1965) pointed the way in a masterful trilogy on the Chinese marketplace system over three great epochs. Smith (197Z 1976) operationalized the approach in a pioneering field study of the very complex marketplace system in highland Guatemala. Subsequently, Appleby used Smith's field methods in studies of the marketplace systems in southern Peru 174 tsfis UrbanC ommerci Counts and Markeplace Censuses (1978) and Senegal and The Gambia (1985), besides this study in Zaire (Poulin, Appleby, and Cao 1987). The central-place methods prsented here are truly an anthropological innovation. 2. This nmthodological problem reflects economic reality: where denand is weak, sellers can better maximize their profits by adding different goods to their arrays than by increasing the diversity of a particular good. 3. The classification of goods and services used here contains an inplicit catego- rization of centers into levels that short-ctuits the elaborate central-place method (Marshall 1969). The categorization is based simply on the numbers and tpes of shops and services. In retail trade, for example, a place with only two small retail shops is less developed (that is, less "central' and of a 'lower order) than a place with 15 small shops and 3 large retailers. This latter center is, in tr, less developed than one with 50 small shops, 15 large retailers, and 5 mer- chants with not only retail, but also wholesale buLking operations. As has been mentioned, a concordance usually exists between the types and extent of com- mertial activity. Higher-level centers have more shops with greater capital, more truckers, more service establishments, and more diverse marketplaces than lower-level centers, and the shifts over time axe usually all in the same direction. Where this concordance does not occur, as sometimes happens, the syste= is more complexly structure and mnay require a more detailed analysis. 4. Analysis at one point in time is basically descriptive, for there are no com- parative materials. Here one is interested in t-he number and level of centers and their spatial patterning, quite possibly for use as a baseline in future monitoring of project impact. Temporal or spatial variation allows a more complex analysis based on the same set of measures. If the system has been studied at two points in time, it is possible to discern changes in the regional system by comparing the number of levels of center and their spatial dispersion at each time (Appleby 1982). If parts of the same system are studied at the same time, one can discem intra-regional variation in much the same way. The only difference is that one is comparing different areas within a single system at one point in time rather than the same area at different points in time (Appleby 1985). In the most compkx use o' this methodology, if one has baseline and post-project data for a region that exhibits sub-regional variation, one can analyze the regional system as it developed over time and discem differential development over space within that system (Poulin, Appleby, and Cao 1987). 5. Naturally enough, people think of marketplaces in terms of the particular goods that one can find there. They do not think of markets in terms of arrays of goods; that is, the inventories carried by individual vendors. To illustrate, one cannot ask whether there is a vendor of plastic pails or hardware (an anray of plastics or of hardware), for the answer will almost always be yes if some vendor caries a few pails or some hinges-mixed into an an-ay predominately of other goods such as lothing. In oier words, a list of all the goods availabIe in a market is not the same, in a central-place perspective, as a census of marketplace arrays. For these reasons, marketplace vendor counts must be carried out by trained teams. Furthermore, retrospective interviews about marketplace compo- sition usually provide little or no usable information. One can ask when the market was successfully established and, maybe, approximately, how large it Gordon Appleby 175 was; one annot usefully ask about the number of types of arrays that were then offered. 6. The team encountered a number of "special considerations" in doing the marketplace censuses. The team arrivedi in Sola just as the market was disband- mag, so the census there was not credible. In Mbululia, it rained suddenly, and the team had to delay its departure unmil after the market reopened. Later that week, the census of the two markets in Kongolo town could only be takcen on a Sunday. Thus% the numbers recorded probably represent the peak business in contrast to the more usual weekday markets studied els-ewhere. Such are the exigencies of field work in a time-bound situation. 7. Some critics nonetheless faulted the evaluation for being unable to demon- strte conclusively the role of the project in changing conditions in north Shaba. Common sens argued persuasively for the importance of the project-alter all, there had been no other development activity in the region'. However, it was then impassible to exclude the possibility of exogenous factors,. such as macro- economiic policies Subsequent design work undertaken in preparation of a fol- low-on project in neighboring central Shaba project provided the opportunity to prove that PNS-and not other factors-accounts for the changes documented here. Commnercial counts done in central Shaba, which would have been subject to the same exogenous factors as north Shaba, documented that this neighboring area scored significantly more poorly on every measure of commnercial develop- menL En fact, central Shaba at that timne (1987-88) looked ver-y much like north Shaba had bjefo the star-up of PNS. Since the same exogenous factors would have been operative in both areas, the difference in commercial development in the two regions can only be explained in terms of project inte-rvention. .. ~~9 Part,icipatory Rural Appraisal: A Case Study from Kenya Charity Kabutha, BarbWra P. Thomas-Slayter, and Richard Ford In this dcapter, Kabutha, Thomas-Slayter, and Ford descibe the application of participatory rural appraisal (PRA) methodoogy to a co-mnunity devlopment initiative. A multidisdplinary team con- -sisting of two social scientists, a biologist, an environmental infor- mation specialist, and a health and nutrition apert undertook the appraisat in a sublocation in eastem Kenya. To gather relevant data and information, the tean conducted key informant and group in- terviews, direct obsation, informal surveys, and literature re- view. ft also developed the village sktch map, village transects, farm sketdces, time line, trend lines, seasonal calendar, farn Frofile, and technical profiles, which were then used by the team to help the community assess its needs, establish development priorities, and formtlate action plans. -The value of participatory rural appraisal lies not only in its use of innoative data collection methods, but also in initiating a pro- cess that empowers the local community by providing it with vital data and other information that can be used for designing and inplementing grassroots development initiative. The participatory rural appraisal described in this capter was conducted in 1988. The chapter was drafted in late 1989. Since that time, much has happened in Mbusyani as a direct result of PRA. Mbusyani Dam has been completey rehabilitated; a new tree nursery has been opened and is operating; sevend hill slopes have been terraced through the organizing and labor of severl women's groups; private farmers have constructed bench teUrraces on their own land, based on the example of the work of the women's groups; a duster of several women's groups has rmised money and pur- 176 Caity Xabutha Barbr P. ThomSlayera and Rida Ford 177 chased a maize mill that has now been running for two years; and plans are well underway for the rehabilitation of a second dam. Further, several villages adjacent fo Mbusyani are now using ee- ments of PRA to hep in their own village planning and prqect design. It has been four years since the original PRA in Mbusyani, and the community groups are carrying on with continued energy. PARTICIPATORY RURAL APPRAISAL (BRA) uses multi-sector teams to join with village leaders to assess village needs and priorities and subsequently, to create village resource management plans. The- plan becomes the basis for action for development projects in the rumral com- munity, and enables local institutions, government units, and nongov- ernmental organizations (NGUIs) to cooperate in their implementation. PRA draws upon the knowledge and skills already resident in the vil- lage. It also: * creates a setting in which local residents exdhange information with one another and the local technical officers; provides a structure for local aspirations and goals to be expressed and implemented; provides a ranked list of village project activities that funding agen- .ies can support; and * sets in place a plan that village leaders and institutions can imple- ment and sustain (Tomas-Slayter 1992). The PRA approach assumes that popular participation is a funda- mental ingredient in development project planning that takes place at the local leveL PRA is rooted in the earlier work of Gordon Conway and Robert Chambers' Rapid Rural Apprisal and draws much of its data collection and ranking techniques from their work. PRA assumes that locally-maintained institutions and technologies, as well as sustainable economic, political, and ecological inputs, are fundamental to reversing the general decline in these spheres that Africa has experienced in the past several decades. The PRA approach is built on the premise that individual rural communities reside in discrete ecosystems or micro- zones-defined by rainfall levels, soil types, elevation and vegetation, and so on-and require particular and unique coqmbinations of fami, health, soil, water, and woodland/grassland management. PRA further assumes that community residents have a good working knowledge of their ecological and development needs, but do not necessarily have the means to make all this information systematic, or-based on this information-to mobilize their communities to take action to solve problems. A combined team of officers from Kenya's National Environment Sec- retariat (NES) and Clark lUniversity (Worcester, Massachusetts), with 178 Participatory Ruraz Appraisal: A Case Studyfrom Kenya assistance from technical officers from Iangundo Division, Machakos District, field-tested Participatory Rural Appraisal (PRA) in Mbusyani Sublocation in July/August 1988. While collaboration on resources management between NES and Clark has been underway for several years, development of the PRA approach is new. The purpose of the Mbusyani study, then, was to leam whether a rapid appraisal team consisting of NES staff, Clark University represen- tatives, technical officers, and community leaders could gather data, de- fine problems, rank solutions, and devise an integrated village plan. for natural resources management in a relatively short time period, withi substantial community participation. The basic questiors to be tested icluded: * Data: To what extent did Mbusyani residents possess sufficient data on which to build a practical resource management plan? * Participation: Would use of participation as a means for gathering data and setting priorities be manageable, both in terms of organizing irastructure and eventually developing an agreed-upon plan? * "Village leaders and institutions: To what extent would a PRA ap- proach caprurc the attention of village leaders who are in a position to manage such an approach? * Donor and NGO assistance: Would the preparation of an agreed- upon village resource management plan provide documentation that would attract NGO or donor assistance? * Sustainability: Would village groups, once m-obilized and familiar with the PRA process, use it to sustahi resource management activity and design new plans? The NES team that carried out the pilot PRA consisted of a social scientist with extensive agriculturl experience (the team leader), a bi- ologist, a social scientist, an environmental infoimation specalist and a village health worker and nutritionist The team worked closely with extension officers for technical assistance, particularly in regard to water resources and conservation. The team also drew heavily on the energy and assistance of local leaders in Mbusyani, including formal leaders such as the assistant chief, as well as informal leaders, such as heads of women s groups. PRA normally has eight clearly-defined steps (Kabutha, Thomas- Slayter, and Ford 1990), though the procedures can vary greatly, de- pending on locaI need and preferences of the team. These steps include: * site seIection and clearance from local administrative officials; * preliminary visit; * data collection: spatial, time-related, social, and technical; focusing especially on problems of the community; s* ynthesis and analysis; Cltariiy Xabudt, Barbara P. Thomas-SIayter, and RicordFord 179 * setting problems in priorty order and eploatirn of opportunities to resolve them; * ranking opportunities by priority and feasibility and preparing a Vil- lage Resource Management Plan (VRMP); * adoption of the VRMP; and * implementation. These eight steps as carried out in Mbusyani incuuded: Site Selection In the case of the Mbusyani study, the head of the farmers' cooperative union for Mbusyani, the assistant chief, and ten leaders from women's groups in the sublocation came to Katheka Sublocation (where NES and Clark University representatives had carried out a research exercse, trainig sessions, and demonstration day3 in 1987 and 1988) to see how village groups here had organized programs and projects to ease pres- sure from human activity on local natural resources. The Mbusyani grup was so impressed with Katheka's achievements that they invited NES to visit their sublocation to determine if similar work could begin in busyani. Thus, NES went to Mbusyani becaus e the local leaders had requested iL The PRA site, Mbusyani, is a sublocation consisting of eight villages in Kenya's semi-arid zone. It lies 90 Ikr east of Nairobi in Kakuyuni Location, Kangundo Division, Machakos DistricL The terrain is billy, the climate dry (average rainfall is 400 to 600 millimeters per year), and the elevation is about 1500 meters. The population in 1990 was esti- mated to be 8,000. The residents of Mbusyani are Akamba, a group of agro-pastoralists who have lived in Machakos since the seventeenth century. Some Mbusyani households have family ties in the sublocation that date back three or four generations. Due to inceasing pressures on the land in neighboring regions, mnany have moved into Mbusyani in recent years, occupying land that previourly was used for rotational grazing. Today, people in the area derive their livelihoods mostly from subsistence agri- culhire and cash remittances, though about 20 prosperous farmers sell coffee as their primary source of income. The extension of farning and. the subsequent constriction of grazing lands are two forces that have led to accelerated resource degradation, including loss of ground cover, soil erosione and reduced water avail- ability n the region in general, and in Mbusyani in particular. These forces, coupled with erosive soils, steep hill slopes, and torrential sea- sonal rainfall have created a situation in which the community's natural resources are vulnerable. There are many dimensions to this predica- * 2~80 Par licipatory RumulAppraisal: ACaseStudy fivnmKnya merit, including health problems such as bilharzia, food storage needs, access to reliable water sources, high rates of soil loss, and declining ''water supplies. NIES made a preliminary visit on May 31, 1988, and spent a half day meeting with leaders. After extended discussions among NES and Mbusyani leaders, all agreed to go ahead with a PRA. The village lead- ership understood what would be expected of them, especially in terms of organizing groups of leaders and residents for discussions with the PRA team. The NES pledged that it would assist with gathering data, o g the data into a format from which village groups could rank priorities, and helping to prepare a village resources management plan. It was stressed that responsibility for implementation of the plan would be placed primarily with comrmunity leaders, with assistance coming from technical extension officers, the NES, and donor or NGO groups that might be identified. Data Collection The team gathered four basic data sets, in addition to routine secondary information (see the appendix). These induded spatial data, time-re- lated information, data on institutions and social structure, and technical information. All turned out to be important in carrying out the PRA and eventually formulating the resources management plan. The total time for the PRA team and community group for the data gathering exercise was six days. Data Synthesis and Analysis Once the du-ta were collected, the PRA team spent a day meeting by themselves to organize the information. They made large charts and tables of trend lines, institutional arrangements, the transect, and so on, and then compiled a list of all the problems mentioned in any of the data-collection exercises. The team then split up into pairs. Each took a particular problem (water, for example) and in each instance suma- rized the problem, in some cases by subdividing by ecological zone; matched opportunities or potential solutions to each problem; and listed opportunities for each problem on large pieces of paper, big enough to be seen in a large meeting. Ranking Problems The next day, villagers met together to rank their problems. The meet- ing was held in a primary school classroom near the sublocation's mar- awrity Kabutha, Barbara P. Thomas-Slayter, and Rickard Ford 182 ket. Data gathered in the preceding exercises were displayed in the form of charts and graphs on the walls, including the trend lines, transect, sketch map, and others. The charts containing the lists of prob- lems and opportunities were placed at the front of the zoom. The NES team leader chaired the meeting. Participants included the six team members from NES; nine technical officers representing soil, water, for- estry, agriculture, village health care, community development, and livestock; a number of village elders and community leaders; the assis- tant chief for the siablocation; and leaders from eight women's groups. There are many ways to rank problems in this contexL The literature on PRA and The Parh'cipatory Rural Appraisal Handbook offer many sug- gestions (Kabutha. Thomas-Slayter, and Ford 1990). In most cases, com- munity groups have simply voted to derive a list of most-severe to least-severe problems. Sonietines, the voting places clusters of prob- lems in categories of severity; other times, it is a numerical ranking of severity. In one case in Pwani (Kenya), the community agreed on one set.of ranldngs the first day and then, after intense discussions, estab- lished a totally different rank order for problems the following day. In the case of Mbusyani, villagers voted. Given that all problems of a community cannot be solved in one effort, there is a need to organize these problems into sets of issues that can be deaIt with step by step over an extended period of time. The ranking process in PRA seemed to help -and has therefore become an important exercise in Mbusyani's ef- forts to help itself. The outcome in Mbusyani was an ordering of prob- lems that villagers and technical officers agreed were the most severe and in greatest need of attention. The task was essentiaTly completed by lunch time. While there were several high-priority problems that were mentioned a few times, the overwhelming and most frequently-cited issue was water. Ranking Opportunities In the aftemoon, the same group reassembled and discussed possible solutions; in this case, to the water problem. The PRA team had previ- ously listed recommended solutions, ranging from boreholes to rehabili- tating rock springs. The NES/PRA team leader chaired the meeting. The means for bringing order and a systematic approach to the ranking was the use of an 'Options Assessment Chart" developed by Gordon Conway and Robert Chambers. 'he team leader used the criteria of stability, equity, productivity, sustainabiIity, and feasibility to help the group place weighted vakues on each possible solution and eventually to arnve at a unanimously agreed-upon set of actions, including develop- ment of a new well, rehabilitating two small reservoirs and water 182 Partidpatory Rural Appraisal: A Cas Sfudyfrim Kenya catchments, terracing a badly-eroded hill slope, and continuing with efforts of reforestation. Technical officers played an important role in this discussion so that solutions would be feasible in technical, eco- nomic, ecological, and social terms. There was some initial difficulty using the Options Assessment Chart, mostly because the terms (for ex- ampIe, sustainability) did not translate well into Kikamba, the local lan- guage. However, the team worked closely with the village leaders and eventually a ranking emerged. Adopting a Village Resource Management Plan (VRMP) According to the original schedule, the group was to reassemble the following moring and create a viUage resource management plan. However, several leaders and especially the water engineer felt there was insufficient technical infonnation to develop a comprehensive plan. The water engineer returned the following week and, joined by the en- tire PRA team and the Mbusyani Resource Management Committee, visited all pbtential water points in the sublocation and expanded con- siderably upon the technical feasibility document. Figure A9" in the appendix includes excerpts from the technical survey. Two weeks Iater, the entire sublocation committee again assembled and, using data that the water engineer's survey had developed, orga- nized a comprehensive water and natural resources management plan for Mbusyani. The plan indicates what tasks are to be carried out, what materials will be needed to do the work, who will do it, and what-if any-extemal assistance will be needed. A brief extract from the plan isWshown in Figure 9.1. Implementation Implementation in Mbusyani has moved forward, not always on sched- ule, but with considerable commitment from all parties involved. The assistant chief has been the pirme mover, with major cooperation and support from many different constituencies in the sublocationL The first task, a well at Kitiini Springs, went quickly, with labor and local materi- als from the community, and cement rings and skilled labor from the Ministry of Water Development No external funds were required. The second project, rehabilitation of a reservoir at Mbusyani Dam, was more ambitious. Women's groups supplied labor and dug dozens of fence post holes to erect a sturdy fence to protect the watershed area. The groups also dug several meters of bench terraces and planted many trees as a means of curbing erosion and siltation in the reservoir. A local NGO heard of the prcect and agreed to provide fence posts, wire fenc- Chaiiy Kiabutha, Barbara P. Thomas-Slay r and Richard Ford 183 Figure 9.1 Vmqae Resources Mfanagement Plan (extract) WATER. Zone I SoUrGe: Estimated Committee By Priority Requimments Responsibility Projected Time Kwa Nzau 1. Hand pump 1. Extemal -mmediately Well 2. Cement rings 2 MOWD 3. Cement and cover 3. Extemal 4. Sand 4. Community 5. Ballast 5. Community 6. Labor 6. Community WATEL Zone II Kwa Kiluli 1. Posts and wire 1. External Work will start on Dam 2. Terracing 2. Community June 20,1989, on the rehabilitation 3. Afforestation 3. Community spillway. 4. Spillway 4. Community protection Kwa Kiluli 1. Hand pump 1. Extemal Immediately Well 2. Cement ing 2. MOWD 3. Cement 3. External 4. Sand 4. Community 5. Ballst 5. Community 6. Labor 6. Community 7. Cover 7. External Yenyeni 1. Dig well deeper 1. Community Work to assemble Springs Well Z Protect site 2. Community materials will sart rehabilitation 3. Technical design 3.MOWD on June 19,1989. 4. Hand pump 4. External Actual building will 5. Sand 5. Community start wfien water 6. Ballast 6. Community level drops, and 7. Cement 7. External digging can begn & Wire and poles 8. External 9. Rings and cover 9. External ing and tree seedlings for Mbusyani Dam. The NMO also wrote a .successful proposal to a UN agency, using data developed during the PRA exercise, and noting how the work on the dam was part of the 184 Participatory Rural Appraisa,: A Case Studyfrom Knya larger village plan to bring sustained production to its natural resources. The funds provided additional fencing and a means for scooping silt from the bottom of Mbusyani and Kakuyuni reservoirs While the paperwork has taken 18 to 20 months, the project is now getting under- way. A third element of implementation was developing soil control on a badly-eroded hill slope. The Ministry of Agriculture donated tools to carry out this work, and an additional allotment of tools has been ob- tained through a.private donor. Women's groups have provided many hours of volunteer labor for the effort and planted hundreds of trees along the new teiraces. Twenty-four months after adopting the plan, Mbusyani has made considerable progress, though the work is not yet finished. It is unclear whether the enthusiasm and work energy that has been maintained over tiis 24-month period will continue. It is equally unclear whether the small external inputs, such as hand tools from the Minstry of Agricul- ture, fencing and posts from an NCO, cement rings from the Ministry of Water Development, and funds from the UN have been critical to the success, or whether the community would have been able to organize altemative means to acquire these inputs on their own. Fndings and Conclusions The introduction of tiis case study. set out five clusters of questions to be tested by the field methodology. Findings from the Mbusyani exercise (and reinforced by subsequent PRA field exercises) are covered in the following sections. Data Mbusyani residents know a great deal about their community. The time line and trend analyses revealed an intimate awareness of what past circumstances had been. The seasonal calendar indicated an inte- grated understanding of past problems and present needs. Yiet m spite of these good data, there were gaps in the information base on topics nornally assumed to be important in development planning. Villagers were generally unaware of quantified information, for example, statis- tics on rates of soil loss or percentage changes in infant mortality. They were also weak in anticipating the econemic and technical feasibility of proposed solutions. Fmally, they were mostly uninformed about ways to find technical and financial support to implement their hopes and aspirations. Even so, these data gaps seemed to pose no serious problem in for- mulating the village resource management plan. Given long-term in- COarit Ka, Barbara P. Thomas-Slayter, and Ricard Ford 185 volvement with the commnunity, villagers had sufficient data to rank problems and consider solutions. Extension officrrs provided technical and economic infornation as needed. As a result of the experience, the community was able to rank solutions in ways that reflected their own knowledge base, integrated with technical and economic considerations provided by members of the PRA team. In the judgment of the PRA team, the villagers, in association with extension officers, had access to sufficient data to formulate sound re- source management plans. Participafion PRA provides an organizational structure that focuses and system- atizes grassroots participatiorL In the present case, Mbusyani residents responded actively at every level. For example, there was no shortage of elders to describe past events and present trends; people were inter- ested in the seasonal calendar exercise; and discussion was prolonged w.hen the relative importance and relationships among institutions in the sublocation were considered. During the formal meeting to finalize Mbusyani's VRMP, discussion was vigorous and often intense. While women were sometimes reluctant to speak in the presence of male ex- tension officers and local officials, they did speak up with conscious encouragement by members of the PRA team and specific support methodologies. By te- end of the sessions, the women participants. made their feelings well known. Two elements seem to have structured the participation. First, data- gathenrng from village groups sent a message that the PRA team had interest in knowing what the community knew. As the participatory data-gathering continued, a momentum began to build. At no point were there ever more than 300 or 400 people involved in the process, out of a total population of 8,000 (or roughly 4,000 adults). Thus, it would be inaccurate to say that the entire community participated. Yet significant numbers did participate on a sustained basis, suggesting that the process was important in attracting the attention of the community. Second, the task-oricntcd and visual nature of the PRA was impor- tant. Village residents could see what the researchers were collecting and felt they could comment, for example, on whether the trend line was being accurately drawnL The interactive and taiigible nature of the data deinstified the research process and made the communitv feel a sense of ownership of the PRA from an early stage.. While there are certainly other elements in explaining why Mbusyani has implemented its resource management activities, the PRA team agreed that the structured and systematized participation has been one of the mest important. 186 Participatory RUral AppdSol: A Case Studyfrom Ke"Ya Village Leaders and Institutions Locally-initiated plans require committed and skilled local leaders to follow up on the recommended initiatives. In the case of Mbusyani, the assistant chief ciemonstrated these qualities admirably. He called meet- ings, organized work groups, attended local committee discussions, and kept good records. He also served as liaison for dealing with extemal groups such as NES or the NGO. His leadership, while not necessarily flashy or charismatic, was steady, reliable, and thorough. This consis- tency in the assistant chiefs leadership in follow-up is a fundamental element in explaining why the PRA research has produced results. It must be stressed, however, that the assistant chief was not the only leader in the formula. To date, he has managed none of the money that has been raised. All funds have been adminstered by one of the exter- nal agencies (for example, the indigenous NGO) or by the Mbusyani Women's Group. Thus, a second crucial quality of local leadership is the skill and commitment that leaders of a half-dozen women's groups have brought to Mbusyani. They have carried out bench terracing fence-post installation, sand and gravel collection, and tree planting They are now coUecting money for a maize grinding mill and are gather- ing fumds for a cost-sharing acquisition of tools. As is the case with the chief, the women's leaders are not necessarily charismatic. Instead, the explanation for their success in leadership lies in simply follouwing through with assigned tasks and delivering services on time. The assistant chief, the women's group leaders, and the community institutions are essential elements in the Mbusyani project's effective management Given the recent shift in the development field in Kenya toward decentralization of development planning, thrugh the District Focus, there is lear evidence that PRA provides a methodology that enables rural institutions to function more effectively. The PRA team agreed that cultivating such leaders and institutions in new commimi- ties will be a fundamental element in expanding the work of local sustainability. Donor and NGO AssiStance A problem prevaiing in much of rural Africa is one of dependency on outside help. Rural communities wait, beg assistance from donors, or implore help from politicians to get water systems, agricultural sup- plies, and the like. This "client relation-hip is demeaning and perpetu- ates a second-dass status for rural communities. PRA suggests there is a great deal that rural communities can do for themselves. The process of ranking needs and designing solutions in Mbusyani placed initiative in the hands of the rural groups in ways that identified: COaity Kabutha Barbra P. Thomas-S aytert and Rihard Ford 187 (1) what steps community groups themselves could take; (2) what ser- vices and materials they could organize from local extension staff; (3) the role of NGOs; and (4) how requested donor assistance could comple- ment items (1) through (3). Mbusyani's experience suggests that when a community raises exter- nal funds, these funds actually reach the intended beneficiaries. At present, donors are not set up to respond to rual requests. While the structures for such support are potentially available, the present process of project identification and design tends to rely on centraIized and external agencies. Thus, while PRA shows that a 'bottom-up" approach in village assessment produces tangible results, there is equal need for donors and governments to restructure a portion of their efforts to re- spond to such locally identified project desigrs. Sustainability The crux of the issue in rural development is how long Mbusyani will carry on without continued encouragement or help from NES, an NGO, or other exteral sources. It is too early for a final judgment However, a number of interesting and perhaps significant results have been ob- sered. First; much of the PRA follow-up has been carried out by the commu- nity itself. Bench terracing, fence-post installation, watershed rehabilita- tion, and well-digging have been completed independently of outside help. Second, a local Kenyan NGO discovered Mbusyani's VRMP and tumned it into a fundable proposal- While delays within the UN bureau- cracy dampened the initial enthusiasm, the funds were eventually alo- cated, and both the NMO and Mbusyani are benefiting. There will probably be additional such "joint" ventures as welL Third, both the Ministry of Agculture and the Ministry of Water Development have been moved to provide assistance that would not necessarily have been forthcoming otherwise. In both cases, they did so because the community was organized and the ministries knew that any supplies made available would be put to good use. Costs The experiment in Mbusyani took place quicldy (in less ian 10 days), was inexpensive (less than $1,000 in new money), participatory, and provided the community abundant and systematized data that has led to the VRMP. If the total cost of this PRA were to be calculated, includ- ing staff time, estimates would include: The time of 3 NES officers, at 15 to 20 days each, - 188 Participatory Rural Apprmsal:A Case Studyfrom Kenya * The fime of 3 extension officers, at 10 days each, * A corps of perhaps 10 village leaders, at 10 days each, * An extended cluster of secondary village opinion-makers and "mov- ersT of about 25 people, at 3 to 4 days each, * A much larger community group of perhaps 400, who attend one or more of the data-gathering or ranldng meetings, at about 2 days each. Thus, the PRA methodology has been effective, at least in one com- * nunity, in turning data into action, using minimal funds. PRA in Perspective PRA is a new way to look at village data, rural priorities and aspirations, and potential interventions. It offers possibilities for restructuring the way rural planning and resources management take place. Yet nothing in this case study should be interpreted to mean that the PRA process can replace national and district planning data collection, and analysis by various government agencies, technical and econonmc feasibility studies, envirornmental inpact statements, or extension services pro- vided by techical officers. Rather, PRA should be considered as a. supplementary methodology that gathers site-specific data, integrates sectors, actively involves beneficiaries in the planning process, links ex- tension services directly with rural communities, and sets in place an "implemenitable" plan. of action that local institutions can take seriously. As the development community searches for means to hitroduce sus- tainable development, the PRA methodology appears to bring a mix of elements not presmt in most donor- and government-initiated method- ologies. To this extent, PRA wanants further testing and refinement in various ecological, cultural, economic, and political settings. References Chambers, Robert 1983. Rural Development Putting the Last First. Essex, U.K: Longman Scienfic and TechuicaL Chambers, Robert, Arnold Pacey, and Lori Ann Thrupp, eds. 1989. Farmer First: Farmer.Lnnovation and Agricultural Researh. London Intrmediate Techinol- ogy Publiatins. Conway, Gordon R., and Edward B. Barbier. 1990. After the Green Revolution: SustRinableAgrid hu reJbr Development. London: Eartbscan. Davis-Case, D'Arcy. 1989. Partidpatory Assessment, Monitoring and Evaluation: A Field ManuaL Rome: Community Forestry Unit of the Food and Agriculture OrganizatiorL 1990. The Community's Toolbox: Te Ide, Methods and Toolsfor Partidpa- fory Assessment, Monitoring and Evaluation in Community Foresby. Rome: Community Forestry Urnit of the Food and Agriculture Organization. ChariNtyKabuthia, Barbara P. Thomas-SlaytEr, and Rhrd Ford 189 :asttman, Ron, Richard Ford, Anne Gibson, and James Toledano. 1990. An Intro- duction to Geographic Information Systemsfor Resources Management. Worces- ter, MA SARSA. Inter!national Institute for Environment and Development January 1990. Manuna on RRA ana Related Approaches. Lonsdon: RRA Notes (Number 8). pp. 30-35. Kabutha, Charity, Barbara P. Thomas-Slayter, and Richard Ford. 1990. Participa- tory Rural Appraisal Handbook. World Resources Institute, in collaboration with Kenya's National Environment Secretariat, Egerton University, an! Clark University. (Also contains the full Mbusyani Vilage Resource Man- agement Plan, pp. 72-80.) Kabutha, Charity, and Richard Ford. October 1988. "Using Rapid Rural Ap praisal to Formlate a Village Resources Management Plan? RRA Notes (Number 2). (Contains a detailed description of the ranking procedure, and an example of the Conway/Chambers Options Assessment CharL) Korten, David C 1990. Getting to the 21st Century: Voluntry Action and Mhe Globrl Agenda. West Hartford, CT.: -Kuarian Press. Mascarenhas, James, et al., eds. August 2991. Participator Rural Appraisal: .Pro- ceedings of the February 1991 Bangtore PRA Trainers Workshop. London RERA Notes (Number 13), IIED. Molnar, Augusta. 1991. vCommunity Forestryn Rapid Appraisal? Rome: Com- munity Forestry Unit of the Food and Agriculture OrgarizatiorL Rocheleau, Dianne. Land-Use Planning with Rural Farm Households and Commnii- ties: Participatory AgroForesty Research. Worldng Paper No. 36. Nairobi, Icmnya: International Centre for Research on Agroforestry. Thomas-Slayter, Barbara P., and Isabella Asamba. 1991. From Cattle to Coffee: Transormation in Ruml Machkaks. Worcester, M& ECOGEN Case Study Series, Clark University. Thomas-Slayter, Barbara P. 1985. Politics, Particpation, and Poverty: Development Through Self-Help in Kenya. Boulder, CO: Westiew Press. : __. November 1989. 'Implementing Effective Local Management of Nahtual Resources: New Roles for NGOs in Africa.' Huntan Organization 51(2):136- 143. 190 Partidpatory Ruml Apprisal:A Case Studyfrom Kenya Appendib Information Gathered Box A91 Inforamation about the Village Sketch Map Definition: The sketch map is a spatial representation of the commu- zity. Purpose: It provides reference points within which data collection, analysis, and planning take place Process: Whereas the assistant chief already had a 1:50,000 topo- graphical map, he did not have a detailed map of the sublocation nor did he have any definition of micro- ecological zones within the sublocation. The team traced a base map, using the 1:50,000 topo map as a guide. Then the 1PRA team, two villatge elders, and the assistant chief drove all roads and lanes in the entire sublocation-a trip of about 2 hours-recording information and talkng with people, as appropriate. Result: The map exercise identified three micro-zones, defined largely by elevation, soils, and rainfall The Upper Zone Has somewhat higher raifall, generally fertile soils, and potential for growing coffea The Lower Zone is generally drier, has few water sowuces, and generally lower agricul- tural potentaL Usefulness: Knowing about ni5cro-zones, disparities in wealth, differ- ences in land use, and variations in resource access pro- vided an opportunity for the assistant chief, women's gup leaders, and the PRA team to locate areas where lo- cal leaders thought there were particular problems. Having this initial visual reference provided common ground for the team and local leaders to exchange informatiorL The sketch map also suggested how the team should go about preparing the transect (next exercise). Assessment Preparing sketch maps as a first step has proven to be a dramatic and visually important way to announce to the community that something is going on. Several PRAs have had exellent responses in having the communities prepare their own sketch maps. The exercise is one of the most important PRA data-gatheing tools. rune: The mapping exercise took half a day. CIaIty lCabnh, Barbam P. honas-layer, and Ridcard Ford 191 Figure A9.l ViDlage Sketch Map LEGE' Tolt MATETANI G~ d ad ro To Kavihiurm .~~~~~ub, \ _uw.thMo< ~bame.be t g&/ XUPPERZONE KAWETHE To Kag rO Ulhini DL { V a /~~~~~K g~~~~~~Y ALK ToKafl S\ X I JFF MEDIUMZONE >;. : T ~~~LOWERZONE \r >4 'S ox \ madam o . E I(YEVA~~~~~~~LU I To Kat 192. Participatory Rural AppraisalA Case Studyfrom Kenya Box A9.2 Village Transect Definitlon: A transect is a cross-section of the communuity, showing eco- logical cultural, economic, and land us conditions. Purpose: The transect identifies types of land use, local perceptions of problems, and village views of opportunities to solve them. It helps the team to verify the validity of the sub-zones, which the sketch map has defined. There are several goals in using the transect. First, the transect confirms sub-zones set out in the sketch map. Normally, the PRA team pencils in a rough track on the sketch map that will cut through the dif- fering micro-zones in the community. Thse zones. may be distinctive in many ways, including their ecology, cropping patterns, economic activity, wealth, ethnicity of residents, or type of land tenure system Using a transect enables the team to double-check the approximate zonal estimates in the sketch map. Second, the transect enables the team to begin looldng for both problems and opportunities on a zone-by-zone basis. The core of FRA is developing collaboration among cornmu- nity institutions to define and act upon their needs, as these institutions perceive them. The transect is the first formal data collection step in this process. Third, the transect exercise is an important entry point for team members to interact with a broad cross section of the community in totally non-threatening ways. As team mem- bers walk the transect line, they will encounter farmers, school children, water caniers, animal cart drivers, people headed for market, cattle herders, and more. The occasion of building the transect allows team members to ask these vil- lagers what kind of yields they had last Year, how prices are changing for maize or fertilizer, whether the water table is rising or falling. and how they descnrbe their most severe problems. Experience in doing several PRAs suggests that these casual conversations yield huge dividends. There are several rea- sons why. For example, a casual conversation on the road and in an anonymous setting may yield an honest response about prices or problems, whereas a formal sit-down inter- view may encourage cautious answers. Further, a chance meeting on a village path may bring together two or three totally disassociated residents candidly discussing, for ex- ample, the current fuel wood oir forestry situation, without continued on next page transect conversations call forth comments fim people who nmight not normally speak up in a community meeting or any * ~~~~kind of an interview. Womnen, impoverished residents, eth- nic minorities, or youths may speak out amiid the anonymityr of the transet path, but remain silenLt in an interview. Process: While some PRA teamns will walk a straight line from one side of a commnunity to the other, intersecting all micro- zones, the Mbusyani team chose a different tactic. The dis- tances were great and the goal of the transect was to understand transactions withiin each micro-Tone, so the team * ~~~~~~broke into three smaller units and explored routes, as noted on the sketch map. T'he sub-groups joined with.elders and asked questions as they walked. In each case,the sub-groups started it the highest point in the micro-zone and fanned out from that area. Results: The transect provided a detailed look at land use practices, present problems, and potential solutions. It also ermabled the team to confimn that considerable variety in ecology and land use practices was present in the subkocation. Llsefulness: Preparing the transect enabled the PRA team to become ac- quainted with details of the inicero-zones. As the sub-groups waalked tromugh the community, they observed, recorded data, anid stopped and talked with whomever they meL A list of problems and opportunitie for the sublocation began to emer-ge. Assessment: The transect is on a par with the sketch map in order of value to the PRA team.L It allows cross-zone comnparisons, high- lights 'diversity in problems and opportunities, and draws out humble elements of the community. It has, like the sketch map, earned a solid place in the arry of PRAAs. infor- mation-gathering tools. Timne: Given that the team split into three sub-groups, it took only hailf a day to gather the data. Then, one member from each sub-grup sat together in the evening and prepared the com- posite trnect that appears in this study (Figure A9.2). 194 Pari d pa toryRu Appraisa. AC4eStudyfivm Kya Figure A92 Composite Transect Son Looo., dep red ai Sandy soils and sma Sauow sandy patches of red soils seat rocky In raostparts Watr About 3/4 of houseolds A rh infOted with Waler In Kilndilrd lave ow wlls, bilhar 2 poorly River salty; River ae also has 3 dams maintaLied dams Kathans, biuha;la; and 1 spring roof catduent Vegefttion AII natual vegetaton High proportion of Natural vegetato deared to give way natusal vegetation, coniting of acada, to settlement acids lana, canaa shrbs, and gass Sodal- 1/2 of houseold 12 of houselds Mainy grass economin heads In wage tin roofs, 1/2 thatched, thatched hous Indkr eplaylmt, brick walls majatymnLabad rxt% brckor stonewas Food aps Mhfze, bean Maize, ban Mahe, bs, bananas,pigeon pigeon peas, uts, banaa, fruits, peas bananas pigeon peas Cash aops coffee Coftee Coffee Achievements Soillcnsevation Sol cosevatioc, Some soil and (ast 5 yea) ree planting, water devlopment water conser- walerdevelopani dam Tatbn wells, roof aschmunt Fresry/ Widespread Minial tme plztitg Very tle tree Apofoestty aro4orestry mangoes papayas planting mangoes,papayas Resourcs TrIcA, A lot of benh Limited soil mangement embankment terran conservation nfeomd with multi-purpos grasse Problems Inadequate water. Water, famine, Wate, trport education, heith inadequate education andfood fad;lte, famines, and health failities lak of dip fcilities Opportunities Rehabliaton 3 dams Water development Water development 1 spring, etal dam, well, roof dams, roof catch- assistance, tools, catchuentt government moot exten markset assinc assstanc Ozarityfabutha.Barbara P.7Thomas~-Slayfeb and Richard Ford 195 Box A9S3 hIformatlon about the Farmn Sketch Definitionc Farm sketches are hand drawings of farm layout and use and include cropping patterns, buildings, tree locations, and wa- ter sources on individual farms. Purpose: They ilustrate relationships between resource management practices and a variety of variables, including income, educa- tion, iand ecology for a representative sample of households in the community. Process: The PEA team broke into smaller units and visited farms along the route of the three tranect paths, paying attention to examples of the variety of ecology, incomes, land uses, and famidly sizes present in the community. Team mewmber-s prepared skeathes by walking around the farms with house- -hold heads. In subsequent PRAs, the teams have combined the exercises of farm sketches, fam interviews, and the transect, just to save time. Results The farm sketches showed individual farm management practices and enabled the team to compare facilities and strategies among the three micro-zones. Usefualness: It became clear that the Upper Zone was, on a relative basis, mtore affluent than the Lower Zone, had better-managed farms, produced more, and provided higher incomes than in the other two zones. The farm sketches (coupled with the Larh interviews noted below) confirmed those differences. Assessment: The utility of the farm sketch depends an the partimar PeA. In most PRA exercises, data have been gathered as a nmean of stimulating and planling community-based enterprises such as water resources, marketing forestry, soil control, transport, etc. There has been relatively little 'on-farm' work resulting fm existing PReAs. Thus, the samnpling tech- hique for picking 6-h0 individual farms for sketches and in- terviews has been generaly qualitative (class, ecological Zone, size of holding, te of land use, family size) rather than precisely quantitative (precise icome, yields per acre, percentage of i acome derived from farm). To date, the gen- eralized sampling has served PRAs well as the site selection can be determined during the transect and sketch map activi- ties. If individual fann strategies al to become a vital element of the PRA vlage action plans, it wil be necessary to con- sider more systematic sampling techniques in choosing fanai sites. Time: Farm skethes took an entire day, even winth the team break- mig into three sub-groups. Combining different data-gather- ing exercises, as noted above, has enabled later PsA teams to gather all interviews and all sketches in ore day. 296 PartidpaloryRuralApprai:A CascStudyfrom Kmnya :Fgure A9.3 Farm Sketch .ALLt . EASAI wI ,~~~ * AAL m5 19* g FALLDw FoiL GgAZIN(a *'''' MA.sG M FM ZS A A6 A 19.4'ta U'APA(A - \~~~~ 0 AVOCADC *°h EIMo4 xUE CEMAIlOD k ; {-9 AC§!{S 6CA-|z ?LAM%ED MCJl FI4EL S SAN A WAS ~ t O 3 PXDUlLT$ TEIUt*Cl;§ P,}^tD PAPAYA C514LDQUN GRE't 51SAL Charity labutha, Barbara P. 7homas-Slayter, and Richard Ford 197 Box A9A Information about the Time Line Definution: The time line Is an aggregate of past events, as Influenced by present conditions. Purpose: The goal Is to leam from the comnnuwity to understand what it considers to be Important in its history. The time line provides an insight into the community's historical perspec- tive on current issues. Process: A PRA team meets with residents to discuss what they con- sider to be the most Important events in the communitys past and to prepare a time line. It is important to note that groups assembled for data-gathering should reflect the age, ethnic, gender, class, and educational diversity of the com- munity. Data are gathered in group meetings with explicit attention paid to-including community residents from differ- ent backgrounds and perspectives. Problems and opportuni- ties are discussed in these meetings. Depending on the size of the community, three or four groups are organized, representng different sub-zones. For example, in MbusyanL with a population of about 8,000, the PRA team split into four groups and asked the assistant dcief to arrange for groups (about half elders for the time line) to come to four local market centers to meet the visitors. The role of elders, both male and female, is important in Africa, where age is so highly respected. Further, the elders carry some of the community's history with them and personify some (but not all) of the values, priorities, and aspirations of the community. They lend a sense of continuity to commu- nity action that has been sorely lacking in many develop- ment interventions. The goal for the time line is to understand what the commu- nity considers to be important in its history. A PRA team leams from discussions among small groups, with emphasis on community elders. These discussions stimulate ex- changes about problems and achievements as far back as the most senior local residents can remember. In Mbusyard, the elders presented a vast store of information (see below) that normally might not be offered to a visiting researcher or project design team. Results: The community emphasized earlier problems that either they had exprienced or that they remembered previous genera- tions describing. Drought is an important eIement of Mbusyani's past, as is famine. The exercise pointed out to cn tinued on next page 198 Participato-y Ruru TEAppraisal: A Case Studyfrom Kenya Box A9.4 (continued) the PRA team that problems of aridity and periodic drought were not new to the conunurity. Usefulness: The ime line exercise worked effectively in Mbusyani. Resi- dents became deeply involved in describing their experi- ences, especially their hardships. The exercise drew heavily on the elderly, who are not always consulted by planners and project designers. The exercise also helped to confinm some of the emerging hypotheses about the nature of Mbusyani's problems, as well as aspirations. Assessment While not as central to PRA as transects and sketch maps, the time line serves an important purpose, that of informing el- ders that their views are important. One of the features of rural African development is the age of the younger, West- ern-trained NCO and extension officers versus the ..ccumu- lated wisdom of village elders. The time line has been an important element in several communities to open dialogue between these two age groups. Time The time involved in MbuLsyani was a full day, though nor- manly half a day is enough. Charity IKabuthz, Barbara P. 7honas-Slaytr, and Richard Ford 199 Figure A9.4 'ne Lne 1836 Yangoyo famiane 1850 Ya Kiasa fanine 1861 Mutulungo famidne 1870 Ngelete famLine 1880 Ndata famine 1885 Kyumbe (Dance), -1897 Rinderpest outbreak 1898 Yua Ya Munyili hMblia and KjIunmi Dance 1898-1900 Mission at Muisuinti-Kangundo 1899 Railway li-ne at lConza 1906-1910 I g on Mbou, Chief Ntheket_ 1910 Ndata la Yaunie 1910-1911 Yua Ya Ndata (abaramgwe) famine 1912-1914 Money introduced 1914-1918 Destockcing by white. man -MAua 19141-1918 Men carried to war 1915-1918 Compulsory school 1918 Diarrhea 1918-1919 Epidemic lung disease of domestic animals-Munyiyli 198-1929 Nzalulangye fanmine 1930-1931 Locusts 1931 Kuthulia sikuyu, fencing with sisal 1939-1940 Muindi Mbirngu campaigned against forced destocicing 1939-1945 World WarIf 1942 Munyoloko famine, enforced conservation measures 11943-1945 Mwolyo 1949-1951 Dams started 1950 Mbua Ya ICanzi, Mabolelo floods 1950-1951 Drought after floods, forced destoclcng 1951-1952 MbYu Ya Kavisi 1952-1954 Emergency 1960-1961 Food for work 1962 Yua Ya Ndeke, bumper hrvest, prcoecs on service dams started 1965 Yua Ya Atta 1973 Drought, eclipse of the sun 1918 E)awrthqae 1980 Nikwa Ngwete 1984 Drought, army wonns 1984-1985 Choleam outbeak, epidemic on lemon reems 1987 Noukengwatie 200 PatidpatonyRun Appmis:.A CarStudy,fium Kmny Box A9.5 Informaton about Trend lines Definitiown Trend Ines plot village perceptions of change over time in key sectors Purpose PRA assumes that local communities have a good grasp of changes over time within thr own villages and that these data are fundametial ihelping communties plan manage- ment strategies for their resources. Trend line discussi bring together all ages mnd groups in the comnmnity, indud- ing men and wom, to elicit thir perspectives on "the way things are goingf The trend lines demonstrate village per- spectives over 20 years on dhanSes in resource issues such as rainfall, crop production, soil loss, deforestation, health, population, and other topics of community concrL They, complement the time lines. APRA team organizes groups of residents aid leaders for this exercise. Proces In recruiting trend line groups, less emphasis is placed on elders than for the time line groups. More atteion is paid to those currently using the land. The reason for this Is that time lines go bak as much as a century, whereas trend lines look only at tie recent past and how it is changing. Normally, villagers talk about trends in some basic sectors such as soil loss, rainfall income, education, populatio, and food production. But the open-ended style of PRA enables villagers to add new categories for trend analysis. For ex- ample, in Muyani, local leaders charted their problemfs and dicussed wheher they had more problems in recent years or earlier. This discussion undocked a rnge of perpecHves that might never have surfaced in a more formal interview or questionnaire setting. Further, the disc n -ammg villag- ers conereing wheier the trend line is going up or down revealed an inner core of villagers feelings about basicissues in youth unemployment, the need for vocational traLinig constaints of land, and the grinling poverty that many vil- lagers ficed. A team leader from NES led the discussion, using chalk and a blackboard (the meeting was held in a school classroom). The groups developed trend lines for population, rainfall/water availability, land productivity, formal education, tree planting, soil conservation, and ero- sion controL Result: Trend aalysis helped the team to: () leam from the commu- nity how they perceive dhange over time in various sectors; (2) integate key change into a viage profile; and (3) orga- nize the range of opportunites for the community to consider. . ~~~~~~~~~~~~~otinued on next page CiWyKabufha, Barbara P. Thomas-S1aySre and Ridzad Ford 201 Box AS5 (continued) Usefiness: Several of the earlier data coiection exers bad focused on opportunities to solve problems. This exercise expanded tte list, daifying local peweptions of important changes within the community. Assessment The trend exerdse turns out to be one of the most important opportuities to open discussions on village problems. Rater than ask direct questions, as with questionnaires and surves, the tend lines allow a PRA team to draw out infor- maion about problems indiectly. Furthe, the trend lines enable commurity members to talc about nterventions that did and did not work in previous attempts at-for example-and productivity, soil erosion, water deveopment and more. It must be bom in mind that these exercses are measuring perceptions, not actual performance. However, given the goals of PRA to use data-gathering and analysis to stimulate communities to plan and to act local perceptions are exceedingly important riM The tend analysis took half a day. 202 PtcipatoRuralAppisa:.A Case Studyfrom Knya Figure A95 Trend Lines IAND PP=Yvzvn nSo Mzo lqg tl,°q0(4 U RSO fl4l~l%OwOL¶SO Ii¶o 1945 A bumper harvest due to good rains 1977 ncaease in yields due to training carried out at Machkos Farm- ers Traming Centre. Free seeds were provided (beans and mize) and free fertlizer as well 1988 hicrease in productivity due to: application of chemicals; manure; fertlizers applied to shambas in 1987 were not used by crops due to shortage of rains plus destruction of crops by cutworms, cridck- ets, and stalk borer RNNIALL mc Rw 1sas nbo mo nfo mqa 1940-1942 Mbulunga famine 1943-1945 Mwolyo famine 1960 Maa Ya Ndege (famine due to floods, food brought by air- plane) 1978 Nkcwa Ngwete famine chariy Kabul1w,Barbara P. Thomas-Slay ferandRk*ad MFord 203 Box A9.6 Information about the Seasonal Calenda Definitiowr A seasonal calendar is a sampshot of village activities, prob- lenms, opportunities, and possible interentions, represented over the course of a year. Purpose: It enables a PEA teami, using group discussions, to learn about land use, hunger seasons, farming activity, times when disease is more prevalent, food surplus, and cash availability. The seasonal calendar also adds to information about village views of problems and opportunities. Procesm: The assistant chief assembled a group representing different age, gender, and leadership perspectives. The PEA team leader asked questions about what activitie were mast im- portant during the year and when they happened. The re- sponses helped the team to prepare a profile of activity in the village on a monthly basis.' One of the valuable contributions of the seasonal calendar is the chance to link problems and opportunities to an annual cycle. For example, in some communities, dilemm-as such as hum-an and animal diseases or seasonal hunger occur virtu- ally every year. In times of drought or blight these problems may start earlier and last longer. But they recur year after year. Other problemis, such as access to fuelwood or the nieed for buildinig miaterials for water storage, persist at a sustained level throughout the year. Results: The seasonal calndar presented larg quantities of divers information in a time frame. It compared vilage activities, month by month, across sectoraI boundiaries. These annual cycles helped to determine, for example, labor availability, diming for projct activity, potentil absorptive capacity for new activities, tizme of disease and food shortage, and varia- tions in cash flow. Usef ulness: The calendar was one of the chief determinants to show that water scarcity seemed to be at the core of many of the community's problems. Regardless of season, water ap- peared to be a persistent problem. The seasonal calendar rantks with the village sketch mnaps, trend analysis, and transct as -one of the four basic exercises to grasp the 'big picture of the commnunity. It may be the mnost helpful tool foDr integrating problems, opportunities, and potential actions into a single visual framework that communurities can discuss. Experience indicates that the sea- sonal calendar is critical to the succes of the PEA. Tme: Group discussions on the calendar took half a day. 204 Pari4ioy RuiraAppraisalt A Ca Stuedyf ma FIgare A9. Seasonal Calendar J P U A J A 1 0 N D I l-W- t I-I I I I I tI -kF Coffee__o e ]_ _ Baans_ - __: [i Papaya _ _ _ _ _ _ _ _ _ _ _ _ _ _ Mmm L] m_ Pumpkin Wf Maiz Beansba t- Pigeonpeas _ T I F'-iger mile _3 . Sorghum _ _ Labor Sdh o l Lalm~11 Cltadey Kabutha,Barbara P. Thoma-Slayter~and Ridhari Ford 205~ Box A9.7 Infomalion about Farm Interviews Pefinition: Farm interviews collect detailed information for a small number of households they supplement the farm setche (see Box A9.3). Purpose The Interviews yield socioeconomic information. Topis in- dude the nature of resource management practics, the char- acteristics of the particular farm household, and the respondents' observations on household and community problems. These interviews off-er the team an opportunity to hold discussions with residents who might not normally be included among the leaders or other group meetings. Process Household beads were the same as those selected for farm sketches. They represente a cross-section of the community by gender, class, and eclologcal niaro-zohe. Results- While the farm interviews yielded good data on family size, income, education levels, numbers of livestock, etc., the prime use was to confirm the growing list of problems and opportunties. The interviews, represenftng the diversity of the commaunity, reinforced the hypotheses that water was a problem for the entire sublocation but was especially severe for households in the lower and poorer zonae Usefulness: The most hpful dimension was to assure that problem defi- dntion and eventwua program activity would be sensitive to issues of equity in the sublocation. Asessment The commentay on farm sketches (Box A93) is equally ap- plicable here. Farm interviews take one day if they are integrated wth the farm sketches A ful questiionaire is found in the PRA Handbook, pp. 43t47. 206 Participatory Rura Appmnisal: A Case Studyfrom Kenya Figure A9.7 Farm Interview (excerpt) This Household Data Form is to be completed for each interview and submitted to the PRA team leader at the end of the day. It records basic household data. The remaining information is to be collected as Field Notes, using the categories described in the Questionnaire Guidelines. Name Position in household. . Zone_ Male_ Female_ Age Marital status Highest leivel of education attained for husband and wife: husbandw wife Place of origins of parents and grandparents? How many children have been born to you? How many children are living? How many children are living with you on the farm? How many people in totaI are living on the farm? Does anyone in this family have ajob outside Kyevaluli? Do they help sometimes with such things as school fees or money for ferilizer? Ch4rityl4bufhaBarbrmP.7Thomas,-SlayterandRichrdFord 207 Box A9.8 Information about Village Institutions * ~~~Definitiorn Groups of residents rank communtity inLstitutions in order of importance and construct diagrams that indicate the relation- * ~~~~~ships between and among village units. Purpose: An understandling of institutional roles and relationships is fundamental to sustainable development The analyses helped the PRA team to: (1) learn about the activities of groups and organizations within the community; (2) under- stand how the community views these institutions and how they rank them according to their contributiont to comimunity development; and (3) assess the relationships among these institutions by creating a diagram of institutional intera- tions. Process: The team first compiled a list of all institutions (church groups, women's organizations, cooperatives, etc.) in the sublocation. Meeting with clusters of men and women in four different sites of the sublocatiork, the team asked resi- dents to rank the importance and cooperation of village insti- tutions. To facilitate this process, the FRA team leader brought 30 to 40 circles, cut from paper. Roughly a third of these were small circle, a thid were mnedium-sized, and a third were lairge. The team leader asked group members to place names of the community's institutions on the labels, u.sing large circles for the influential groups and smaller circles for the less important ones. This exercise alone can frequently consume one or two hours, as there is often in- tense discussion among villagers about which groups are the most important Next, the group leader asked villagers to amange the circles to show how different institutions in the community cooperate to get things done. If two groups wor-ked dlosely together, the circles would be placed to overlap one another. If the groups had nio record of collaboration, circles would be placed separately from each other. Results The Mbusyanui discussions created four diagrams, one each from the four discussion groups, showing inLstitutional rela- * ~~~~~~tionships, While details varied, they all identified the impor- tant role of women's groups, as well as of government institutions in Mbusyani. Usefulness Responses on the social and institutional antalysis are reveal- ing, in part as they provide direct information (institutional structure) and indirect data (which groups wilr work to- continued on next page 208 Participatorj Rumt ApprwLial: A Case Study fiom Kenya Box A9.8 (continued) Jether). For Mbusyani, they confirmed that the womenes groups were the best bet to supervise follow-up work, raise and nmange funds to do the work, and to cooperate with the assistant chief topIan new activities. In another community (not Mbusyarui), the institutionkal analysis revealed that no villag group trusted the assistant chief and would work with him. Many other examples of such hinsghts could be cited. The point- of the village soial analysis is the depth of understanding and the action imperatives that can be de- rived from a half-day discussion of the village's social and political profile, as perceived by representatives from the commundty- Assessment While the sketch miap and seasonal calendar reveal impor- tant physical information, the inLstitutional diagram has be- come a central device for gathering social and institution data. The results have offered valuable insights into the interworkings andoften closely guarded details of comma- tity life. The structuring of an implementation plan often reflects the institutional information revealed in this exercis wi The exercise took half a day. O-tyKam Baubara P. 7homas-Ster andJRihardFord 209 Figure A9S8 Institutional Vlagrawzz (Adminisadm - / ~Mweffy Business Heatlt Cuc Elucation( Agricultue Livestock 210 Particpatory Run) Appraisal: A CotStudy from Kenya * ~Box A9.9 Information about the Technical Survey In addition to the tlme~, spatial, and social data, technkIca officers on a PRA team as'emble information on economic and technical feasibility, ,, wa- ter, soils forestry, agriculture, etc. needed to help vilager rank project activity. Technical data are futndamental to the successful design and Implemen- tation of an action plan. Most PEA teams wi ll nlude two or three techni- Cal officers who can do surveys of potentials in water, fore-stry, crop adaptation.s, marketing, etc. In cases In which a particular skill Is lacking, the team PEA specialist has been able to identify a local expert, in most cases for no additional mioney other than providing transport and some- times the cost of food and lodging for a few days. Technical assessments consider the econkomic, ecological, and technical implications of alternative interventions (Figure A9.9 is an example for water in Mbusyami). In the case of Mbusyari, the technical survey was' initiated during the PRtA and a few more detailed investigations con- cluded a few days later. After conducting several PRAs, it has become clear that: (1) the technical data are among the most important to initegrate into the villags' socioeconomic and institutional capabilities; (2) villagers are the key agents, in cooperation with technical extension and NGO offic- ers, to carry out the implementation; and (3) the technical interventions that work best are those that commnunity leadens can understand and man- age with very little external help. The technical survey aiay take two or three days and may not require active involvement of vilage grops. However, previous PRA experience hms shown that it is helpful for village leadens or elders to work closely with the technical groups in developing the knformatiorL I; ::. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ------- Oruarify Kabutha,Barbara P.7Thomas-Stayter,andRichard Fond 211 Figure A9.9 Technical Survey Water Sources in Mbusyani (excerpt) Kithin Springs: Located in the upper area of the sublocation; flows all year; serves 1,000 people. The spring is located off the road In a high, hilly area. ICakuyunl Dam Builtby self-help effortsIn 1969-70. It is located along a tributary of the Syananuu River (sea- sonal). The dam was constructed with large rocks and cement. It no longer holds much wa- ter because of leakage. There is still some waler being retained in the dam, as witnessed by the cattails growing In the catchmnent area ankd by vegetables growing alontg the water access way. If rehabilitated, this dam could supply wvater for the secondary schooL, Kakuyuni market and households in the area. Kivuenani Welt This water source is caled a well because there is no outet Water avaiability is seasonal, as the source is replenished only by rainfalL The communal usage is not high because this source has a low flow and is in the Upper Zone where other water sources are more plentiful. Symnamu Sub-Surface Near the coffee factory south of Kakuyuni mar- Dam: ket. It is a sub-surface dam in good condition. The cement and rock wal has recently been completed. However, there are probleEm with the quality of the water. It is saline and unsuit- able for human consumption, but is used for cattle watering and for the coffee factory. There is also bilbazia in the water. ICwa Muikuyu Spring- A stream and rock dam located in the middle part of the sublocation along a ributary of the Syanramu River. The spring area has been sur- vemyd and is demarcated as a commal water - resource. The spring is highly productive and serves many people in the sublocation, despite high prevalence of bilharzia. -wa Kituma Welh Built by the coffee society. The water is pumped into the weil through a chanel origi- nated in the Syanamu River. This is poor qual- ity water and is only used by the coffee factory. Selected Bibliography Burges, R.G., ed. 1982. Field Research: At Source Book and Manual. London AHlen and Unwin. Casley, Dennis J., and Krishna Kumar. 1988. Project Monitoring and Evaluation in Agrculture Baltimore, Maryland, and LWondor: Johns Hopkins University Press. 1. 989. The CollWton, Analysis, and Use of Monitoring and Evaluation DatL Baltimore, MUryland, and Londom Johns Hopkins University Press. Chamber Robert 1981. "Rapid Rural Appraisa: Rationale and Repertoire. Pufbic Administraiin and Deedopment 1 (2)95-106. 1992 Rural App=isal: Rapid, Relxed and Participatory. Sussex, Engbnd: t-itutW for *Dcv.lopment Studies. Cnmway, GordonL 1985. -Agroecosystem Analysis. Agricultunl Administrtion 20:31-55. Fmstebusch, Kurt 1976a. "Demonstating the Value of Mini-Surveys in Social Research! Sociaologicl Methods and Research 5(1). 197b. Mini-Surveys: An Underemployed Research Tool Socia Science Reserch 5(1). Kreger, Richard A. 1988. Focus Groups: A Practad Guide for Applied Resear*. Beverly Hills, California: Sage Publications. Kunar, Krishna. 1987. Conducting Group Interviews in Devloping Countries. Washington, D.C.: Agency for International Development 1 1987. Rapid, Low-Cost Data Colledon Methdsfo rA.LD. Washington, D.C: Agency for International Development _._* 1988. Conducting Key Infomant In ternvs in Developing Con triae Wash- igtn D.C.: Agency for Jhternational Development _ 1990. Conducting Mini-Surveys in Devdoping Countries. Washington, D.C.: Agency for Intnational Development International Conference an Rapid Rural AppraisaL Papers presented at the con- ference in Khon Kaen, Thailand, September 2-5,1985. Lincoln, Yvonna S., and Egon C. Guba. 1985. Nauradistic Inquiry. Newbuny Park, CaEfoniiz Sage Publications. 1Mies, Matthew B, and A. Mdhael HubermanL 1984t Qualitative Data Anaysis: A Sourmebok ofNew Methods. Bevery Hills, Caiiorni Sage Publications. 213 214 Bibliography - - Patton, Miachel Q. 1980. Qualitative Evaluation Mthkods. Beverly Hills, California Sage Publications. 1982. Pracical Evaluation. Beverly Hills, California: Sage Publicatiors Spracley, James P. 1979. 7he Etmnofgnphic Interriew. New York Holt, Rinehart and WinstorL Sudman, Seymour, and Norman MI Bradbur l1988. Asking Questions. San Fran- cisco and London: Jossey-Bass Publishers. Van Maahezi, John, ed. 1983. Qualitative Methodology. Beverly Hills, California: Sage Publications. Whyter, William Foote, ed. 1991. Participafory Action Researi. London and New Delhi: Sage Publications. Yin, Robert 1984. Case Study Researdr Design and MetJods. Beverly Hills, Califor- nia: Sage Publicatiorns. Contributors GORDON APPLEY received hI-s Ph.D. m anthropology from Stanford University and has over 20 years of experience in international devel- opment He has worked for the USAID/S&lYs Offie on Agriculte on small-farm research and extension, headed the planning unit of a river basin development organizatin in West Africa, and consulted an a number of projects in agriculture, rural development, and the naturAl wsources. More recently, Dr. Appleby was the director for the Environment and Natural Resources unit at the Academy for Educatiorial Development (Washingtmn D.C.), where le managed a project usig social :iaketing techniques for agriltural extension. Presely, he is workdng as a short-term consultant for the World BanIk USAID and oher iternational agen JILL E ARM5iRONG is an associate professor and a community nutri- tion specalist in the Department of Food Science and Human Nutri- tion at Wasdngtn State University. Her research interests include community nutrition, social marketing, and interatioiial develop- ment. Dr. Armstrong has conducted commurnty nutition assess- rents in Antigua, the Westidies among the HImong people in the Pacific Northwest; and in cross-culbtral studies with multi-ethnic groups- STEWART BLUMENFELD received hi doctorate from the University of California at Los Angeles, where he subsequtly served on the fac- ulty for ten years. From 1979 to 1983, he was seconded from UCLA to USAHYs Office of Nutrition bo develop and manage a project aimed at developing an improved methodology for evaluating nutri- tion intervention programs. In 1983, Dr. Blumenfeld joined the Uni- versity Research Corporation's Center for Human Services (Bethesda, Maryland), and is curently the Deputy Director of the Center's Quality Assurance Project 215 216 Coifnbutors CECLIA CABANRERO-VERZCSA, a health communications expet on the staff of the Academy for Educational Development, has managed multinational health interventions on diarrheal disease control, im- muruization nutrition, family plannrig, water, and sanitatiorL In 199Z she became Deputy Director for Technical Services of a global communications program, Health Communication for Child Survival (HEALTHCOM). The countries in which she has conducted field work or managed projects include Bangladesh, China, Honduras, In- dia, Indonesia, Kenya, Malaysia, NepaL Nigeria, Peru, the Philip- pines, Sri LankL, Swaziland, and Thailand. Ms. Cabafhero-Verzosa received her M1A in public administrtionfrom the University of the Philippines. DENNIS J. CASLEY retired from the WorId Bank as chief of the Opera- tions Monitoring Unit in the Central Operations Department. Earlier, he headed the Monitoring and Evaluation Unit in the Bank's Agcul- ture and Rural Development Departent. Mr. Casley has been i- volved in development programs for about three decades, particularly in Africa. His most recent publications (co-authored with Krishna Kumar) are Pn4ect Monitoring and Evaluation in Agriculture and Te Colection, Analysis, and Use of Monitoing and Evaluation Data, published by the Johns Hoplins University Press for the World Bank. RICHARD FORD is a professor of history and interational develop- ment and a director of the International Development Research Pro- gram at Clark Unmversity (Worcester, MA). He has caried out extensive field work and published numerous articles and assess- ments of ommunty-based development in several parts of Afica, as well as in India, the South Pacfic, and among Arab settlements in IsraeL He was a member of the first Participatory Rural Appraisal Team in Kenya and co-author of the PRA Handbook and Implemnfing PRA, as wel as of other evaluations, case studies, and handbooks on PRA. ROBERT J. HAGGERTY, trained in food science and human nutrition, has been involved since 1985 in the development of a systematic method of post-harvest loss assessmenl This work has been done with the Postharvest Institute for Perishables at the University of Idaho and Washington State University. Mr. Haggerty presently co- ordinates intemational activities in food science and human nutrition in the College of Agridculture at the Urniversity of Idaho. JOHN S. HOLTZMAN, an agricultural economist, is the technicl direc- tor of the agricultural group at Abt Assodates, Inc. (Bethesda, Mary- lanc) and also served as the research director of the Agriculturl Marketing Improvement Strategies (AMIS) ProjecL AMIS used rapid appraisal metiods in maxny of its diagnostic studies of agricultural Contbuitors 217 marketing in developing countries. Before joining Abt Associates, Dr. Holtzman was on the faculty of the Department of Agricultural Eco- nonics at Michigan State University, where he worked on the food security research program funded by USAID. CECILE M. JOHNSTON received her Ph.D. from the University of Michigan in experimental psychology. She coordinated the formative research effort an the USAID-funded HEALTHCOM II Proect She has extensive research expertise in marketing anaIysis, strategic plan- ning, advertising effectiveness, communications, and consumer re- searcL CHARIrY KABTJTHA is the former deputy head of the planning unit for Kenya's National Environment Secretariat, Ministry of Environ- ment and Natural Resources. In that capacity, she served as the leader of the origiiia PRA team for severn! Kenya-based studies, in- cluding Mbusyani She is also the co-author of several artidles and reports on PRA, including Participatory Rural Appraisal Handbook Co- ducting PRAs in Kenya. Ms. Kabutha now works for UNICEF, where she is program officer for projects i mother and child health in westem Kenya. OLABODE 0. KAYODE is a lecturer in health education at the Univer- sity of Ilorin, Nigeria. He received his MYP.H. degree in health edu- cation hrom the University of North Carolina and has consulted with many TUSAID-unded intemational development projects, including HEALTHCOM. Mr. Kayode has authored reports on health educa- don, particularly nutrition, and on weaning foods and water-related parasitic diseases. KI-SHNA KJMAR is a senior social scientist in USAID's Center for Development Information and Evaluation. He has taught at Michi- gan State University and the Fast-West Center, and has consulted extensively with intenational donor agencies. Dr. Kumar has pub- lished eight books, dozens of monographs, and numerous artidles in professional joumals on development problems and evaluation meth- odology. His two recent books (co-authored with Dennis J. Casley) are: Project Monitoring and Evaluafion in Agriculture and The Collection, Analysis, and Use of Monitoring and Evaluation Data, both published by the Johns Hopkins University Press for the World Bank. MANUEL ROXAS received his MD. degree from Manila Central Uni- versity and his MLPEL from the University of the Philippines Institute of Publc HeaIth. He has worked at various levels of the PhiIippine Department of HeaIth, from direct service provider, early in his medi- cal career, to the highest policy-making level as Undersecretary of Health for Public Health Services. Dr. Roxas was co-principal investi- gator for the PRICOR Project in the Philippines. 228 Conributors MARICOR DE LOS SANTOS, a sociologist, has designed, managed, and supervised field data collection activities and data analysis for a num- ber of prcjects. In 1987, she joined the PRICOR/Philippine Project as a technical advisor and helped design and manage the field phase of research as described in the case study in this book. Ms. de los Santos provided technical assistance to local investigators carrying out op- erations research studies for the Philippines Departnent of Health under the PRICOR Project. THAYER SCUDDER is a professor of anthropology at the California Institute of Technology, and has directed the Institute for Develop- -ment Anthropology (Binghamton, NY) for several years. He has con- sulted extensively with major international donor agencies, particularly USAID, the World Bank, and FAO, and has publishdd numerous artidles and several authoritative books based on his exten- sive field work in Africa and Asia. His most recent book African IRiver Basin Developmet, was published by the Westview Press in 1991. BARBARA P. THOMAS-SLAYTER is an associate professor of interna- tional development at Clark University and director of the Interna- tional Development Program there. She has published on participation, resource management, and gender, with paricular em- -phasis on Africa and Asia. Her book, Politics, Participation, and Pov- erty: Development Through SeIf-Help in Kenya, assessed community mstitutions and their capacities to- carry out self-help activities. Dr. TIhomas-Slayter was a member of the first PRA tean and co-authored the PRA Handbook. T H E WO R LD BA N K Introduction by DnnisJ. Caskey,former Chief of the World Bank&s Operations Monitoring Unit -S?01g Other World Bank Regional and Sectoral Studes Nongovern mental Organizations and the World Bank: Cooperation fir DevelopmenA edited by Samuel Paul and Arturo Isracl Unfair Advantage: Labor Market Discrimiination in Developing Countrics, cdited by Nancy Birdsall and Richard Sabot Education in Asia: A Comparative Study of Cost and Financing, Jee-Peng Tan and Alain Mingat c-I-k Hcalth Care in Asia: A Comparative Study of Cost and Financing, Charles C. Gdffin Bolivia's Answer to Poverty, Economic Cris4, and Adjustment: the Emergency Social Fund, edited by Steen Jorgensen, Margaret Grosh, and Mark Schactcr Commodity Price Stabilization and Policy Reform: An Approach to the Evaluation of tbc Brazilian Price Band Proposals, Avishay Braverman, Ravi Yanbur, Antonio $;alazar P. Brandao, Jeffrey Hammer, Mauro de Rczendc Lopes, and Alexandra Tan Crop-Livestock Interaction in Sub-Sabaran Afica, John McIntire, Daniel Bourzat, and Prabhu Pingali The Tranitioj from Socialism in Easten Europe: Domestic Restrcturing and Foreign Trade, edited by Aryc L. Hillman and Branko Milanovic Women's Employmcnt and Pay in Latin America: Overview and Metbodoloqg} George Psacharopoulos and Zafiris Tzannatos Case Studies on Women's Employm ent and Pay in Latin A merica, edited by George Psacharopoulos and Zafiris Tzannatos .'