Social Development Papers Paper Number 36 June 2001 Social Analysis Selected Tools and Techniques Richard A. Krueger Mary Anne Casey Jonathan Donner Stuart Kirsch Jonathan N. Maack First printing: June 2001 This publication was developed and produced by the Social Development Family of the World Bank. The Environment, Rural Development, and Social Development Families are part of the Environmentally and Socially Sustainable Development (ESSD) Network. The Social Development Family is made up of World Bank staff working on social issues. Papers in the Social Development series are not formal publications of the World Bank. They are published informally and circulated to encourage discussion and comment within the development community. Copies of this paper are available from: Social Development Department The World Bank 1818 H Street, N.W., MSN MC5-507 Washington, D.C. 20433 USA Fax: 202-522-3247 E-mail: sdpublications@worldbank.org Contents Acknowledgments iv Contributors v 1. Introduction to the Issues 1 2. Designing and Conducting Focus Group Interviews 4 Richard A. Krueger and Mary Anne Casey 3. Using Q-Sorts in Participatory Processes: An Introduction to the Methodology 24 Jonathan C. Donner 4. Ethnographic Methods: Concepts and Field Techniques 50 Stuart Kirsch 5. Scenario Analysis: A Tool for Task Managers 62 Jonathan N. Maack iii Acknowledgments This publication is a collaboration between We gratefully acknowledge the valuable members of the Social Development Family in comments provided by David Marsden, Anis the World Bank and academicians outside the Dani, and Judith Edstrom. Alicia Hetzner Bank. It was made possible in part by the edited the volume, and it was desktopped by Danish Trust Fund, which has supported the Gaudencio Dizon. Social Development Family in the development of tools and techniques for social assessments. iv Contributors Mary Anne Casey Richard A. Krueger Consultant Professor and Evaluation Leader 1494 Idaho Avenue, West University of Minnesota St. Paul, MN 55108 St. Paul, MN 55108 USA USA Tel.: (651) 647-4952 Tel.: (612) 624-6754 Fax: (651) 647-4307 Fax: (612) 625-2798 Email: Casey016@tc.umn.edu Email: Rkrueger@umich.edu Jonathan C. Donner Jonathan N. Maack ontheFRONTIER, a Monitor Group Company Consultant 2 Canal Park Social Development Department Cambridge, MA 02141 World Bank USA 1818 H Street, N.W., Rm. MC5-244 Tel.: (617) 252-2540 Washington, D.C. 20433 Fax: (617) 761-3603 USA Email: jdonner@onthefrontier.com Tel: (202) 458-7463 Fax: (202) 522-3247 Stuart Kirsch Email: Jmaack@worldbank.org Visiting Assistant Professor Maackj@hotmail.com Department of Anthropology University of Michigan 500 S. State Street Ann Arbor, MI 48109 USA Tel.: (734) 764-2292, -7274 Fax: (734) 763-6077 Email: skirsch@umich.edu v vi 1. Introduction to the Issues Increasing importance is being attached to scenario analysis--will enable diverse groups of facilitating dialogues among stakeholders in stakeholders to identify the key drivers of development projects and programs, to devel- change behind a development process and is opment interventions, and to increasing the particularly useful in translating stakeholders' voice of the poor in policymaking at all levels. expressed needs into development strategy. This volume selects four tools and techniques These techniques are four among many in that provide rigorous methods for eliciting use by the World Bank and other development information from stakeholders to ensure that partners. The methods already in use for the information can feed into World Bank development include Rapid Rural Appraisal, projects and programs. Participatory Rural Appraisal, Gender Analy- Social analysts working in development face sis, Appreciative Inquiry,2 Systematic Client three challenges. The first is to draw out Consultation, Beneficiary Assessment, and information from stakeholders, who can SARAR.3 These methods all have their own provide understanding both of disenfranchised literature and practice, which are not replicated groups and how power works in any social here. Most of these methods are well docu- context.1 The second challenge is to identify mented in The World Bank Participation and verify patterns in the data and to under- Sourcebook.4 The techniques presented in the stand the underlying logic that results in a current volume also can be used to flesh out the pattern's being reproduced. Understanding the approaches developed by the Bank's Poverty logic underlying patterns will enable the Reduction and Economic Management Net- analyst to identify the sets of incentives neces- work (PREM) Public Sector, and Poverty sary to change patterns of behavior--the key Divisions. These PREM approaches include task of development. The third challenge faced assessing political commitment, poverty- by analysts is to translate for development mapping, and participatory poverty assess- agencies what the actors' expressed interests ments. The four techniques presented in the mean in terms of development objectives. this volume provide means to implement the The first of the four techniques presented in PREM approaches. this volume--the focus group--provides a In choosing to use one technique rather popular and flexible way to meet the first than the other, a task manager must make a challenge, and guidance on ways to system- trade-off. The choice of technique will vary atize the data received. The Q-sort methodol- according to the stage of application in the ogy will not provide as much guidance on how project cycle, the budget and the time avail- to ask questions. However, it provides a able, and the breadth and depth of analysis rigorous methodology by which to analyze required. Using more than one method can patterns in the survey data and reveal the increase the reliability of the results. There is a mental models of the survey participants. danger in single technique fundamentalism; Ethnographic methods provide guidance on all none of these techniques is a panacea for all three challenges. The fourth technique-- situations. The tools described here are not 1 Social Analysis: Selected Tools and Techniques Table 1. Strengths and Weaknesses of Techniques Reliability No. of of How many Prevalence analysts/ Strengths Limitations informa- respon- Speed of of actors/ facilitators of of Method tion dents? analysis Cost analysts required technique technique Q-sorts High 10­100 Rapid Modest ­ Analyst- 1 person To force Not a medium driven fully trained choices and research Validation is Full (if actors in method- reveal method; rapid, analysis already are ology, or 3 mental questions because within 24 assembled (facilitator, models are pre- actors are hrs. on site) analyst, and determined on site. computer analyst) Focus Medium 6­8 per group, Medium Medium ­ Analyst- 4­5 To Does not groups repeated 3­4 expensive driven depending understand build Depends on times for each on no. of how people consensus nos. and unit of repetitions think or feel depth of analysis (i.e., Does not interviewing 18­32 people) provide empirical reality Scenario Not subject 10­25 Slow Expensive Analyst- 4­6 To identify Does not analysis to reliability facilitated, key drivers identify next testing, actor-driven and steps because it indicators is imagining Will not futures To survey all understand stakeholder High complex interests reliability in situations identifying drivers of To build change consensus Ethno- High micro- Depends on Slow Modest Actor-driven 1 person To Inferring graphic level breadth of minimum; understand patterns methods reliability in inquiry ideally, how people from limited team of 3-5 think or feel individual spheres in different data may locations not be Medium-low reliable macro-level reliability blueprints for action. Each practitioner must (Berkeley and Los Angeles: University of California refine them and adapt them to the context, so Press, 1984). that they enhance the quality of the particular 2. Appreciative Inquiry is a technique that has project or program. been used with great success to find local solutions, Table 1 sets out some of the characteristics of locate excellence in organizational culture, and build coalitions on the basis of past success. The technique the techniques presented in this volume. is fully explained in C. Elliott, Locating the Energy for Change: An Introduction to Appreciative Inquiry Notes (Winnipeg: International Institute for Sustainable Development, 1999). It is also concisely explained in 1. Stakeholders are active social actors with three S. A. Hammond, The Thin Book of Appreciative Inquiry key characteristics: (1) practical consciousness (Plano, Tex.: Thin Book Publishing Company, 1996), derived from life experience; (2) their own theories so is not included as a separate chapter in this about social relationships and power; and (3) volume. See also M. Grindle, "Divergent Cultures: systems of reflective monitoring on how things When Public Organizations Perform Well in Develop- work. See A. Giddens, The Constitution of Society ing Countries," World Development 25 (4) (April 1997). 2 Introduction to the Issues 3. SARAR is a participatory approach to purpose is to (a) provide a multisectoral, multi- training. The acronym stands for five attributes or level approach to team building through training, abilities that are critical to achieve full and com- (b) encourage participants to learn from local mitted participation in development: self-esteem, experience rather than from external experts, and associative strength, resourcefulness, action (c) empower people at the community and agency planning, and responsibility for follow-through. levels to initiate action. SARAR is a highly experiential methodology 4. The World Bank Participation Sourcebook (Wash- whose central strategy is group process. Its ington, D. C.: 1996). 3 2. Designing and Conducting Focus Group Interviews Richard A. Krueger and Mary Anne Casey Market researchers have used focus groups to to generate diverse ideas but not so many search for ways to improve and market their participants that they do not have a chance products to consumers since the 1950s. In the to share. last 20 years, government agencies, nongovern- · Hold three or four separate focus groups for mental organizations (NGOs), academics, and each type of participant one desires to use as nonprofit organizations have started using a unit of analysis. For example, to compare findings from focus group interviews to help how men and women view a particular make decisions about their products and issue, one should conduct three or four services. International public health organiza- groups with men and three or four groups tions were quick to make use of focus groups, with women. particularly in social marketing efforts. Educa- · Avoid power differentials among partici- tional and environmental organizations also pants. All participants in a group should feel have used the technique to listen to their comfortable talking with one another. employees as well as to their current and potential users. Create a Comfortable Environment The techniques we share are based on what we have learned while running focus groups · Hold focus groups in familiar or neutral with nonprofit organizations. settings such as office buildings, libraries, schools, homes, cafes, or community gather- Characteristics of Focus Group Interviews ing spots. · Seat people so that they can easily see one A good focus group has the following characteris- another. tics: carefully recruited participants, interacting · Interview people in their language. Do not in a comfortable environment, led by a skillful use an interpreter in the group. Using an moderator, followed by systematic analysis interpreter stilts the discussion and turns and reporting. When conducting focus groups: the process into serial interviews rather than a lively discussion among partici- Carefully Recruit Participants pants. It is better to train someone who knows the language to moderate the · Invite individuals who have the characteris- groups than to have someone who knows tics, experience, or knowledge needed to how to moderate work through an inter- provide rich information on the topic. preter. After the group is over, if necessary, · Limit the size of the group to six to eight translate the discussion back into the people. It is desirable to have enough people analyst's language. 4 Designing and Conducting Focus Group Interviews · Record the discussion for analysis. Take field Roles in Conducting Focus Groups notes and, if possible, make an audio tape recording. Sometimes a laptop computer can There are different roles in a focus group study: be used to make a transcription during the organizer (leads planning and developing focus group. Explain why the focus group is questions), recruiter (invites participants), being recorded and who will have access to moderator (leads the groups), assistant mod- the data. erator (handles logistics and captures data), and analyst/reporter (summarizes the data and Choose a Skillful Moderator prepare reports). One person can fill all the roles. However, · Pick moderators who make people feel we prefer working with a team of four or five comfortable and who are good at listening. people. Team members work together to For groups of people who are used to being complete the study, but individuals may take in powerful positions, pick a moderator who primary responsibility for certain tasks. For can keep the group on track and control example, one may have multiple moderators dominant participants. who speak different languages to better reach · Use predetermined questions. The modera- the groups being addressed. Each of these tor should be prepared to ask a set of ques- moderators would be responsible for the tions designed to get the information needed outcome of his or her focus group. by decisionmakers. Finally, although one person may be respon- · Establish an open environment. It is the sible for organizing the study, the plan and moderator's job to create the feeling questions are usually stronger when several that it will be safe to talk in the focus people are involved in their development. group. Planning Record and Analyze Data Systematically The main challenge during the planning stage · Develop a systematic approach to recording is to design a study that will answer the key and analyzing data. The analyst should be questions within the time and budget con- sure that he or she can describe the process straints. The task manager and the team must to others. be clear about the purpose of the study. The · Use a verifiable process. This means the task manager must decide whether focus analysis process should be documented and groups are the appropriate method. If focus there should be evidence to support the groups are the best method, then the task findings. Another researcher should be able manager must decide how many to do and to to review all the data and see how the whom he or she wants to listen. analyst arrived at the findings and conclu- sions based on those data. First Steps with Focus Group Studies A focus group is working well when partici- In developing and planning a focus study pants begin to talk to one another and build on group, five key steps should be followed: one other's comments rather than continually responding directly to the moderator. Ideally, 1. Decide whether focus groups are appropriate. participants become engaged, and the focus Focus groups work particularly well for the group becomes a forum for their own discus- following tasks: sion. The moderator should begin to play a less · Understanding how people see needs and central role as participants share experiences, assets in their lives and communities. debate ideas, and offer opinions. Some groups · Understanding how people think or feel arrive at this point quickly; others never about an issue, idea, behavior, product, or reach it. service. 5 Richard A. Krueger and Mary Anne Casey · Pilot testing ideas, reforms, or projects. 3. Decide what types of people to listen to the Focus groups can be used to get reactions to (target audience(s)). What types of people have plans before large amounts of money are the experiences or characteristics that will allow spent in implementation. them to provide input on the study topic? The · Evaluating how well programs or projects target audience may not consist of the most are working and how they might be im- highly educated or the most influential people proved. in an area, but its members have direct experi- · Developing other research instruments, such ence with something about which the task as surveys or case studies. manager wants to learn more. For example, young people who drop out of school know a Focus group interviews are not meant to be great deal about what it would take to keep used as: other young people in school. Teachers, counse- lors, and parents may give the task manager · A process for getting people to come to different perspectives on the same issue. consensus The task manager should consider the · A way to teach knowledge or skills usefulness of listening to a wide variety of · A test of knowledge or skills. people. These include elected officials, influen- tial local figures, the people most affected by If the task manager answers yes to any of the change, the people who must buy into the the following questions, he or she likely will change before it can happen, and employees need to consider other methods to use in (both frontline staff and management) of the conjunction with, or instead of, focus group organizations that will implement programs or interviews: services to support the change. · Do you need statistical data? Focus groups 4. Get advice from the target audience(s). The cannot provide statistical data to project to a task manager should find a few people similar population. The number of people listened to to those whom he or she wants to invite to the is too small to be statistically representative. focus groups and tell them about the study. The · Will harm come to people who share their task manager can ask for their advice on ideas in a group? Although the task manager several issues: Who can ask these kinds of can guarantee that he or she will keep questions, that is, who should moderate the information shared in the group confidential, focus groups? Where might the groups be the moderator cannot promise that other held? What days or times would work well for participants will do the same. If harm may people? How does one find people with these come to people who openly share in the characteristics? What would it take to get group, choose another method, such as people to come? individual interviews. · Are people polarized by this topic? It is 5. Put thoughts in writing. The task manager difficult to conduct focus groups if people should develop a plan that includes the study's holding opposing views on controversial purpose, the number of groups, the potential issues are in the same group. questions, a timeline, and a budget. This plan · Is there a better, more efficient way to get the will clarify one's thinking and provide a basis information? for further discussions. Then this plan should be shared with colleagues and their feedback 2. Clarify the purpose of the study. Team mem- invited. bers may disagree about the information that the study should produce and what should be Sampling and Number of Groups done with the results. They should come to agreement on a clear purpose for the study. The basic sampling strategy is to conduct three This will make the entire process simpler. or four focus groups for each audience category 6 Designing and Conducting Focus Group Interviews that is of interest. If after the third or fourth Developing a Set of Questions group the team is still hearing new informa- tion, the team might continue conducting A set of questions developed for the focus focus groups until no new information is groups is sometimes called a "questioning elicited. route." In a two-hour focus group, the re- Do not use a statistical formula to determine searcher can expect to ask about a dozen sample size. Instead, use the concepts of questions. The questions should be written in redundancy or theoretical saturation, in which the the form that they will be asked in the groups. researcher continues interviewing until no new There are three steps to develop a questioning insights are presented. In effect, the researcher route: (1) hold a brainstorming session, (2) use will have exhausted the range of views on the the brainstorming questions to draft a ques- topic, and continuing the interviews would tioning route, and (3) send the draft question- only yield more of what the researcher already ing route to the team for feedback (box 1). knows. Theoretical saturation tends to occur regularly in focus group research after three or 1. Hold a brainstorming session. The researcher four groups with one audience. should begin by inviting four to six people who For example, suppose a researcher were are familiar with the study to a one- or two- doing a study in a country with several larger hour meeting. Ask these people to suggest urban areas and a sizable rural population and questions that should be answered in the study. wanted views that reflected the entire country. Questions may be lightly discussed but do not He or she might decide to conduct three or four get stuck debating the merits of a single ques- focus groups in the cities and three or four in tion. Have one person record all these ques- the rural areas. If the country had a multilin- tions, and adjourn when ideas dry up. gual and multiethnic population, the researcher likely would want to conduct three or four 2. Use the brainstorming questions to draft a groups for each language and each ethnicity in questioning route. Groups are good at brain- both the urban and rural areas. Clearly, the storming, but they are not efficient at develop- number of focus groups needed can multiply ing questioning routes. Have one person use quickly, and the budget and timeline will the questions generated in the meeting as the force the researcher to restrict the size of the basis to draft a questioning route. Select the study. questions that seem most likely to provide useful information. Rephrase these questions Questions according to their order in the questioning route. Sequence the questions in a logical flow Developing questions carries several chal- from general to specific. lenges. The aim is to ask questions that address Say the questions out loud. Are they easy to the purpose of the study. Nevertheless, many ask? Do they seem like questions the target study teams get swept away inventing ques- audience will be able to answer? tions that would be interesting to ask but There is no magic in having approximately whose answers would not address the study's 12 questions. However, beginning focus group purpose. A good set of questions focuses on researchers often develop questioning routes getting information that directly relates to the with 20 to 30 questions--which is far too many. study's objectives. The questions must be The result of too many questions will be shal- conversational and easy for the participants to low data. Participants will not have enough understand. The researcher must have the right time to go into depth on any of the questions. number of questions--neither too many nor too Once we have a draft questioning route, we few. He or she must start with questions that estimate how much time we think the group allow the participants to get ready to prepare should spend on each question. Not all ques- the participants to answer the most important tions deserve the same amount of time. Some questions. questions are simple, or not as important, and 7 Richard A. Krueger and Mary Anne Casey Box 1. Examples of Questioning Routes Pilot testing new materials 6. What other advice do you have about the 1. Take a few moments and look over the materi- program? als (the materials presented to the group should include a brief description of a program Evaluating a service for children and examples of handouts that participants 1. Introduce yourself and tell us how you learned would receive). about this service. 2. What one thing do you like the best? 2. Think back to when you first became involved 3. What one thing do you like the least? with these services. What were your first 4. If you could change one thing about the impressions of the service? materials, what would it be? 3. What has been particularly helpful about the 5. What would get you to participate in this services your family has received? program? (Under what conditions would you 4. What has been frustrating about the services? participate?) 5. What has your child liked about the experi- 6. Suppose that you were trying to encourage a ence? friend to participate in this program. What 6. What has your child not liked about the experi- would you say? ence? 7. Do you have any other advice for us as we 7. Some of you may have had experiences with introduce this new program? other services for your child. How does this approach compare with other services you've Basic questions for formative program evaluation experienced? Is it any different? How so? 1. Tell us how you participated in the program. 8. What would make the services work better? 2. What did you like best about the program? 9. Is your child any different because they have (What has been most helpful to you?) received these services? If so, how? 3. What did you like the least about the program? 10. Is your family life any different because you (What was least helpful to you?) received these services? How? 4. What should be changed? 11. If you had a chance to give advice to director of 5. What should be continued... as is (keep it the this program, what would you say? same) or with revision (continue it but fine 12. Based on your experiences, would you recom- tune). mend these services to other parents? can be easily covered in five minutes. Key ques- researcher might ask students a question that tions may be complex or include activities. A key he or she does not want to ask parents or question can take 15 to 20 minutes to answer. teachers. Nevertheless, the researcher needs to Once we have estimated times for each of the keep a core set of questions the same in each questions, we add up the total to determine questioning route so that responses can be whether we should add or delete questions. compared across audiences. 3. Send the draft questioning route to the team Phrasing and Sequencing Questions for feedback. Ask the brainstorming team the following questions: Will these questions get us Care is needed in the phrasing and sequencing the information we need? What have we of the questions. Focus group questions are missed? What can be deleted? Are these the distinctive. When developing questions, the words participants would use? Does the flow researcher should keep in mind several guide- make sense? Then the researcher can revise the lines: questioning route based on feedback. Use open-ended questions. Most of the ques- It is important to remember that the same tions in the focus group are open-ended or non- questions are asked in all the interviews with directive. These questions deliberately give the the same audience. However, if separate participants as much latitude as possible for groups are going to be conducted for different their responses. Be cautious of phrases such as audiences, a slightly different questioning route "how satisfied" or "to what extent" because might be used for each. For example, the these words limit the range of responses. 8 Designing and Conducting Focus Group Interviews Examples of open-ended questions include: Box 2. Examples of Ending Questions What did you think of the program? How did you feel about the conference? Where do you Several questions are particularly effective at the get new information? What do you like best end of the focus group. These questions help the about the proposed program? researcher get a final viewpoint from participants on key topics. Consider using one or more of these Avoid questions that can be answered with a ending questions: "yes" or "no." If the question can be answered All-things-considered question. This question with one word, the researcher should revise the asks participants to reflect on the entire discussion question. One-word answers usually lack the and then offer their positions or opinions on topics desired detail. of central importance to the researchers. For example: "Suppose that you had one minute to talk Avoid "why?" questions. "Why?" seems to the head of X about this topic. What would you demanding and makes people defensive. say?" Or "Of all the things we discussed, what to Instead, the researcher can ask about attributes you is the most important?" and influences. Attributes are characteristics or Summary question. The assistant moderator features of the topic. Influences are things that often gives a short summary of the focus group prompt or cause action. Try questions such as: near the end. After the brief oral summary, the "What prompted you to try...?" moderator asks: "Is this a good summary of what was said here today?" Use "think back" questions. Take people back to a specific time to get information based on Final question. The moderator reviews the purpose of the study and then asks the partici- experience. "Think back to the last time you pants: "Have we missed anything? Is there visited the clinic...." "Think back to when you anything we should have discussed that we had your first baby...." "Think back to when didn't?" you first planted...." Use different types of questions. Five types of Focus the questions, sequencing them from questions are asked in focus groups: (1) opening general to specific. As a rule, the questions tend questions (answered by everyone), (2) introduc- to begin with general and broader topics and tory questions, (3) transition questions (move move to more specific categories. For example, participants to key questions), (4) key questions if a researcher were doing a focus group on a (address one of the fundamental issues of the specific brand of cola, he or she might begin by study), and (5) ending questions (get a final asking about beverages, then sodas, then colas, viewpoint from participants on key topics). Not and finally about the specific brand. all questions are the same. Some are designed to get people talking. They are easy for every- Recruiting one to answer. Others move participants to the key questions that are the most important. At The researcher has the plan and knows the least half of the focus group's time should be questions that he or she is going to ask. The spent on key questions. Ending questions help next challenge is to find the right people and wrap up the discussion (box 2). get them to attend the focus groups. The first step in recruiting participants is to Use questions that get participants involved. identify as precisely as possible the characteris- Participants can do more than talk. Ask them to tics of the target audience. A basic principle of try a product or a task and talk about it. Or ask focus groups interviewing is that the researcher them to draw a person who uses the program controls attendance. The researcher invites or service that is the subject of the focus group. people because they meet the "screens" or Have them rate different ideas. Remember to qualifications of the study devised in the take into consideration reading and writing planning stages. Participants are selected and abilities before using certain types of questions. invited because they have certain experiences 9 Richard A. Krueger and Mary Anne Casey or qualities in common, not simply because Developing a Recruiting Procedure they are interested in attending. Participants may have taken part in a community program Successful recruiting has two distinct qualities. that the researcher is evaluating; they may be First, the process should be personalized. Each residents in a community in which the re- invited person should feel that he or she has searcher is doing a needs assessment; or they been personally asked to attend and share his may be farmers who have adopted improved or her opinions. Second, the invitation process agricultural practices. is repetitive. Invitations are given not just once, One of the challenges of focus group re- but two or three times. A typical recruiting search is getting people who are not interested process follows: in the study to participate. They may be apa- thetic, indifferent, or even consider the topic to Set meeting dates, times, and locations for group be irrelevant. However, if the researcher limits interviews. Most groups with adults are sched- the study to those who show interest in the uled for two hours. Focus groups with children topic, the results may be biased. To be success- are usually shorter. Do not schedule more than ful, the researcher will need to do three things: two groups in one day unless you have mul- (1) find a pool of participants, (2) develop a tiple moderators. sound recruiting procedure, and (3) devise incentives to increase attendance. Recruit potential participants by telephone or in person. Before beginning the recruiting, the Finding a Pool of Participants researcher needs to be clear on how he or she will describe the study. People will want to Typically, we find a pool of people who meet know the purpose of the discussion, who wants our selection requirements, and then we ran- the information, what the sponsor of the study domly select individuals to invite from that is going to do with the information, and why pool. For example, we might invite every fifth they are being asked to participate. name on a list or every tenth person who enters Usually, we do not use the word "focus a health clinic. Here are several different ways group" when inviting participants, as the term to find a pool: can be intimidating. Instead, we say we are getting together a few people to talk about · Find a list of people who fit the selection criteria. the topic. Do not use jargon in the invitation. Think about community members (or In most cases, the invitation should sound like organizations) who might have a list. it will be an easy, comfortable, interesting · Piggyback on another event that attracts the type conversation. of people desired. Do all farmers in a certain Think about who should offer the invitation. area get together for a particular event? Will people be more willing to participate if · Recruit on location. Invite every fifth person someone from their community or village who arrives at the clinic. invites them than if a stranger invites them? Or · Encourage nominations. Ask key people, such would people feel honored to be invited by a as elders, educators, or service providers, head of a local organization? People are more whom they know who fit the selection likely to say yes if someone they know and criteria. respect invites them to participate. If that is not · Snowball samples. Once the researcher finds possible, it helps to be able to refer to a person some people who fit the selection criteria, whom they know and respect as supporting the ask them for names of other people who fit study. An invitation with this phrasing brings the selection criteria. Put the names in a results: "The deputy minister (or the commu- pool. nity health nurse) said you might be able to · Place advertisements in newspapers and on help us. We are getting together some people to bulletin boards. talk about (name of topic)." 10 Designing and Conducting Focus Group Interviews Soon after the person has agreed to participate, · Compliment (others value your insights). send him or her a personalized letter. Do not use a · Honor (we value your opinions). generic salutation such as "Dear friend." This · Enjoyment (you will have a nice time). letter should thank the person for agreeing to · Community (your participation will help the participate and confirm the date, time, and community). place of the focus group. In certain cases, the recruitment and the meeting time are close Moderating together in time. In these cases, the personal- ized letter is not possible, but it may be helpful The challenge of moderating is making people to give those agreeing to participate a one-page feel comfortable enough to share in a group fact sheet with the details of the study. what they think and how they feel. Participants must trust the moderator, the process, and the Telephone or contact each person the day before sponsoring organization, and they must believe the focus group to remind him or her of the group. that the results will be used in a positive way. "I'm looking forward to seeing you tomorrow The moderator must know when to wait for at...." more information and when to move on. The moderator must be able to control dominant Sometimes recruiting is done just before the speakers and encourage hesitant participants. focus group is held. For example, the re- The moderator must respect the participants, searcher might find the pool of participants at a listen to what they have to say, and thank them festival or community event at which many for their views even when the moderator may people have gathered. As people arrive at the personally disagree with those views. event, a limited number can be invited to join the discussion. If many qualified participants Moderator Skills are present, the researcher might randomly select those who will participate. In this sce- The moderator should have enough knowledge nario, it would be a good idea to arrange for a about the topic to understand what the partici- special place to hold the focus group where the pants are saying. He or she does not need to be participants will not be disturbed by the an expert on the topic but should understand surrounding events. common terms that will be used in the discus- sion. Getting People to Attend: Incentives It sometimes helps to have a moderator who looks like the participants. This can make the The researcher should think about what might participants more comfortable and give the make it hard for people to attend and try to impression that "this person will understand eliminate these things. If appropriate and what I have to say." The researcher should possible, provide transportation and consider things like gender, age, race, and childcare. ethnicity. For some topics, these issues may not The researcher should also think about what matter, but for other topics they are very might entice people to participate and offer important. For example, women may be more some or all of these things. He or she can ask a willing to talk about breastfeeding with a few people who are similar to the target audi- woman than a man. As mentioned before, the ence what it would take to get them to come. moderator should be fluent in the participants' Incentives that have been used to encourage language. people to participate include: Other things that the moderator should do include: · Money (we will pay you). · Food (there will be something to eat). Be mentally prepared. · Gifts (we have a gift for you). · Be alert and free from distractions. The 11 Richard A. Krueger and Mary Anne Casey moderator should arrive early so he or she is · Small-talk topics should be easy to discuss. relaxed and ready to listen. They can focus on what is happening locally · Have the discipline to listen. Beginning (if it is off the topic of the focus group). If the moderators are often delighted that people moderator is new to the area, he or she can are talking and do not notice that the partici- ask participants about the weather, geogra- pants are not really answering the questions. phy, transportation, places to eat, or their As the moderator listens, he or she needs to families. The moderator's job is to make determine whether participants are really people feel welcome. answering the question. If not, the modera- tor should refocus their attention on it. Give a short and smooth introduction. · Be familiar with the questioning route. · Welcome everyone. Know which questions are the most impor- · Give an overview of the topic. tant and which can be dropped if time is · Provide any ground rules for the discussion. running out. · Ask the first question. Work with an assistant moderator. Use pauses and probes to draw out more · An assistant moderator improves the quality responses. of the groups. · Be comfortable using a five-second pause. · The assistant helps by handling details (for Beginning moderators are sometimes un- example, taking care of refreshments, moni- comfortable with silence. However, pauses toring recording equipment, integrate encourage people to add to the conversation. latecomers into the group). · Use probes to get more detail. Generally, · More importantly, the assistant helps ensure more detailed information is more useful. the quality of the analysis by taking careful Consider using these questions and com- notes, summarizing the discussion at the ments: "Would you explain further?" "Can end, and acting as another set of eyes and you give an example?" "I don't understand." ears for analysis. "Tell me more." Record the discussion. Control one's reactions to participants. · It is impossible to remember everything that is · Do not lead participants by giving verbal or said in a focus group. The focus group can be nonverbal clues as to what you do or do not recorded through field notes, tape recording, like. The moderator should avoid showing or with a laptop computer. The moderator signs of approval or disapproval. For ex- will not be able to take comprehensive notes ample, it is often tempting for the moderator and guide the discussion at the same time. to give a broad smile and nod his or her head · The responsibility for recording the focus when hearing certain comments. Participants group falls to the assistant moderator. quickly spot this behavior and then assume that more of these "approved" comments are Use purposeful "small talk" before the group wanted. begins. · Avoid head nodding and verbal cues like · Create a warm and friendly atmosphere. "That is good" or "Excellent." While the moderator is waiting for partici- · Do not correct participants during the group pants to arrive, he or she can engage those discussion. If they share information that is who arrive first in "small talk." These harmful, offer the correct information at the informal discussions precede the focus end of the discussion. group interview, help put participants at · Do not become defensive if participants say ease, and foster conversation among the that they think the program is horrible. group. Small talk often occurs while people Instead, the moderator should try to get are standing around and before the official information that will help him or her under- discussion begins. stand their perspective. 12 Designing and Conducting Focus Group Interviews · Save unplanned questions for the end. The recommended pattern for introducing Sometimes new and unanticipated questions the group discussion includes a welcome, an will occur to the moderator. There is a risk in overview of the topic, the ground rules, and the asking these questions during the focus first question (box 3). groups, as they might interrupt the sequence of the planned questions and throw partici- Expectations of the Assistant Moderator pants off topic. Save these questions and ask them at the end of the focus group discus- It is helpful to have a second member of the sion. research team present in the focus group. We call this person the assistant moderator or the Use subtle group control. reporter. Because there is so much going on in · The moderator's job is not to make sure the focus group the moderator often is not able everyone speaks the same amount in a to lead the discussion, observe, and take notes group. However, everyone should have the at the same time. The assistant helps with the opportunity to share. Some people will have arrangements, takes careful field notes, and more to say. If they are answering the ques- assists with the analysis. tion and giving new and useful information, Assistant moderators typically take responsi- let them continue. bility for the following: · Control dominant talkers by thanking them for their input and asking for others to share. · Equipment. Ensure that needed equipment is Remind the group that it is important to hear available and working. This includes record- from everyone. ers, microphone, tapes, and handouts. · Call on quiet participants. They are often · Refreshments. Arrange for food (either com- reflective thinkers and have wonderful plete meals or snacks) and beverages to be things to offer. Invite them to share with available on time. something like, "Maria, I don't want to leave · The room. Arrange chairs and table and be you out of the discussion. Is there something attentive to background noises that would you would like to add?" affect the audio recording as well as room temperature and lighting. Conclude with a summary and final questions. · Welcoming participants as they arrive. Act as · Summarize the key points of the discussion the hosts and make participants feel wel- and ask for confirmation. Usually, the come and comfortable. assistant moderator does this. (Do not · Sitting in a designated location. Sit outside the summarize the entire focus group, but circle, opposite the moderator, and closest to instead summarize three to five of the most the door, so that they can greet those arriv- important points.) ing late, briefly telling them what has been · Review the purpose and ask if anything has discussed so far, and find them a place to sit. been missed. · Taking notes throughout the discussion. Cap- · Thank the participants and conclude the ture the details of the group interaction in session. their notes. · Not participating in the discussion. Speak only Beginning the Focus Group Discussion if invited to by the moderator. Control their nonverbal actions no matter how strongly The first few moments in a focus group discus- they feel about an issue. sion are critical. In a brief time, the moderator · Asking questions when invited. At the end of must create a thoughtful, receptive atmosphere; the discussion, ask questions of amplifica- provide ground rules; and set the tone of the tion or clarification when invited to by the discussion. Much of the success of group moderator. interviewing can be attributed to creating an · Giving an oral summary. Provide a brief oral open environment. summary (about three minutes) and invite 13 Richard A. Krueger and Mary Anne Casey Box 3. Example of a Typical Introduction "Welcome! Thank you for joining us to talk about want to miss any of your comments. People often say education. My name is Richard Krueger, and this is very helpful things in these discussions, and we Peter Wyet. We are working with the Department of cannot write fast enough to get them all down. We Education. We know that most teenage boys from this will not use any names in our reports, and we will school district do not graduate from high school. We keep what you say confidential. want to find out what can be done to help these young "I have about a dozen questions to ask. My job is people stay in school. The Department of Education will to listen and to make sure everybody has a chance use the information we get from these discussions to to say what he wants to say. You do not need to design new programs specifically for boys. respond directly to me all the time. Feel free to "You have been invited because you are all follow-up on something someone else says. You may fathers of junior high school boys. Also, each of your want to add something or share a different experi- sons has missed several days of school in the past ence. I will ask a question, and then you can feel free month. We would like your thoughts on what might to have a conversation about it. be causing this and what could be done to keep your "Well, let us begin. Let us find out some more sons in school. We will also be talking to junior high about one another by going around the table. Please age boys, mothers, teachers, and community tell us what grade your son is in and one of his leaders. favorite things to do when he is not in school." "There are no wrong answers. We expect that you will have differing points of view. That is okay. You do Note: After the first question is completed, it is helpful to not have to agree. Please feel free to share your point say, "We will not be going around the circle anymore, so of view even if it differs from what others have said. just feel free to jump into the conversation whenever you "You have probably noticed the microphone. We want." If the moderator continues to go around the table, are tape-recording the session because we do not discussion becomes boring, and participants tune out. and can be prone to distortion. Memory will participants to offer additions or corrections be one of the methods used, but it should to the summary. not be the only method. · Discussing the focus group with the moderator. · Field notes can capture meaningful quotes Discuss overall impressions, notable quotes, plus the content of the discussion. The field key ideas or insights presented, and how this notes of the assistant moderator should be group compared to other groups. expected to capture details of the group, · Giving feedback on the analysis and report. Offer whereas the field notes of the moderator will valuable insights into the analysis. In some be sketchy. circumstances, assistant moderators may · Audio tape recording is recommended when- actually provide leadership for the analysis. ever possible. Standard audiocassette At a minimum, the assistant should be in recorders are low cost and reliable. To close contact with the person coordinating ensure the best sound quality, use a micro- the analysis and be one of the first readers of phone that is separate from the audiocas- the finished report. sette recorder. Capturing the Data · Video cameras are useful in some circum- stances, but be cautious because there is a Thought must be given to capturing the com- tendency for people to be apprehensive and ments of the focus group participants. Multiple less candid on video. methods are recommended because no single · Laptop computers used by a fast typist can be method is perfect. An examination of alterna- used to capture a nearly complete transcript tive methods of capturing data follows: in real time. · Memory is the most fallible. Memories of the The researcher's choice of methods will moderator and the assistant fade quickly depend on resources and circumstances. At 14 Designing and Conducting Focus Group Interviews minimum, we recommend memory, field notes, only one person. This is possible only if the and audio tape recording. analyst has a clear understanding of the pur- pose of the study. Analysis Systematic Analysis Process The analyst must take the data and find what is meaningful to the purpose of the study, so that Focus group analysis should be systematic. the results can be used. Not all studies require Analysts should develop a protocol that fol- the same level of analysis. One of the skills that lows a predetermined and verifiable set of beginning analysts must learn is to match the steps. There is no single "best" systematic level of analysis to the problem at hand. Some process. Box 4 gives an example of a systematic of the choices are listed in table 1. A complex analysis process that we have often used. study in which the researchers are trying to Notice that the process begins while the first understand how different types of people think focus group is still being conducted and contin- or feel about a cultural practice may require ues past the last group. transcript-based analysis. However, if research- ers are trying to determine which of three sets Focus Group Analysis Tips of educational materials is more appealing, analysis based on notes may be all that is When analyzing focus group data, the analyst required. No matter what level of analysis is needs to consider many different aspects of the selected, breaking analysis into manageable focus group and its participants' responses, chunks keeps the material from becoming including words the participants use in the overwhelming. The analyst must look for the discussion, context, internal consistency of the major themes that cut across groups as well as participants' views, frequency of comments, a key insight that might have been shared by degree of agreement on a topic, intensity of Table 1. Analysis Choices Memory-based Note-based Tape-based Transcript- based Analysis type analysis analysis analysis analysis Description Moderator Moderator prepares Moderator prepares Analyst prepares analyzes based on a brief written written report based written report based memory and past description based on an abridged on complete experiences and on summary transcript after transcript, with gives oral comments, field listening to tapes, some use of field debriefing to client. notes, and selective and consulting field notes and review of tapes. notes and modera- moderator tor debriefing. debriefing. Oral or written Usually oral report Usually oral and Usually oral and Usually oral and reports only. written report. written report. written report. Time required Very fast: within Fast: Fast: Slow: per group minutes following within 1-3 hours per within 4-6 hours per about 2 days per the discussion. group. group (includes group (includes time time for completing for completing full abridged transcription). transcription). Perceived level Minimal. Moderate. Moderate to high. High. of rigor Risk of error High. Moderate--depends Low. Low. on quality of field notes. 15 Richard A. Krueger and Mary Anne Casey Box 4. Example of a Systemic Analysis Process The analyst(s) should: · Listen to the tape, review field notes, and read transcript if available. 1. Start while still in the group. · Use transcripts as the basis for the next steps. · Listen for vague or inconsistent comments and probe for understanding. Option 2: Analysis without transcripts · Consider asking each participant a final · Prepare summary of the individual focus group preference question. in a question-by-question format with amplifying · Offer an oral summary of key findings and quotes. inquire if the summary is correct. · Share report with other researchers who were present at the focus groups for verification. 2. Prepare for analysis Immediately after the · Use summaries as the basis for the next focus group. steps. · Draw a diagram of seating arrangement. 4. Analyze the series of focus groups within days · Spot check tape recording to ensure the of the last group. recorder picked up the discussion. If the tape does not work, immediately take time to expand · Analyze groups within a category (for example, written notes. Recreate as much of the discus- first analyze the groups of parents, then the sion as possible. groups of teachers, then the groups of stu- · Turn the tape recorder back on and record the dents). observations of the moderator and assistant · Analyze groups across categories (for example, moderator. compare and contrast the parent groups with the · Discuss issues such as: What seemed to be the teacher and student groups). key themes of this discussion? What was · Look for emerging themes by question and then surprising? How did this group compare with overall. prior groups? Do we need to change anything · Construct typologies or diagram the analysis if before the next group? Note hunches, interpre- appropriate. tations, and ideas. · Describe findings and use quotes to illustrate. · Label and file field notes, tapes, and other materials. 5. Prepare the report. 3. Analyze the individual group within hours of · Consider narrative style versus bulleted style. completion of the focus group. · Use a few quotes to illustrate each important point. Analysts can proceed with or without transcripts. · Sequence could be either question by question or by theme. Option 1: Transcript-based analysis · Share report with other researcher(s) for · Make back-up copy of tapes and send tape to verification of findings. transcriber if necessary. · Revise and finalize report. feeling toward a topic, specificity of re- Context. Participant responses were triggered sponses, and "big ideas" that emerge from by a stimulus--a question asked by the mod- the discussion. erator or a comment from another participant. Examine the context by finding the triggering Words. Think about both the actual words stimulus and then interpret the comment with used by the participants and the meanings of its environment in mind. The response should those words. Some words are powerful, color- be interpreted in light of both the preceding ful, or very descriptive. Different participants discussion and the tone and intensity of the will use different words and phrases, and the oral comment. analyst will need to determine the degree of similarity among these responses. Internal consistency. Participants in focus 16 Designing and Conducting Focus Group Interviews groups sometimes change, and even reverse, weight than responses that are vague and their positions after interaction with others. impersonal. To what degree can the respondent This phenomenon rarely occurs in individual provide details when asked a follow-up probe? interviews due to a lack of interaction from Greater attention is often placed on responses other participants. When there is a shift in that are in the first person as opposed to those opinion, the researcher typically traces the in the hypothetical third person. For example, flow of the conversation to find clues that "I feel the new practice is important because I might explain the change. What seems to have have used it and been satisfied" has more prompted the change? weight than "These practices are good and people in the area should use them." Frequency. Frequency is the measure of how often a comment was made. Frequency alone Finding big ideas. One of the traps of analysis does not tell us how many different people is focusing so much on the detail that the made this particular comment. Indeed, the analyst misses the big ideas. Step back from the same person may have made the same, or discussions by allowing extra time for big ideas similar, comments 10 times within the course of to percolate. For example, after finishing the a single discussion. Do not assume that fre- analysis, the researcher might set the report quency is an indicator of importance. It is not aside for a brief period and then jot down the necessarily true that items that are discussed three or four of the most important findings. most often are most important. Assistant moderators or others skilled in qualitative analysis might review the process Extensiveness. Extensiveness is the measure and verify the big ideas. of how many different people made a particu- lar comment. This measure gives the analyst a The Old-Fashioned Analysis Strategy: Long Tables, sense of the degree of agreement on a topic. Scissors, and Colored Marking Pens Unfortunately, it is impossible to determine extensiveness using only the transcript unless An analyst who has not analyzed focus group names are attached to comments. If the analyst data before may want to try this strategy. It is was present in the focus group, he or she will a concrete way of categorizing and "seeing" have a sense of the degree of extensiveness, and the data. After the analyst understands this this can be captured in the field notes. In focus method, it is easier to understand how this group analysis, extensiveness is a more useful process can be accomplished using computer concept than frequency. software. The equipment needed includes two copies of all transcripts, scissors, tape, lots of Intensity. Occasionally, participants talk room with long tables and possibly chart about a topic with a special intensity or depth stands, large sheets of paper (flip charts, of feeling. Participants will sometimes use newsprint paper), colored marking pens, and words that connote intensity or tell the mod- stick-on notes. This analysis strategy has 10 erator directly about their strength of feeling. components: Intensity may be difficult to spot with tran- scripts alone because the voice tone, speed, and 1. Prepare the transcripts for analysis. The emphasis on certain words are key to commu- analyst will save time and agony later if he or nicating emotion. Individuals will differ in how she is careful in preparing the transcripts. Be they display strength of feeling. Strong emotion sure they follow a consistent style. For ex- may be evident in fast, excited speech in some ample, single-space comments and double- people and in slow, deliberate speech in others. space between speakers. The moderator's Pay attention to what is said with passion or comments should be easily identifiable by intensity. bolding, capitalizing, or underlining. Specificity. Responses that are specific and 2. Make two copies of each transcript. One will based on experiences should be given more be cut up, and the other one stays intact for 17 Richard A. Krueger and Mary Anne Casey later reference. At this point, the analyst may will be several different points of view, and want to follow a tip that will be useful at later the analyst can cluster the quotes around stages of the analysis. Consider printing tran- these points of view. The quality and rel- scripts on different colors of paper and color- evance of quotes will vary. In some groups, coding by audience type or category. For the analyst may find that almost all of the example, copy the comments of teenagers on quotes can be used, but in other groups, there blue paper and those of parents on green paper. will be few usable quotes. Set the unused In addition, use a marker to put one line down quotes aside for later consideration. If a the right margin of each page of the first parent participant's comments are really addressing transcript, two lines down each page of the another question, tape the comment under the second parent transcript, three lines down each question it addresses. page of the third parent transcript. This way, once the analyst has all of the transcripts cut 7. Move similar quotes into categories or "piles." up, he or she can easily see that a quote on As the analyst reads each quote, he or she green paper with two lines down the side came needs to reflect on whether it is similar to or from the second parent group. To take this one different from other quotes already assembled step further, most word processing programs and put similar quotes together. If a quote allow numbering each line. raises different issues or ideas, then create a new category and a separate pile for this 3. Arrange transcripts in an order. The order information. can be based on the sequence in which the groups were conducted, but transcripts are 8. Write a statement about each question. Look more likely to be arranged by category of over the quotes and prepare a paragraph that participant or by the demographic screening describes responses to that question. A number characteristics of participants (for example, of possibilities may occur. For example, the users, non-users, and employees, or teens, analyst may be able to compare and contrast young adults, and older adults). This arrange- differing categories. There may be a major ment helps the analyst to be alert to changes theme and a minor theme. The analyst may that may be occurring from one group to discuss the variability of the comments, or even another. the passion or intensity of the comments. Following the overview paragraph, several 4. Read all transcripts at one sitting. This quick additional paragraphs may be needed to reading reminds the analyst of the entire scope describe subsets of views or to elaborate of the focus groups and refreshes his or her selected topics. Compare and contrast how memory of where information is located, what different audiences (for example, parents, information is missing, and what information teachers, students) answered the question. occurs in abundance. (If the analyst color-coded the transcripts, then the colors easily help him or her "see" 5. Prepare large sheets of paper. Use a large how the different audiences answered the sheet of paper for each question. Place the large questions.) When finished, go on to the next sheets on chart stands, on a long table, or on the question. floor. Write the question at the top of the sheet. 9. Take a break. Get away from the process for 6. Cut and tape. Read responses to the same a while. Refocus on the big picture. Think question from all focus groups. Cut out rel- about what prompted the study. It is easy to get evant quotes and tape them to the appropriate sidetracked into areas of minor importance. Be place on the large sheet of paper. Look for open to alternative views. Be skeptical. Look quotes that are descriptive and capture the over the pile of unused quotes. Think about the essence of the conversation. Sometimes there big picture. 18 Designing and Conducting Focus Group Interviews 10. Prepare the report. Invite a research col- time will vary with the typist's speed, the league to look over the report and offer feed- quality of the tape recording, the length of the back. session, the experience of the typist with focus groups, and the complexity of the topic. Transcribing Focus Group Interviews Special Topics In transcribing focus group interviews, keep these suggestions in mind. Involving the Community in Focus Group Research Use quality playback equipment. The typist In the past decade, a variation of focus group should avoid tape players with small speakers research has emerged that engages the commu- and awkward buttons. Earphones sometimes nity in the study. In these studies, the re- provide greater sound clarity. Focus group searcher takes on a different role. Instead of interview tapes always have background noise, being the outside expert coming in to conduct and participants speak with different tones and the study, the researcher becomes the facilita- voice levels. Therefore, these tapes will require tor, teacher, coach, and mentor. The researcher concentration and the best quality playback works with, teaches, and guides one or more equipment that can be obtained. If possible, use people from the community who, in turn, equipment with a tape speed control and foot- recruit participants, conduct focus groups, and operated rewinding. help with the various aspects of the focus group study. These community members can Minimize distractions. Type transcripts in a be volunteers but often are employed for the place with minimal distractions or interrup- duration of the study. tions. Many of these community-based studies have been on topics relating to policy develop- Identify moderator statements. Use bold print ment, community development, or prevention for the moderator's statements and questions. (drugs, teen use of alcohol, teen pregnancy, If needed, type the name of each speaker violence). In these studies, the researcher followed by his or her comment. (If the person trained and supervised local people who took doing the analysis was not in the groups, it will on specific tasks in the focus group study. As be helpful to have the names attached.) Single- mentioned earlier, if the researcher does not space the comments and double-space between know the language of the target audience, he or speakers. she should seek out a local person who is trusted by the target audience and fluent in Type comments word for word. People do not the local language. This person should receive talk in complete sentences. Insert punctuation training and coaching from the researcher where it seems appropriate. Avoid the tempta- and then be sent out to conduct the focus tion to add or change the words, or to correct groups. the grammar. If some of the words are unintel- Some tasks are easier to share with people in ligible, type an ellipsis ("...") to indicate that the community than others. Among the easier words are missing from the transcript. tasks are: Note special or unusual sounds that could help · Recruiting. Invite people to attend the focus analysis. For example, note laughter, loud group (in person or on phone) with written voices, or shouting in the transcript in paren- follow-up reminders. thesis. · Moderating. Guide the discussion in several focus groups using predetermined questions. Allow sufficient time. Typically, it takes about · Recording. Take notes, listen to the focus 8 hours to transcribe a 60-minute tape. But the groups, and assist the moderator. 19 Richard A. Krueger and Mary Anne Casey Other tasks can be more complicated and However, involving the community in focus require more time, skill, and practice. These group research has some disadvantages. For include: example, the studies may take more time because the researcher must recruit, train, and · Planning. Develop the overall strategy and supervise the team. In addition, the research determine the time schedule, budget, and team members may interpret results differently groups to be conducted. than would the expert researchers (although · Developing questions. Determine the questions this could also be an advantage). to be used for a focus group study. Involving the community in focus group · Testing questions. Pilot test the focus group research often works best in situations in which questions with a specific audience. local people are able to obtain information · Transcribing. Type the results of a focus otherwise unavailable to researchers, help group while listening to the audio tape. stretch scarce resources, increase the likelihood · Reporting. Share written results and present that the results will be used, and develop new an oral report with a specific audience. skills or insights about their community. Still other tasks are among the most compli- Telephone Focus Groups cated and difficult to share. These skills require considerable time to develop and perform, and Focus group discussions can be conducted on they include: the telephone. With a conference call, a mod- erator can carry on a focus group discussion · Analyzing. Prepare a three- to five-page with people scattered around the country. The question-by-question summary of the focus primary advantage of a telephone focus group group or analyze across the groups and is that it allows greater potential for partici- write the report. pation. This is particularly true for busy · Coordinating. Overall coordination of the people or people who are geographically entire project must include all budgets, time- dispersed. schedules, logistics, and equipment. The principle disadvantage of telephone focus groups is that the researcher misses the The researchers need to weigh what is nonverbal communication. Much is gained in gained and what might be lost if the commu- focus groups by bringing people together and nity becomes engaged in the study. Involving being able to observe the participants' reac- the community gives several major tions--head nodding, signs of boredom, smiles, advantages: frowns, alertness, interest in the topic--all of which are lost on the telephone. In effect, a · Focus groups can be conducted in different telephone focus group will lack the richness of languages and dialects by moderators who in-person focus groups. are trusted by the participants. In comparison to in-person focus groups, · Focus group participants (and especially the telephone focus groups are shorter and have research team) feel a sense of ownership of fewer participants and questions. In most the study. situations, two hours is too long to be on the · Because the community members have telephone. We recommend one-hour telephone muted the bias that might occur from the focus groups. Because we have less time, we sponsoring organization, results can be more recruit only four to six people for a phone focus authentic. group and limit the number of questions. We · The results are deemed more believable and also send out the questions ahead of time, trustworthy by local residents because they which seems to make the limited time more were familiar with the people conducting the productive. The advance preparation helps the study. participants know the direction in which we · Local team members gain skills. are heading, know what they want to say, and 20 Designing and Conducting Focus Group Interviews stick with us mentally, even when we are not situations. The focus group, which has gained together physically. considerable popularity among Western devel- Some have argued that telephone focus oped countries, has been found to have distinct groups are less intimidating because partici- advantages in developing countries as well: pants cannot see one another. As a result, there may be greater clarity of language because of · Participants do not need to be literate. the absence of eye contact, gestures, and · Focus groups do not depend on mail service nonverbal signals. Participants cannot detect or telephone systems. signals of approval or disapproval from other · The process is familiar in that it resembles participants or the moderator. The difficulty is the decisionmaking process of many cul- that the moderator has limited ability to detect tures. Having a small group of people talk whether someone is "tuning out" of the conver- about a current topic of interest is a very sation and paying more attention to reading basic interaction. the newspaper, reviewing documents, or working on a computer. In summary, the However, the task of conducting focus moderator has no reliable indicator of the groups in developing country settings is beset degree of attention that a participant is giving by a number of challenges that researchers to the discussion. should consider. If the researcher chooses to use telephone focus groups as the discussion format, it is Power differentials. The first and most impor- useful to keep several suggestions in mind (see tant challenge is the power differential between box 5). focus group participants and the sponsor. The sponsors of most international focus groups are International Focus Groups organizations that control resources, people, or power. This power differential in conjunction Increasingly, focus group research is being with cultural differences has the potential to conducted in cross-cultural and international create communication problems. These power differentials almost always cause difficulties in focus group research. For a Box 5. Tips for Telephone Focus Groups focus group to work, participants must be able to talk without feeling threatened or fearing · Invite a small group of only four to six people. future reprisal. The researcher must set up the · Send questions or discussion topics to the focus group so that there is no obvious advan- participants in advance of conducting the tage or disadvantage to providing either focus group. Include any visuals, with each item labeled and in the order that they will be positive or negative information. The re- used. searcher must clearly communicate that all · Limit questions to five to eight. points of view are valued and appreciated. This · If voices are not recognizable, ask partici- clear communication of openness begins with pants to identify themselves before speaking. the first contacts with community leaders, · Limit discussion to about 60 minutes. continues as participants are recruited, and is · Call on people who are not audibly participat- repeated when the group begins. Researchers ing. · Consider asking participants to reflect for a should anticipate that they might hear points of moment or to jot down something on a piece view that are critical of the sponsoring agency of paper and then read the comments. It is or upsetting to themselves. The moderator fine to include questions that ask participants must take care not to show emotional reactions to rate or assign values to ideas or things. when participants offer negative views on a Keep the scales simple and logical. program. It is often advisable for the moderator · Include ending questions such as the "all not to be a person who has traditionally repre- things considered" or the "have we missed anything?" question. sented the international organization, because there is a tendency for participants to hold back 21 Richard A. Krueger and Mary Anne Casey from giving negative feedback to people in worldview. Questions need to be constructed influential or powerful positions. carefully, and the study introduced in a manner In some cultures, the norm is to avoid appropriate to the cultural environment. criticism, particularly when outsiders are Therefore, the researcher needs to plan the present. There are at times severe consequences study carefully and talk to locally influential for openly sharing one's views, and such figures early in the planning process. It is also sharing is restricted to the closest relationships. important to listen to local wise people about A questioning strategy that sometimes works is timing, locations, and other factors relevant to to depersonalize the answer by asking what the study. others are saying. For example, "What do others say about this topic?" or "When people Sense of time. It is important to be sensitive to are critical of this program, what do they say?" how the participants relate to punctuality and The researcher should think carefully about time schedules. Starting times and the length of who is the best person to moderate the group the group may take on different levels of and avoid anyone who occupies a position of importance in various countries. power. Sometimes local residents are able to lead the group discussion, and in other situa- Feeling of exclusion. When doing focus groups tions the international guest researcher is quite in villages or close-knit communities, some capable and non-threatening. special problems may emerge. One concern is that some people will feel left out of the pro- Local language. Language also can be an cess. People may be offended because they issue. It is important that the researcher con- were not invited. Suppose the researcher is ducts the groups in the primary language of doing a study on new farming practices and is the participants and avoids interpreters. This inviting farmers who have been slow or reluc- means that the moderator should be fluent in tant to adopt new practices. Progressive farm- the language. If the moderator is not fluent, ers may feel upset because they were not then the researcher must find someone who is invited to talk about this topic that has been of fluent and train him or her to lead the group. great interest to them. Locally important Notes should be taken in the same language figures may also be upset because traditionally that the moderator is using, and then the notes they have been involved in local decision- or tape recording translated back into the making, and they were not involved in the language of the report. focus group. Finally, some people do not feel that the results are believable unless they Sense of control. One area in which variation personally have given their views. can occur is the participants' sense of control. The researcher should give thought to these Some individuals feel that they have control potential problems and consider one or more of over their social, political, and economic the following solution strategies: environment and that individuals can and ought to make needed changes. Others feel that · Do focus groups with local experts or influential someone has control, but that they personally figures even if they are not the target of the study. do not. Still others feel that no one has control, They may have valuable insights into the or that one should not tamper with fate but problem that could enrich the study; more- rather accept what occurs as the will of a over, they could block progress if they are higher power. If the researcher is examining a not involved. (Later, the analyst will want to public program or the consequence of a policy keep the groups separate and not aggregate decision, focus groups might offer different results across groups.) perspectives based on their philosophical · Gather information in other ways. The re- orientation. In fact, we argue the participants searcher can use a combination of focus may not even understand the questions posed groups, individual interviews, surveys, and to them because they are so different from their group meetings. He or she can explain that 22 Designing and Conducting Focus Group Interviews people are being involved in a variety of References ways: some will be in focus groups, while others will be asked to be in individual Debus, M. 1990. Handbook for Excellence in Focus interviews or surveys. Group Research. Washington, D. C.: Academy for · Add several additional focus groups at the end of Educational Development. the study. These focus groups are different in Glaser, B. G., and A. L. Strauss. 1967. The Discovery of that they are open to anyone in the commu- Grounded Theory: Strategies for Qualitative Research. nity. Concentrate the analysis on the early New York: Aldine De Gruyter. focus groups that are conducted using careful procedures. At these final groups, Goldman, A. E., and S. S. McDonald. 1987. The Group listen particularly for points of view that Depth Interview: Principles and Practice. Englewood have not been expressed in the earlier focus Cliffs, N. J.: Prentice-Hall. groups. Some caution is needed in interpret- ing the comments given at group meetings Greenbaum, T. L. 1998. The Handbook for Focus Group Research. Thousand Oaks, Cal.: Sage. because the tendency is for certain individu- als to "perform," to give positions for politi- Guba, E. G., and Y. S. Lincoln. 1990. Fourth-Generation cal reasons, or to give views to impress Evaluation. Newbury Park, Cal.: Sage. others in the community. · Invite the community to attend a meeting at Hayes, T. J., and C. B. Tatham, eds. 1989. Focus Group which the results of the focus groups are pre- Interviews: A Reader. 2d ed. Chicago: American sented. Present the themes and trends found Marketing Association. in the focus groups to the community and Krueger, R. A. 1998a. Developing Questions for Focus invite the members of the community to Groups. Thousand Oaks, Cal.: Sage. comment on the findings and discuss strate- gies to deal with the issues raised. ------. 1998b. Moderating Focus Groups. Thousand Oaks, Cal.: Sage. Benefits for the participants. Finally, consider how the study can benefit the participants. Will ------. 1998c. Analyzing and Reporting Focus Group study results be shared with participants? Will Results. Thousand Oaks, Cal.: Sage. the study inform policy or future decisions? Krueger, R. A., and M. A. Casey. 2000. Focus Groups: A Practical Guide for Applied Research. 3d ed. Summary Thousand Oaks, Cal.: Sage. Focus groups can be a powerful learning tool. Krueger, R. A., and J. A. King. 1998. Involving They can help organizations listen to their Community Members in Focus Groups. Thousand employees and to the people whom they serve. Oaks, Cal.: Sage. They can be used to assess needs and assets, Merton, R. K., M. Fiske, and P. L. Kendall. 1990. The develop social marketing efforts, pilot test Focused Interview. 2d ed. New York: The Free ideas and products, and evaluate services or Press. programs. However, researchers must carefully recruit participants, create a comfortable Morgan, D. L. 1997. Focus Groups as Qualitative environment so that people feel safe to talk, Research. Newbury Park, Cal.: Sage. choose the right moderator for the group, and record and analyze the data systematically. Patton, M. Q. 1990. Qualitative Evaluation And Research Methods. 2d ed. Newbury Park, Cal.: Above all, the researchers must respect the Sage. participants. The researchers must believe they have something to learn from the participants Tesch, R. 1990. Qualitative Research: Analysis Types and and be open to hearing what is shared. Software Tools. New York: Falmer. 23 3. Using Q-Sorts in Participatory Processes: An Introduction to the Methodology Jonathan C. Donner It is not that simple. 1980; McKeown and Thomas 1988; Stephenson She sees the issue differently than you do. 1953). Q-methodology allows a researcher to It is a challenge to get our arms around this explore a complex problem from a subject's complex problem. point of view: in a Q-sort, participants weight statements, in response to a question, in accordance with how they see the issue at As a professional working in international hand. Since the same Q-sort can be given to development, you probably hear these phrases different people, a researcher can look at the all the time. If you are involved in social patterns of responses to uncover and name development, participation or civic engage- distinct "points of view," even within small ment, you probably utter these phrases all the groups. Because the results of a Q-sort analysis time. When tackling complex, multidimen- capture the subjective "points of view" of sional development challenges, like prioritizing participants, and because the data are easy to economic plans for a region, articulating a gather, easy to analyze, and easy to present, vision for a village, or clarifying a mission for a Q-methodology is good not only as a research department or a project, it seems that there can tool but also as a participatory exercise be as almost as many perspectives on an issue (Donner 1998). as there are stakeholders. This is a brief chapter. It is neither an in- For many of us, it is critical to make these depth statistical guide nor a theoretical review. different perspectives discussible, as an early Instead, it is a practical, step-by-step introduc- step in a collaborative effort to help construct tion to the methodology, such that someone action plans that most stakeholders can em- with a personal computer (PC), this chapter, brace. Qualitative tools to capture these per- and a little patience can construct, administer, spectives can be detailed, rich, contextual--and and analyze a Q-sort of his or her own. To also messy, time-consuming, and difficult to ground the instructions in an example, the administer consistently. Quantitative surveys chapter will draw on a Q-sort administered to can be clear and methodical--and also over- members of the World Bank's Social Develop- simplified, rigid, and unwieldy. Certain situa- ment Family (SDV) in November 1999. At the tions call for a tool that combines the richness time, Monitor Country Competitiveness (now of interviews with the standardization of a called ontheFRONTIER) was working with the survey. leadership of the SDV family to help them This chapter is an introduction to a tech- refine their position and strategic priorities nique called Q-methodology, which can help within the Bank. We developed the Q-sort in fill this gap in a practitioner's toolkit (Brown close conjunction with them, as one input in 24 Using Q-Sorts in Participatory Processes this change process. It was administered and particular statement or concept versus another analyzed at the start of a five-day offsite at does not need to be known or even hypoth- Monitor's headquarters.1 esized in advance. Neither do the elements have to be mutually exclusive nor completely Overview exhaustive of all the possible concepts that could apply. The elements are assumed to be After a brief overview, roughly sketching the simply a subset of the possible concepts that methodology, this chapter will present four may be important to the issue at hand, just as sections: construction, administration, data the participants may be considered a subset of analysis, and interpretation. At each stage, the the possible stakeholders. SDV data will be used to illustrate the steps Let us take a simple example to begin. If we involved. wanted to know more about at what kind of Q-methodology was developed by William restaurant people would like to eat on a Friday Stevenson (1953) in the 1930s and has benefited after work, we could arrange a Q-sort. We over the past couple of decades by the develop- would ask the umbrella question: "Which ment of statistical packages that make its statements best describe a restaurant you'd like factor-analysis calculations easier to perform. to visit on a Friday after work?" We would For a detailed methodological and statistical populate the sort with 30-odd elements ranging exploration of Q, refer to McKeown and from "costs less than $15 per person" to "is Thomas (1988). near to my home" to "has a hip clientele." Q-method requires participants to prioritize Participants would be asked to sort the 30 a set of 20 to 50 elements or statements in order elements in ascending order from least to most from least to most desirable. The statements are attractive. A completed sort may look like table often presented as multiple possible answers to 1, with the numbers corresponding to each a given umbrella question, such as: "What is an element written in the answer sheet as such. attractive outcome?" or "What is important for Q requires each participant to assign an exact this group to study?" Q-methodology does not number of elements to each potential value. In require a large number of participants--even the example above, 2 elements were required at one is worthy of review--and meaningful, each of the extremes (-4 and 4), while 6 were discernible groups can be found with as few as required at the neutral point of 0. There are a dozen participants. It is quite rare to adminis- usually an odd number of column values to ter a Q-sort to more than 100 participants. allow for the neutral column. It is also common Although the root of the Q is the prioritiza- to lump more statements in the middle. tion of elements, one of its strengths is that the Each person uses his or her own subjective pattern or logic that drives the weighting of a criteria to evaluate the relative attractiveness of Table 1. Example of Completed Sort Guide to sorting statements -4 4 Value Least -3 -2 -1 0 1 2 3 Most attractive attractive Number of 2 3 3 4 6 4 3 3 2 elements 4 25 12 1 10 3 19 13 14 Statement 5 11 7 20 9 8 15 6 17 numbers 26 21 29 27 2 18 23 16 24 22 28 30 25 Jonathan C. Donner each element. Participants' logic (perspective) holders perceive the issue you are confronting. is their own; but since you are asking the same Instead of hypothesizing relationships between question, using the same elements, to be placed items in advance and testing that structure, the in the same format, you can compare these researcher gleans the relationship between the subjective perspectives with more rigor than items only once the sort has been completed normal qualitative methods would allow. (McKeown and Thomas 1988). Along with the The results of a small Q-sort like this example easy and novel administration and fast turn- could be ready to analyze within an hour or so, around, these theoretical elements make Q using the freeware program PQMethod.2 The especially good for participatory processes, in output (discussed in more detail in the analysis which the participants and eventual beneficia- section below) would tell us: ries of the analysis are one and the same (Argyris, Putnam, and Smith 1985). A bibliogra- · Which criteria or elements were rated at phy of Q-related materials, compiled by Steven roughly the same level (either high, low, or R. Brown and available on Peter Schmolk's neutral) by most participants. We call these QMethod Page is an excellent place to assess "consensus items." the breadth of research topics that have been · Which elements garnered a real split deci- addressed using Q.3 sion, meaning they were highly agreeable to In the case of the Social Development Fam- some participants and disagreeable to ily, the multidimensionality of the issues others. We call these contention items. surrounding of SDV's strategy were clearly · And, perhaps most interestingly, the analysis evident. SDV managers were concerned with would identify distinct subgroups within the choices regarding their role as advocates in the set of participants who share a similar Bank, the introduction of training, the ideal pattern of responses (meaning members of location for SDV staff (field or in Washington, the subgroup agreed on the contention D. C.), and their capacity to offer independent items). Each subgroup can be said to share a products directly to clients. Depending on to similar perspective or voice about the topic. whom we spoke, some or all of these issues were salient--and their relative importance in It is up to us to infer the meaning of these charting a course for SDV was not clear. The outcomes. We will get to the details of that issue of how SDV managers imagined the process in the interpretation section. optimal course for SDV's development was ripe for further exploration via a Q-sort. Construction Umbrella Question To create a Q-sort, you need a research topic, an umbrella question, and some elements. The umbrella question to a Q-sort should be broad enough to hold all the statements Research Topic underneath. Thus, something as simple as: "Sort the statements from least attractive to Although the Q can fit a variety of research most attractive" imposes no additional topics--from restaurant choices to program constraint on the participants. If you prefer, plans to beliefs about fairness and justice, it is or if your research goals require it, you can especially well suited to situations in which a use the umbrella question to bound the single "issue" is made out of subdimensions, conditions of the sort in some way. "Where and in which you are not necessarily sure how would you like to go on a Friday night," for all these sub-dimensions fit together. Consider example, puts participants in a frame of mind it more exploratory than confirmatory, more of to evaluate attributes of restaurants for a an opener than a conclusion to a process of particular event. Fill-in-the-blank questions social inquiry. Q is effective as a way to discern can also be used, such as "In order for this the lay of the land in terms of the way stake- department to be most effective, it must 26 Using Q-Sorts in Participatory Processes __________" (participants are then asked to Elements can be created in a variety of ways. sort phases according to how well they com- A researcher can create all the elements "from plete the sentence). The SDV example de- scratch" if he or she desires. Literature reviews scribed here assumes that the same statements may be of assistance, or an external list may be will be evaluated once by a range of individu- used. Another approach is to involve partici- als, using a single umbrella question, to deter- pants in the element-generation stage. In mine what distinct perspectives may exist on advance, a researcher can ask some or all the issue. However, you may also choose to participants to submit a few elements for the look at how one or two participants rank-order sort. If an explicit invitation for submission is the same statements under a variety of condi- not possible or desirable, a researcher can still tions. This can be achieved by varying the draw on elements generated by the stakeholder umbrella question accordingly. community by pulling comments or phrases In the case of the SDV Q-sort, we wanted from sources that stakeholders have previously everyone to consider what an "effective" SDV produced (flyers, papers, books, headlines). would look like in the future but also wanted Even quotes from interviews can make good to ensure that each participant was considering elements. Keep in mind that elements do not the same time horizon. So, our umbrella ques- need to be lengthy sentences. Pictures or tion set the scene: images, simple phrases, or single words can work. This may be especially helpful with less After an especially long journey back to literate populations. Brown's bibliography, DC for a Social Development meeting, mentioned earlier, is a good place to review you awake in the morning to find you've different approaches to element design.4 really overslept. It is December 2002. You If a researcher has a few ideas in mind arrive at 1818 H St. and find your col- about dimensions that may be critical, he or leagues to see what they've been up to. she can "seed" the elements with items that As it turns out, they have made great represent different levels of the dimension progress over the three years you were (Brown 1993). slumbering. . . . There is no clear rule of thumb for the Which of the following statements number of elements that should be included, describe this new and improved Social but sorts with as few as 20 or as many as 60 Development Family, as it appears to you items are possible. One simply adjusts the in 2002? Sort these statements from "least structure of the answer curve accordingly. describes" to "best describes." There are a few rules of thumb you can use to craft a good set of elements. First, try to Elements choose elements that mean different things. Items that are nearly repeats of one another are The most challenging part of designing a Q- confusing to participants, as are pairs of items sort is not in defining the umbrella question, that are the exact inverse of each other (for but rather in selecting the elements that the example, restaurant takes credit cards; restau- question asks participants to sort. The best rant does not accept credit cards). thing to keep in mind is that while no list of Next, try to avoid extreme elements, so good statements is perfect, no list has to be. Since (or so repulsive) that everyone you sample what is really of interest are the tacit, underly- could be expected to either agree (or disagree) ing criteria and perceptions people use to with, to the exclusion of prioritizing other consider an issue, the elements themselves are items. Instead, elements should be plausible of secondary importance. Their job is to be competitors with one another, such that some broad enough (and clear enough) to set these participants may be attracted to them and tacit criteria and perceptions to work, and to others disinclined to choose them. give the researcher insights about them once Finally, try to keep the elements parallel in the sort is complete. style. Pick all sentences or all phrases. Avoid 27 Jonathan C. Donner double negatives. Be clear. However you as small as possible while keeping the font construct your list of elements, you may want legible and try to keep all the chips the same to pre-test the items with a couple participants, size. Approximately 30 brief statements will fit for clarity and general comparability. on an 8-1/2 x 11-inch page if the margins are In the case of the SDV Q-sort, the Monitor set as wide as possible.6 Print them onto card consultants initially developed the elements, stock or heavy paper, and use a paper cutter to drawing on materials written by SDV. Then, a separate the elements from each other. Bind team composed of SDV managers and the them together with an elastic band or put them Monitor consultants reviewed the items, in an envelope. Double-check to make sure that suggested revisions for clarity, and debated each set of chips is labeled consistently and additions and subtractions until a list that was contains exactly one of each element. Duplicates satisfactory to all was developed. Although no or missing items at this stage can ruin a sort. pre-determined dimensions were specifically The guide strip helps participants structure "seeded" into the sort, a list of sub-issues that the sort according to the pattern determined the group felt SDV was facing was created, and in advance by the researcher. The guide strip items were written to touch on them. See should be wide enough to accommodate the appendix 1 for a list of the items selected. chips underneath. You can print out two pages horizontally and tape them together to Administration yield a strip about 18 inches wide and 3 inches high. The guide for the SDV Q-sort With the elements prepared and tested, and the (22 elements) looked like the one in table 2. umbrella question crafted, the administration The answer sheet can be a single page, with becomes a matter of preparing the instrument the umbrella question on the top and the guide and instructing the participants. strip (and answer spaces) underneath. It would be a blank version of table 1, with the question Preparing the Instrument written at the top. Make sure the number of spaces matches the number and shape of the The instrument has four components: (1) chips, elements as presented in the header. If you (2) guide bar, (3) answer sheet, and (4) instruc- have any other tracking questions (group tions. Since the sort should be tactile and affiliation, name, date administered, or any iterative, it is difficult to avoid using (and other survey questions), you can place them on creating!) little chips. There is PC software that the same page as the answer sheet. The sheet can represent the sort on a screen using an we used for the SDV sort is attached as appen- Internet browser,5 and it is possible to adminis- dix 2. ter extremely brief sorts using a conventional Finally, it is helpful to include an instruction survey instrument. However, basically, sheet for participants. Although spoken instruc- nothing so far beats the cards. The rest of the tions are best, the backup is useful, especially chapter assumes a "conventional" sorting as the number of simultaneous participants technique. increases. The instructions we used for the SDV The chips are what participants actually sort. sort are attached as appendix 3. Write or type one element on each chip, and Enclose the guide strip, the chips, the answer number the chips from 1 to X. Make the chips sheet, and the instructions in a folder, and you Table 2. A Guide Strip Guide to sorting statements Least Most important important -3 -2 -1 0 1 2 3 2 statements 3 statements 4 statements 4 statements 4 statements 3 statements 2 statements 28 Using Q-Sorts in Participatory Processes are ready to turn to the participants and the like, until they arrive at a first pass on the setting. sort, with the right number of statements arranged under each column of the guide Preparing the Setting and Instructing Participants strip. 5. Take a step back, review the sort, and make You will need to consider the setting and the changes. Again, the process is intended to be instructions you will deliver to participants. iterative (as participants consider the attrac- The Q-sort can be administered in group or tiveness of each element, relative to the other solo settings. In either case, participants need elements). Sometimes this second look will a wide, flat workspace large enough to lay yield changes. out the guide strip, sort the statements, 6. Record the sort. Participants can record their visually review the entire sort (to make sorts by writing the number corresponding adjustments), and then record their answers to each element in the space on the answer without disturbing the sort. Sorts can be sheet that represents its place on the sort on handled fairly quickly; allot 20 or 25 minutes the table. The answer sheet is a "snapshot" for a sort of 30 statements. of the sort on the table. The instructions should make clear to 7. Check the recorded sort. Have participants participants the purpose of each of the four double-check their own entry for double- components in the pack. If in a group admin- counts or blank spaces. They will not catch istration, verbal instructions should be given every mistake but will catch a few. immediately upon--if not before--distribu- tion of the pack, since curious participants Be available for help as participants com- will take things out of the pack and start plete the sort. working with them as soon as they can. The As you collect the responses, try to run your seven basic steps that participants should be eyes over each sort to check for mistakes and instructed to follow are: blanks. Mistakes can be hard to remedy later, and if one item is wrong, the participant's 1. Count the chips. Make sure the proper num- whole sort is invalid. Once the sheets are ber of chips are there. collected, you can take back the chips and 2. Review the question (the instructions or header. Carefully check the number of chips context for the sort). Make it understood that before reusing them. each chip represents one element that can be interpreted as "more" or "less" important to Data Analysis the question at hand. 3. Review the guide. See the shape into which Analysis can take as little as a couple of hours the elements must be sorted. There is no or considerably longer, depending on the impact on the order within columns, only number of iterations you wish to pursue and between them. the degree of depth to which you want to work 4. With the question in mind, sort the items in a with the data. A normal statistical package with multi-stage process. Most people cannot assign factor analysis capability is sufficient to per- a number to each statement on the first pass. form the analysis, but a specialty program such It is much easier to do a rough sort first, as PQMethod (mentioned earlier) makes things categorizing items into three piles: less much easier. important, neutral, and more important. It This section will present the minimal num- does not matter how many statements go ber of steps necessary to yield a Q analysis. It into each pile at first. If the participant then will presume and require little statistical works to prioritize statements within each background and will be linked to the use of the pile in turn, the sort will begin to emerge. free PC (DOS) program PQMethod. Research- Participants should be encouraged to adjust ers interested in a more in-depth treatment of statements on the fly, as many times as they the statistical underpinnings of Q, the various 29 Jonathan C. Donner options and nuances of the method, and the 1. Launch PQMethod. It is free. Download it at range of possibilities should examine McKeown the Q-method page (mentioned earlier). It is and Thomas (1988), or Steven R. Brown's available for PC (DOS), Mac, or Unix. These primer on Q-methodology (1993). The latter instructions refer to PQMethod for PC (DOS). appears in Operant Subjectivity, a journal dedi- Make sure to get the manual, as it will help cated almost exclusively to Q-methodology. with the downloading and installation. This Finally, this summary assumes a Q targeted at chapter is not a tutorial on DOS, nor is it a multiple participants at a single administration. substitute for the PQMethod manual. What it Slightly different methods would be used to will do is guide you through the steps with an look at a single participant's sorts under eye toward getting to the three key outputs as different umbrella questions. easily as possible. Starting with the end in mind, a Q-sort creates three basic outputs for analysis and 2. Enter the statements and data. Once discussion: PQMethod is running, you will be asked to name the project for DOS. This is straightfor- 1. The distinct groups, or common "perspec- ward, as long as you do not mind saving to the tives" that appear in the sample. Using a form "projects" folder within the PQMethod folder. of factor analysis, the 10, 20, 30, or 50 partici- Otherwise, you will have to specify a path pants who completed the sort are compressed name. Keep the study name to 8 characters. I into 3, 4, 5, or more subgroups, each reflecting a chose SDVV2000 for this project. You then see common pattern of responses. Each of these the main dialogue screen for PQMethod (table subgroups can be portrayed with a "snapshot," 3)--the one you will return to between each summarizing the average sort of the partici- operation. pants in that subgroup. It is useful to name Once you have named the project, the first these subgroups with an evocative summary stage in the PQMethod analysis is to enter the title. statements routine: STATES. These correspond 2. Contention elements are elements that to the elements you have used for the sort. You distinguish one subgroup from another. The can type them by hand into the DOS editor, farther apart the rankings of any two sub- specify a different text editor open automati- groups on an item, the more contentious that cally, or tell the software to find a file. The third element is likely to be. Identification of these option may be the easiest: prepare the file in items can prompt excellent discussions in participatory sessions. 3. Consensus elements are elements generally Table 3. Main Dialogue Screen for PQMethod agreed on by members of most groups (every- one agrees, or everyone disagrees). If the Current Project is ... c:/pqmethod/projects/sdv2000. discussion of the contention items gets out of Choose the number of the routine you want to run and enter it. hand, it is handy to have the consensus items in your back pocket. 1 ­ STATES - Enter (or edit) the file of statements. 2 ­ QENTER - Enter Q-sorts (new or continued). To generate these three outputs, there are six 3 ­ QCENT - Perform a Centroid factor analysis. steps to be performed using PQMethod: 4 ­ QPCA - Perform a Principal Components factor analysis. 5 ­ QROTATE - Perform a manual rotation of the 1. Load and launch PQMethod. factors. 2. Enter the statements and data. 6 ­ QVARIMAX - Perform a varimax rotation of the 3. Extract initial factors. factors. 4. Rotate the factors. 7 ­ QANALYZE - Perform the final Q analysis of 5. Group participants. the rotated factors. 6. Generate the data run(s). 8 ­ View project files sdv2000.* X ­ Exit from PQMethod. 30 Using Q-Sorts in Participatory Processes advance with Microsoft Word or Notepad and · These factors are rotated to arrive at clearer save it as a text-only file (.txt). Then, rename it representations of distinct patterns of obser- with the correct extension for PQMethod, vations. replacing .txt with .sta. As you enter state- ments, keep two things in mind: (1) limit There are slight differences between a factor your statements to 60 characters each, and analysis (in which variables that exhibit similar (2) make sure you enter the statements in the patterns are grouped) and Q-analysis (in which correct order, since the program assigns num- people who exhibit similar patterns of re- bers to each statement automatically. Separate sponses are grouped), but for details on this statements with a simple hard return (line topic, please visit two texts that handle the break). seemingly bottomless nuances of Q-statistics Now you are ready to enter the data, using better than this author could: McKeown and routine #2: QENTER. To build the data matrix, Thomas (1988) or (Brown 1993). PQMethod will ask for four pieces of data: The first two phases of this triad of opera- tions are completely automatic for beginners. 1. A title for the project Running routine #4 QPCA will derive the 2. The number of statements that were sorted correlations and then perform an initial factor 3. The leftmost and rightmost column values in analysis using the principal components the sort (for example, -3 and 3) method. This will give you the raw building 4. The number of statements that appear under blocks for the evaluation step (rotation) that each column (for example, 2, 3, 4, 4, 4, 3, 2). will follow. Skip routine #3 QCENT. QCENT is an alternative method of extracting initial Once the software has the shape and size of factor, which would run a factor analysis using the curve, its process for guiding you through centroid extraction. The difference between the data entry is fairly painless. Starting with the two approaches is negligible for most Q-sorts identification number, PQMethod will ask you (McKeown and Thomas 1988). for the numbers for each statement that ap- The main benefit of QPCA over QCENT is pears under each column. If you mis-type, or if that QPCA gives you eigenvalues for each your data are flawed, the system will not let factor, which can help you decide how many you continue. You can also re-visit or edit sorts groups you eventually want to build. Eigenval- you have previously entered. The system is not ues are a measure of the relative contribution of perfect--there are many hard returns between a factor to the explanation of the total variance you and a completed data matrix. However, in the correlation matrix. Factors with an the system is better than alternatives in SPSS or eigenvalue greater than one explain more Excel because it allows you to enter the data in variance than a single variable would. Thus, the shape in which they were gathered. If you the maximum number of factors you would gathered other information while doing the want to carry into the rotation step is equal to sort (gender, age, attitude data), this system is the number of initial factors with eigenvalues not as effective, but you can handle those greater than one. Table 4 displays the eigenval- variables in Excel or any statistical package ues extracted by QPCA on the SDV data. In this once the Q-sort is complete. case, the maximum number of factors to carry into the next step would be 5 (although #5 is 3. Extract the initial factors. At the core of a Q- just barely above one). sort is set of three procedures that look very much like a factor analysis (Kim and Mueller 4. Rotate the factors. This is the first stage in 1978): which there is real flexibility in the process for beginners, since the rotation stage allows you · Correlations between observations (partici- to specify the number of factors (subgroups of pants' sorts) are derived. participants) you want to carry forward into · Initial factors are extracted. the analysis. As mentioned above, the QPCA 31 Jonathan C. Donner procedure yields eigenvalues for each initial mizing the variance between each of the factor extracted. Using this information (dis- factors. (It adjusts the weights given to each played in table 4), you should have identified element on each factor.) Launch the procedure the maximum number of factors you will rotate and specify the number of factors you want to (based on all factors with an eigenvalue greater rotate. The DOS version of PQMethod will ask than one). However, you should not be guided you whether you want to use the newer graphi- by the eigenvalues alone. It may be the case cal procedure. You should opt to do so. that choosing fewer factors will yield sufficient Table 5 illustrates roughly what the output dispersion to make the story clear. The more from the rotated factor solution will show.7 factors you have, the more fragmented the data Each factor is a column; each participant is a will become. You will have to judge for your- row. The numeric value in each cell indicates self whether a 3-, 4-, or 5- (or more) factor the strength of the participant's "loading" onto solution is most compelling. Thus, you will the factor. Loads can be as high as 1 (indicating probably want to run the analysis more than perfect agreement), or as low as ­1. In the case once, starting with the smallest number factors of negative values, participants can be said to you think you will accept, and ending at the disagree with the factor. So, in the case of the maximum number you think you can rotate, SDV data, participant #3 (R3) loaded highly based on the eigenvalues. (0.8) onto factor 1, and relatively strongly (-0.4) PQMethod offers two procedures for onto factor 3. The more a participant loads rotating factors. Routine #5, QROTATE, offers cleanly (disproportionately) onto a single functionality to rotate factors by hand and by factor, the better that factor represents that sight. This is a more advanced procedure, participant's sort--and subjective perspective which will not be covered in this chapter. on the issue at hand. Skip routine #5: QROTATE. The second procedure for rotating factors is 5. Group participants. The final aspect of the #6: QVARIMAX. Commonly applied through- rotation sequence is to assign participants to out the social sciences, and likely to yield an factors. This is the step at which you are creat- acceptable result, this varimax procedure is the ing the subgroups and establishing the founda- one you should use. Varimax rotation clarifies tion for the calculation of the distinct "voices" the structure of your chosen factors by maxi- present among your participants. The software Table 4. Output from PQMethod on SDV Data: Results of QPCA Procedure Cumulative Eigenvalues As percentages percentages 1 5.4953 32.3251 32.3251 2 2.1647 12.7337 45.0588 3 2.0397 11.9982 57.0570 4 1.6746 9.8508 66.9078 5 1.0443 6.1427 73.0505 6 0.9452 5.5603 78.6108 7 0.8041 4.7302 83.3410 8 0.6988 4.1108 87.4517 9 0.5912 3.4777 90.9294 10 0.4695 2.7620 93.6914 11 0.3299 1.9407 95.6321 12 0.2774 1.6316 97.2637 13 0.2509 1.4758 98.7395 14 0.0889 0.5227 99.2622 15 0.0605 0.3561 99.6183 16 0.0380 0.2236 99.8419 17 0.0269 0.1581 100.0000 32 Using Q-Sorts in Participatory Processes Table 5. Example of Rotated Factor Solution: of the automatic pre-flagging on the left for the SDV Data SDV data, and the manual edits I made on the right. Fac1 Fac2 Fac3 Fac4 In this case, the only manual change I made R1 0.38 0.10 0.80 0.74 to the automatic pre-flagging was to switch the R2 0.14 0.76 -0.14 0.30 affiliation of participant R15 from factor 2 to R3 0.80 0.32 -0.40 -0.10 R4 -0.15 0.78 0.27 0.14 factor 4. Negative loads, although clear in R5 -0.10 0.21 0.88 -0.50 theory, are difficult for participants to under- R6 0.80 0.42 0.57 0.22 stand. Imagine this conversation: "Well . . . Bill, R7 0.68 0.20 0.32 0.17 Bob, and Mary belong to subgroup 2. But as for R8 0.73 -0.20 0.70 0.45 Joe, Joe's defining characteristic is how little he R9 0.29 -0.11 0.64 0.13 agrees with Bob, Bill, and Mary." The 0.49 load R10 0.29 0.50 0.16 0.62 R11 0.87 0.30 0.16 0.16 of R15 on group 4 is not great, but not bad R12 0.13 0.30 0.42 0.50 either: the minimum threshold for loading on a R13 0.30 -0.19 0.71 0.26 group is just about 0.50. The first tip, therefore, R14 0.55 -0.60 0.20 0.58 in terms of how to manually adjust automatic R15 -0.42 -0.67 0.60 0.49 pre-flags is to avoid assignment to groups R16 -0.10 0.70 0.20 0.80 based on negative loads. R17 0.12 0.17 -0.30 0.66 The second tip is to avoid or ignore people who do not load cleanly onto a factor. For can begin this process for you by performing example, in the SDV data, R14 sits at 0.55 on an "automatic pre-flag." All it will do is mark factor 1, but also at 0.58 at factor 4. I chose to the cases that load cleanly onto a factor. After not assign that participant to a group. Again, that, you may want to manually adjust the the more factors you run, the less likely it is flags to compensate for close calls or other that you will encounter double- or triple- nuances in your data. Table 6 shows the results loading and the cleaner your factors are likely Table 6. Automatic and Manual Flags of SDV Factor Data Assigning participants to factors: "Flagging" assignments to subgroups marked with * B. With manual adjustments (changes A. Automatic from automatic are shaded) Fac1 Fac2 Fac3 Fac4 Fac1 Fac2 Fac3 Fac4 R1 0.38 0.10 0.80 0.74* 0.38 0.10 0.80 0.74* R2 0.14 0.76* -0.14 0.30 0.14 0.76* -0.14 0.30 R3 0.80* 0.32 -0.40 -0.10 0.80* 0.32 -0.40 -0.10 R4 -0.15 0.78* 0.27 0.14 -0.15 0.78* 0.27 0.14 R5 -0.10 0.21 0.88* -0.50 -0.10 0.21 0.88* -0.50 R6 0.80 0.42 0.57* 0.22 0.80 0.42 0.57* 0.22 R7 0.68* 0.20 0.32 0.17 0.68* 0.20 0.32 0.17 R8 0.73* -0.20 0.70 0.45 0.73* -0.20 0.70 0.45 R9 0.29 -0.11 0.64* 0.13 0.29 -0.11 0.64* 0.13 R10 0.29 0.50 0.16 0.62* 0.29 0.50 0.16 0.62* R11 0.87* 0.30 0.16 0.16 0.87* 0.30 0.16 0.16 R12 0.13 0.30 0.42 0.50* 0.13 0.30 0.42 0.50* R13 0.30 -0.19 0.71* 0.26 0.30 -0.19 0.71* 0.26 R14 0.55 -0.60 0.20 0.58 0.55 -0.60 0.20 0.58 R15 -0.42 -0.67* 0.60 0.49 -0.42 -0.67 0.60 0.49* R16 -0.10 0.70 0.20 0.80* -0.10 0.70 0.20 0.80* R17 0.12 0.17 -0.30 0.66* 0.12 0.17 -0.30 0.66* Results: 4 participants in group 1, 2 in group 2, 4 in group 3, and 6 in group 4. One participant (R14) is unassigned. 33 Jonathan C. Donner to be. However, you will also have more · Correlations between each factor. This demon- subgroups to discuss at the end, and fewer strates how similar each factor (subgroup) is consensus statements left in the data. to each other factor. · Normalized factor scores for each factor 6. Generate the data run(s). Once you have (subgroup). These tables go factor by factor, assigned participants to subgroups, you are listing all the statements in descending order ready to submit the prepared data back to the of ranked importance. You will use these PQMethod software for analysis. Select routine pages to determine the overall character of a #7, QANALYSE, and let it write the results to a subgroup's perspective. The Z-scores show list file. You can print it out or save it in how far from the overall mean (measured in Microsoft Word, according to what you need. standard deviations) each item is for the At this stage, it is easy to run two or three group. different analyses at once, with different · Array of differences between factors. This block numbers of factors and assignments to groups, of tables allows for per-item comparisons and then print them all to compare side by between each factor. side. At that point, you can close PQMethod. · Factor Q-Sort values for each statement. These two snapshot pages give you the "voice"-- Interpretation average sort--for each factor group. For easy interpretation, the Z-scores are translated The PQMethod file you generate for each run back into the original scale used in the sort. will contain a number of items and is likely to These are sorted in order of statement be over 20 pages long. There is more detail than number, then by degree of agreement be- you need as a beginner, but it is quite conve- tween the groups. nient to have everything generated at once and · Factor characteristics. This reports the number in one place. Items generated include the of defining variables (statistically distinct following (items needed for a basic evaluation from other groups) and other indexes of how of the results are marked with an asterisk): well the factor holds together. · Distinguishing characteristics for each factor. · Correlation matrix. This shows the correlation These tables highlight contention statements between individual pairs of Q-sorts. that subgroup members have ranked signifi- · Unrotated factor matrix. This will contain cantly differently from other subgroups eight factors if you choose QPCA, plus the (higher or lower than overall average). Use accompanying eigenvalues and the percent- these tables to help define the key differ- age of variance each factor represents. ences among subgroups. · Rotated factor matrix, with the number of · Consensus statements. These items do not requested factors, the item loadings, and distinguish between any pair of sub- percent of variance explained by each factor. groups. (The format looks similar to table 5 above. Unlike the display for the unrotated matrix, The first key output from the analysis-- this table will contain information only for profiles of the perspective of each distinct the factors you selected (in this case, four)). subgroup--is derived using three sources: the · Free distribution data results. The participant- factor Q-sort values, the normalized factor by-participant summaries of the mean and scores, and the distinguishing characteristics of standard deviations of their sort should be each factor. We will visit the contribution of the same for each case and are therefore of each in turn. little use. The Factor Q-Sort Values for each statement · Rank statement totals for each factor. The provide a good snapshot of how each group totals tell you how each factor (subgroup) ranked the items. Table 7 contains the example ranked each statement. It can be useful, but from the SDV data. The values in the columns the same information will be available in represent the "archetypal" pattern sort of the clearer formats later in the analysis packet. participants assigned (flagged) to that factor. 34 Using Q-Sorts in Participatory Processes Table 7. Factor Q-Sort Values for SDV Fac1 Fac2 Fac3 Fac4 Num Element n=4 n=2 n=4 N=6 SDV is composed of people with a mix of educational and professional 1 backgrounds with management and client-interface skills that match their 2 0 -1 -2 technical proficiencies. SDV is a leading resource within the Bank for easily applicable solutions to 2 social issues facing country teams, such as participatory methods or local 3 1 2 3 capacity building. SDV manages a network of external links to NGOs both in developing and 3 developing nations, which enables SDV to play a role in coordinating 0 2 -2 -2 actions among multiple actors in the development arena. SDV delivers stand-alone projects to country teams, complete with staff or 4 -2 0 -1 -3 connections to local staff. SDV is a provider of region-level solutions to problems; with a staff and 5 0 1 2 -1 budget that resides primarily with the regions. SDV is involved, through an allocated staff member, on nearly every 6 -3 1 -1 1 relevant Bank project. 7 SDV has its own products that can be offered directly to clients in countries. -2 2 0 -2 SDV is respected and in demand within the Bank as a source for efficient 8 -1 -3 -3 0 and fair compliance reviews. SDV serves as the primary advocate within the Bank for the needs of 9 -2 -1 1 2 countries' most vulnerable populations. SDV affects change in the Bank by persuasion; its voice and 10 -1 -1 3 1 recommendations are respected by Bank audiences. SDV is the primary locus of policy formation for certain crosscutting social 11 3 3 3 2 issues. SDV is able to quickly anticipate and adjust its offerings based on the needs 12 1 -2 0 -1 of client teams. SDV is a leading resource within the Bank for knowledge on a broad range 13 2 2 0 2 of crosscutting social issues. SDV serves as a center for training and development of other Bank staff on 14 2 -3 -2 0 social issues. SDV challenges and pushes the Bank to maintain a focus on programs that 15 -1 1 0 3 reduce poverty. 16 SDV is a driver of quality assurance for Bank projects. 0 0 -2 -1 SDV is a source of world-class research in issues related to social 17 -3 -1 1 -3 development. SDV attracts and retains the most skilled social scientists working in 18 1 -1 1 -1 development. SDV is a constant innovator; developing a series of new products and 19 1 -3 2 0 services to offer within the Bank. SDV serves as a center for training and development of non-Bank staff 20 0 -2 -3 0 (government, other NGOs) on social issues. SDV works on the front lines, allocating staff to designing and implementing 21 -1 3 1 1 projects on the ground in developing countries. SDV serves as a clearinghouse for a core set of assessment tools that are 22 1 0 -1 1 used by the majority of Bank project teams. From this Q-sort values page alone, you Now the profile is clear: items at the top of could create "snapshots" of each group's the table are ranked as more important by the perspective on the issue. However, it is difficult group. Items at the bottom of the page are to get a sense of the relative priorities of any ranked as less important. one group without sorting for that group. So, The final piece of the profiling sequence is to returning to output 7 (normalized factor scores) is isolate the elements in the sort that are distinc- helpful. Table 8 contains the normalized factor tive for each group. Output 11 (distinguishing scores for first subgroup in the SDV sort. characteristics for each factor) is helpful here. 35 Jonathan C. Donner Table 8. Normalized Factor Scores for SDV Subgroup 1 Num Statement Z-score SDV is a leading resource within the Bank for easily applicable solutions to 2 social issues facing country teams, such as participatory methods or local 1.812 capacity building. SDV is the primary locus of policy formation for certain crosscutting social 11 1.764 issues. SDV is a leading resource within the Bank for knowledge on a broad range of 13 1.223 crosscutting social issues. SDV is composed of people with a mix of educational and professional 1 backgrounds with management and client-interface skills that match their 0.871 technical proficiencies. SDV serves as a center for training and development of other Bank staff on 14 0.798 social issues. SDV is able to quickly anticipate and adjust its offerings based on the needs of 12 0.779 client teams. SDV serves as a clearinghouse for a core set of assessment tools that are used 22 0.707 by the majority of Bank project teams. SDV is a constant innovator; developing a series of new products and services 19 0.588 to offer within the Bank. SDV attracts and retains the most skilled social scientists working in 18 0.281 development. SDV serves as a center for training and development of non-Bank staff 20 0.209 (government, other NGOs) on social issues. SDV is a provider of region-level solutions to problems; with a staff and budget 5 0.165 that resides primarily with the regions. 16 SDV is a driver of quality assurance for Bank projects. -0.212 SDV manages a network of external links to NGOs both in developing and 3 developing nations, which enables SDV to play a role in coordinating actions -0.358 among multiple actors in the development arena. SDV works on the front lines, allocating staff to designing and implementing 21 -0.447 projects on the ground in developing countries. SDV affects change in the Bank by persuasion; its voice and recommendations 10 -0.561 are respected by Bank audiences. SDV is respected and in demand within the Bank as a source for efficient and 8 -0.658 fair compliance reviews. SDV challenges and pushes the Bank to maintain a focus on programs that 15 -0.659 reduce poverty. 7 SDV has its own products that can be offered directly to clients in countries, -0.872 SDV serves as the primary advocate within the Bank for the needs of countries' 9 -0.965 most vulnerable populations. SDV delivers stand-alone projects to country teams, complete with staff or 4 -1.147 connections to local staff. SDV is involved, through an allocated staff member, on nearly every relevant 6 -1.504 Bank project. 17 SDV is a source of world-class research in issues related to social development. -1.812 Table 9 illustrates the distinguishing character- networks) and item 6 (involved on every istics for subgroup 1 in the SDV data. project) lower than average. This table shows that the four participants in To create the summary profile of a sub- subgroup 1 rate item 1 (skill mix), item 14 group, combine the three outputs discussed (training for Bank staff) and item 12 (speed of above. Take the factor values for the subgroup adjustment) higher than the average of the from table 7, put them in the order of the other subgroups. They rate item 3 (NGO factor-specific sort from table 8, and flag the 36 Using Q-Sorts in Participatory Processes Table 9. Distinguishing Characteristics for SDV Subgroup 1 Group 1 Statements significantly different than Factor 1 Factor 2 Factor 3 Factor 4 overall mean @ p<0.05 (bold n=4 n=2 n=4 n=6 @<0.01) Num Statement RNK Z RNK Z RNK Z RNK Z SDV is comprised of people with a mix of educational and professional backgrounds with 1 2 0.87 0 0.00 -1 -0.46 -2 -0.74 management and client-interface skills that match their technical proficiencies. SDV serves as a center for 14 training and development of 2 0.80 -3 -1.30 -2 -1.19 0 -0.08 other Bank staff on social issues. SDV is able to quickly anticipate 12 and adjust its offerings based on 1 0.71 -2 -1.27 0 -0.05 -1 -0.63 the needs of client teams. SDV manages a network of external links to NGOs both in developing and developing 3 nations, which enables SDV to 0 -0.36 2 1.27 -2 -1.06 -2 -0.98 play a role in coordinating actions among multiple actors in the development arena. SDV is involved, through an 6 allocated staff member, on nearly -3 -1.50 1 0.65 -1 -0.60 1 0.16 every relevant Bank project. significant variables from table 9 to tell the does not feel the need to be involved via an story of the worldview of the four participants allocated staff member on "nearly every" who you categorized into subgroup 1. Table 10 Bank project. (Mention the items the sub- is an example of such a summary profile, again group ranked as defining low.) using subgroup 1 from the SDV data. · Like many others, members of subgroup 1 We're finally at the moment at which real see "world class research" and "stand-alone interpretation can commence. Using this products" as less important. (Mention the summary chart, a few patterns emerge: subgroup's absolute lows, whether distin- guishing or not.) · Like others in the sample, members of subgroup 1 want SDV to be a leading re- At this point, a researcher could show the source within the Bank for knowledge about data chart above, the bullet points, or some and solutions to "social issues" and to be combination. If there were demographic clues involved in policy formation. (The highest to the composition of the group, there may items are worth mentioning, even if they are be additional points of differentiation to not unique.) discuss. · Subgroup 1 is particularly interested in We often find that naming the group with a developing a mixed skill-set (including brief, evocative, sometimes colorful name can management skills) and in supplying train- help anchor the group in audience's minds. On ing on social issues to other Bank staff. (Note the other hand, it can be a great interactive the defining high items, even if they are not exercise to have the participants name the top-ranked.) group together. Often, they will have a person · Subgroup 1 is relatively less interested than or an archetype in mind that helps this particu- others in managing a network of NGOs and lar worldview come alive for them. It also 37 Jonathan C. Donner Table 10. Summary of Factor 1 (n=4) Num Statement Score Note SDV is a leading resource within the Bank for easily applicable solutions to 2 social issues facing country teams, such as participatory methods or local 3 capacity building. SDV is the primary locus of policy formation for certain crosscutting social 11 3 issues. SDV is a leading resource within the Bank for knowledge on a broad range of 13 2 crosscutting social issues. SDV is composed of people with a mix of educational and professional 1 backgrounds with management and client-interface skills that match their 2 High technical proficiencies. SDV serves as a center for training and development of other Bank staff on 14 2 High social issues. SDV is able to quickly anticipate and adjust its offerings based on the needs of 12 1 client teams. SDV serves as a clearinghouse for a core set of assessment tools that are used 22 1 High by the majority of Bank project teams. SDV is a constant innovator; developing a series of new products and services 19 1 to offer within the Bank. SDV attracts and retains the most skilled social scientists working in 18 1 development. SDV serves as a center for training and development of non-Bank staff 20 0 (government, other NGOs) on social issues. SDV is a provider of region-level solutions to problems; with a staff and budget 5 0 that resides primarily with the regions. 16 SDV is a driver of quality assurance for Bank projects. 0 SDV manages a network of external links to NGOs both in developing and 3 developing nations, which enables SDV to play a role in coordinating actions 0 Low among multiple actors in the development arena. SDV works on the front lines, allocating staff to designing and implementing 21 -1 projects on the ground in developing countries. SDV affects change in the Bank by persuasion; its voice and recommendations 10 -1 are respected by Bank audiences. SDV is respected and in demand within the Bank as a source for efficient and 8 -1 fair compliance reviews. SDV challenges and pushes the Bank to maintain a focus on programs that 15 -1 reduce poverty. 7 SDV has its own products that can be offered directly to clients in countries. -2 SDV serves as the primary advocate within the Bank for the needs of countries' 9 -2 most vulnerable populations. SDV delivers stand-alone projects to country teams, complete with staff or 4 -2 connections to local staff. SDV is involved, through an allocated staff member, on nearly every relevant 6 -3 Low Bank project. 17 SDV is a source of world-class research in issues related to social development. -3 begins the process of having the participants pants, the last clue that makes helps in the take the perspective of another, as they try to naming and interpretation of this profile is ascertain what the members of subgroup 1 other profiles. Seeing how subgroup 1 looks were thinking as they prioritized the items. different from subgroups 2, 3, and 4 can make This is a key benefit of the playback/participa- interpreting subgroup 1 a little easier. Tables tory process. 11-13 show the profiles for the other three Whether naming is done by the researcher subgroups, along with some explanatory ahead of time, or in collaboration with partici- bullets. 38 Using Q-Sorts in Participatory Processes Table 11. Summary of Subgroup 2 (n=2) Num Statement Score Note SDV is the primary locus of policy formation for certain crosscutting social 11 3 issues. SDV works on the front lines, allocating staff to designing and implementing 21 3 High projects on the ground in developing countries. SDV is a leading resource within the Bank for knowledge on a broad range of 13 2 crosscutting social issues. SDV manages a network of external links to NGOs both in developing and 3 developing nations, which enables SDV to play a role in coordinating actions 2 High among multiple actors in the development arena. 7 SDV has its own products that can be offered directly to clients in countries. 2 High SDV is involved, through an allocated staff member, on nearly every relevant 6 1 Bank project. SDV challenges and pushes the Bank to maintain a focus on programs that 15 1 reduce poverty. SDV is a leading resource within the Bank for easily applicable solutions to 2 social issues facing country teams, such as participatory methods or local 1 capacity building. SDV is a provider of region-level solutions to problems; with a staff and budget 5 1 that resides primarily with the regions. 16 SDV is a driver of quality assurance for Bank projects. 0 SDV is composed of people with a mix of educational and professional 1 backgrounds with management and client-interface skills that match their 0 technical proficiencies. SDV delivers stand-alone projects to country teams, complete with staff or 4 0 connections to local staff. SDV serves as a clearinghouse for a core set of assessment tools that are used 22 0 by the majority of Bank project teams. SDV serves as the primary advocate within the Bank for the needs of countries' 9 -1 most vulnerable populations. SDV affects change in the Bank by persuasion; its voice and recommendations 10 -1 are respected by Bank audiences. SDV attracts and retains the most skilled social scientists working in 18 -1 development. 17 SDV is a source of world-class research in issues related to social development. -1 SDV serves as a center for training and development of non-Bank staff 20 -2 Low (government, other NGOs) on social issues. SDV is able to quickly anticipate and adjust its offerings based on the needs of 12 -2 client teams. SDV serves as a center for training and development of other Bank staff on 14 -3 social issues. SDV is respected and in demand within the Bank as a source for efficient and 8 -3 fair compliance reviews. SDV is a constant innovator; developing a series of new products and services 19 -3 Low to offer within the Bank. · Like others in the sample, the two members of · Subgroup 2 is relatively less interested in subgroup 2 want SDV to be a leading resource training non-Bank staff. Most interestingly, within the Bank for knowledge about (but not they feel strongly that "constant innova- necessarily solutions to) "social issues" and to tion" may be less important than other be involved in policy formation. goals. · Subgroup 2 is particularly interested in · Like some others, subgroup 2 is not as con- going to the front lines and in offering its cerned with training Bank staff or with SDV's products directly to clients in countries. role in compliance reviews within the Bank. 39 Jonathan C. Donner Table 12. Summary of Subgroup 3 (n=4) Num Statement Score Note SDV is the primary locus of policy formation for certain crosscutting social 3 11 issues. SDV affects change in the Bank by persuasion; its voice and recommendations 3 High 10 are respected by Bank audiences. SDV is a provider of region-level solutions to problems; with a staff and budget 2 High 5 that resides primarily with the regions. SDV is a leading resource within the Bank for easily applicable solutions to 2 2 social issues facing country teams, such as participatory methods or local capacity building. SDV is a constant innovator; developing a series of new products and services 2 19 to offer within the Bank. 17 SDV is a source of world-class research in issues related to social development. 1 High SDV attracts and retains the most skilled social scientists working in 1 18 development. SDV works on the front lines, allocating staff to designing and implementing 1 21 projects on the ground in developing countries. SDV serves as the primary advocate within the Bank for the needs of countries' 1 9 most vulnerable populations. 7 SDV has its own products that can be offered directly to clients in countries. 0 Low SDV is able to quickly anticipate and adjust its offerings based on the needs of 0 12 client teams. SDV challenges and pushes the Bank to maintain a focus on programs that 0 15 reduce poverty. SDV is a leading resource within the Bank for knowledge on a broad range of 0 Low 13 crosscutting social issues. SDV delivers stand-alone projects to country teams, complete with staff or -1 4 connections to local staff. SDV is composed of people with a mix of educational and professional -1 1 backgrounds with management and client-interface skills that match their technical proficiencies. SDV serves as a clearinghouse for a core set of assessment tools that are used -1 22 by the majority of Bank project teams. SDV is involved, through an allocated staff member, on nearly every relevant -1 Low 6 Bank project. 16 SDV is a driver of quality assurance for Bank projects. -2 SDV manages a network of external links to NGOs both in developing and -2 3 developing nations, which enables SDV to play a role in coordinating actions among multiple actors in the development arena. SDV serves as a center for training and development of other Bank staff on -2 14 social issues. SDV is respected and in demand within the Bank as a source for efficient and -3 8 fair compliance reviews. SDV serves as a center for training and development of non-Bank staff -3 Low 20 (government, other NGOs) on social issues. · Like others in the sample, the four members sion and providing solutions at the regional of subgroup 3 want SDV to be a leading level. They are also relatively more inter- resource within the Bank for solutions to ested in SDV's role in producing world-class (but not necessarily knowledge about) research. "social issues" and to be involved in policy · Subgroup 3 is least interested in training formation. non-Bank staff. They are also less interested · Subgroup 3 is particularly interested in in getting involved on all Bank projects and affecting change in the Bank through persua- in having stand-alone projects. Finally, they 40 Using Q-Sorts in Participatory Processes Table 13. Summary of Subgroup 4 (n=6) Num Statement Score Note SDV challenges and pushes the Bank to maintain a focus on programs that 15 3 High reduce poverty. SDV is a leading resource within the Bank for easily applicable solutions to 2 social issues facing country teams, such as participatory methods or local 3 capacity building. SDV serves as the primary advocate within the Bank for the needs of countries' 9 2 High most vulnerable populations. SDV is the primary locus of policy formation for certain crosscutting social 11 2 issues. SDV is a leading resource within the Bank for knowledge on a broad range of 13 2 crosscutting social issues. SDV affects change in the Bank by persuasion; its voice and recommendations 10 1 High are respected by Bank audiences. SDV serves as a clearinghouse for a core set of assessment tools that are used 22 1 by the majority of Bank project teams. SDV works on the front lines, allocating staff to designing and implementing 21 1 projects on the ground in developing countries. SDV is involved, through an allocated staff member, on nearly every relevant 6 1 Bank project. SDV serves as a center for training and development of other Bank staff on 14 0 High social issues. SDV serves as a center for training and development of non-Bank staff 20 0 (government, other NGOs) on social issues. SDV is respected and in demand within the Bank as a source for efficient and 8 0 fair compliance reviews. SDV is a constant innovator; developing a series of new products and services 19 0 Low to offer within the Bank. SDV attracts and retains the most skilled social scientists working in 18 -1 development. 16 SDV is a driver of quality assurance for Bank projects. -1 SDV is a provider of region-level solutions to problems; with a staff and budget 5 -1 Low that resides primarily with the regions. SDV is able to quickly anticipate and adjust its offerings based on the needs of 12 -1 client teams. SDV is composed of people with a mix of educational and professional 1 backgrounds with management and client-interface skills that match their -2 technical proficiencies. SDV manages a network of external links to NGOs both in developing and 3 developing nations, which enables SDV to play a role in coordinating actions -2 among multiple actors in the development arena. 7 SDV has its own products that can be offered directly to clients in countries. -2 SDV delivers stand-alone projects to country teams, complete with staff or 4 -3 connections to local staff. 17 SDV is a source of world-class research in issues related to social development. -3 are less interested in being a source for cross- subgroup 4 want SDV to be a leading re- cutting knowledge for the Bank of social source within the Bank for solutions to (but issues. not necessarily knowledge about) "social · Like some others in the sample, members of issues" and to be involved in policy subgroup 3 are not as concerned with SDV's formation. role in compliance reviews within the Bank · Subgroup 4 is particularly interested in and in training Bank staff. challenging the Bank to retain its focus · Like others in the sample, the six members of on poverty and to serve as the primary 41 Jonathan C. Donner advocate within the Bank for the needs of uncover different perspectives concerning countries' most vulnerable populations. It SDV's future. These data became one of the key also rates "affecting the Bank by persuasion" inputs to the change process during the five- higher than others in the sample. day offsite. · Subgroup 4 is less interested in continually At times, you may want to downplay the innovating and getting involved in all Bank exact characteristics of groups, and focus projects, and in providing solutions at the instead on individual items. Contention and region level. consensus items are easy to list and discuss. · Like some others in the sample, members of Table 15 shows the items we found in the case subgroup 4 are not as concerned with SDV of the SDV group. delivering stand-alone projects, and in SDV's Looking at the sorts on a per-item basis being a source for world-class research. yields a slightly different view. Three items were consensus high. SDV can most likely Based on these patterns, summarized below agree that it should remain (or become) a locus in table 14, the SDV team and the Monitor for policy formation on certain social issues consultants arrived at the following labels-- and should be a source for knowledge and representing four distinct visions for SDV (in solutions about social issues. This approach 2002, after a long slumber...). Clearly, in a identifies actionable consensus items. small group of 17 managers, who work In terms of contention items, there were lots closely (and well) together, we were able to of items with a reasonable gap (4 of 6 possible Table 14. Summary of Subgroups Found in the SDV Q-Sort Group More important Less important Name 1 · Source for knowledge and · Managing a network of NGOs Knowledge Shop solutions for social issues · Involvement in many projects · Policy formation · World-class research · Management skills · Stand-alone products · Training non-Bank staff 2 · Source for knowledge about · Training non-Bank staff In touch with the social issues · "Constant innovation" field · Policy formation · Training Bank staff · On the front lines · Compliance reviews · Offering products to clients 3 · Source for solutions to social · Training non-Bank staff Country team issues · Involvement on many projects influencers · Policy formation · Stand-alone products · Affecting change through · Source for knowledge about persuasion social issues · Providing solutions at the · Compliance reviews the Bank regional level · Training Bank staff · World-class research 4 · Source for solutions to social · Constant innovation Advocates issues · Involvement on many projects · Policy formation · Providing solutions at the · Challenging the Bank to retain regional level its focus on poverty · Stand-alone products · Advocate for the needs of · World-class research countries' most vulnerable populations · Affecting change through persuasion Note: Items in italics are distinguishing statements for the subgroups. 42 Using Q-Sorts in Participatory Processes Table 15. Summary of Contention and Consensus Statements Num Element Fac1 Fac2 Fac3 Fac4 Gap Note SDV is the primary locus of policy formation for certain Consensus 11 3 3 3 2 1 crosscutting social issues. high SDV is a leading resource within the Bank for easily applicable solutions to social issues facing country Consensus 2 3 1 2 3 2 teams, such as participatory methods or local capacity high building. SDV is a leading resource within the Bank for Near 13 knowledge on a broad range of crosscutting social 2 2 0 2 2 consensus issues. high SDV serves as a clearinghouse for a core set of 22 assessment tools that are used by the majority of Bank 1 0 -1 1 2 Neutral project teams. SDV attracts and retains the most skilled social 18 1 -1 1 -1 2 Neutral scientists working in development. SDV is a driver of quality assurance for Bank projects. Near 16 0 0 -2 -1 2 consensus neutral SDV serves as a center for training and development of 20 non-Bank staff (government, other NGOs) on social 0 -2 -3 0 3 issues. SDV is respected and in demand within the Bank as a 8 -1 -3 -3 0 3 source for efficient and fair compliance reviews. SDV is a provider of region-level solutions to problems; 5 with a staff and budget that resides primarily with the 0 1 2 -1 3 regions. SDV is able to quickly anticipate and adjust its offerings 12 1 -2 0 -1 3 based on the needs of client teams. SDV delivers stand-alone projects to country teams, 4 -2 0 -1 -3 3 complete with staff or connections to local staff. SDV challenges and pushes the Bank to maintain a Contention 15 -1 1 0 3 4 focus on programs that reduce poverty. item SDV serves as the primary advocate within the Bank Contention 9 -2 -1 1 2 4 for the needs of countries' most vulnerable populations. item SDV affects change in the Bank by persuasion; its Contention 10 voice and recommendations are respected by Bank -1 -1 3 1 4 item audiences. SDV works on the front lines, allocating staff to Contention 21 designing and implementing projects on the ground in -1 3 1 1 4 item developing countries. SDV is involved, through an allocated staff member, on Contention 6 -3 1 -1 1 4 nearly every relevant Bank project. item SDV is composed of people with a mix of educational and professional backgrounds with management and Contention 1 2 0 -1 -2 4 client-interface skills that match their technical item proficiencies. SDV manages a network of external links to NGOs both in developing and developing nations, which Contention 3 0 2 -2 -2 4 enables SDV to play a role in coordinating actions item among multiple actors in the development arena. SDV has its own products that can be offered directly to Contention 7 -2 2 0 -2 4 clients in countries. item SDV is a source of world-class research in issues Contention 17 -3 -1 1 -3 4 related to social development. item SDV serves as a center for training and development of Contention 14 2 -3 -2 0 5 other Bank staff on social issues. item SDV is a constant innovator; developing a series of Contention 19 1 -3 2 0 5 new products and services to offer within the Bank. item 43 Jonathan C. Donner points). The three items concerning advocacy Q helps plumb knotty, multidimensional and challenging the Bank fell in this category. problems in interactive and participatory ways, Whether SDV should be an internal training in a short time, with just a dozen or so partici- center was of even greater contention, as was pants. It is a great tool to have in your tool kit. whether SDV needs to be constantly innovating Give it a try! (in terms of developing new products). Notes Conclusion 1. Thanks to Gloria Davis, Judith Edstrom, James This chapter summarizes the results of one Edgerton, Anis Dani, and Ashraf Ghani, among particular four-factor Q-sort solution using others, for leading the SDV team that was evaluating SDV's data. As with any Q-sort, there is a great its strategy, and to James, Ashraf, and Anis especially deal to explore by varying the number of for their help designing the Q-sort itself. The SD data factors and the assignment of participants to displayed below represent a slight change from the groups. As a first step, however, this chapter analysis originally presented to the SD group, but should be a helpful guide to the process of the basic composition of the groups and topics conducting a Q-sort. It really is easy; the addressed were not affected by the re-analysis of detailed statistics are not a barrier, thanks to these data. The re-analysis was necessitated by the choice to use PQMethod as the recommended PQMethod. Q-methodology is a very powerful software package in this chapter. tool, not only for analysis and research but also 2. PQMethod and its manual can be downloaded as a starting point for interventions and partici- at no charge at Peter Schmolk's QMethod Page patory exercises. It can bring into clarity a http://www.rz.unibw-muenchen.de/~p41bsmk/ small number of dimensions to an issue that qmethod. It is available for DOS/Windows (PC), previously felt too messy or thorny to ap- MAC, and UNIX platforms. All commentary in this proach. Similarly, it can enable groups who chapter refers to PQMethod for DOS 2.09. seem divided into factions by political or social PQMethod was updated and maintained by Peter differences to see some commonalities of Schmolk, based on a mainframe program by John beliefs across factions, and some differences Atkinson. There are also other programs available. within them--a key step in beginning to reach See the Qmethod page. 3. The bibliography's link is http://www.rz.unibw- group consensus. muenchen.de/~p41bsmk/qmethod/syllabus00.pdf. Participants like the novel administration 4. The bibliography's link is http://www.rz.unibw- method, enjoy the apparent magic of the sorts, muenchen.de/~p41bsmk/qmethod/syllabus00.pdf. and are always interested in seeing their 5. WebQ can be found at: http://www.rz.unibw- opinions translated into patterns and quanti- muenchen.de/~p41bsmk/qmethod/webq. fied. With skilled facilitation, the resulting 6. I cannot show you an example from the SDV conversations can be lively, are rooted in data, sort since the margins would be outside what is and help participants consider not only the acceptable for a chapter like this. uniqueness of their own perspectives on an 7. Screen capture from the graphical procedure issue, but also the perspective of others who within PQMethod is not possible, so I have created participated in the sort. tables that approximate the outputs. 44 Using Q-Sorts in Participatory Processes Appendix 1. Items Used in the SDV Q-Sort After an especially long journey back to DC for a Social Development meeting, you awake in the morning to find you've really overslept. It is December 2002. You arrive at 1818 H St., N.W., and find your colleagues to see what they've been up to. As it turns out, they have made great progress over the three years you were slumbering.... Which of the following statements describe this new and improved Social Development Family, as it appears to you in 2002? q1 SDV is composed of people with a mix of educational and professional backgrounds with management and client-interface skills that match their technical proficiencies. q2 SDV is a leading resource within the Bank for easily applicable solutions to social issues facing country teams, such as participatory methods or local capacity building. q3 SDV manages a network of external links to NGOs both in developing and developing nations, which enables SDV to play a role in coordinating actions among multiple actors in the development arena. q4 SDV delivers stand-alone projects to country teams, complete with staff or connections to local staff. q5 SDV is a provider of region-level solutions to problems; with a staff and budget that resides primarily with the regions. q6 SDV is involved, through an allocated staff member, on nearly every relevant Bank project. q7 SDV has its own products that can be offered directly to clients in countries. q8 SDV is respected and in demand within the Bank as a source for efficient and fair compliance reviews. q9 SDV serves as the primary advocate within the Bank for the needs of countries' most vulnerable populations. q10 SDV affects change in the Bank by persuasion; its voice and recommendations are respected by Bank audiences. q11 SDV is the primary locus of policy formation for certain crosscutting social issues. q12 SDV is able to quickly anticipate and adjust its offerings based on the needs of client teams. q13 SDV is a leading resource within the Bank for knowledge on a broad range of crosscutting social issues. q14 SDV serves as a center for training and development of other Bank staff on social issues. q15 SDV challenges and pushes the Bank to maintain a focus on programs that reduce poverty. q16 SDV is a driver of quality assurance for Bank projects. q17 SDV is a source of world-class research in issues related to social development. q18 SDV attracts and retains the most skilled social scientists working in development. q19 SDV is a constant innovator; developing a series of new products and services to offer within the Bank. q20 SDV serves as a center for training and development of non-Bank staff (government, other NGOs) on social issues. q21 SDV works on the front lines, allocating staff to designing and implementing projects on the ground in developing countries. q22 SDV serves as a clearinghouse for a core set of assessment tools that are used by the majority of Bank project teams. 45 Jonathan C. Donner Appendix 2. Answer Sheet for the SDV Q-Sort Answer Sheet for the Q-Sort After an especially long journey back to DC for a Social Development meeting, you awake in the morning to find you've really overslept. It is December 2002. You arrive at 1818 H St., N.W., and find your colleagues to see what they've been up to. As it turns out, they have made great progress over the three years you were slumbering. . . . Which of the following statements describe this new and improved Social Development Family, as it appears to you in 2002? Once you have finished your sort, write the number on each of your chips in the corresponding space below. -3 -2 -1 0 1 2 3 Least Most Important Neutral Important Once you have completed the Q-sort, please double-check that there is one number in each cell and no duplicates. 46 Using Q-Sorts in Participatory Processes Appendix 3. Instruction Sheet for the SDV Q-Sort Detailed Instructions (We will review this with you in person, as well.) 1. Count the blue chips. You should have 22 statements and one "header" with numbers ranging from -3 to 3. Alert one of the Monitor team if you are missing any chips. 2. Each statement on the chip concerns one element of a vision for SDV's future. After an especially long journey back to DC for a Social Development meeting, you awake in the morning to find you've really overslept. It is December 2002. You arrive at 1818 H St., N.W., and find your colleagues to see what they've been up to. As it turns out, they have made great progress over the three years you were slumbering. . . . Which of the following statements describe this new and improved Social Development Family, as it appears to you in 2002? 3. Look for a minute at the diagram on the next page and at the "header." To complete the exer- cise, you should organize the chips in the shape indicated by the diagram: 2 statements go under the column marked "-3: least important", 3 statements under "-2", 3 under "-1", 4 under "0: neutral", and so on. There is no difference among the importance of items as long as they are placed in the same column. 4. Most people find it difficult to do this prioritization in one step. Instead, it is easier to start by separating the statements into three piles: not important (or disagree), neutral, and important. 5. Once you have three general piles, focus first on the "important" pile. Leave the others aside and simply try to organize the chips you considered important into the appropriate number of slots for "1", "2" and "3: most important." 6. Now turn to the general pile you called "not important" and perform the same sort. 7. Next sort out any statements you left in the "neutral" pile in the beginning. 8. You should now have a complete sort that matches the number of statements listed on the header. Feel free to look at the whole picture and make any changes you want. 9. When you are comfortable with the sort, write the NUMBER of each statement in the corre- sponding slot on the answer sheet on the next page. 10. If you have any questions, ask one of the Monitor team for help. 47 Jonathan C. Donner Appendix 4. Data Matrix from the 17 SDV Q-Sorts Respondents Sort value 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 -3 4 17 6 19 20 20 3 6 4 6 4 3 20 17 7 1 7 -3 17 12 10 20 8 14 17 17 22 17 17 17 4 4 5 7 5 -2 16 14 17 18 14 1 20 3 7 16 9 20 17 6 11 17 18 -2 3 9 9 14 3 6 7 4 6 19 8 8 21 7 4 12 3 -2 8 8 21 8 16 15 9 7 14 20 6 22 16 18 22 4 4 -1 6 19 4 12 6 4 16 15 3 4 7 4 3 20 1 19 17 -1 5 16 7 22 1 3 8 19 20 12 10 6 1 19 3 18 14 -1 1 10 15 10 13 12 15 21 9 3 15 12 7 22 13 16 9 -1 19 5 12 4 22 8 6 22 12 7 16 5 8 3 16 14 12 0 21 20 18 1 12 18 21 5 16 18 5 18 22 5 2 21 8 0 18 18 14 2 18 10 14 8 17 13 3 10 15 14 21 3 20 0 12 7 1 9 15 5 5 16 13 10 21 13 14 15 20 8 19 0 7 1 8 17 9 7 18 18 21 1 20 14 9 16 14 20 2 1 14 4 3 16 21 16 4 1 15 14 22 1 12 9 19 22 10 1 20 22 16 6 7 21 19 9 2 21 13 21 13 10 18 11 22 1 9 6 20 15 19 19 10 10 18 11 18 11 18 12 17 5 1 1 10 15 5 3 4 17 13 12 8 5 19 7 6 21 8 13 6 2 22 13 2 21 2 13 1 11 1 15 1 19 5 11 10 15 21 2 11 2 22 5 17 2 11 14 19 8 12 15 2 13 6 10 16 2 2 11 19 13 5 22 12 20 10 22 14 16 11 8 12 6 13 3 13 3 11 7 11 9 2 2 11 2 11 2 19 2 9 9 15 3 15 21 13 11 10 11 22 13 5 9 2 9 10 1 15 2 11 48 Using Q-Sorts in Participatory Processes Appendix 5. Links and References Links The Q-Method Page http://www.rz.unibw-muenchen.de/~p41bsmk/ qmethod WebQ http://www.rz.unibw-muenchen.de/~p41bsmk/ qmethod/webq PQMethod Manual http://www.rz.unibw-muenchen.de/~p41bsmk/ qmethod/pqmanual.htm Steven R. Brown's Q Bibliography http://www.rz.unibw-muenchen.de/~p41bsmk/ qmethod/syllabus00.pdf References Argyris, C., R. Putnam, and D. McLain Smith. 1985. Action Science: Concepts, Methods and Skills for Research and Intervention. San Francisco, Ca.: Jossey-Bass. Brown, S. R. 1980. Political Subjectivity: Applications of Q Methodology in Political Science. New Haven: Yale University Press. _____. 1993. "A Primer on Q Methodology." Operant Subjectivity 16:91-138. Donner, J. C. 1998. "Making Mental Models Explicit: Quantitative Techniques for Encouraging Change." In Monitor Company Seeds for Change Series. Kim, J., and C. W. Mueller. 1978. Factor Analysis: Statistical Methods and Practical Issues. Beverly Hills, Ca.: Sage. McKeown, B., and D. Thomas. 1988. Q Methodology. Beverly Hills, Ca.: Sage. Stephenson, W. 1953. The Study of Behavior: Q-technique and Its Methodology. Chicago: University of Chicago Press. 49 4. Ethnographic Methods: Concepts and Field Techniques Stuart Kirsch Poverty is not a certain small amount of goods, nor with two additional techniques. The first are is it just a relation between means and ends; above the participatory research methods that are all it is a relation between people. normally used in rapid appraisal exercises. -- Anthropologist Marshall Sahlins (1972:37) These methods, which emphasize local partici- pation and making research results accessible Poverty is much more than a lack of income. Pov- to community members, can be used to assist erty also means [the poor] having no the poor by providing them with valuable "voice" in influencing key decisions that knowledge and skills. The second recommen- affect their lives, or representation in state dation is derived from the model of commu- and national political institutions. nity-based research, in which the skills and -- World Bank (2000) resources of the researcher are put in service of community goals. The resulting collaboration can increase local understanding of, and This chapter provides guidance on the applica- control over, the processes of development. tion of qualitative methods developed by These methods can be used to stimulate the anthropologists for ethnographic research to following innovations in development policy World Bank project design, implementation, and practice: and evaluation.1 It offers task managers practi- cal guidelines for using ethnographic methods · Developing agendas that are responsive to of participant-observation, qualitative and community needs semistructured interviewing techniques, and · Integrating complementary knowledge bases case study analyses. It also encourages task · Providing information and research tools to managers to use ethnographic methods to local communities and encouraging them to supplement quantitative data collected by the develop their own analyses. World Bank.2 Ethnographic methods may be used at the While listening to the poor is an essential level of local institutions as well as in processes strategy for analyzing the causes and conse- that transcend local contexts. These methods quences of poverty and devising appropriate can also be used to "study up," to provide interventions, it poses new methodological information about development projects and challenges for Bank task managers. Ethno- the workings of capital. graphic methods, with their focus on local This chapter recommends that task manag- knowledge and understanding, provide impor- ers supplement ethnographic research methods tant analytical tools to help address the prob- 50 Ethnographic Methods: Concepts and Field Techniques lems of poverty. Applied in conjunction with · Gender lines, for example, increasing participatory research methods and the model women's workloads or restricting their of community-based research, these methods autonomy. can provide the poor with new skills and · Generational lines, for example, providing information that will help them to analyze, the younger generation with the means to understand, and alter their predicaments. challenge the authority of their seniors. This chapter is organized in the following · Class lines, for example, landowners may sections: oppose land reforms favored by peasants, or agro-industry may object to water diversifi- 1. Use of ethnographic research methods cation projects. 2. Key principles in conducting ethnographic · Linguistic lines, for example, creating barri- research ers in the flow of critical information. 3. Limitations of ethnographic methods · Religious lines, for example, non-Catholics 4. Identification of the right data for one's may resent support for Catholic community needs organizations. 5. Appropriate interview strategy 6. How to conduct effective ethnographic Project Planning interviews 7. How to organize an ethnographic account Ethnographic research can also assess local 8. The case study method responses to program assumptions, objectives, 9. How to combine ethnographic methods and techniques. If incorporated in the early with participatory research analysis stages of planning, ethnographic research can 10.How to combine ethnographic methods have beneficial impact on decisionmaking and with community-based research results. Successful applications of participant- observation and other ethnographic methods in 1. Use of Ethnographic Research Methods development contexts include primary health care planning in Nepal (Justice 1986), the Ethnographic research methods may be pro- problems of involuntary displacement (Cernea ductively employed by World Bank task 1993), and the evaluation of a World Bank- managers in the following contexts: (a) prelimi- sponsored public housing project in Ecuador nary research to assess differences within and (Salmen 1987). among communities, especially the unequal Ethnographic investigation may be particu- distribution of power, (b) project planning, and larly useful in identifying problems inherent in (c) project evaluation. planning assumptions, as Gardner and Lewis (1996:147-51) argued in their analysis of a Assessing Difference within Communities project to construct a fish farm in rural South Asia. The initial project was strongly biased Existing rifts and fault lines in a community toward the introduction of new technology and may well become impediments to successful failed to adequately investigate the social project implementation if they are not clearly context. Yet, "by opening up avenues for identified in advance. Ethnographic investiga- discussion with local people, and identifying tion can reveal local sources of power and some of the potentially contradictory interests inequality, including gendered differences in and needs of different classes and groups, political participation, unbalanced land distri- better decisions [were] made about . . . what bution, local alliances and class interests, and the project has to offer to specific categories controls over information. Projects may also of person" (p. 150). By integrating local inadvertently transform these differences knowledge and technologies in project into new forms of inequality or exacerbate planning, the anthropologists also increased existing inequalities. These differences may the acceptance--as well as the efficacy--of occur along: the proposed project. 51 Stuart Kirsch Project Evaluation Direct Observation Should Be Combined with Interviews Ethnographic methods may also be used to evaluate ongoing development projects, as What people do and say are often different (see Gardner and Lewis (1996:140-41) observed in Bernard and others 1984). This requires a focus reference to a training program for rural on practice, on behavior and action, in addition cooperatives in South Asia. Their analysis to how people describe their own actions or focused on how the hierarchical organization of explain their motivations. The distance that power within the development project im- comes from being an external observer may peded the distribution of knowledge about reveal patterns of behavior that may not be different aspects of the project. They recom- visible to the participants themselves. Because mended building stronger relationships be- people do not always describe their own tween expatriate and local staff, as well as behavior accurately, observation is an impor- between project staff and their clients. They tant corrective to research methods that rely on also advised the project to develop stronger self-reporting in questionnaires, surveys, and links between the contextual knowledge of interviews. project clients and the complementary desk- based knowledge of project staff. Furthermore, Focus on Local Knowledge and Categories their analysis of how local interests affected participation created the opportunity for Ethnographic methods focus on local concepts enhancing participation in the project. and understandings. Development initiatives should be responsive to local knowledge and 2. Key Principles in Conducting perspectives, rather than recapitulating and Ethnographic Research imposing the views of external observers (box 1). Framing projects in local terms also There are six basic principles of ethnographic facilitates participation. research: (a) research should be conducted primarily in natural settings; (b) direct Direct Personal Engagement with Community observation should be combined with inter- Members Is Essential views; (c) research should emphasize local knowledge and categories; (d) direct personal Ethnographic research relies on direct contact engagement with community members is with the subjects of study, rather than indirect essential; (e) all of the major segments or interaction through a research instrument such factions of the group under study should be as a survey or a questionnaire. Rapport with included; and (f) researchers must ensure that informants are not harmed by the research. Box 1. Reversing Roles Research Should Be Conducted Primarily By wagging the finger, holding the stick, sitting on in Natural Settings the chair behind the table; by dominating and overwhelming thought and speech; by being Ethnographic research should be conducted rushed and impatient; by demanding information wherever people are actually engaged in the and answers; by believing that we know and they process under study. Research on agriculture, are ignorant, that they are the problem and we are for example, should take place (at least in part) the solution; by failing to sit down with respect and interest and listen and learn--in these ways we in the fields and the marketplace. Information have impeded expression of knowledge and gathered in natural settings such as these is creative analysis by rural people. richer and more contextualized than data obtained solely through formal and structured Source: R. Chambers 1992, 298­99. approaches. 52 Ethnographic Methods: Concepts and Field Techniques informants in ethnographic research differs continually undergo change. Thus, the prin- significantly from the standards of objective ciples of reliability and generalizability must be formality and distance that characterize the conceptualized differently by focusing on relationships between other social scientists corroboration and consistency within ethno- and the persons that they study. These relation- graphic accounts. ships can be facilitated by explaining the Another way to overcome these limitations objectives of the research project to community is by making use of complementary methods, members and soliciting their input. Such two- including surveys and questionnaires. These way communication is the cornerstone of methods may be applied after one has carried ethnographic research and is an appropriate out preliminary ethnographic research to starting point for any form of collaboration, establish the most important issues and ques- including development initiatives. tions. Alternatively, ethnographic methods can be used to collect and interpret data in selected Include All of the Major Segments or Factions locales following the implementation of rapid of the Community under Study assessment or participatory appraisal exercises. Ethnographic research methods can also be This may require meeting separately with used to supplement quantitative studies by different groups of people, for example, with ground-truthing or spot-checking more gener- youth, women, or political leaders. The views alized findings. They can also be used to of laypersons must be considered in addition to interpret key research results. those of local experts. This makes it possible to map or track the distribution of local knowl- 4. Identification of the Right Data edge, paying attention to divergent views as for One's Needs well as bottlenecks in the flow of information, rather than to simply the dominant (or most Ethnographic research produces several kinds common) understandings. of data. It is important to identify one's needs in advance. Ensure That One's Informants Will Not Be Harmed by the Research · Methodological conclusions indicate how to collect relevant data on a particular subject. The most important tenets of the American For example, what variables will give one an Anthropological Association's code of ethics is efficient measure of household poverty in a that the ethnographer: (1) is obligated to ensure particular community? Data collection can that the research will not cause any harm to the then be replicated in other project areas to subjects; (2) must respect local rights to pri- identify regional variation or to monitor vacy; and (3) is required to make public the change over time. results of his or her research (American Anthro- · Relational data consider how different factors pological Association 1998; see also Association or variables are interconnected. For example, of Social Anthropologists of the Common- how does the price of fuel affect the deci- wealth 1999). sions that people make regarding the mar- keting of produce? 3. Limitations of Ethnographic Methods · Sociological data examine who participates in particular social processes and why. Such The relatively small sample size of ethno- information may be critical in identifying the graphic research projects means that these factors that inhibit participation in the studies do not provide the same guarantees of formal economic sector as well as relative generalizability as large, controlled studies. rates of success. Furthermore, ethnographic research is not · Interpretive information considers the mean- generally replicable, given differences between ings given to specific actions or behaviors by researchers and the fact that research settings the actors themselves, providing an impor- 53 Stuart Kirsch tant middle ground between raw data and In contrast, unstructured interviews allow analytical models. the informants to shape the discussion, so that · Explanatory data provide information on they direct the researcher toward the most how a particular process works. For instance, important issues from their perspectives. how are prices determined by competing Successful ethnographic research design should individuals in a market? It is important to retain the capacity to surprise the researcher note that on their own, ethnographic meth- with unexpected results. The contribution of ods cannot determine causality, that is, that the researcher is in identifying the relevance of one behavior is the cause or consequence of this information. Comparability and focus are another. They can, however, be used to compromised with this interview strategy, formulate hypotheses that may be scientifi- although this method has the potential to cally tested. produce the most novel results. This type of interview strategy is used in Appreciative 5. Appropriate Interview Strategy Inquiry and Scenario Analysis (see chapter 5). Qualitative interviewing is one of the primary Single vs. Group Interviews techniques employed in ethnographic research. Ethnographic studies depend on smaller Interviews with individuals provide the oppor- sample sizes than questionnaires or surveys, tunity to learn about their personal circum- which may interview larger numbers of people stances and perspectives in detail and to in less depth. In comparison with these other discuss issues that would be difficult (or methods, however, participants in qualitative inappropriate) to address in group situations. interviews may provide a substantial quantity In contrast, group interviews reduce the of information. There are important choices pressure on the informant to respond to every that must be made by the analyst, including question. Participants can build on the remarks structured versus unstructured interviewing, of others and may encourage one another to individual versus group interviewing, and contribute more. A wide range of views can be multiple, in-depth interviews versus a larger recorded in a relatively brief period. sample size. The disadvantage of group interviews is the loss of privacy for informants. Another is that Structured vs. Unstructured Interviews subjects will respond in part to their peers rather than to the researcher, reducing the In structured interviews, one plans the ques- researcher's control over the interview process, tions in advance. Structured interviews gener- particularly when working with an interpreter. ate standardized kinds of information; for Groups may be assembled randomly or se- example, they are particularly useful for lected systematically to allow for a more collecting census information, genealogies, and focused discussion of a particular issue. data on exchange practices. They are most often used when collecting information from Multiple Interviews vs. Larger Sample Size either the entire population or a representative sample. Research to identify the appropriate The interviewer must also choose between questions should always be undertaken first, conducting in-depth interviews with a limited followed by constructing and field-testing a number of subjects and opting for a wider preliminary questionnaire, which should be range of participants. Multiple interviews with revised until the appropriate data are elicited. the same person can enhance rapport and These interviews are perhaps most appropriate encourage the subject to respond more natu- for topics that are already well defined. These rally to questions. The process may also teach interviews can be structured using the Focus key informants about the information that the Group methodology (see chapter 2). researcher seeks, enabling the informant to 54 Ethnographic Methods: Concepts and Field Techniques become a better source of information. How- (b) interviewer strategies, (c) recognizing ever, these benefits must be balanced by the cultural differences, (d) choosing the right need to obtain a sufficiently large and diverse informants, (e) recording the information and sample. (f) reporting the results. Optimizing Interview Strategies Setting For development contexts, semi-structured The interview itself should be relaxed and interviewing that combines both structured comfortable, not confrontational. A neutral and unstructured elements is generally recom- setting is desirable. An institutional setting mended (Rietbergen-McCracken and Narayan may make the subject uncomfortable, whereas 1997a). Sanal Ece describes semi-structured if the setting is too relaxed, it may hinder interviews as a "low-cost" and "rapid" way to productivity. collect information "from individuals and small groups," recommending that they be Interviewer Strategies kept "conversational enough to allow partici- pants to introduce and discuss issues that they It is helpful to present oneself as informed, but deem to be relevant." Group interviews are an independent. The interviewer should avoid efficient means of documenting a wide range talking too much, as an interview is not a of views in a short time. This technique can be conversation, although some turn-taking, that supplemented with one or more interviews is, sharing of examples from other research with several carefully chosen individuals. contexts, may be warranted on occasion. This These secondary interviews are particularly can also be helpful in terms of illustrating the useful in drawing out the full implications kind of information that one seeks. of the material presented in the group interviews. Recognizing Cultural Differences The best approach to interviewing is to combine strategies. One suggestion is to begin The researcher should keep in mind that there research on a new project by carrying out may be culturally appropriate or inappropriate unstructured interviews that generate a list of strategies for asking appropriate questions, for questions and topics to be pursued further. example, that Euro-American executives like to These initial interviews can be very time- be asked about what they have accomplished, consuming, but generally reveal unanticipated or that people are often reluctant to divulge but critical variables. Subsequent interviews "private" information about their earnings. should be started with a several short questions Seek clarification of what one does not under- that elicit background information for com- stand, but do not restrict the responses of one's parative purposes. Next, several questions informants only to terms with which one is (derived from the preliminary research) could already familiar. (A common mistake in inter- be used to facilitate the transition to an open- viewing strategies is to ask informants about ended interview. Unlike formal survey meth- their understanding of Euro-American catego- ods, it is appropriate to use information from ries, for example, leasing of land, before inves- previous interviews to guide the discussion tigating local property models). and even to solicit commentary on this infor- mation. Choosing the Right Informants 6. How to Conduct Effective A good informant has a formal role in the Ethnographic Interviews community and/or regularly participates in activities that expose him or her to the informa- The major factors in conducting effective tion being sought, for example, the person who ethnographic interviews are: (a) the setting, takes goods to the market in a project on 55 Stuart Kirsch marketing, or the user of a particular service. · It can document the steps in a process, for (Violation of this principle is a common cause example, the operation of a collective irriga- of gender bias in research). In addition to being tion system. knowledgeable on the subject, the informant · It can produce a holistic account of a process must be willing to share this information and or an event, for example, how a particular be able to communicate effectively. The re- commodity is produced, marketed, and sponses of a good informant should also be consumed. internally consistent. Rather than select infor- · It can examine how events are interpreted, for mants on the basis of their impartiality, the example, how rural villagers view pollution interviewer must learn to evaluate the interests caused by a mining company, in contrast to and stakes of the persons participating in their scientific assessments of the damage down- research. stream. · Qualitative interviewing can bridge inter- Recording the Information subjectivities, conveying a situation from the insider's perspective, including the use of Semi-structured interviews generate more first-person narrative description and information than it may be possible to record analysis. by hand. Write notes as the conversation · Finally, qualitative interviewing can identify develops, trying not to interrupt the flow of the variables and frame hypotheses for subsequent discussion, although it may be necessary to quantitative research (Weiss 1994:9-11). slow the pace. An interview should be directed back on track when the subject speaks in 7. How to Organize an general terms about things about which he or Ethnographic Account she lacks first-hand knowledge. Many anthro- pologists tape their interviews, although Ethnographic description relies on discerning working with tapes and transcripts is very the "experience-near" concepts that people labor-intensive. Leave time after each interview naturally and effortlessly use to define what to write up the results while they are still fresh, they think, feel, or imagine, in contrast to the including questions that one did not think to "experience-distant" categories that specialist ask. observers use to advance their scientific or practical objectives (Geertz 1983:57-58). Dis- Reporting the Results of Qualitative Interviews crimination, poverty, and hunger are condi- tions that people experience directly, whereas The interview material may be difficult to social stratification, structural inequality, and categorize, with the result that the analysis economic disincentives to production are relies more on interpretation, summary, and examples of concepts formulated by analysts. integration than quantification and correlation. For example, consider the presumably straight- Findings are usually supported by quotations forward task of collecting economic data from and case descriptions rather than statistical families in a rural community. It is critical that summaries and tables (Weiss 1994:3). While research take local definitions, such as for organizing interview data, keep the following "household," into account. In a village census organizational frameworks in mind: in Papua New Guinea, for example, one may obtain very different answers to the simple · Qualitative interviewing can produce detailed questions of "Whose lives in this house?" and descriptions of events and processes. "Who sleeps in this house?" The answer to the · It permits the integration of multiple perspec- first question includes all of those persons who tives that no single individual can possess in have the right to live in a particular house, even totality, for example, how the welfare system if they were born and reside elsewhere, whereas works from the perspective of legislators, the second question will elicit a list of current case workers, and recipients. occupants. 56 Ethnographic Methods: Concepts and Field Techniques Here are some guidelines for organizing an the accessibility of microcredit schemes tar- ethnographic account of a specific behavior or geted to individual women? event: Ethnographers also employ a variety of heuristic devices that assist them in organizing · Analysis of the specific behavior or event complex information in a manner parallel to starts with the identification of the behavior in the problem at hand. This list is suggestive question as well as related questions about rather than exhaustive or exclusive: the form(s) that it might take. · Then one determines the frequency of the · Use specialized local vocabulary that sug- behavior, how often it occurs, as well as its gests different concepts or categories. magnitude, strength, or importance. · Structure the material in terms of the steps of · One should also investigate the place of this the analysis. practice within a larger sequence of events, · Organize the argument by following a including its origins (looking backward in sequence of events. time) and consequences (projecting forward · Use social hierarchy as a frame of reference, in time), or in both directions through for example, from lower to upper class, or longitudinal study (from beginning to end). from workers to management. · Also important is the complementarity · Organize the analysis with reference to of related events, whether symmetrical spatial organization. (a change in one prompts a comparable · Track the process under study, for example, change in the other) or asymmetrical the steps involved in agriculture, from (the change in one prompts an equivalent preparing the soil to harvest. change in the other, but in the opposite · Trace the social life of a particular object direction). from raw material to manufacture to sale to · Finally, there are the contributing or associated use to discard. factors, the conditions under which the · Focus on life histories (of a person or a behavior is more or less likely to occur. family) or social history (of an institution or a period of time). Consider a hypothetical study of women · Emphasize the boundaries and borders that marketing produce from their gardens. One define groups and difference, or the criteria must investigate how often this practice occurs, for entering and leaving a group. and its economic significance to the partici- pants (measured in terms of resources gained 8. The Case Study Method as well as their effect on female autonomy) and their households. It is also necessary to con- The case study method "involves the investiga- sider which events encourage direct marketing tion of a relatively small number of naturally by women, including questions of origins (for occurring (rather than researcher-created) cases" instance, planting with surplus production in (Hammersley 1992:185). It follows the parties mind) and consequences (whether women involved, including their interactions and reac- control the money they earn), as well as longi- tions, as well as the consequences of the events tudinal study for the long-term consequences for the people and their activities. Kane (1995:176) of this behavior. What is the relationship presents a detailed and very useful example of between these behaviors and other related how the case study method operates: events? For example, how does the marketing of agricultural produce affect women's Suppose you have found that while workloads, including child-care responsibili- most girls from poor rural backgrounds ties? Finally, what factors assist women in grow up to live in poverty and to do the marketing produce directly, for example, same work that their parents performed, improving safety at the marketplace, the effects others become professionals, large-scale of government subsidies or price supports, or entrepreneurs, political figures, and so on. 57 Stuart Kirsch What made the difference in these special permit is now required to make women's lives? Case studies use almost all jaggery (unrefined sugar lump) by tradi- the research techniques in the social tional methods rather than to give cane to scientist's tool kit. Let us say that you did the sugar factories. Peasants trying to a survey of these women and found build a small dam by selling some of the certain patterns emerging: they had a sand from a dried-up river running dynamic headmistress at school, a sup- through their village had to engage in a portive parent, were good at a particular four-year struggle with the state govern- subject, took a particular approach toward ment to get the rights to prevent the sand life, and so on. One factor is unlikely to be being auctioned off to contractors the cause. This is what a case study is (Omvedt 1993: 119). good for: showing how factors and cir- cumstances come together over time. You Presumably the economic policies that can select your case studies from among Omvedt describes were well intentioned, women who seem to illustrate the pattern, although their impact on the poor may not and build a picture of them. Interview the have been adequately understood when they women, their families, their teachers, and were implemented. Using case study methods, anyone else who seems relevant. Look at anthropologists can document the processes documents such as their old school that create and perpetuate the cycle of poverty. records. Not only can they give you information, they can tell whether the 9. How to Combine Ethnographic Methods timing was important. Was a particular with Participatory Research Analysis headmistress present at a particular stage in their schooling? Observe how the When ethnographic research methods are women behave, their attitudes, how they combined with participatory techniques go about things now. Do these give you (Chambers 1992, 1998; Rietbergen-McCracken any insights into the qualities that have and Narayan 1997a), important information helped them? Using material from all can be shared with community members (box these techniques and sources, you try to 2). Mascarenhas (1992) and Chambers (1992) show how these factors worked in real described a number of strategies to involve life. community members in the research process. Local research assistants and other community The case study method focuses simulta- members can map primary resource use (land, neously on structure and agency, or practice water, and forests) and other economic pat- (van Velsen 1979). In other words, it takes into terns. These variables can be given historical consideration both the structural--political, depth by constructing a time line of events that organizational, legal, economic--factors in a can be applied to past resource use. This society and individual choices, which combine information can be gathered in part by inter- to produce particular outcomes. The concrete- viewing older people about changes to the ness of the case study method makes it particu- landscape at different points in time. Seasonal- larly relevant for research in applied settings. ity can be documented by diagramming Returning to the question of poverty, con- patterns of rainfall, employment, income and sider the following observation made by an expenditure, credit and debt, food and nutri- Indian activist: tion, and disease. Value ranking techniques may be constructively applied to the economic In Maharashtra, where I live, farmers importance of plants and animals or different are not permitted to follow traditional economic activities. Wealth and inequality may methods of separating cotton lint from be documented by establishing the economic seeds under the Cotton Monopoly Pur- position of community members. By participat- chasing Scheme; in sugarcane areas a ing in the data-gathering, community members 58 Ethnographic Methods: Concepts and Field Techniques · Harris and others also raised questions of Box 2. Participatory Research Methods (PRA) propriety. The rubric of participation has been criticized by anthropologists for masking · Interviews/discussions. Individuals, house- differences within communities and for holds, focus groups and community meetings legitimizing project objectives rather than · Mapping. Community maps, personal maps, empowering project participants (Gardner institutional maps and Lewis 1996:110-16). · Ranking. Problem ranking, preference ranking, wealth ranking · Trend analysis. Historical diagramming, These shortcomings suggest that the practical seasonal calendars, daily activity charts techniques associated with PRA­rather than their rapid application­should be integrated Source: Rietbergen-McCracken and Narayan, 1997: 4. within a broader plan for ethnographic re- search and in project preparation and imple- mentation. have the opportunity to learn about these issues, a process that can be enhanced by 10. How to Combine Ethnographic Methods visually sharing the research results in the form and Community-based Research of maps, diagrams, and graphic representations of quantified data. World Bank task managers may also put While Kane (1995) argued that "rapid ethnographic methods in service of local assessment and participating learning ap- communities themselves. Here I follow the proaches" produce "reliable, timely and cultur- innovative agenda for community-based ally sensitive information," these methods have research proposed by Sclove and others (1998), been critiqued on several grounds. Harris and which may be applied to development con- others (cited in Ervin 2000:195-97) have identi- texts. They note that "community-based fied several potential shortcomings: research processes differ fundamentally from mainstream research in being coupled rela- · They questioned the accuracy of the resulting tively tightly with community groups that are data. By focusing on objective, tangible eager to know the research results and to use criteria--things that can be mapped, mea- them in practical efforts to achieve constructive sured, and diagrammed--they may neglect social change." This takes the popular notion of local meanings, interpretations and motiva- "participation" to its logical conclusion, by tions. By choosing the village as their unit of putting research--and development initia- analysis, they may downplay or overlook the tives--in service of the poor. Community-based important interconnections to regional, research can be used to stimulate the following national, and international political and innovations in development practice: economic systems. · They also raised questions about utility and Agendas That Respond to Community Needs feasibility. By operating rapidly, they rarely have the time to follow through on ambigu- Ethnographic research can document the needs ous or anomalous findings that might signal of community members, paying attention to key issues. They do not provide the opportu- differences as well as shared interests. It can nity to interact with local informants on a identify the factors that prevent community sustained basis that is the cornerstone of members from achieving their goals. These ethnographic methods. Data collection may be: exercises by outsiders may not always be well received, particularly when the re- · Institutional. For example, the inability to searchers' requests for assistance precede resolve conflicts, requiring training in conflict any sort of a relationship and little or noth- resolution and/or the establishment of ing concrete is offered in return. structures that can mediate future disputes 59 Stuart Kirsch · Informational. For example, lacking crucial Notes knowledge of markets or how to succeed in the context of free trade 1. This chapter benefited substantially from the · Infrastructural. For example, lacking key recommendations of Daniel Friedheim, Ashraf resources, such as start-up capital, or the Ghani, Lynn Morgan, Elisha Renne, and especially opportunity to learn skills in demand. James Trostle. Thanks also to Clare Lockhart and Alicia Hetzner. The views represented here are those of the author and not the World Bank or the Univer- Ethnographic research can be applied in sity of Michigan. social audits and participatory monitoring and 2. For additional information on ethnographic evaluation that help to make sure that develop- methods see Agar (1996), Bernard (1994), Pelto and ment projects are responsive to the needs of Pelto (1988), and Schensul (1999). For references on their stakeholders. anthropology and sociology in development, consult Cernea and others (1994). On rapid assessment Integration of Complementary Knowledge bases procedures, see Chambers (1998). Community members may have greater experi- General References ential knowledge that will be crucial to a project's success or failure, while World Bank Agar, M. 1996. The Professional Stranger: An Informal task managers may have greater technical and Introduction to Ethnography. 2d ed. New York: Academic Press. theoretical knowledge. Integration of the two American Anthropological Association. 1998. Code of types of information will provide important Ethics of the American Anthropological Association. resources for both groups. Community mem- Available at: . through alternatives, while task managers will Association of Social Anthropologists of the Com- be able to ground-truth their proposals. Setting monwealth. 1999. Ethical Guidelines for Good up a partnership or a collaborative relationship Research Practice. Available at: Bernard, H. R. 1994. Research Methods in Anthropol- cerns about dependency, while maximizing the ogy: Qualitative and Quantitative Approaches. 2d ed. contributions of task managers. Thousand Oaks, Calif.: Sage Publications. Bernard, H. R., P. D. Killworth, D. Kronenfeld, D. Providing Information and Research Tools for Local and L. Sailer. 1984. "The Problem of Informant Communities to Develop Their Own Analyses Accuracy: The Validity of Retrospective Data." Annual Review of Anthropology 13:495-517. Involving the poor in ethnographic research Cernea, M. M. 1993. Anthropological Approaches to Resettlement: Policy, Practice and Theory. Boulder: can provide them with new ways to under- Westview Press. stand local as well as translocal processes. Cernea, M. M., with A. Adams. 1994. Sociology, Research results should be made accessible to Anthropology, and Development: An Annotated local communities, rather than monopolized by Bibliography of World Bank Publications, 1975-1993. the institutions or agencies responsible for their Washington, D. C.: World Bank. production, so that community groups may use Chambers, R. 1998. Whose Voice? Participatory them in practical efforts to achieve social Research and Policy Change. London: Intermediate change. Experimentation with format (for Technology. _____. 1992. "Rapid but Relaxed and Participatory example, emphasizing graphic or pictorial Rural Appraisal." In N. S. Scrimshaw and G. R. representations, along with oral presentations) Gleason, eds. Rapid Assessment Procedures: Qualita- may be necessary to facilitate access. Ethno- tive Methodologies for Planning and Evaluation of graphic research organized in response to Health Related Programmes. Boston: International community concerns may also contribute to Nutrition Foundation for Developing Countries more appropriate project design. (INFDC), 295-305. 60 Ethnographic Methods: Concepts and Field Techniques Ece, S. "Semistructured Interviews." Available at: Rietbergen-McCracken, J., and D. Narayan. 1997a. "Participatory Rural Appraisal." In Participatory Ervin, A. M. 2000. Applied Anthropology: Tools and Tools and Techniques: A Resource Kit for Participation Perspectives for Contemporary Practice. Boston: and Social Assessment. Module III. Washington, Allyn and Bacon. D. C.: World Bank. Gardner, K., and D. Lewis. 1996. Anthropology, _____. 1997b. "SARAR" [Self-esteem, associative Development and the Post-Modern Challenge. strength, resourcefulness, action planning and London: Pluto Press. responsibility]. In Participatory Tools and Tech- Geertz, C. 1983. "`From the Native's Point of View': niques: A Resource Kit for Participation and Social On the Nature of Anthropological Understand- Assessment. Module IV. Washington, D. C.: World ing." Local Knowledge: Further Essays in Interpretive Bank. Anthropology. New York: Basic Books, 55-70. Sahlins, M. 1972. Stone Age Economics. New York: Hammersley, M. 1992. What's Wrong with Ethnogra- Aldine Publishing Company. phy? Methodological Explorations. New York: Salmen, L. F. 1987. Listen to the People: Participant- Routledge. Observer Evaluation of Development Projects. New Jackson, B. 1987. Fieldwork. Urbana: University of York: Oxford University Press for the World Bank. Illinois Press. Schensul, J., ed. 1999. Ethnographer's Toolkit. Seven- Justice, J. 1986. Policies, Plans and People: Culture and volume series. Walnut Creek, Cal.: AltaMira Press. Health Development in Nepal. Berkeley: University Sclove, R. E., M. L. Scammell, and B. Holland. 1998. of California. Community-based Research in the United States: An Kane, E. 1995. Seeing for Yourself: Research Handbook Introductory Reconnaissance. Amherst: The Loka for Girls' Education in Africa. Economic Develop- Institute. Also available at ment Institute Learning Resources Series. Wash- van Velsen, J. 1979. "The Extend-Case Method and ington, D. C.: World Bank. Situational Analysis." In A. L. Epstein, ed. The Craft Mascarenhas, J. 1992. "Participatory Rural Appraisal of Social Anthropology. New York: Pergamon Press. and Participatory Learning Methods." In N. S. Weiss, R. 1994. Learning from Strangers: The Art and Scrimshaw and G. R. Gleason, eds. Rapid Assess- Method of Qualitative Interview Studies. New York: ment Procedures. Boston: International Nutrition Free Press. Foundation for Developing Countries (INFDC), World Bank. 2000. "`Voices of the Poor' New Study 306-21. Offers Unique Human Insight Into Living With Narayan, D., with R. Patel, K. Scafft, A Rademacher, Poverty." World Bank News Release no: 2000/ and S. Koch-Schulte. 2000. Voices of the Poor: Can 248/S on Narayan, D., with R. Patel, K. Scafft, A Anyone Hear Us? New York: Oxford University Rademacher, and S. Koch-Schulte. 2000. Voices of Press for the World Bank. the Poor: Can Anyone Hear Us? New York: Oxford Omvedt, G. 1993. Letter to the editor. The Ecologist 23 University Press for the World Bank. Available at: (3):119. < http://www.worldbank.org/poverty/voices/ Pelto, P. J., and G. H. Pelto. 1988. Anthropological reports.htm> Research: The Structure of Inquiry. 2d ed. New York: Cambridge University Press. 61 5. Scenario Analysis: A Tool for Task Managers Jonathan N. Maack Scenario analysis has been used by the private Managing risk. Scenarios that look at future sector for the last 25 years to manage risk and paths for a country help decisionmakers develop robust strategic plans in the face of an manage risk and develop concrete contingency uncertain future. Its success in helping firms plans and exit strategies. Describing how and manage large capital investments and change why possible futures might occur enables corporate strategy has made it a standard tool decisionmakers to reflect on how political, of medium- to long-term strategic planning. social, and economic changes affect the opera- Scenarios have helped public sector agencies tion and to plan accordingly. plan for population growth and regional development, state transportation investments, Building consensus for change. Scenarios are a and the distribution of landfills. In the develop- logically rigorous, transparent means to give ing world, scenarios have been used to high- stakeholder representatives a role in planning. light the opportunities, risks, and trade-offs in Because a number of possible futures are national policy debates. created, many perspectives can be included, and the discussion does not revolve around the Uses advocacy of fixed positions. All participants take part in formulating the core elements of all Scenario analysis is different from the high, scenarios, and later break into smaller groups medium, and low cases for lending considered to write each of the four or five scenarios. This as part of Country Assistance Strategy (CAS). process helps decisionmakers share their High, medium, and low scenarios look at thinking about uncertainty and risk, develop different rates of progress along a path that is mutually understood contingency plans, and based on a single set of projections. Scenario defuse blame-casting when forces outside their analysis focuses on the areas of greatest uncer- control lead to a change in strategic direction. tainty for a country or an operation, systemati- cally develops several plausible alternative Augment understanding about the future. future environments in which the operation Scenarios are designed to bring up issues that might be implemented, and determines how otherwise would not be considered by expos- they would affect its success. This structured ing the underlying forces in a sector or region. approach to thinking about the future could help They are most effective when dealing with big World Bank country teams make strategic choices issues and strategic directions, rather than about where and how to direct lending over the tactical decisions, and should not be used for medium term in several ways: short-term planning. 62 Scenario Analysis: A Tool for Task Managers Monitoring progress and scanning changes in and technological forces that will impact on a the environment. Scenario analysis can help project or strategy. Scenarios are arrived at by a establish indicators that create a framework to team composed of key decisionmakers, experts, monitor the execution of a strategy. Decision- and stakeholder representatives during two or makers then "scan" the external environment three one-day workshops held over a period of for the agreed indicators that the country or weeks or months. Because they focus on areas region is beginning to move toward a different of uncertainty and the potential for unexpected scenario (or to a different stage along a scenario future discontinuities, scenarios provide a path). The focus on key drivers of change perspective not captured through projections makes explicit the assumptions underlying based on past data. lending and speeds mobilization of resources in response to changes in the external environ- Scenario analysis involves constructing or ment. developing scenarios (steps 1-4 below), and integrating the content of scenarios into decision- Scenario analysis is a tool with a strong track making (steps 5-8 below). During the exercise record and significant potential, but it needs to itself, it is important that both of these elements be adapted and piloted before it can be more receive equal attention and that mutually widely used in the Bank. This chapter describes acceptable triggers and measures of success be a scenario process designed to be the corner- agreed as outcomes of the process.1 stone of a strategic plan. It also suggests ways to use the key concepts of scenario thinking in 1. Define a focal issue or decision. Scenarios are shorter exercises designed to help decision- best suited to looking at the future through the makers minimize risk, address key uncertain- lens of a specific issue, such as the likely ties, and more effectively share their thinking outcome of a national election leading to a with stakeholders about a lending program transfer of power or a currency devaluation. and the country's future. Without this grounding, there is a danger that they will be too general. Process 2. Identify driving forces. These are the social, Scenario analysis follows a systematic process economic, environmental, political, and techno- to create a set of four to five plausible and logical factors that are most relevant to the vividly contrasting narratives that describe focal issue. They should be prioritized by the possible evolutions of key areas of uncertainty scenario team according to their level of pre- (figure 1). These narratives, known as sce- dictability and importance in affecting the narios, examine the social, political, economic, desired outcome. Figure 1. Diagram of the Scenario Process Choose scenario group Preliminary interviews First scenario 4-5 draft scenarios Verify scenario Write full-length workshop(s) assumptions scenarios Preliminary research 4-5 final scenarios Indicators for Final scenario Review and revise scenarios Implement operation Communicate external events workshop(s) and monitor progress scenarios in each scenario Strategic implications of Legend : each scenario Core scenario team Full scenario team 63 Jonathan N. Maack 3. Write scenario plots. These are the stories formed on the basis of the scenarios. Use or that explain how driving forces interact and modify existing systems to monitor progress what effects they have on the operation or toward operational goals as well as changes in strategic direction being discussed. the external environment. 4. Flesh out scenarios. Good scenarios combine Scenarios in Practice a solid understanding of relevant present trends with a clear focus on the outcomes The two primary situations in which scenario sought by decisionmakers. They should incor- analysis has been used by the public and porate lessons revealed by analysis of quantita- private sectors are: tive and qualitative data relevant to the 1. When considering a significant capital assumptions underlying each scenario. investment decision or sectoral strategy 2. When developing or reformulating na- 5. Look at implications. After scenarios have tional or corporate strategy. been fully developed, decisionmakers should study their implications for the outcomes being Scenario Analysis for Capital Investments sought by the operation. Scenario analysis is extensively used in the 6. Choose "leading indicators." These indica- private sector by companies that make large tors should help decisionmakers monitor investments in productive capital or research changes in the external environment as well as and development. Utilities, hospitals, oil developments in the project. companies, and pharmaceutical companies all invest significant amounts of money in re- 7. Disseminate scenarios. Once scenarios have been search, development, and site construction. built and refined, they should be written in succinct, They need to make all possible efforts to ensure easy-to-read language and disseminated within that their investments will remain viable over implementing organizations and to the public. the long term. Scenario analysis has helped them examine the political and regulatory 8. Integrate scenario outcomes in daily proce- environments, the likely attitudes of consum- dures. Change the incentive system in affected ers, and other relevant factors to attempt to agencies and areas to ensure concerted move- ensure that their investments continue to be ment toward the strategic goals that have been profitable as conditions change (box 1). Box 1. Scenario Analysis for Capital Investment: Shell Oil Company Scenarios to assess the riskiness of an investment Another looked at a future in which the pressure of strategy need not be simplistic. The process of high prices and environmental organizations led to identifying the real focal issue behind an investment technological breakthroughs that reduced the is often quite revealing. For example, Shell Oil dependence of Western Europe on oil and gaso- Company has used scenario analysis since the line. 1960s. When Shell began looking into investing in All of these scenarios were plausible, and, new oil fields in the late 1970s, it asked its scenario indeed, all did come to pass to a degree. Shell team to evaluate the investment in terms of forces planned its investments to take advantage of that would affect the price of oil in Western Europe. potential technological breakthroughs and to grow The team came back with several scenarios. One through a flexible series of small expansions. When cited the risk of the collapse of the Organization of OPEC was no longer able to control enough of the Petroleum-Exporting Countries (OPEC) due to oil supply to keep prices high, Shell's conservative internal tensions and external pressure. Another position allowed it to rise from seventh in the industry looked at the demographics and political situation in to third. Its small investments remained profitable the Soviet Union and noted that a relaxing of while competitors lost significant amounts of money tensions with the West could lead to large amounts as riskier oil fields tapped during a period of high of oil from Central Asia being exported to Europe. prices failed to perform. 64 Scenario Analysis: A Tool for Task Managers In the public sector, scenarios have been used makers and key stakeholders agree on substan- to guide large capital investments in transpor- tive messages about possibilities for future tation, landfill development, and city planning change. The process helps create informal (box 2). These scenarios deal with potential networks and understanding among the changes in user behavior, the utility of capital participants that can lead them to become investments to a mobile population, and the champions of the policy recommendations that economic and social pressures likely to develop come out of scenario discussions. The second on a national scale over time. Special emphasis function of public sector scenarios is to engage is also placed on likely voter pressures, larger public opinion. The completed scenarios are political and demographic trends, and ways to simplified and broadcast in the mass media, ensure continued political support for projects explained at local meetings, and highlighted in once they are underway. Scenarios can help leaflets and policy bulletins. Shaping public create the best long-term strategies for contin- debate about issues of national concern is made ued growth and flexibility for states and easier by the flexibility of the scenario method- regions. ology. Scenarios are stories that bring together the key elements of policy in a narrative that Scenario Analysis for Long-Range can be understood by the public. Because each National Strategy scenario can illustrate plausible changes along many dimensions, discussion begins to focus Significant scenario development also has been on the interactions among forces, rather than on done in the public sector at the national policy specific issues. level. Scenarios have been used to shape agendas for change in Colombia, Guatemala, Adapting Scenario Analysis and South Africa (box 3). In these countries, to World Bank Interventions which were mired in turmoil or facing funda- mental socioeconomic crises, leaders from In the World Bank, scenarios can provide academia, business, trade unions, the military, assistance in formulating goals, designing rebel groups, political parties, and peasant projects, building ownership, and monitoring unions came together to discuss steps forward. progress. Scenarios are most valuable when When used in the public sector, strategic they are brought in at the point in the project scenarios have the goal of helping decision- cycle at which the overall direction of the Box 2. Scenario Analysis for the Public Sector: New Jersey Department of Transportation In the mid-1990s, the State of New Jersey used a experts, academicians, and activists. The scenarios participatory scenario process to develop a vision for focused on economic growth and citizen attitudes the future that would help it deal with intransigent (see box 7 for full scenario matrix). Special attention transportation issues. The core decision was was paid to regional growth, because New Jersey is a whether to focus on sustaining existing roads, major transportation corridor between the two large expanding the highway system, or building more light metropolitan areas of New York and Philadelphia. rail. To develop the scenarios, the scenario team The scenarios were widely publicized and looked at the likelihood of regional and national sparked significant public debate. All of the sce- economic growth, citizen priorities, environmental narios highlighted the importance of innovation and pressures and political will. Research, which cooperation with the public sector and the need for included a values survey of a cross-section of environmentally sound development. Through citizens and businesses, studies of existing travel innovative public-private partnerships, the depart- and commerce patterns, and a study of conventional ment of transportation leveraged federal funds for its forecasts, was carried out as part of the process. In long-term plan of mixed road-rail development. addition, consultations were held in communities Business practices were also changed to allow for across the state. more citizen interaction. The scenario team included state politicians, department of transportation decisionmakers, Source: Adapted from Bonnet, 308-24 65 Jonathan N. Maack Box 3. Scenario Analysis for National Strategy: South Africa In 1990, four distinct--and evocatively titled-- deep economic slump that leaves the country in scenarios for South Africa's post-apartheid future worse condition than it was during the transition. were developed during three 3-day workshops. The 4. Flight of the flamingos. A smooth transition process was initiated by a local university and took followed by a move toward sustainable policies to place during the heated national debates between increase growth and inclusion. This story, named the National Party and the African National Con- after the slow take-offs and close flight patterns of gress (ANC). The participants looked at the possible flamingos, shows steady economic growth, biparti- actions of various stakeholders in the political san priority setting, and a greater increase in long- process and came up with four scenarios. term benefits to the poor. 1. Ostrich. A closed transition process in which The scenarios were developed by economists, conservative parties attempt to forge a nonrepre- businesspeople, politicians from all major parties, sentative compromise government rather than academicians, and NGOs. They received heavy reach a negotiated settlement. The outcomes of press coverage, and F. W. de Klerk, leader of the this policy, named after the bird's habit of sticking National Party, was quoted as saying that he was its head in the sand in the face of danger, are "not an Ostrich." The Icarus scenario, which critiqued political and economic isolation and deepening unsustainable government spending, was endorsed ethnic tensions. by key left-wing economists and comforted the local 2. Lame duck. A half-hearted and prolonged and international business community. The transition transition of power that creates a coalition govern- government that eventually came into power was ment stripped of any real authority. The outcomes of notable for its restrained economic policies and its this scenario are a deteriorating political climate, success at using tested, sustainable methods to lack of sufficient investment in economic redevelop- reach the poor. The scenario process was an ment, and the risk of continued isolation. excellent way to broaden the involvement of stake- 3. Icarus. A populist national government takes holders in policy dialogue without creating chaos in power and adopts an unsustainable policy of high the decisionmaking process. government spending and taxation. The outcome of this policy is short-term prosperity followed by a Source: Adapted from Kahane, 325-32. intervention is being designed or significantly analysis make it well suited to this approach. rethought. If it is meant to be the key strategic Scenarios are also valuable for monitoring the framework for an operation, the scenario progress of both the client and the Bank toward process should be integrated with the budget- the goals outlined in the CAS. ing, negotiation, and implementation of a project rather than treated as a stand-alone 2. Formulation of participatory national exercise. strategies. The emphasis on participation Such an application of scenario analysis makes scenarios useful for planning pro- would add significant value in five strategic cesses that need dialogue and agreement areas: among multiple stakeholders, such as the Poverty Reduction Strategy Paper (PRSP) 1. Formulation of Country Assistance Strategies. and the Comprehensive Development Frame- The CAS is the Bank's business plan in a work (CDF). A country-driven scenario nation, the closest analog to the corporate process is an excellent way to develop owner- strategies around which scenario analysis ship and incorporate the political situation techniques were developed. Scenarios can and relevant external forces in PRSP and CDF increase the selectivity and realism of a CAS by documents. Because the scenario process can focusing discussion on likely future opportuni- be led by the country, as it was in Colombia ties and constraints. The technique has been and South Africa, the Bank can concentrate presented in clinics for Bank task managers on on its comparative advantage: leveraging the formulating CAS documents. The combination input of macroeconomic, political, and of a focus on areas of key uncertainty and a sectoral experts to add depth and believabil- strong participatory element in the scenario ity to the scenarios. 66 Scenario Analysis: A Tool for Task Managers 3. Formulation of a medium-term framework for Adapting and Shortening program loans. Scenario analysis is most effec- the Scenario Process tive in looking at medium- to long-term time horizons. It is thus well suited to helping the Scenarios can be adapted to enhance strategic team prepare a medium-term framework that planning at a number of stages in the project looks at the desired outcomes of lending and cycle. Scenarios are, above all, a way of thinking the uncertainties that it will face. Because the about the world and can be used in shortened signposts of progress toward different sce- form to change the terms of dialogue and intro- narios are formulated by a group, the process duce a new way of interacting. A brief, participa- makes the framework more transparent and tory scenario workshop can be arranged early in increases the motivation of both Bank and the design phase to build scenarios and gather client to act on strategic recommendations. input quickly. Alternately, a more extensive Scenarios also aid in the development of exit process with supporting research can be carried strategies based on mutually understood out during the development of a risk manage- changes in the agreed indicators. ment framework and used to develop a system for monitoring and evaluation. 4. Developing benchmarks and goals for an An example of a streamlined scenario Adaptable Program Loan. Scenario analysis can implementation may clarify how the technique provide an effective and inclusive method to can be adapted. If the task manager of an create a long-term set of benchmarks for adjustment loan wanted to use scenarios as one extended interventions, such as the Adaptable input during project design, but did not want Program Loan (APL). By including all stake- to execute the full process, he or she could set holder representatives in the risk analysis and up scenario workshops with relevant stake- indicator development process, scenarios holders during preparation and appraisal to establish a clear and consistent set of bench- build simple models of relevant forces. These marks that are widely understood. Due to the basic scenarios could be sketched out by the need for regular review and approval in the group in two one-day workshops during a two- face of emerging needs, the APL project cycle week preparation mission as a form of brain- favors the regular monitoring and revision that storming and negotiation. They could then be make scenarios most useful. integrated in the early stages of project plan- ning. If there were enough time and money, the 5. Analysis of risks and impacts of adjustment period between preparation and appraisal loans. Adjustment operations frequently in- could be used to flesh out the scenarios with volve institutional change. Scenarios are an statistical projections and other relevant infor- excellent way to look at the potential outcomes mation. The final workshop, in which the of changing the formal and informal rules in a completed scenarios would be further ques- sector or an industry. Changes in the way tioned by the group and applied to create people do business can have unexpected contingency strategies, could be held during results. Disciplined scenario analysis enables the appraisal mission. decisionmakers in the Bank and the client government to clarify the drivers of change in a Running a Scenario Exercise given sector and explore potential interactions among those drivers and the impacts of The following description of the stages of a changes. Because they build consensus among scenario exercise will give task managers a labor, public, and private sector leaders about background in its practical requirements. This what needs to be changed and why, scenarios description should enable task managers to can help task managers shape the public lead a simplified scenario analysis process, dialogue about adjustment operations by make a significant contribution to a scenario clarifying the objectives and reasoning behind team, or work knowledgeably with private an intervention. sector consultants. 67 Jonathan N. Maack Stages of the Scenario Process Building the Scenarios There are no fixed rules for how to carry out a The first step of the scenario workshop is to scenario analysis, but several stages occur in clarify the decision focus of the exercise. This almost every case. The actual creation of should have been outlined to individuals when scenarios takes place with a group in a work- they were invited to participate, but it is shop. The organizers need to carry out a important for the group to work together to number of activities before the first and last clarify the issue. As this discussion can become workshops. The four broad stages to the contentious, it is best to state clearly within the process follow. The specifics of each stage will first 10 minutes of the meeting the behavior be discussed later in the chapter. An illustration expected during the meeting. Propose, or ask of a sample process from start to finish can be the group to agree on, whether a facilitator is found in appendix 2. needed, and, if so, who it will be; a system to recognize speakers; a time limit on the length 1. Preparation. A number of tasks need to be of comments; and other basic procedures of the carried out before the scenario workshops. They meeting. include defining the scope of scenario use, Discussion of the decision focus forces all choosing and interviewing the scenario team, participants to concentrate on the goals sought and preparing background material for the by the development intervention rather than on workshops. These are crucial steps in the process, the specific process that will be used to reach because they help the task manager build confi- those goals. This separation of means from objec- dence in the scenario exercise and gain an early tives is key to giving the group the conceptual understanding of the issues that will be most space needed to come up with new approaches. important to participants. The specifics of this Several key issues should be addressed when process are detailed in appendix 1. deciding the decision focus: 2. Building the scenarios. This process can 1. Desired outcome of the intervention. This occur in one workshop, or two held back to discussion should begin with input from the back. Together the scenario team determines organizers of the event but then will be open to the focus of the intervention, identifies and further discussion. The group may come up prioritizes the key external factors that will with a different formulation of the desired affect its success, sets them in a matrix to outcome than was originally brought to the differentiate their effects, and writes simple table. This reformulation will guide the group's scenario plots. understanding of the forces that will affect it. 3. Investigating and writing the scenarios. This 2. Desired outcome of the scenario process. Once exercise is carried out between workshops. The the outcome of the intervention is clarified, the task manager and core team gather relevant role that scenarios will play should be ex- qualitative and quantitative information to plained. Will they be used as a way to test determine whether the assumptions made in existing strategies for riskiness, as a tool to the scenarios are accurate. Decisionmakers formulate new strategies, or as a means to from the scenario team should write more build a shared vision for governmental or finished drafts of the scenarios. organizational reform? The organizers should clarify how they envision the scenarios being 4. Using the scenarios to plan strategy. In this applied to strategy, and explain any internal workshop, the scenario team goes over the business tools that will be used to put that finished scenarios and looks at the implications strategy into practice. For example, if the goal for the strategic decision being made. Specific is the creation of a medium-term framework to external "signposts" of changes in the key guide the sequencing of a program of adjust- forces are also identified at this stage. ment loans, the group needs to understand 68 Scenario Analysis: A Tool for Task Managers how the program of loans works and where storming. The goal of this stage is to build a scenarios fit in. good conceptual model of the forces that are most relevant to the decision focus. 3. Time frame. The time frame will signifi- There are two large categories of driving cantly affect the range of issues a scenario can forces. External forces are the social, economic, address. The technological and economic shifts environmental, and political forces in society that should be considered in a 20-year time that are relevant to the topic of the scenario horizon may have no relevance for a 5-year discussion but are outside the participants' plan. Scenarios should be conceived of as ways control. Organizational forces are the product of to shape strategy within the medium term. The the actions of the Bank and other stakeholders time frame should be long enough for signifi- and can be affected by the project. These driv- cant changes to occur in the driving forces, ing forces will become the key foci of further ideally 3 to 5 years. scenario development, and they can be identi- fied intuitively by using the visual clustering Key Factors method described in appendix 1 or separated into a series of logical groups and analyzed. After the decision focus has been discussed and analyzed, the team should brainstorm key External Forces: The SEEPT (Social, factors for the success of the strategy or opera- Economic,Environmental, Political and tion. This session should be led by a person Technological) Framework with experience in facilitation. Key factors for project success can include: The simplest way to systematically address external forces is to organize the issues identi- · Specific events, for example, the passage of fied in the brainstorming sessions and prelimi- legislation that gives legal status to NGO nary interviews in five broad categories: microcredit providers · General trends, for example, a decline in the 1. Social cost of fertilizer 2. Economic Improvements in service delivery measured 3. Environmental by positive beneficiary feedback as well as 4. Political number of persons served. 5. Technological. To get as many dimensions of success as The SEEPT framework (table 1) is an efficient possible, the facilitator should encourage all way to obtain a holistic view of the many forces members of the group to participate during that will affect a project's success. Rather than this phase. After a brief brainstorming session, focusing on a single dimension, such as eco- the clusters of logical association and key nomic performance, the group can look at how factors developed by the analysts after the the forces relate to and affect one another. The preliminary interviews (see appendix 1) can be specific events identified in the initial brain- added to the list. These can become building storming of key factors can be thought of as blocks for discussion and make it easier for symptoms resulting from these deeper forces. different groups to understand one another. For example, the strain on urban infrastructure that results from migration to cities is inti- Two Driving Forces: External mately related to demographic, social, and and Organizational economic pressures. It may also be tied to political pressures or regional instability. A The free association used to identify key factors simple device like the SEEPT framework can can lead to a jumbled list. The next phase in the help decisionmakers look at all the dimensions process consists of the group's looking at the that affect an urban development project at underlying causes of the issues that have come once. As the group thinks about how these up in individual interviews and group brain- forces could plausibly change and interact in 69 Jonathan N. Maack Table 1. Illustration of the SEEPT (Social, Economic, Environmental, Political and Technological) Framework for Identifying External Forces This is an illustrative SEEPT framework for a national-level scenario exercise. This chart can serve as a menu for scenario teams or as a stand-alone example of the sort of questions that can be posed in each of the SEEPT categories. The Social Indicators of Development, Country Political and Institutional Assessment information, United Nations Development Programme (UNDP) Human Development Indicators, and the World Development Indicators are useful sources of data for these domains. Domain Category Examples Social Social factors Education levels, social priorities, cultural and class tensions, land and water rights, differentiated membership in groups and associations, gender issues Demographic patterns Age, family, household, and ethnic structures; regional and national migration patterns; wealth distribution, including regional and national poverty rates Economic Macroeconomic GNP, balance of trade, rate of inflation, exchange rate; current and future conditions relations with international financial markets, current debt levels; governmental expenditures, deficits Changes in the economic structure of nation (dependence on single export, percentage of exports in finished goods), formation of new regional trading blocs Microeconomic Change in size, type and ownership of firms; formal and informal labor force conditions structure by region; changes in economies of scale/structure of key industries Market forces Spending patterns of consumers (urban/rural, national/regional), international demand for key exports Distribution and efficiency of rural and urban markets, impact of the informal sector, sources of competition (national, regional, international) Impacts of global Volume of assistance from multilateral and bilateral agencies, conditions for economy and assistance (policies, requirements)/harmonization development Risk tolerance and conditions for entry and exit by international firms, stake in local economy by international firms Environmental Physical environment Air/water/land pollution trends and locations, environmental quality issues (global warming) Natural resources Energy prices and availability (likelihood/impact of an oil shock), raw materials (rate of depletion, ease of access), land use (farming methods, erosion levels), sustainability (strategic use of resources) regional distribution of natural resources Political Geopolitical Trends in international relations; relationship with other nations in region (regional trading blocs, military alliances); levels of tension, conflict (regional, international); trade and protectionism National Change in governmental development strategy and policy (privatization); changes in legislation (including regulation, creation of enabling environment); changes in structure and responsibility of ministries; changes in rules governing formation and functioning of parties; stability of government, likelihood of change/overthrow Technological Infrastructure Level of technology in key industries, emerging technologies, capacity to manufacture technology for export Future directions Basic research and technical education trends in nation; "digital divide" ­ computer and telecom infrastructure/trends; potential for the rapid diffusion of new technologies from abroad Source: Adapted from Wilson 1987. 70 Scenario Analysis: A Tool for Task Managers the future, the group could create differentiated Apparent direction of these forces today. What stories about how broad future trends and are the current trends in these forces and developments could affect the operation. why? One of the most critical tasks for sce- nario analysis is a good understanding of the Organizational Forces causes of present trends and conditions. Present trends are a good base from which to Organizational forces are the specific actions extrapolate future interactions among driving and general positions taken by actors (indi- forces. viduals, agencies, and organizations) that affect project success. These actors may include Relevance to the decision focus. What is the government agencies, the private sector, magnitude of the impact of these forces on the political parties, NGOs, and international future course of key factors for project success? agencies. General trends in governmental Are reforms dependent on a political coalition actions might include a move toward that draws its support from a constituency privatization of state industries or trade liberal- whose influence is shrinking? ization. As in the case of external forces, the behavior of actors is interconnected. If the Future prospects. How much, in what ways, government and trade unions work together to and how fast might these trends change in the create an enabling environment for trade, future? At this stage of analysis it is integral international businesses might begin to make that the scenario team differentiate among serious investments or solidify their presence. trends that are: The actions of agencies and organizations should make sense within the economic and · Predictable. Certain things, such as current political environment of a scenario. If organiza- demographic trends, are already known and tional forces are used as key scenario drivers, will need to be taken into account in all they should be consistent with past behavior in planning. that organization. Radical changes of position · Unpredictable/uncertain. Market prices, on critical issues should not be incorporated currency rate fluctuations, demand for unless the rest of the scenario environment export goods, and changes in political values makes such changes likely. are all unpredictable elements. Prioritizing and Analyzing External Forces The impact-uncertainty matrix is an easy way to combine key factors and driving Identifying key factors and assembling a list of forces in a clear, legible format. The matrix relevant forces are the first steps. This list can ranks the forces on a simple "high-medium- yield as many as 50 driving forces and key low" rating system. The best way to deter- factors. The next steps are analysis and mine prioritization is through a vote. A vote prioritization, which reduce this larger set of gives all participants equal voice and leads forces to those that are most relevant to the to a picture of the key forces that incorpo- decision focus, are predictable vs. unpredict- rates multiple viewpoints. The facilitator able, and will respond to actions by participat- should give each person in the group ap- ing agencies and groups. Complex analysis is proximately 20 chits to "spend" as "votes" not required. This is a logical exercise based on and ask each individual to grade each force rigorously thinking through the forces and according to: trends already noted and identifying the ones that are most important for the decision. At a · Its impact on the key decision factors (a minimum, this discussion should include: participant should "spend" more chits on the forces he or she sees as most important) and Identification of cause-and-effect relationships. the degree of uncertainty (each participant How do these forces interact? What impact do should "spend" more chits on the forces that they have on other forces? are uncertain) 71 Jonathan N. Maack · The number of chits that equal a "high, Creating a Scenario Matrix to Outline Four "medium," or "low" rating should be Distinct Scenarios decided on by the group based on the number of forces that the group has devised The results of the impact/uncertainty matrix and the number of people in the group. are seldom conclusive. A number of forces will cluster in the upper right hand corner (highly The impact/uncertainty matrix (table 2) is an unpredictable/highly relevant). The group illustration of how the ranking system can be needs to pick the two most relevant of these forces used to narrow the list of forces to the most and set up a scenario matrix using them as the relevant for differentiating scenarios. Scenario axes. The group will then work out the four plots should be built around high-impact/low- permutations of these forces and build the uncertainty issues (highly relevant issues with scenarios around them. predictable future outcomes for which current The scenario matrix (box 4) ensures that each planning must prepare) and high-impact/high- scenario will be different in a logical, nonran- uncertainty issues (issues that could shape dom way, because the top scoring factors will different futures which planning should take be key drivers in all scenarios. The scenario into account). team discusses each permutation of the two Table 2. Example of Impact/Uncertainty Matrix Table 2. Example of Impact/Uncertainty Matrix Degree of uncertainty Low Medium High Critical planning Important scenario Critical scenario issues drivers drivers Highly relevant and fairly Extremely important and Factors and forces High predictable (can often be fairly certain. Should be essential for success and based on existing used to differentiate highly unpredictable. projections). Should be scenarios. Should be Should be used to taken into account in all based on projections differentiate scenario plots scenarios. but potential discon- and trigger exit strategies. tinuities also should be investigated. Level Important planning Important planning Important scenario of issues issues drivers im Relevant and very Relevant and somewhat Relevant issues that are Med pact predictable. Should be predictable. Should be highly uncertain. figured into most present in most scenarios. Plausible, significant shifts scenarios. in these forces should be used to differentiate scenario plots. Monitorable issues Monitorable issues Issues to monitor and Related to the decision Related but not crucial to reassess impact focus but not critical. the decision focus. Should Highly unpredictable Low Should be compared to be monitored for forces that do not have an projections as scenario is unexpected changes. immediate impact on the implemented. decision focus. Should be closely monitored. Note:Note: Shaded areas indicate key focus. Shaded areas indicate key focus. Source: Adapted from Wilson 1989. Source: Adapted from Wilson 1989. 72 Scenario Analysis: A Tool for Task Managers most relevant forces. The decisions made about · "Critical Scenario Drivers" from the Impact/ how these forces are likely to behave sets the Uncertainty Matrix. Each of these factors tone for each of the four different scenarios. should play a role in at least one scenario. This process enables the quick formulation of That role should work naturally with the distinct, logical scenarios. Because the decision other forces in play in each scenario, adding on the two axes of uncertainty is group-driven, the twist of giving decisionmakers a chance it encourages group members to challenge to explore how a particularly positive or one another and seek justification that a negative development might affect the certain force is the best way to differentiate a operation/strategy as a whole. scenario. Narrowing the discussion enables · Systems of interaction among forces. The the group to explore ways in which these two numerous relevant forces and issues devel- forces could develop and how these develop- oped by the group should be explored ments can help focus the other forces around systematically in each of the different sce- a distinct plot. narios. Changes in the two key axes of uncertainty can be thought of as stones What Makes a Good Scenario Plot? tossed in a pond: their aftereffects will ripple through the host of other forces that also The scenario matrix approach is designed to bear strongly on the decision focus. cut through many complex issues and isolate two sources of great uncertainty and great An easy way to start the process is to think importance for the success and sustainability of of different static "end states" that could result the work being planned. This process of reduc- from the interaction of two forces, and work tion may leave members of the scenario team backwards to explore how forces would need uncomfortable, because not all issues can be to interact to reach that point. Some of the key effectively addressed in terms of these two key factors cited by the group earlier in the exercise axes of uncertainty. Although these forces will will be precisely this sort of "end state." It is remain the skeleton of each future narrative, important to remember, however, that sce- the next phase involves the addition of detail to narios are not descriptions of a static future the stories and the inclusion of a larger number world described in great detail. They are of driving forces. A good plot combines driving narratives about how events might unfold forces in a dynamic, coherent story. between now and a future date given the ways in which the forces that the scenario team has Key Elements identified interact. Scenarios should be: The key elements of a good scenario plot are: 1. Plausible. The events in the story should be · "Critical Planning Issues" from the Impact/ possible, and the narrative should be credible Uncertainty Matrix. These are the compara- (the descriptions of what happened, and why tive sureties in the future that are particu- and how it happened should be believable). larly relevant to the decision focus. The critical planning issues may concern specific 2. Distinctive. Each scenario should focus on a aspects of the issue that the task team is different combination of the key forces. Sce- studying and should have a role in all of the narios should be clearly differentiated in stories about the future. Issues such as structure and in message, not variations on a demographic trends should be implicit in all single theme. Multiple scenarios should be used plots, but the role they play may be quite to explore how different permutations of the different depending on how political, social, same key forces can yield very different worlds. and economic factors affect issues such as education, employment, emigration, and 3. Consistent. Each scenario should have a consumption. strong internal logic. The goal of scenarios is to 73 Jonathan N. Maack Box 4. Scenario Drivers in the New Jersey Department of Transportation The decision focus was whether transportation was more urban and environmentally conscious, investment should focus on sustaining existing wanted to revitalize cities, and trusted government infrastructure, expansion of highways, or the more. development of light rail. The group agreed on the 3. Government leadership. The government was three most important driving forces for the sce- modeled as oriented to the short term and crisis narios: management, or oriented to the long term and able to stay with a stable plan. 1. Rate of regional and national economic growth. The team looked at high, medium, and low levels of Each of the scenarios looked at the relationships GNP growth and the rise or fall of the two major among the three driving forces and their effects on urban areas that affected the state. They studied the factors such as the environment, investment, nature of growth (technological or industrial) and the education (what is the workforce doing?), and traffic effect of the economy on private investment in flows. Scenario logics were consistent. A low-growth transportation and public sector revenue. scenario had a greater split between rich and poor, 2. Values and attitudes of citizens. Citizens were an emphasis on safety, and a government that modeled as inwardly oriented and individualistic, or focused on short-term solutions due to constrained outwardly oriented and communal. The inward- resources. In another scenario, technology drove looking citizen had a continuing love of cars and strong national and regional growth; immigrants were single-family homes, a strong distrust of government, middle-class; and convenient transportation was and was rent-seeking. The outwardly oriented citizen emphasized in more densely developed urban areas. Scenario Matrix: New Jersey Department of Transportation High Growth Gateway to the world: Pushing the envelope: A high-tech future A sustainable future ····Private/public Demand for efficiency, strong leaders Movement to urban areas Modest GNP growth Strong public transportation, good leaders Education matches skills needed ···Economic growth based on sector invests in sustainable technologies highway transportation Inward-looking Citizens Outward-looking Citizens Bad News: Worst Muddling through: possible future Status quo continues ·····Large Class and race hostilities grow Investment decisions are made politically Public distrust of government ··Public transportation is seen as important Decline of New York City hurts NJ but only partially funded Growth slow, high unemployment gap between rich and poor ··Gap GNP grows slowly, low public spending between rich and poor grows Low Growth Source: Adapted from Bonnet, 308-24. 74 Scenario Analysis: A Tool for Task Managers explore the way that forces interact, and each · Winners and losers. These scenarios explore action should have a reaction. Neither actors the future in terms of ascendant versus nor external factors should completely over- declining organizations, nations, or social turn the evidence of current trends and posi- forces. Examples include scenarios that look tions unless logical explanations for those at trading blocs, focusing on dominant changes are a central part of the plot. For groups and exploring how different nations example, it is highly unlikely that there will be would fare in a climate of struggle and low inflation and high growth, or that a regula- change. tory agency that was formerly very strict will · Good news/bad news. These scenarios use significantly loosen its requirements without large external factors, such as currency some extenuating circumstances. devaluation, a massive increase in corrup- tion, or breakdown of the rule of law, to 4. Relevant. Each scenario, and the entire set encourage rethinking existing strategies and of scenarios taken as a whole, should contrib- priorities. These stories ask decisionmakers ute specific insights into the future that relate "what if?" questions about the logic of their to the decision focus chosen by the group. It is long-term plans in the face of key forces that important for the author to return frequently to can move in unexpected directions. the objectives of the exercise as she or he builds · States of change. This model is one of additive the scenario and assesses which of the external change, a world in which a series of alter- and institutional factors created in brainstorm- ations feed off one another to move society ing should be explored. into a different mode. These alterations can involve a series of innovations leading to 5. Creative. The emphasis on logic and rigor improvement, or a series of mistakes leading in the process should not stamp out innovative to political gridlock and economic stagna- thinking. Part of writing a scenario is enticing tion. Some models focus on structural the reader into the scenario world. The author change in a nation (the ending of apartheid can do this by telling his or her story through in South Africa), an industry (the introduc- parables, citing examples, or using famous tion of software to produce online reports, figures as stand-ins for complex ideas. removing the need for specialized printers), or a sector (environmental movements spur 6. Challenging. Scenarios should challenge strong pressure to build public transporta- conventional wisdom about the future. When tion and reduce pollution) and how the thinking about the major sources of uncer- change will affect different actors and the tainty, the author should try to explore alterna- viability of investments. tives that will significantly alter the basis for · Cycles. Another way of looking at change is the assumptions that go into the "official through cycles, both economic and political. future." It is often worthwhile to write a "wild In this model, the rise and fall of social card" scenario that uses some of the key forces groups or stakeholders can be charted in not listed in the scenario matrix. This gives the relation to the economic cycles of relevant set of scenarios as a whole more breadth and industries or international trends. strength. · Wild cards. Wild-card stories explore how catalytic developments could completely Scenario Plot Types reshape a society. For example, the develop- ment of the internet and its effect on busi- Scenario plots should differ from one another ness in the United States is a wild-card without being wildly positive or negative. It is scenario that is still being played out. Other important that they challenge assumptions examples might include the unexpected while remaining balanced enough that they are discovery of oil reserves or other resources. not dismissed out of hand. Here are some plot An example of this is Tuvalu, an impover- "types" that can help create stories about the ished South Pacific island nation, which future: discovered it had a unique asset. Tuvalu's 75 Jonathan N. Maack internationally designated internet country · Synthesizing quantitative work. Projections of domain is ".tv." Tuvalu has sold the rights to trends for all of the key forces should be set its domain name to a Canadian company up on a single chart. How do they interact in that will market it to television and radio existing projects? Do they move together? websites. This sale will earn the 10,000 How does this "official future" compare with Tuvalu islanders as much as $50 million over that shown in the scenarios? Compare the next 5 years. national, regional, and international trends along the driving forces. If possible, eco- Investigating and Writing the Scenarios nomic models for the predicted forces should be created and looked at for how A thorough investigation of the trends underly- they compare to past data. ing the scenario plots is essential to writing · Carrying out original qualitative work. This credible scenarios. If scenarios are not based on may involve conducting stakeholder inter- significant quantitative data and a careful views to analyze the strength and impor- review of a country's political and cultural tance of political or cultural tensions, situation, the scenario process amounts to a working with local universities to create brainstorming session. Writing the final sce- profiles of likely user groups, or other field narios should include a process of triangulation work. Other tasks might include disaggre- whereby the ideas and principles discussed in gating household surveys, and analyzing the the workshops are investigated through origi- geographic distribution of the poor and best nal research and consultations with experts. practice projects. Qualitative and quantitative research on relevant issues should be carried out before the Writing Full-length Drafts of Each Scenario scenarios are written. The former is based on interviews and follow-up questions with Once the scenario plots have been sketched out participants and key stakeholders identified in by the whole scenario team in a workshop, the scenario meetings. The latter includes they should be thoroughly researched and economic projects, financial sustainability written up as full-length drafts. These drafts research, demographic studies, and statistical should be written by key decisionmakers, analysis. ideally the task manager and a highly placed Key research tasks follow: government official in the client country. It is essential that the authors be present for the · Interview experts. Speak with academics, entire scenario discussion and have a clear sectoral and regional experts in the Bank, stake in developing and implementing effective and other specialists on the best way to scenarios. Decisionmakers will be asked to shape address technology, the political situation, the strategy for a project or a nation using these social issues, and trade and economics. scenarios, and the best way to ensure that they own, Interviewing is an excellent first step be- understand, and believe in them is to give them cause interviews can be combined with the authorial control. scenario team's requests for help in identify- A scenario should have a creative title that ing the best way to quickly gain an under- sums up its central message. It is best to begin standing of a subject or the nation. the scenario with a brief summation that · Assemble and disaggregate existing information. explains the title and sums up the elements of These steps involve reading relevant stories the story agreed in the workshop. The finished in the local press, conducting quick literature drafts of the scenarios should be smooth reviews on key subjects, collecting best narratives that are easy to read and under- practice cases, and assembling quantitative stand. Each scenario will be built around a information on the predictable and relevant series of specific events, but the underlying forces in the scenario. forces and their interactions also should be 76 Scenario Analysis: A Tool for Task Managers made clear. This phase of writing involves Using Scenarios to Make interweaving the social, economic, political, Strategic Decisions and technological scenario drivers into a smoother whole. The final task of the scenario team is to explore The scenario in box 5 was created by a the relationships among the plausible futures scenario team funded by the Colombian pri- they created and the strategic decision at hand. vate sector and led by private consultants. It The strategic options presented by each sce- reflects the input of economists, politicians, nario should be discussed in a workshop with academicians, military officials, guerilla lead- the whole team present. This process enables ers, church officials, and peasant and student decisionmakers to reap the full benefits of leaders. It is a good example of a smooth group involvement in the process. When narrative form for a complex story. It moves looking at each scenario, the scenario team from economic events to political and social should take the following actions: issues and explores the way that forces influ- ence one another over time. The entire scenario 1. Step into the narrative. This involves re- is written in the first person plural (we) to reading the scenarios and questioning them a immediately connect the reader to the events second time. The team is thereby able to re- and decisions being described. It works chro- engage intellectually and emotionally with the nologically, highlighting the impact of a failure material. The team members should envision of leadership along economic, political, and themselves in this future world and think social lines as the country moves from bad to about how the conditions that define it would worse. change their professional and personal lives. Box 5. Destino Colombia Scenario:"When the Sun Rises,We'll See" Weariness, laziness or inability to face problems are all justified by the phrase "When the sun rises, we'll see." Darkness turns into a pretext for dreams and apathy, but the clear light of dawn, rather than inspiring important decisions, simply gives rise to a new period of uncertainty. In the face of the country's crisis, this irrational confidence in unexpected, miraculous outcomes, this recourse to halfway solutions, this generalized tendency to put off basic actions until later, have combined to the point of becoming a collective alternative. Before its enormous and rapidly growing problems, Colombia seems to be saying "When the sun rises, we'll see.'" -- Quote from the Destino Colombia scenarios As crises repeatedly rock the country, exhaustion seeps into the government and the populace. Continual crises lead to a focus on solving problems day to day. The government wastes valuable time and energy negotiating without a strategy and is captured by pressure groups including the wealthy, various regional warlords, and corrupt officials. This "law of the strongest" leads to the extraction of the greatest costs from the most successful industries, the growth of the drug trade, the rise of regional governments with their own taxation systems, and the end of a national oil industry and economic independence. As the situation deteriorates, the poor suffer the most. International organizations step in but are able to provide only stopgap solutions. Ultimately, increasing human rights violations and economic disorganization lead to the country's becoming economically and politically isolated. This isolation leads to military threats to the borders by neighboring countries and increasing levels of emigration. The scenario closes with the comment that the disintegration was slow enough that people simply "became numb to the proliferation of their misfortunes." Source: Adapted from Kahane, 11-14. 77 Jonathan N. Maack 2. What are the implications of this world? Each current strategy of the Bank and the client. In scenario provides a different world for the the case of a specific loan, this encourages the intervention. This step involves a discussion of scenario team members to look at how what the implications of each scenario for the central they have learned about likely impacts on the strategic decisions that need to be made. A intervention fits into the national development standard set of questions designed to apply to strategy. these decisions should be created for this phase When scenarios are designed to create a of discussion and used for each scenario. A framework for a series of loans, it is more team might want to look at which ministries appropriate to compare the conclusions are most affected by each scenario or which reached to the consensus about development in geographic areas would experience the most the country. This consensus can be identified change. through Country Assistance and/or Poverty Reduction Strategies as well as through com- 3. What is the best strategy for this scenario? parisons with the national strategies of other Each scenario presents risks and opportunities. donor agencies. The team should focus on the broader strategic The team should look at the contingency context and answer the following questions: plans suggested by the scenario team and identify: · What is the best strategy for dealing with this situation? The scenario team should look first 1. Which alternatives seem to be suggested by a at how the intervention can most effectively majority or all of the scenarios? These should be move toward its desired outcome in this key parts of any strategic plan. They reflect an future world. What general strategic position interaction of forces that is very relevant to the should be taken? Should it be one of inter- intervention and that the scenario team feels is vention in certain areas, or restraint to allow likely to occur. the situation to run its course? · What are the major opportunities and threats in this 2. Which alternatives challenge most strongly the scenario? This may involve looking at how assumptions underlying the current strategy? The different sectors will be affected by the changes team should look to these alternatives for outlined in a scenario. It may also involve guidance in rethinking strategic orientation. looking at the effects that different scenarios Even if the scenario from which they are drawn have on specific social groups, identifying the does not come to pass, they highlight a blind winners and losers in each story. spot in current plans. When making final · If we know this scenario will come to pass, the strategic decisions, the scenario team should Bank/client should (name one action)? It should look for creative ways to include these insights stop doing (name one action)? These questions in the intervention. concretize the discussion by relating it to the specific set of actions that are being per- 3. Which alternatives are logical extensions of formed (or proposed) as part of the interven- the current strategy? These alternatives give tion. In addition, the scenario team should decisionmakers an idea of how alternate future develop exit strategies for all or part of the developments could be leveraged to push intervention based on each scenario. What forward an agenda or program that is already situations will trigger the abandonment of a in place. project or set of projects? Each alternative strategy presented by the By answering these questions, the scenario scenario team should be carefully considered team has in effect developed a series of simple during the workshop. The final step in the contingency plans for each potential future. exercise is deciding which strategic alternatives The next phase of the process is to assess how should be adopted. Again, a simple set of much these strategies have in common with the questions can serve as a guide: 78 Scenario Analysis: A Tool for Task Managers 1. What events would trigger each strategy? nomic changes. These indicators should be What impact (positive or negative) would tracked throughout the life of the project and those events have on the project? How effective compared to the scenarios developed at its is the strategy at addressing these issues? outset to determine progress toward a given future. Monitorable indicators can include 2. What is the evidence to support the assump- (1) economic figures, such as export and tions underlying the strategic suggestion? What growth levels; (2) demographic trends, such aspect of the scenario serves as the underpin- as rate of migration to geographic areas; and ning of the strategy? (3) employment trends, such as the number and type of jobs created in the formal sector. 3. Is it feasible for the organizations involved to · When choosing trends to monitor, scenario execute the alternative? What would prevent team members should look at where they them from being able to do so? can obtain data, how much it costs to pro- duce that data, and how feasible it is to Develop Monitorable Progress Indicators regularly track it. Another strategy is to look toward Each Scenario at how data that is already collected would be influenced by the driving forces of a Once each scenario has been tied to a set of scenario. By tracing the likely impact of a strategic actions, the team must develop a way scenario on an already monitored set of to look at what is happening in the country and demographic or poverty indicators, the team the world to determine which scenario is can ensure that the information will be coming to pass. The changes described in recorded in a standardized and reliable way. different plots (unified vs. disintegrated poli- Thinking about how each scenario would ties, outbreaks of ethnic violence) do not occur translate into migration, poverty rates, or overnight. The scenario team should identify other affected data helps make the world the both indicators based on fixed events and scenario team describes more real. indicators based on trends. Make It a Reality · Fixed events. Fixed events are pivotal inci- dents that push groups into conflict or create The final stage of the scenario process is the economic opportunity. They can be external dissemination of the message and its imple- or internal events and may be time-sensitive mentation on the ground. These two goals are (elections) or ongoing external processes fundamentally linked. One of the keys to (currency fluctuations due to the collapse of successful implementation is giving the public markets in Asia). The group should identify and key stakeholders (especially those who at least four "turning points" for each sce- will be carrying out the strategy) a good nario. Events such as oil prices reaching an understanding of the lessons learned from the all-time high or a 20 percent decline in scenarios and the changes in policy that have tourism can serve as signs that the world is been made as a result of them. developing along the lines of the scenario. The basic steps are to: One way to identify such turning points is to invent newspaper headlines from the future Simplify scenarios and prepare them for dissemi- that sum up crucial turning points or sym- nation. Reduce each scenario to a 1- to 2-page bolic indications of the forces at work in the narrative that can be read or explained to an scenario. As these events occur, it is impor- audience in 2 minutes. Use newspaper inserts, tant to look at how their outcomes are related radio and television broadcasts, and videos to the scenarios that have been created. shown during clinics and seminars to dissemi- · Trends. Indicators of trends can include nate the scenarios. In Colombia, a video was measurable changes in attitudes, demo- made with various team members explaining graphic shifts (urbanization), and macroeco- their scenario with the aid of cartoons. 79 Jonathan N. Maack Develop support for the scenarios. Once the The team also should rethink scenario plots in scenarios have been written, it is essential to light of new developments (ideal at Mid-Term involve larger groups of stakeholders to dis- Review) and adjust them so that they corre- cuss them. During the dissemination period, spond to the most recent information. the scenario team should build relationships with local politicians, influential figures, and Common Pitfalls in Developing Scenarios important civil society organizations. The team can organize clinics and seminars for client Despite its usefulness, scenario analysis is not a country officials, local political leaders, and panacea. Scenarios are a process, not a product. other figures and use handouts and videos to To become truly effective, they need to be used explain the scenarios and the strategy that and refined over time. This requires sustained resulted from them. It is important to monitor commitment on the part of high-level local and media responses to the scenarios and decisionmakers and a skilled and dedicated adjust for them in presentations and seminars. scenario team. It also requires the government to carefully develop and track indicators to Build accountability systems. The scenarios help decisionmakers tie what is going on in the should lead to the definition of clear goals and outside world to the scenarios. pathways to alternate scenarios. It is important Scenario development is not easy. The to involve the individuals who will generate process demands significant effort, thought, the data in designing the implementation. The and creativity of the scenario team. To help the team should develop incentives and mandates scenario team avoid mistakes made by others, for data collection and reporting. Responsibili- common pitfalls are listed below. ties should be clear, and the relationship linking the data collected, the scenarios, and 1. Failure to gain the support of key decision- strategy should be explained. For example, makers. To be credible, scenarios need to be insights from scenarios should be tied to the integrated in the decisionmaking apparatus of choice of baseline survey during the design the organization in which they will be imple- phase and to the strategy for monitoring and mented. Without real buy-in at the top, the evaluation during supervision. scenarios will not be implemented. Implement. Clarify the expected results of the 2. Unrealistic goals and expectations. Scenarios intervention and define a clear strategy. Re- do not produce action plans; they help solve residual conflicts between previous ways decisionmakers envision what will happen. of doing business and the current system. The methodology is not suited to addressing Target priority areas for strategic resource specific tactical issues. It is meant to provide a allocation. Provide guidance and training broad view of the uncertainties facing an (including a background on the scenarios) to all intervention. Strategic decisions flow from this of the line managers who will implement the understanding, but they are not a direct prod- program. uct of the exercise. Monitor progress and test results. Tie decisions 3. Failure to develop a clear map of the future with about proceeding with the intervention to the monitorable indicators. It is essential that the team indicators of progress toward certain scenarios. develop clear, monitorable milestones of progress Scenario indicators should be a part of the Mid- toward the various scenarios. These milestones Term Review process and should be examined should have a direct relationship to the goals and during supervision missions to chart progress. planned outcomes of the intervention. Re-examine environment and strategy. The 4. Scenarios that are not credible. Scenario work- scenario team should review for major prob- shops are not brainstorming sessions. Scenarios lems, adjust objectives based on observed must be based on solid quantitative as well as results, and revise time tables and priorities. qualitative projections if they are to be credible to 80 Scenario Analysis: A Tool for Task Managers those implementing them. Because scenarios do frames. Each scenario should be a smooth not assign probabilities or project against current narrative that makes intuitive sense to the trends, it is important to make sure that they are reader. The main aspects of the future should based on strong research. be internally consistent; the outcomes postu- lated for the two key uncertainties should be 5. Scenarios that are not tied to the planning able to coexist; and the actions of stakeholders process. The indicators and thinking in sce- should be compatible with their interests. narios must be directly built into the way that an intervention is planned. Scenario indicators 10. Lack of diversity of inputs. If the scenario should be closely monitored and associated team members are of homogeneous educa- with explicit changes in strategy, including exit tional backgrounds and institutional affilia- strategies. Similarly, scenarios should be tions, they will be much less likely to come up related to the client's and the Bank's budget with innovative solutions. To build successful and policy cycles. scenarios, the participation of a diverse group of people is essential. 6. Not enough time to carry out the scenario process. The process requires discipline and Conclusion attention. It can be divided in two phases: scenario building and relating scenarios to strategy. If scenario development is carried out with Although these two phases may take place at commitment and purpose, it can be a powerful separate times to conform with the Bank's tool for improving development interventions. project cycle and the client's policy cycle, the Scenarios are useful tools for task managers scenario team should have enough time to operating in environments in which long-term think through the logic of the scenarios and thinking, flexibility, and the inclusion of stake- ensure that they are properly researched. holders in decisionmaking are needed. Sce- narios bring insight to the potential futures in 7. Inappropriate time frame and scope. Scenarios which an intervention will operate, what it will that focus on current crises and existing prob- need to do to succeed in each future, and what lem areas rather than looking at the interaction needs to be done to make that success possible. of broader forces do not generate the kind of new By addressing rather than minimizing uncer- thinking necessary to jump-start an agenda. tainty, scenarios spur innovative and robust solutions. Because they are developed with a 8. Mistaking projections for scenarios. Projec- team of knowledgeable stakeholders, they are tions are based on past data and often posit a an effective way to gain buy-in for strategies. continuation, or slight variation, of current Furthermore, the research and discussion that trends. Projections are not well suited to dealing accompanies scenario development promote with the potential for significant discontinuities collaborative learning by both World Bank brought on by external events. Scenarios are staff and the client. Ideally, all members of designed to highlight "what if?" situations for the team walk away with a better under- decisionmakers. Scenarios should be built standing of the interactions among the around the forces that shape society. Trends are country's social, economic, political, and symptoms of these forces rather than their technological forces. cause. These unexpected changes cannot be predicted with a study of current trends. Note 9. Failure to tell a dynamic, internally consistent 1. The steps that follow owe a significant debt to story. Scenarios should be movies, not still P. Schwartz, The Art of the Long View. 81 Jonathan N. Maack Appendix 1. Preparation The first step in the scenario analysis process is preliminary interviews, analyze the country for the task manager to define the intervention situation, and ground-truth the assumptions in with a counterpart in the client government. each scenario. The task manager decides how scenario analy- sis will be used in the project and works out a Assembling the Scenario Team: general plan to incorporate it in the project A Preliminary Checklist cycle. She or he then assembles the scenario team in consultation with the client govern- Although it may not be possible to include ment and local and international NGOs. Once multiple representatives from each group listed the team is assembled, the task manager below, the best scenario teams are diverse. conducts a series of one-on-one, preliminary Below is a list of individuals likely be part of a interviews to determine the team members' scenario team: points of view regarding the intervention. These interviews can help the guide the 1. Decisionmakers. The task manager and a organizers of the exercise in assembling senior representative from the borrower gov- useful background material before the first ernment should jointly lead the process and workshop. write the final scenarios. 2. People with a thorough knowledge of the Scenario Team World Bank and its role in the country. A represen- tative from the Country Office or the task In the broadest sense, the scenario team is the manager can fill this role. group of leaders and experts who gather to 3. People with an understanding of the borrower create four or five distinct narratives about the government and its role in the sector/field (when future over the course of several workshops. appropriate). This includes senior staff at the Each of the members gives his or her time, appropriate ministry and officials who are energy, leadership experience, and technical interested in the scenario process and are active expertise to the group for the duration of the in the field. workshops. Team members should be chosen 4. Experts and specialists. This includes based on their ability to represent distinct Bank, engineers/technical specialists, local viewpoints on the issue being discussed, be it academicians, and other relevant experts technical or political. Ideally, all also will be from other donor agencies or international champions of the scenario process and generate NGOs. support for its ideas among their colleagues 5. Line ministry staff and/or implementing and communities. NGOs. These individuals will carry out the There is also a smaller group within the operation if it is an investment loan. In a scenario team that plays an active role before, strategy context, the net can be cast more between, and after the series of workshops broadly, but there is nonetheless a clear need during which the scenarios are created. This for practitioners to give input on the realities of smaller group, known as the core scenario team, implementation. should be led by key decisionmakers at the 6. Leaders from the private sector and from trade Bank and in the country. The finished scenarios unions. In an investment loan, these representa- need to be authored by, or in collaboration tives should come from the industry or field with, the people who will be asked to lead the most closely related to the project. In a strategy implementation of the intervention. Other context, a diverse group of dynamic business- members of this smaller core team will conduct persons can be considered. It is important that 82 Scenario Analysis: A Tool for Task Managers both local and international interests be repre- clarify what has been said. The challenge in this sented when they will be equally affected by sort of interviewing is to remain unobtrusive the intervention. and affect the interviewee's train of thought as 7. Leaders from NGOs, activist groups, religious little as possible. The following list of open- organizations, and beneficiary organizations. ended questions should help interviewers to 8. Politicians. Members of all relevant politi- move the conversation without directing it. All cal parties should be included in strategy questions should be phrased so that they apply discussions. In Colombia, politicians included primarily to the organizations/situations with guerillas and revolutionaries. which the interviewee has the greatest interac- tion/association. Preliminary Interviews Clairvoyant: If I could answer three questions about the future for you, what would they be? Preliminary interviews engender comfort with This question helps interviewers identify the the project in the participants, give the team a greatest sources of uncertainty while encour- sense of individual styles, and help identify aging the interviewee to prioritize their potentially explosive issues. It is essential that concerns. all participants in the scenario process be Good scenario: Imagine that the future were interviewed. This includes Bank staff and very good, happening exactly as you would wish. members of the client government. The key How would you answer the three questions that you outcome of these interviews will be a better asked the clairvoyant? What would have to happen understanding of the mental models (the to cause this future? This question encourages outlook and expectations) of each member of the interviewee to revisit the three uncertainties the scenario team. he/she introduced in his or her first question Most interviews will take between one and and resolve them while creating a story in two hours to conduct. Interviews can be con- which all the elements interact. ducted by Bank staff, members of the client Bad scenario: Imagine that the future developed government, or staff from an external agency or along the worst possible lines. How would you NGO that is running the process. The ideal answer the three questions you asked the clairvoy- interviewer would be a locally based indi- ant in this case? What would have to happen to vidual or team with experience in facilitation create this future? and an understanding of the policies of both Lessons from the past: What does (the country/ the client government and the Bank. Interview- the government, your agency/NGO/the World ers should be fluent in the language of those Bank, the agency that provides you services/the interviewed (interviews should not be con- sector or field) need to forget? What does it need to ducted through an interpreter unless absolutely remember? necessary) and be fully conversant with both Important decisions ahead: What are one or the details of the project and the scenario two critical strategic decisions on the immediate process. Interviewers must establish trust and horizon for your organization/group? credibility with participants from the begin- Constraints in the system: What are the ning. This can be accomplished by clearly obstacles to this process? To the "good scenario"? explaining the goals of the process and high- This question should elicit strong responses lighting the fact that the data collected will and will be closely tied to the organization the remain confidential. interviewee knows best. Epitaph: What would you like to be remembered Interview Questions for having accomplished? This very open-ended question allows for identification of the role of Interviews are meant to be conversations rather the organization and individual's goals and than formal surveys. The interviewer should commitment. participate in a reactive rather than active role, Closure: What should I have asked you that I feeding back responses and asking questions to didn't? 83 Jonathan N. Maack Visual Clustering to Analyze Interview Results group. The final stage consists of using the "clustered" statements from each related group Immediately after the interview, interviewers of interviews, which should not total more than should sit down for at least 30 minutes to go 15 statements per group of interviews, as the through their notes to identify important basis for a last clustering to form a final, overall observations to be processed further. A simple map of interview responses. guideline for identifying key points is: "Is the view expressed relevant and significant in the Visual Clustering Process context of the organization's relationship with beneficiaries and the external environment?" The core scenario team should write the state- Issues that are related to internal processes of ments from the interviews on adhesive the organization should be included only notecards and place them on a display surface insofar as they affect the way that the inter- or a wall. Statements should be written in viewee or organization will react to the outside large, heavy characters so that they can be read world. Interviews normally yield from 40 to 60 from a distance and text should be limited to a statements. Each statement should be reduced few (at most 8) words. As team members read to a single line. the statements, they should be able to note In the next stage of analysis, the interviewer patterns emerging. The idea is to organize the combines the statements from all of the inter- separate comments into higher-level concepts views to create logical cause-and-effect chains that can be related to one another. of reasoning. At first, statements will seem like random, unconnected thoughts, but as inter- 1. When clusters are finalized, team mem- viewers read through them, patterns and bers should rephrase each one into a one-word connections should emerge. Clustering, or statements that sum up the logic of the various grouping these statements, serves three pur- statements brought together to form it. For poses: (1) it forces the analysts to pay attention example, a key internal constraint to educa- to the conceptual meaning of what has been tional planning is the lack of efficient disburse- said; (2) it organizes the thoughts of a diverse ment mechanisms, which affects both teacher group into more coherent stories; (3) it is an pay and procurement. iterative process that promotes serial appraisal 2. Statements that do not fit naturally into of the outlooks of each of the people inter- existing categories should be put aside and viewed. This technique is best suited to cluster separate categories created for them later. comments from a set of 5 to 10 interviews. It is 3. As the process progresses, team mem- important to remember that with only 9 inter- bers begin to arrange the clusters hierarchi- views, the core scenario team could be dealing cally according to higher level concepts. For with as many as 450 statements. example, it may be possible to sum up 15 However, up to 25 persons can participate in statements about disbursement in a cluster, a scenario exercise. The interviewer can simply which should then be organized under a take the statements from several related inter- larger cluster focusing on, say, internal views and cluster them separately. Say, for budgetary issues. example, there are 20 interviews, and 5 interviewees are from the Bank, 6 are from At the end of the process, there should be NGOs, 4 are from the client government, and five to six larger concepts and enough clusters the other 5 have no particular affiliation. The to ensure that no cluster holds more than 15 or statements of NGO representatives could be so statements. Statements should flow from clustered separately, as could those of the other simple clusters to larger ideas. related groups. Clustering enables the inter- viewer to work with a manageable number of Assembling Background Information statements. Each related group of interviewees will then yield a smaller number of "clustered" The clustered results of the preliminary inter- statements that synthesize the thinking of the view should give the core scenario team an 84 Scenario Analysis: A Tool for Task Managers idea of the expertise of the group and the and composition, and population growth information that team members are likely to rates. need to make informed judgments when · Unpredictable, relevant forces. Unpredictable building scenarios. If possible, the core team forces such as the price of oil, international should draw together basic information on investment flows, aid levels, and the sustain- relevant issues and present it to the group as ability of debt can significantly affect an a packet. This background can help the intervention. Research into these topics is group start off with the same basic reference more involved. Industry and academic points. experts could be consulted, and historical The collection of relevant information trends affecting these forces could be regarding these forces should focus on the assembled to aid comparisons with the country itself, its neighbors, trading partners, current situation. and (when appropriate) regional trends. The · Sociopolitical information. A focused analysis of general areas for preliminary data gathering the historical and political situation in the are: country as it relates to the intervention should be assembled for the benefit of outsiders. It · Predictable, relevant forces. This information could become the focus of a brief discussion might include short-term economic forecasts, during the workshop to enable people from relevant demographic data, household type the affected areas to add detail. 85 Jonathan N. Maack Appendix 2. A Roadmap to the Scenario Analysis Process Preparation ···· Schedule Assemble group that is representative and diverse Conduct interviews to determine mental models of participants Carry out preliminary research based on key issues revealed and arrange workshops Build Scenarios Workshop I Workshop II ···· Write Determine decision focus (1 hr) Rethink scenario logics (1-2 hrs) Identify key factors and driving forces (3-4 hrs) Flesh out scenarios in plenary session (2 hrs) Prioritize forces using scenario matrix (2 hrs) basic scenario plots (1 hr) ···· Present Break out in groups to finish scenarios (4 hrs) scenarios to groups and wrap up (1 hr) Investigate and Write Scenarios ·· Write Conduct quantitative and qualitative research (3 wks) full-length drafts of each scenario (2-3 wks) Use Scenarios to Plan Strategy Workshop III ···· Highlight Correct, revise, and amend draft scenarios in plenary sessions (1 hr) Explore strategic implications of each individual scenario (3 hrs) Develop monitorable indicators for project success in each scenario (2 hrs) "trigger" events that will signal environment is moving toward one of the scenarios (2 hrs) Implement Strategy Disseminate scenarios Set up implementation strategy · Simplify scenarios and prepare them for dissemination ·· Implement Build accountability and incentive systems changes on the ground (ongoing) · Gain support from public and those who will implement scenarios (ongoing) Reiterate and perfect ·· Reexamine Monitor progress and test results (ongoing) environment and strategy (ongoing) 86 Scenario Analysis: A Tool for Task Managers References Kaplan, R. S., and D. P. Norton. 1996. The Balanced Scorecard: Translating Strategy into Action. Cam- Bonnet, T. W., and R. L. Olson. 1998. "How Scenarios bridge, Ma.: Harvard Business School Press. Enrich Public Policy Decisions." In Learning from Noonan, P. S., and M. S. Tenaglia. 1998. "How the Future: Competitive Foresight Scenarios, edited Information Technology Helps Scenarios by L. Fahey and R. M. Randall. New York: John Advance from Consensus to Decisiveness." In Wiley & Sons. 308-24. Learning from the Future: Competitive Foresight Butcher, J. 1999. "Advance Interviewing for Scenario Scenarios. 384-401. Analysis." San Francisco: Global Business Net- Ringland, G. 1998. Scenario Planning: Managing for the work. (Unpublished presentation.) Future. New York: John Wiley & Sons. Fahey, L., and R. M. Randall, eds. 1998. Learning from Schoemaker, P. J. H. 1998. "Twenty Common Pitfalls the Future: Competitive Foresight Scenarios. New in Scenario Planning." In Learning from the Future: York: John Wiley & Sons. Competitive Foresight Scenarios. _____. 1998. "What Is Scenario Learning?" In 422-32. Learning from the Future: Competitive Foresight P. Schwartz. 2000. "The Official Future, Self-Delusion Scenarios. 3-21. and the Value of Scenarios." Financial Times. Fahey, L. and R. M. Randall. 1998. "Integrating "Mastering Risk," Part Two. May 2. Strategy and Scenarios." In Learning from the _____. 1996. The Art of the Long View: Planning for the Future: Competitive Foresight Scenarios. 22-38. Future in an Uncertain World. New York: Georgantzas, N. C., and W. Acar. 1995. Scenario- Doubleday. Driven Planning: Learning to Manage Strategic Schwartz, P., and J. A. Ogilvy. 1998. "Plotting Your Uncertainty. Westport, Conn.: Quorum Books. Scenarios." In Learning from the Future: Competitive Godet, M., and F. Roubelat. 1985. "Creating the Foresight Scenarios. 57-80. Future: The Use and Misuse of Scenarios." Long- Van der Heiden, K. 1998. "Setting the Agenda." San Range Planning 29 (2):164-71. Francisco: Global Business Network. (Unpub- Kahane, A. 1999. "Destino Colombia." Deeper News 9 lished.) (1). Berkeley, Cal.: Global Business Network. Wilson, I. 1998. "Mental Maps of the Future: An _____. 1998. "The Mont Fleur Scenarios." Deeper Intuitive Logics Approach to Scenarios." In News 7 (1). Berkeley, Cal.: Global Business Learning from the Future: Competitive Foresight Network. Scenarios. 81-108. 87