OnTrack: Review of pilot implementations Annex 2. Evaluation methodology Methodology overview ............................................................................................................. 2 Introduction .......................................................................................................................... 2 OnTrack’s theory of change .................................................................................................. 3 Assessment areas and indicators .......................................................................................... 4 Data collection matrix ........................................................................................................... 6 Sampling .............................................................................................................................. 11 Analytical framework .......................................................................................................... 12 Limitations to the evaluation .............................................................................................. 14 Data collection materials and sources .................................................................................... 15 Tools and guides used for data collection........................................................................... 15 PIU interview guide......................................................................................................... 15 Staff interview guide ....................................................................................................... 16 Top level government authorities interview guide ........................................................ 18 Guide for focus group discussions with project beneficiaries that have participated in informational sessions about OnTrack ........................................................................... 19 Guide for focus group discussions with project beneficiaries that have NOT participated in informational sessions about OnTrack ....................................................................... 21 Staff online questionnaire .............................................................................................. 22 SMS survey of project beneficiaries that have sent a report using OnTrack ................. 26 Questionnaire for face-to-face survey with PAR beneficiaries in Bolivia ....................... 27 Survey of PAR beneficiaries - Methodological report ......................................................... 35 List of consulted documents ................................................................................................... 46 List of project documents facilitated by the OnTrack team and partners .......................... 46 Other documents/Bibliography .......................................................................................... 48 List of consulted stakeholders ................................................................................................. 49 Staff interviewed ................................................................................................................. 49 Staff surveyed online .......................................................................................................... 50 Project Implementing Unit representatives interviewed ................................................... 50 Interviews with Civil Society Representatives .................................................................... 51 Focus group discussions carried out ................................................................................... 51 Methodology overview Introduction The World Bank Institute (WBI) contracted Keystone to carry out a formative evaluation of the OnTrack Platform for Citizen Feedback and design a methodology for evidence-based improvement in implementation going forward. As stated in the terms of reference (ToR) for this assignment and emphasized by project leader, Natalia Agapitova, the overall approach is to learn from the past in order to ensure that evidence for real-time improvement is available for future projects that seek to establish ICT-enabled citizen feedback mechanisms. The assignment had two main objectives:  Evaluate the quality the OnTrack program and distill lessons of experience about identification, piloting and scaling out of innovations in the WBG including evidence-based decision making process; Identify the initial conditions and factors that should be in place for the successful deployment of OnTrack and other innovations.  Develop a framework for adaptive management of digital engagement initiatives using the experience and findings from the assessment of OnTrack. The work started on 5 December 2013 (formal contract from 27 December 2013) with virtual and in-person discussions with key members of the OnTrack team and ODTA to set the expectations and the parameters for the work to be carried out. A focal point was established within the OnTrack team (Alexandra Endara) who provided Keystone with internal documents relevant to the evaluation, as well as necessary explanations and clarifications regarding the platform and the way the work of OnTrack is structured and organized. Keystone’s approach to reviewing the OnTrack pilots has been to start by clarifying the program’s theory of change. On the basis of that, a series of indicators for the evaluation were developed and the accompanying data sources and data collection methods were identified. They comprised a combination of quantitative and qualitative inquiry, covering the whole spectrum of involved stakeholders, from end project beneficiaries to WB senior management. The tools used for data collection, as well as the list of consulted documents and stakeholders are included in this annex. Keystone’s Constituent Voice methodology has been used as the analytical framework for the review. OnTrack’s theory of change On 17 December 2013, an in-person 3-hour session was held at WB offices. The session was facilitated by Keystone’s David Bonbright, with participation from OnTrack and ODTA team members that have a stake in the evaluation. The purpose of the session was to gather input on the project’s theory of change, which can be used by Keystone as a basis for setting the indicators and metrics for the evaluation and evaluation framework. We have used this input to sketch out a theory of change for OnTrack, which is presented in the following section. OnTrack aims to create the conditions for effective feedback loops between users, governments and public service providers. This will lead to improved public services and, ultimately, to better development outcomes from World Bank finance and advice. OnTrack is based on the premise that the use of ICTs enables broader citizen participation and contributes to making the feedback process more efficient and effective. The key constituents of the theory of change are:  Users of public services: They monitor the state and quality of public services in their area and report complaints and suggestions using the OnTrack platform. They also provide stories on project implementation. These people are identified as the beneficiaries of the World Bank supported projects in which the OnTrack program is being tested.  Project Implementation Units/public service providers: Their role is to ensure that the OnTrack platforms are in place and functioning correctly. They promote awareness of OnTrack among service users and are in charge of the management of the platform. They ensure that the feedback provided by citizens is used to take actions to improve the quality of the public services. They also report back to citizens on actions taken in response to their feedback.  OnTrack program staff: Their role is to act as innovators, facilitators and advisors in establishing effective feedback mechanisms. They provide an advisory service for WB supported projects to set up and run platforms for citizens to provide feedback on the services they receive and for governments/public authorities to respond to it. Two other constituent groups play an important role in ensuring the preconditions for OnTrack’s success:  The government authorities responsible for the PIUs. They have the last word on key decisions regarding targeted public services and are in a position to create the enabling environment. They also provide the necessary authorization for PIUs to implement OnTrack effectively.  Project Team Leaders and WB country offices. Their buy-in is crucial in developing and implementing OnTrack in WB supported projects. They provide key insights on local and project-specific context, as well as ensuring in-country support and managing relations on the ground. In order for all these actors to contribute effectively to the OnTrack feedback loop, a series of key preconditions and intermediate outcomes are necessary. These are shown in the following graphic representation of OnTrack’s theory of change: Figure 1 OnTrack's Theory of Change Assessment areas and indicators The following table presents the assessment areas and indicators that were used to guide the review of the OnTrack pilots. Table 1 Assessment areas and indicators Evaluation Assessment Indicators question areas 1. How likely Quality and Relevance to broad WB goals would OnTrack relevance of contribute to design Criteria for selecting the pilot countries and better results projects of public service Mobile phone/internet penetration in pilot delivery to sites beneficiaries in the pilot Sensitivity to process and culture countries? What are the Openness and inclusiveness of the platform related predictive Directness of feedback process indicators and evidence? Adaptation of OnTrack to in country conditions Contribution Level of awareness by project beneficiaries to development Beneficiary usage rate results Responsiveness to feedback Timeliness of response to feedback Fix rate 2. How Process OnTrack quality of advice and support efficient is the efficiency OnTrack in PIU quality of service enabling and tracking PIU/public service providers capacity behavioral changes of key New needs identified by the implementation of stakeholder the program groups including Human, material and financial resources citizens, invested by different constituents (PIU, CMU, service OnTrack, broad ODTA) providers and Monitoring Existence and implementation of a monitoring policy-makers? of behavioral and learning system changes Quality of relationships between different constituents (including management of expectations) 3. What are the Sustainability Buy in by PIUs/public service providers key factors that of OnTrack need to be Buy in by government authorities from which taken into depend the PIUs account in OnTrack Phase Buy in by TTLs/CMUs 2 rollout and in for "off Perceptions on the value of beneficiary loading" of feedback by PIU/public service providers OnTrack Platform as Beneficiary usage rates part of the broader citizen OnTrack quality of advice and support engagement agenda? Clear definition of OnTrack’s role going forward PIU quality of service PIU/public service providers capacity Replication Interest expressed by other actors in WB (i.e. of OnTrack internal demand for OnTrack or similar systems) Completeness of feedback system as product (Have all aspects of design and operations been finalized and tested? Technical aspects of the platform solved? Feedback mechanism process and operations fully explained by guidelines that have been tested?) Data collection matrix This matrix refers to the data collection process that took place during the assignment. Indicator Data collection Data source Instrument used method Relevance to broad Desk review WB strategic N/A WB goals documents OnTrack staff; WBI Online survey Management; Staff survey ODTA staff; CMU; questionnaire Project TTLs Criteria for Semi-structured OnTrack staff; Staff interview selecting the pilot interviews WBI guide countries and management; projects Pilot country directors; Project TTLs Partner interview PIUs guide PIUs/public PIU/service Indicator Data collection Data source Instrument used method service providers providers interview guide Mobile Desk review Studies carried N/A phone/internet out as part of penetration in pilot OnTrack design sites Publicly available studies Semi-structured Staff survey interviews OnTrack questionnaire staff/Project TTLs Face-to-face PAR beneficiaries survey with PAR project questionnaire smartphones beneficiaries, Santa Cruz Bolivia Sensitivity to Semi-structured OnTrack staff; Staff interview process and culture interviews WBI guide management; Pilot country directors; Project TTLs PIU/service PIUs/public providers service providers interview guide Focus group Project Focus group guide discussions beneficiaries that for beneficiaries have participated that have received in informational direct information Face-to-face sessions about survey with OnTrack smartphones PAR project PAR beneficiaries beneficiaries, questionnaire Santa Cruz Bolivia Adaptation of Same as above Same as above Same as above OnTrack to in country conditions Openness and Focus group Project Focus group guide inclusiveness of the discussions beneficiaries that for beneficiaries platform have participated that have received in informational direct information sessions about OnTrack Focus group guide Indicator Data collection Data source Instrument used method Project for beneficiaries beneficiaries that that have not have not received direct participated in information SMS survey (test informational carried out in sessions Project PAR project, beneficiaries Bolivia) Project survey beneficiaries that questionnaire have sent a report via SMS Directness of Semi-structured OnTrack staff; Staff interview feedback process interviews Pilot country guide directors; Project TTLs PIU/service PIUs providers interview guide Level of awareness Face-to-face PAR project PAR beneficiaries by project survey with beneficiaries, questionnaire beneficiaries smartphones Santa Cruz Bolivia Project Project beneficiaries that beneficiaries focus Focus group have/have not group guides discussions participated in informational sessions about OnTrack Beneficiary usage Desk review OnTrack platform N/A rate monthly reports (Bolivia and Nepal) Responsiveness to Desk review OnTrack platform N/A feedback Online survey OnTrack staff; Staff survey CMU; Project questionnaire TTLs Focus group Project Project discussions beneficiaries that beneficiaries focus have/have not group guides participated in informational sessions about OnTrack Indicator Data collection Data source Instrument used method Timeliness of Same as above Same as above Same as above response to feedback New needs Same as above Same as above Same as above identified by the implementation of the program Fix rate Desk review OnTrack platform N/A Online survey OnTrack staff; Staff survey CMU; Project questionnaire TTLs Semi-structured PIU/service interviews PIUs/public providers service providers interview guide OnTrack quality of Online survey OnTrack staff; Staff survey advice and support WBI questionnaire Management; ODTA staff; CMU; Project TTLs Semi-structured PIU/service interviews PIUs/public providers service providers interview guide PIU quality of Semi-structured PIUs/public PIU/service service interviews service providers providers interview guide OnTrack staff; Pilot country Staff interview directors; Project guide TTLs PIU/public service Same as above Same as above Same as above providers capacity Human, material Desk review OnTrack work N/A and financial plans and budgets resources invested by different OnTrack staff; constituents (PIU, Online survey WBI Staff survey CMU, OnTrack, Management; questionnaire broad ODTA) ODTA staff; CMU; Project TTLs Semi-structured PIUs/public PIU/service interviews service providers providers interview guide Existence and Online survey OnTrack staff; Staff survey implementation of CMU; Project questionnaire Indicator Data collection Data source Instrument used method a monitoring and TTLs learning system Semi-structured PIU/service interviews PIUs/public providers service providers interview guide Quality of Online survey OnTrack staff; Staff survey relationships WBI questionnaire between different Management; constituents ODTA staff; CMU; (including Project TTLs management of Semi-structured PIU/service expectations) interviews PIUs/public providers service providers interview guide Focus group discussions Project Project beneficiaries that beneficiaries focus have participated group guide in informational sessions about OnTrack Buy in by Semi-structured PIUs/public PIU/service PIU/public service interviews service providers providers providers interview guide Buy in by Semi-structured Top level govt Top level govt government interviews authorities authorities authorities from interview guide which depend the PIUs Buy in by TTLs and Semi-structured OnTrack staff; Staff interview CMUs interviews Pilot country guide directors; Project TTLs Perceptions on the Semi-structured PIUs/public PIU/service value of project interviews service providers providers beneficiary interview guide feedback by PIU/public service providers Interest expressed Online survey OnTrack staff; Staff survey by other actors in WBI questionnaire WB Management; ODTA staff; CMU; Project TTLs Semi-structured PIU/service interviews PIU providers interview guide Completeness of Online survey OnTrack staff; Staff survey feedback system as WBI questionnaire Indicator Data collection Data source Instrument used method product/Level of Management; readiness of ODTA staff; CMU; feedback system Project TTLs Semi-structured Staff interview interviews OnTrack staff; guide Pilot country directors; Project TTLs Sampling Purposeful sampling was used for the interviews with World Bank staff and PIUs, as well as for the online survey of World Bank staff, aiming to collect information and perceptions from as many as possible of those stakeholders that have been directly involved in the implementation and/or supervision of the OnTrack pilots. A mix of convenience and purposeful sampling was used for focus group discussions held with project beneficiaries in Santa Cruz, Bolivia and Kapilvastu, Nepal. They included both beneficiaries that have and have not received orientation regarding OnTrack. For the face-to-face survey of PAR beneficiaries in Santa Cruz Bolivia, stratified sampling with random selection of alliances and members was used to design a representative sample. A detailed methodological report prepared by the company who carried out the data collection on the ground (Ipsos) is included at the end of the next section. Table 2 Summary of consulted stakeholders Bolivia Nepal Zambia Washing ton DC Interviews with WB Sr 1 1 - 3 Management Interviews with TTLs 2 1 1 - Interviews with other in- 3 4 3 - country WB staff Interviews with OT team - - - 4 Interviews with PIUs 7 4 - - Online survey of WB staff 7 - 5 7 Focus groups with project 8 41 - - beneficiaries beneficiar beneficiar ies ies 5 12 extension Partner workers organizat ions Face-to-face survey with PAR: 439 - - - project beneficiaries The list of consulted stakeholders is included at the end of this annex. Analytical framework In order to examine OnTrack’s relevance and potential in achieving its stated outcomes, Keystone’s work was guided by its core methodology, Constituent Voice1. We use the Hirschman Voice Cycle as a framework against which to assess OnTrack program quality. The Hirschman Voice Cycle is a distillation of over 7 years of work in implementing successful citizen feedback systems. This evaluation framework is not rigid, and is not applied in a mechanistic way. Rather, it is an exhaustive checklist, albeit an organized one, of the features one would expect to see in an effective feedback mechanism. It is not assumed that OnTrack will meet the Hirschman Voice Cycle elements in any particular way. The evaluation, however, describes precisely how OnTrack addresses each of the features in the five-step cycle: design, collect, analyze, dialogue, and course correct. 1 For more details on Keystone’s Constituent Voice method, please see Constituent Voice - Technical Note 1. Figure 2 The Hirschman Voice Cycle Specifically, for each step of the cycle we analyze a series of elements that speak directly to OnTrack’s likelihood to contribute to better results for public service delivery and its efficiency in enabling and tracking behavioural changes of key stakeholder groups, as stated in the evaluation questions included in the TOR. The table below presents the elements that are examined under each step. Table 3 Elements of analysis 1. Designi 2. Collect 3. Analyze 4. Dialogue 5. Course correct Rigor Adequacy and Segmentation Publishing Utility of data Sensitivity adaptation of Triangulation and reporting and findings to process method to Benchmarking back Use of data and context/feedback Interpretation Sense making and findings culture providers of findings with Improvements Cost Management of stakeholders in service Utility expectations Follow up Frequency investigations Independence Anonymity Limitations to the evaluation The following are the main limitations that we encountered in the process of reviewing the OnTrack pilot implementations:  Lack of information on project design There is little information available, at least in written form, regarding the design of the OnTrack program. We are lacking documents that clearly state the objectives and goals of the project and that lay out the strategy and assumptions behind it. Similarly, we are lacking a monitoring plan for the project with the corresponding indicators. This circumstance poses a limitation to the evaluation in the sense that additional inquiry has been required for clarifying the project’s theory of change, which includes sense making and pulling each element from different documents and sources. Furthermore, no monitoring data are available except from monthly reports on website traffic.  Limited use of the platforms OnTrack operated in 2 out of 4 pilot countries, where it was live for just over a year. As we discuss in the findings, there were almost no addressable issues reported by users on any of the 3 project platforms. This gave us little information to work with, especially in correlating the reporting of issues with actual fixes/development results.  Difficulties in consulting with certain key stakeholders Namely, a key member of the OnTrack team has not made themselves available for an interview with the Keystone team. Furthermore, we were not put in touch with the company that has developed the platform, as their contract with the WB was being renegotiated and the team considered that it was not adequate to include them in the process. This has led to having to rely solely on information provided by OnTrack staff regarding the process for developing and technical specificities of the platforms. It has also implied the impossibility of running an online survey of website visitors, as the collaboration of the developers was necessary (see below). Limitations regarding reaching the designed sample in the survey of PAR beneficiaries in Bolivia, are discussed in Ipsos methodological report included at the end of the next section.  Difficulties for remotely surveying platform users OnTrack staff have brought to our attention the following limitations for carrying out an online and SMS/phone survey: o Privacy issues/lack of authorization by users to contact them using the phone numbers and email addresses registered on the OnTrack platform o Very limited use of the platforms o Impossibility to make changes on the platform support (website), so not possible to place a banner for the online survey. o Political sensitivity by PIUs who may not want a third party to contact directly their constituents for a survey Due to these difficulties we were only able to carry out a test SMS survey for PAR beneficiaries in Santa Cruz, Bolivia. However this test was unsuccessful as the phone numbers extracted from the back log of the platform were mainly those of agricultural extension workers (PAR employees) that had registered their personal phone numbers for demonstration purposes during the orientation sessions they had held with beneficiaries. Data collection materials and sources Tools and guides used for data collection PIU interview guide Instructions:  Semi-structured interview with approximate duration of 1-1,5 hour  Questions may be shared with respondents prior to the interview so that they can prepare  Ask question as in the guide and continue with follow up questions to make sure that there is no ambiguity in our understanding  Order of questions is not rigid, should be adapted to the flow of the conversation.  When relevant and available, ask respondent to provide with documents supporting the opinions provided.  Indicator column is included here for showing the correlation of the questions with the evaluation indicators and as a reference for analysis. Question Indicator 1. Please explain your role in relation to OnTrack General 2. Why do you think that your service/unit was chosen for 2. Criteria for selecting the pilot piloting OnTrack? countries and projects 3. What was your service/unit’s interest in participating in OT? 20. Buy in by PIU/public service providers 4. In what way is citizen feedback important for you? 23. Perceptions on the value of project beneficiary feedback by PIU/public service providers 5. Throughout the process of design and implementation of OT, 4. Sensitivity to process and did you feel that you were adequately consulted and that your culture opinions where taken into account? Can you explain/provide examples? 6. Do you feel that adequate attention has been given to process 4. Sensitivity to process and and to sensitivity towards the local context and culture in the culture; 5. Adaptation of design process? Is the OT design adequately adapted to the in- OnTrack to in country conditions country conditions? 7. To what extent do you believe that OT effectively enables 7. Directness of feedback direct feedback - from user to provider? What are the process challenges? How can this be improved? 8. Can you provide examples of actions taken/improvements 13. Fix rate made in your service in response to feedback provided by citizens? Was the feedback provided through OT or by other means? 9. What kind of support do you need from WB staff in order to 14. OnTrack quality of advice effectively implement OT? Do they have the necessary and support capacity to provide this support? Are you satisfied with the support received? What are the areas that need improvement? 10. What capacity elements are crucial for you (the PIU) to be able 15. PIU quality of service; 16. to effectively operate the platform and fully implement the PIU/public service providers feedback loop? Do you believe that you currently have that capacity capacity? Where is capacity lacking and what could be done about it? 11. How many people from your service/unit work on OT? Is it full 17. Human, material and time or part time? Since the beginning, how much time have financial resources invested by they dedicated to OnTrack? different constituents (PIU, CMU, OnTrack, broad ODTA) 12. Could you give an estimate of the material and financial 17. Human, material and resources that you have dedicated to OT since the beginning of financial resources invested by the pilot? [this might be a sensitive question] different constituents (PIU, CMU, OnTrack, broad ODTA) 13. Do you feel that the level of human and financial resources 17. Human, material and that have been invested so far on OnTrack (both by the WB financial resources invested by and you) is proportionate to the results it is expected to different constituents (PIU, deliver? Why? CMU, OnTrack, broad ODTA) 14. Is a monitoring and learning system in place for OnTrack? 18. Existence and What kind of data is being collected and analysed? How are implementation of a monitoring these being used? and learning system 15. How would you describe your relationship with the staff in WB 19. Quality of relationships for implementing OT? Do you feel that there is mutual between different constituents understanding and respect? 16. What kind of expectations did you have from your 19. Quality of relationships participation in piloting OT? Were these met? Did you feel that between different constituents you were promised things that were later not followed through? 17. Are you aware of any expressions of interest for replicating OT 24. Interest expressed by other in any other projects that are supported by the WB in your actors in WB country? In any other projects not supported by the WB? 18. What have been OT’s achievements to date? General 19. What have been OT’s challenges? General Staff interview guide Instructions:  Semi-structured interview with approximate duration of 1-1,5 hour  Questions may be shared with respondents prior to the interview so that they can prepare  Ask question as in the guide and continue with follow up questions to make sure that there is no ambiguity in our understanding  Order of questions is not rigid, should be adapted to the flow of the conversation.  Online survey of staff will take place previous/simultaneously to interviews. If responses to survey are available, make sure to use them as lead in for the questions.  When relevant and available, ask respondent to provide with documents supporting the opinions provided.  Indicator column is included here for showing the correlation of the questions with the evaluation indicators and as a reference for analysis. Respondent Question Indicator All 1. What is your role in relation to OT? General Management 2. How does OT fit in the broader WB goals and 1. Relevance to strategy? Any docs to point us to? broad WB goals All (except 3. What has been the process for selecting the pilot 2. Criteria for ODTA) projects and countries? What were the criteria used? selecting the pilot countries and projects; 4. Sensitivity to process and culture OT 4. Do you feel that adequate attention has been given 4. Sensitivity to staff/CMUs/TTLs to process and to sensitivity towards the local process and context and culture in the design process? Is the OT culture; 5. design adequately adapted to the in-country Adaptation of conditions? OnTrack to in country conditions OT 5. Was mobile phone/internet penetration adequately 3. Mobile staff/CMUs/TTLs assessed in the pilot sites? How? In your opinion, phone/internet what level of penetration is necessary for an OT-like penetration in initiative to be successful? pilot sites OT 6. Is a monitoring and learning system in place for 18. Existence and staff/CMUs/TTLs OnTrack? What kind of data is being collected and implementation analysed? How are these being used? of a monitoring and learning system OT 7. To what extent do you believe that OT effectively 7. Directness of staff/CMUs/TTLs enables direct feedback - from user to provider? feedback process What are the challenges? How can this be improved? OT 8. What capacity elements are crucial for the PIU to be 15. PIU quality of staff/CMUs/TTLs able to effectively operate the platform and fully service; 16. implement the feedback loop? Do you believe that PIU/public they currently have that capacity? Where is capacity service providers lacking and what could be done about it? capacity OT 9. What are the conditions that need to exist in country For real-time staff/CMUs/TTLs in order for OT to be effective? adaptive management method Respondent Question Indicator OT staff 10. Do you feel that there is sufficient commitment from 22. Buy in by TTLs TTLs and CMUs to the success of OT? To what extent and CMUs do you believe that such commitment is needed? How can this commitment be strengthened/motivated? CMUs/TTLs 11. To what extent do you consider citizen feedback to 22. Buy in by TTLs be an important factor to the success of your and CMUs project? Why? What motivates you in relation to CE? What turns you off? What conditions need to be in place for your commitment to stronger? OT 12. What are the elements that need to be in 25. Completeness staff/CMUs/TTLs place/developed in order to consider OT as a of feedback complete product? Can you prioritise between system as them? To what extent have these been achieved so product/Level of far? What are the critical elements that need to be in readiness of place before scaling up and out? feedback system; For real-time adaptive management method All 13. In what is OT distinct from other CE initiatives? General All 14. What have been OT’s achievements to date? General All 15. What have been OT’s challenges? General All 16. We have been assigned to develop a real-time For real-time adaptive management method for OT going forward. adaptive In your opinion, what are the key issues that such management method must address? What characteristics should it method have? What are your expectations? Top level government authorities interview guide Instructions:  Semi-structured interview with approximate duration of 1-1,5 hour  Questions may be shared with respondents prior to the interview so that they can prepare  Ask question as in the guide and continue with follow up questions to make sure that there is no ambiguity in our understanding  Order of questions is not rigid, should be adapted to the flow of the conversation.  When relevant and available, ask respondent to provide with documents supporting the opinions provided.  Indicator column is included here for showing the correlation of the questions with the evaluation indicators and as a reference for analysis. Question Indicator 20. Please explain your role in relation to OnTrack General 21. Why do you think that the particular service/unit was chosen 2. Criteria for selecting the pilot for piloting OnTrack? countries and projects 22. What was your government’s interest in participating in OT? 21. Buy in by government authorities from which depend the PIUs 23. In what way is citizen feedback important for you? 21. Buy in by government authorities from which depend the PIUs ; 23. Perceptions on the value of project beneficiary feedback by PIU/public service providers 24. Throughout the process of design and implementation of OT, 4. Sensitivity to process and did you feel that you were adequately consulted and that your culture opinions where taken into account? Can you explain/provide examples? 25. Do you feel that adequate attention has been given to process 4. Sensitivity to process and and to sensitivity towards the local context and culture in the culture; 5. Adaptation of design process? Is the OT design adequately adapted to the in- OnTrack to in country conditions country conditions? 26. To what extent do you believe that OT effectively enables 7. Directness of feedback direct feedback - from user to provider? What are the process challenges? How can this be improved? 27. What kind of support does your government need from WB 14. OnTrack quality of advice staff in order to effectively implement OT? Do they have the and support necessary capacity to provide this support? Are you satisfied with the support received? What are the areas that need improvement? 28. What capacity elements are crucial for the PIU to be able to 15. PIU quality of service; 16. effectively operate the platform and fully implement the PIU/public service providers feedback loop? Do you believe that you currently have that capacity capacity? Where is capacity lacking and what could be done about it? 29. Do you feel that the level of human and financial resources 17. Human, material and that have been invested so far on OnTrack (both by the WB financial resources invested by and your government) is proportionate to the results it is different constituents (PIU, expected to deliver? Why? CMU, OnTrack, broad ODTA) 30. How would you describe your relationship with the staff in WB 19. Quality of relationships for implementing OT? Do you feel that there is mutual between different constituents understanding and respect? 31. What kind of expectations did you have from your 19. Quality of relationships participation in piloting OT? Were these met? Did you feel that between different constituents you were promised things that were later not followed through? 32. What have been OT’s achievements to date? General 33. What have been OT’s challenges? General 34. Are you aware of other government departments that are General interested in implementing OT or something like it? If yes, what departments (not for contacting them but to understand the type of department)? Guide for focus group discussions with project beneficiaries that have participated in informational sessions about OnTrack Instructions:  6-8 participants  approx. duration 1-1,5 hr  carry out separate groups for men and women. If women come with children make sure that there is a separate space where a person can stay with the children while their mothers participate in the discussion  sit in a circle, at the same level  have refreshments available  ensure that all have a chance to speak. Directly ask questions to the more timid ones  the order of the questions should be adapted to the flow of the conversation  when relevant, ask for examples  if use of interpreter is necessary, rehearse first  1 person facilitating and 1 taking notes Introduction:  Welcome, thank you for your time, it is very appreciated  We wanted to talk to you because you participated in a session about the OnTrack and we would like to hear about your experience. Your opinion is very important for us to be able to make the system better  Round of introductions  Feel free to say what you think. In our report we will not use your names. There are no correct or wrong answers, we just want to know your opinions. Let others finish what they are saying, do not interrupt. Everyone will have a chance to speak Question Indicator 1. Do you remember the session you had about OT? Can you explain 8. Level of awareness what you talked about/did during that session? by project beneficiaries 2. OT is about letting the government (or use the name by which the 4. Sensitivity to PIU is known in that area) know about things that are not working process and culture; well and asking them to fix them. How do you feel about that? In 5. Adaptation of your culture, do you feel comfortable complaining when things OnTrack to in country don’t work well? Do you believe that things will change if you report them? 3. During the session, did you try to send an SMS for reporting a 6. Openness and problem? What did you think about it? Was it easy? inclusiveness of the platform 4. Ever since, have you tried sending a report through an SMS? 5. What about the website? Is it clear for you how to use it? Have you 6. Openness and used it since? inclusiveness of the platform 6. If you have sent in a report using OT, did you receive a response? 10. Responsiveness to What did you think of it? // If not: What would you expect to feedback; 11. happen if you sent in a report? Timeliness of response to feedback 7. Do you think that OT is a good way for making sure the government 12. New needs (or use the name by which the PIU is known in that area) knows identified by the Question Indicator about the local needs? Is it better than other ways (for ex. implementation of Community meetings, surveys)? the program 8. Looking back at that session, are you satisfied about how you were 19. Quality of treated by the people who ran the session? Was it worth your time? relationships between Were your expectations met? different constituents 9. Have you told other people about OT? To whom? What did they 8. Level of awareness think? Do you know if they have used it or would like to use it? by project beneficiaries 10. Could you put us in contact with them? We would like to have a similar discussion with them? It might not be practical, due to time constraints, to schedule the focus group with people that have not participated in the information sessions after this FG is done. So, it might be better that this information is solicited at the moment that they are invited to participate in the FG. Guide for focus group discussions with project beneficiaries that have NOT participated in informational sessions about OnTrack Instructions:  6-8 participants  approx. duration 1-1,5 hr  carry out separate groups for men and women. If women come with children make sure that there is a separate space where a person can stay with the children while their mothers participate in the discussion  sit in a circle, at the same level  have refreshments available  ensure that all have a chance to speak. Directly ask questions to the more timid ones  the order of the questions should be adapted to the flow of the conversation  when relevant, ask for examples  if use of interpreter is necessary, rehearse first  1 person facilitating and 1 taking notes Introduction:  Welcome, thank you for your time, it is very appreciated  We wanted to talk to you about OT. Your opinion is very important for us to be able to make the system better  Round of introductions  Feel free to say what you think. In our report we will not use your names. There are no correct or wrong answers, we just want to know your opinions. Let others finish what they are saying, do not interrupt. Everyone will have a chance to speak Question Indicator 11. Have you heard about OT before today? If yes, can you explain what 8. Level of awareness it is? by project beneficiaries After they have answered, give an explanation of what OT is about and how it works. 12. OT is about letting the government (or use the name by which the 4. Sensitivity to PIU is known in that area) know about things that are not working process and culture; well and asking them to fix them. How do you feel about that? In 5. Adaptation of your culture, do you feel comfortable complaining when things OnTrack to in country don’t work well? Do you believe that things will change if you report them? 13. Do you send SMS? Have your ever used SMS to report a problem in 6. Openness and your area? Do you think that it is easy to do? Would you do it? inclusiveness of the platform 14. What about the internet? Do you use it? Would you use it for 6. Openness and reporting problems? inclusiveness of the platform 15. What would you expect to happen if you sent in a report? 10. Responsiveness to feedback; 11. Timeliness of response to feedback 16. Do you think that OT is a good way for making sure the government 12. New needs (or use the name by which the PIU is known in that area) knows identified by the about the local needs? Is it better than other ways (for ex. implementation of Community meetings, surveys)? the program Staff online questionnaire Email for sending out survey: Dear [FirstName] At the Annual Meetings last year, World Bank President Jim Kim said " "[We] must become a better listener. Last year, we had beneficiary feedback on 34 percent of our projects. We promise that for our projects with clear beneficiaries, we will get feedback – from every single one of them, 100 percent." Keystone Accountability has been assigned by WBI’s Innovation Labs to review the experience of its innovative online tool for beneficiary feedback -- OnTrack. In particular, we are studying OnTrack experiences of Nepal, Bolivia, and Zambia to create a real-time adaptive management methodology that OnTrack and other ICT-enabled feedback mechanisms may use going forward. As part of this assignment we are carrying out an online survey of all World Bank staff that have been involved in the design and implementation of OnTrack. Please take 15 minutes to take the survey using this link [SurveyLink] Please answer the questions based on your experience so far with OnTrack. If you do not know the answer to a question, please choose “I don’t know” or leave it blank. All individual responses will be treated confidentially. We will however acknowledge the contribution of the different staff members that respond to the survey (hopefully ALL of you!). Should you have any questions regarding the survey or this assignment, please do not hesitate to contact me at natalia@keystoneaccountability.org or the Project Lead from WBI: Natalia Agapitova at nagapitova@worldbank.org. Thank you for your time, ___________________ 2 Question Options Indicator 1. What is your position in the WB? N/A 2. Where are you based? Washington DC N/A Bolivia Ghana Zambia Nepal 3. What is your gender? Male N/A Female 4.Please, briefly describe your role in relation to N/A OnTrack: Please indicate the extent to which you agree or disagree with the following statements: 5. OnTrack is aligned with the broader WB goals 0- Completely 1. Relevance to broad disagree WB goals 1 Please explain your answer: 2 3 4 5 6 7 8 9 10- Completely agree Don’t know 6. OnTrack’s design ensures that users who 0- Completely 10. Responsiveness to provide feedback on public services always disagree feedback receive a response ….. 2 Indicator column is included here for showing the correlation of the questions with the evaluation indicators and as a reference for analysis. It will not be included in the survey to be sent out to respondents 2 Question Options Indicator 10- Completely Please explain your answer: agree Don’t know 7. In my experience, providers of public services 0- Completely 10. Responsiveness to involved with OnTrack always respond to disagree feedback feedback received by users through OnTrack ….. 10- Completely agree Don’t know 8. In my experience, providers of public services 0- Not at all 10. Responsiveness to involved with OnTrack are committed to using committed feedback feedback from users in order to improve their ….. services 10- Completely committed Don’t know 9. I know of at least one example where a public Yes 10. Responsiveness to service provider has taken a concrete action in No feedback; 13. Fix rate response to feedback provided by a user through OnTrack If yes, please provide the examples you know of: 10. The use of SMS and internet technology by 0- Completely 11. Timeliness of OnTrack enables providers of public services to disagree response to feedback respond to user feedback in a practical and ….. timely way 10- Completely agree Don’t know 11. I know of at least one example where a public Yes 11. Timeliness of service provider has given a timely response to No response to feedback feedback provided by a user through OnTrack If yes, please provide the examples you know of: 12. OnTrack has the potential to be an effective 0- Completely 12. New needs mechanism for identifying new needs regarding disagree identified by the public service provision ….. implementation of 10- Completely the program agree Don’t know 13. OnTrack is an effective mechanism for 0- Completely 12. New needs identifying new needs regarding public service disagree identified by the provision ….. implementation of 10- Completely the program agree Don’t know 14. I know of at least one example where a new Yes 12. New needs need was identified by a user through OnTrack No identified by the implementation of If yes, please provide the examples you know of: the program 15. To my knowledge, the OnTrack team has the 0- Completely 14. OnTrack quality of necessary human resources for providing high disagree advice and support quality advice and support to the PIUs and ….. CMUs for the implementation of OnTrack 10- Completely agree Don’t know 16. To my knowledge, the OnTrack team has the 0- Completely 14. OnTrack quality of 2 Question Options Indicator necessary material resources for providing high disagree advice and support quality advice and support to the PIUs and ….. CMUs for the implementation of OnTrack 10- Completely agree Don’t know 17. Please explain your answer to the previous two 14. OnTrack quality of questions: advice and support 18. [ONLY ANSWER THIS QUESTION IF YOU DO NOT 17. Human, material WORK FULL TIME ON ONTRACK] Please provide and financial a rough estimate of the number of full work resources invested by days that you have dedicated to OnTrack since different constituents the beginning of your involvement with it: (PIU, CMU, OnTrack, broad ODTA) [ONLY ANSWER THIS QUESTION IF YOU WORK FULL 17. Human, material TIME ON ONTRACK] Please provide a rough estimate and financial of the number of full work days that you have resources invested by dedicated to the following different pieces of work: different constituents 19. Selection of pilot sites (PIU, CMU, OnTrack, 20. Negotiation with Project TTL and CMUs for broad ODTA) ensuring agreement to pilot OnTrack 21. Negotiation with PIUs for ensuring agreement to pilot OnTrack 22. Technical assistance/Capacity building of CMU (including field visits) 23. Technical assistance/Capacity building of PIU (including field visits) 24. Building of the OnTrack platform/website (including customization and troubleshooting) 25. Testing the feedback system 26. I consider that the level of human and financial 0- Completely 17. Human, material resources that have been invested so far on disagree and financial OnTrack is proportionate to the results it is ….. resources invested by expected to deliver 10- Completely different constituents agree (PIU, CMU, OnTrack, Please explain your answer: Don’t know broad ODTA) 27. I feel that the relationships between different 0- Completely 19. Quality of WB staff involved in OnTrack are based on disagree relationships between mutual understanding and respect ….. different constituents 10- Completely Please explain your answer: agree Don’t know 28. I feel that the relationships between different 0- Completely 19. Quality of WB staff involved in OnTrack and the PIUs are disagree relationships between based on mutual understanding and respect ….. different constituents 10- Completely Please explain your answer: agree Don’t know 29. Have there been any expressions of interest Yes 24. Interest expressed from other parts/units/projects within the WB No by other actors in WB for replicating OnTrack? If yes, please tell us about them: 2 Question Options Indicator 30. OnTrack is a complete feedback system, with 0- Completely 25. Completeness of all the technical and process aspects fully disagree feedback system as designed and tested ….. product/Level of 10- Completely readiness of feedback Please explain your answer: agree system Don’t know SMS survey of project beneficiaries that have sent a report using OnTrack We have decided to adapt the questions, as the only users that we were able to get from the website, were people that participated in trainings. See below for adaptation Question Options Indicator 1. Sometime ago you sent It was complicated 6. Openness and inclusiveness an SMS to report a It was somewhat easy of the platform problem with project It was very easy XXX. Was it easy to do that? 2. Are you a man or a Man 6. Openness and inclusiveness woman? Woman of the platform Prefer not to say 3. What is your level of I never went to school 6. Openness and inclusiveness education? I went to school for less than 4 of the platform years I went to school for 5-9 years Finished secondary school Went to university but not graduated I graduated from university Don’t Know 4. When you sent in the Yes 10. Responsiveness to feedback SMS, did you receive a No response? Don’t know 5. Was the problem that Yes 13. Fix rate you reported fixed? No Don’t know 6. If yes, was it fixed Yes 11.Timeliness of response to quickly? No feedback Don’t know 7. Are you happy with how I am not happy General the issue was fixed? I’m somewhat happy I am very happy 8. How likely are you to 0=Not at all likely General recommend OnTrack to a … friend or relative? 10=Very likely Adaptation for test in Bolivia: Question Options Indicator 1. We are doing a survey regarding OnTrack. Please answer the questions and you will receive a link with XX$ free airtime. Respond by marking 1 to continue. 2. Sometime ago you sent It was complicated 6. Openness and inclusiveness an SMS to report a It was somewhat easy of the platform problem with project It was very easy XXX. Was it easy to do that? Mark 1 for complicated, 2 for somewhat easy and 3 for easy 3. Since then, did you send Yes 13. Fix rate another SMS to OnTrack? No Mark 1 for Yes, 2 for No 4. Would you recommend No General OnTrack to a friend or Not sure relative? Mark 0 for No, 1 Yes for not sure, 2 for Yes 5. Are you a man or a Man 6. Openness and inclusiveness woman? Mark 1 for man Woman of the platform and 2 for woman Spanish translation: (accents cannot be used in Spanish as the SMS survey software - Telerivet- breaks it into multiple messages) Questions: 1. Estamos haciendo una encuesta sobre la plataforma EMPODERAR. Responda y recibira un enlace con XX$ de crédito para su tel. Responda marcando 1 para continuar 2. Hace un tiempo mando un mensaje para informar de un problema con EMPODERAR. Como fue? Responda 1 por COMPLICADO, 2 por BASTANTE FACIL y 3 por MUY FACIL. 3. Desde entonces ha mandado algun otro mensaje a la plataforma EMPODERAR? Responda 1 por SI y 2 por NO. 4. Recomendaria el uso de la plataforma EMPODERAR a un amigo o vecino? Responda 0 por NO, 1 por NO ESTOY SEGURO, 2 por SI 5. Es usted hombre o mujer? Responda 1 por HOMBRE y 2 por MUJER Questionnaire for face-to-face survey with PAR beneficiaries in Bolivia This survey was built collaboratively with WB external consultants Fredrik Sjoberg and Jonathan Mellon, who have been working with the WBI to develop a series of guidelines for the evaluation of ICT-enabled citizen engagement initiatives. ENGLISH DRAFT Introduction: We are doing a survey and we would like to know your opinion regarding using your mobile phone and the internet for dealing with issues related to your work. Your contact has been facilitated to us by the head of the Alliance [say name]. Your responses will be kept anonymous. The survey will take approximately10 minutes. Please answer with as much honesty as you can. There are no correct or incorrect answers. If you don’t know something, please say so. 1. Imagine that you participate in a’ program run by the government that supports local producers. With your mobile phone you could [treatment: ‘anonymously’ (50%) - include instructions for interviewers] report a problem regarding that program. how likely is it that you would do it?  Very likely  Likely  Unlikely  Very unlikely  Don't know 2. What would be your most preferred way for submitting a problem report? (choose 1)  Email  Website  SMS  Phone call  In-person  None of the above 3. If you had a problem, suggestion or opinion regarding PAR would you know in which way to report it?  Yes  No - skip to 7  Not sure - skip to 7 4. If yes, what ways do you know of that are available for you to do this? (tick all that apply)  Send an SMS  Call a free phone  Send a letter  Write down the issue and put in a box in the PAR office  On a website specifically created for this  Go to the PAR offices in person  Discuss with the agricultural extension worker (acompañante) 5. [If they choose SMS and/or online] How did you hear about it? (tick all that apply)  A person that works with the project (project coordinator/contact person)  Participated in a training  Saw an advertisement/poster  Heard about it on the radio  A friend/relative/ neighbor/ colleague told me about it  I don’t know 6. [If participated in training] How easy did you find the system to be?  Complicated  Somewhat easy  Very easy 7. Please indicate whether you have done any of these actions [specific for PAR] Would never Have done Might do Don't know do Present myself for election in the administration of a producers’ association Take part in a vote during a meeting of the producers’ association Present issues to be discussed in meetings of the producers’ association Participate in a peaceful demonstration Go to the office of a civil servant Send an SMS to report an issue with PAR Go online to share a story regarding PAR File an official complaint regarding a public service* If “have done”: how? For which service?: ________________________________ 8. What would motivate you to send an SMS to report an issue with PAR or to go online and share your story on the PAR’s website? [Open question] 9. Please indicate whether you agree or disagree with the following statements: Strongly Strongly Don't Agree Disagree agree disagree know I feel that I have a pretty good understanding of the important political issues facing our country. People like me can influence what the government does 10. In what year were you born? (enter 4-digit birth year; for example, 1976) 11. What is the highest level of education you have completed?  None/Illiterate  Primary incomplete  Primary complete  Secondary incomplete  Secondary complete  Superior technical incomplete  Superior technical complete  Superior university incomplete  Superior university complete  Postgraduate 12. Do you/your family own the following items  Car/Jeep/Van/Motorcycle  Refrigerator  Air conditioner  Personal computer/laptop  Smartphone  None of these 13. How often do you use the Internet?  Daily  Weekly  Monthly  Less than monthly - if less than monthly, when was the last time? ________________  Never - skip to end  Don't know - skip to end 14. Which of the following things have you done on the Internet in the last 12 months?  Check the price of crop/cattle online  Log in to Facebook  Check my email  Read information about a PAR project  Read the news 15. How many SMS do you send per week on average (on a normal week)? [insert number] *************************************** By enumerator Gender of the respondent  Male  Female Name of the alliance they belong to: _____________. Generated by the Software GPS coordinates Timestamp (start and finish of interview) SPANISH VERSION USED Ipsos Bolivia S.A. C. Pedro Salazar esq. C. Andrés Muñoz # 634, Sopocachi, Telf.: (2) #ENC. 2167676, La Paz C. Eduardo Caba # 337 entre América Oeste y Parque Lincoln, Telf.: (4) 4038393, Cochabamba Av. Velarde # 438 casi Segundo Anillo, Telf. (3) 3330600, Santa Cruz Línea gratuita: 800-11-7676 50-02-14 PROYECTO DE ALIANZAS RURALES VERSIÓN_6 GENERADO POR EL SOFTWARE Coordenadas GPS Hora de inicio ____ ____ : ____ ____ Fecha ____ ____ / ____ ____ / 2014 Hora de finalización ____ ____ : ____ ____ PRESENTACIÓN (LEER) Buenos días / tardes. Mi nombre es… (MOSTRAR CREDENCIAL) y soy encuestador(a) de Ipsos Bolivia, empresa especializada en realizar estudios de opinión pública y de mercado. En esta oportunidad nos encontramos realizando una encuesta para conocer su opinión sobre el uso y acceso a telefonía móvil e Internet en su alianza productora. Su contacto nos ha sido facilitado por el jefe de la Alianza… (MENCIONAR NOMBRE DE LA ALIANZA). Sus respuestas se mantendrán anónimas. La encuesta tardará aproximadamente unos 10 minutos. Le rogamos que conteste con la mayor franqueza posible. No hay respuestas correctas o incorrectas. Si no sabe algo, por favor dígalo. 1. (APLICAR P1SÓLO A LA MITAD DE LA MUESTRA) (MOSTRAR TARJETA P1) Imagínese que está participando en un programa del gobierno que apoya a productores locales y que con su teléfono móvil o celular puede informar de manera anónima de problemas que tiene con el programa. ¿Cuán probable es que use su celular para informar de los problemas que tiene con el programa? (LEER OPCIONES, RESPUESTA ÚNICA) No tiene teléfono Nada probable Poco probable Probable Muy probable celular (NO LEER) 1 2 3 4 97 1.b. (APLICAR P1b SÓLO A LA MITAD RESTANTE DE LA MUESTRA) (MOSTRAR TARJETA P1) Imagínese que está participando en un programa del gobierno que apoya a productores locales y que con su teléfono móvil o celular puede informar de problemas que tiene con el programa. ¿Cuán probable es que use su celular para informar de los problemas que tiene con el programa? (LEER OPCIONES, RESPUESTA ÚNICA) No tiene teléfono Nada probable Poco probable Probable Muy probable celular (NO LEER) 1 2 3 4 97 2. (MOSTRAR TARJETA P2) ¿De qué maneras preferiría informar de los problemas que tiene con el programa? (LEER OPCIONES, RESPUESTA MÚLTIPLE) Ninguna de estas maneras Por correo electrónico 1 Por teléfono 4 96 (NO LEER) A través de una página de 2 En persona 5 No sabe (NO LEER) 98 Internet Otro (NO LEER, ESPECÍFICA) Por mensaje de texto 3 _________________________ 3. Si tuviera un problema, sugerencia u opinión relacionados con el Proyecto de Alianzas Rurales, ¿sabría como reportarlo? No estoy seguro (PASAR Sí (CONTINUAR) 1 No (PASAR A P7) 2 3 A P7) 4. (APLICAR SÓLO SI RESPONDE COD. 1 EN P3, MOSTRAR TARJETA P4) ¿Qué maneras conoce que están a su disposición para reportar un problema, sugerencia u opinión sobre el Proyecto de Alianzas Rurales? (RESPUESTA MÚLTIPLE) En una página de Internet Enviar un mensaje de texto 1 específicamente creada para 5 Ninguna (NO LEER) 96 esto Ir en persona a una oficina del Llamar a un teléfono gratuito 2 6 No sabe (NO LEER) 98 Proyecto de Alianzas Rurales Enviar una carta 3 Hablarlo con un acompañante 7 Escribir el asunto y ponerlo Otro (NO LEER, ESPECÍFICA) en una caja de sugerencias 4 de una oficina del Proyecto _________________________ de Alianzas Rurales 5. (APLICAR SÓLO SI RESPONDE COD. 1 O 5 EN P4) ¿Cómo se enteró de que puede enviar mensajes de texto o usar el Internet para reportar problemas, sugerencias u opiniones sobre el Proyecto de Alianzas Rurales? (LEER OPCIONES, RESPUESTA MÚLTIPLE) Usted participó en una 1 Escuchó sobre ello en la radio 4 Ninguno (NO LEER) 96 capacitación A través una persona que trabaja en el proyecto Un amigo / familiar / vecino / 2 5 No sabe (NO LEER) 98 (coordinador del proyecto / compañero le informó sobre ello persona de contacto) Otro (NO LEER, ESPECÍFICA) Usted vio un anuncio / cartel 3 _________________________ 6. (APLICAR SÓLO SI CONTESTO COD. 1 EN P5, MOSTRAR TARJETA P6) Cuando participó en la capacitación, ¿cómo le pareció el sistema para reportar problemas sobre el Proyecto de Alianzas Rurales? (LEER OPCIONES, RESPUESTA ÚNICA) Muy difícil Difícil Fácil Muy fácil 1 2 3 4 7. (APLICAR A TODOS, MOSTRAR TARJETA P7) Por favor indique si ha hecho, tal vez haría o nunca haría las siguientes acciones. (LEER OPCIONES, RESPUESTA ÚNICA POR FILA) Sí lo ha Tal vez lo Nunca lo No sabe hecho haría haría (NO LEER) Presentarme como candidato en las elecciones para la administración de una asociación de 1 2 3 98 productores. Participar en la votación durante una reunión de la 1 2 3 98 asociación de productores. Indicar temas a ser debatidos en reuniones de la 1 2 3 98 asociación de productores. Participar en una marcha pacífica. 1 2 3 98 Ir a la oficina de un funcionario público. 1 2 3 98 Enviar un mensaje de texto para reportar algún asunto relacionado con el PROYECTO DE 1 2 3 98 ALIANZAS RURALES. Utilizar el Internet para compartir alguna historia relacionada con el PROYECTO DE ALIANZAS 1 2 3 98 RURALES. Realizar una queja oficial en relación a un servicio 1 2 3 98 público. 7a. (APLICAR SÓLO SI REALIZÓ UNA QUEJA OFICIAL EN RELACIÓN A UN SERVICIO PÚBLICO) Cuando Ud. realizó la queja oficial en relación a un servicio público. ¿Cómo realizó esta queja y en relación a qué tipo de servicio público? (ESPONTÁNEA, RESPUESTA MÚLTIPLE) _____________________________________________________________________________________ _________________________ 8. (APLICAR A TODOS) ¿Por qué motivos usted enviaría un mensaje de texto o utilizaría Internet para informar sobre algún asunto relacionado con el Proyecto de Alianzas Rurales o para compartir una historia en la página de Internet del Proyecto de Alianzas Rurales? (ESPONTÁNEA, RESPUESTA MÚLTIPLE) _________________________________________________________________________________________ _____________________ 9. (MOSTRAR TARJETA P9) Por favor indique que tan de acuerdo o en desacuerdo está con las siguientes afirmaciones. (LEER OPCIONES, RESPUESTA ÚNICA POR FILA) Muy en En De Muy de No sabe desacuerdo desacuerdo acuerdo acuerdo (NO LEER) Considero que tengo un buen entendimiento de los acontecimientos políticos importantes a los 1 2 3 4 98 que se enfrenta nuestro país La gente como yo puede influenciar lo que hace 1 2 3 4 98 el gobierno. LEER: Ahora, con la finalidad de agrupar sus respuestas con las de otras personas de similares características a las de usted, nos gustaría que responda a las siguientes preguntas: 10. ¿En qué año nació? (CUATRO DÍGITOS): _____ _____ _____ _____ 11. (MOSTRAR TARJETA P11) ¿Cuál es el máximo grado de instrucción alcanzado por Ud.? (RESPUESTA ÚNICA) Ninguno / Analfabeto 1 Secundaria completa 5 Superior universitaria completa 9 Primaria incompleta 2 Superior técnica incompleta 6 Post grado 10 Primaria completa 3 Superior técnica completa 7 Secundaria incompleta 4 Superior universitaria incompleta 8 12. ¿Usted o algún miembro de su familia posee los siguientes artículos? (LEER OPCIONES, RESPUESTA ÚNICA POR FILA) Sí No Carro / Todoterreno / Camioneta / Motocicleta 1 2 Refrigerador en buen estado, es decir, que funcione. 1 2 Aire acondicionado 1 2 Computadora / Laptop 1 2 Teléfono celular inteligente / Smartphone 1 2 13. (MOSTRAR TARJETA P13) ¿Con qué frecuencia utiliza Internet? (RESPUESTA ÚNICA) A diario / todos los días (IR A P14) 1 Menos de una vez al mes (IR A P13a) 4 Alguna vez a la semana (IR A P14) 2 No usa internet (IR A P15) 97 Alguna vez al mes (IR A P14) 3 No sabe (NO LEER) (IR A P15) 98 13a. (APLICAR SÓLO SI CONTESTÓ COD. 4 EN P12) ¿Cuándo utilizó Internet por última vez? (RESPUESTA ÚNICA) _________________________________________________________________________________________ _____________________ 14. ¿Cuáles de las siguientes actividades ha realizado en Internet durante los últimos 12 meses? (LEER OPCIONES, RESPUESTA MÚLTIPLE) Mirar el precio de un producto agrícola / 1 Revisar su correo electrónico o e-mail 4 ganado Leer información sobre algún proyecto del 2 Revisar su cuenta de Facebook 5 Proyecto de Alianzas Rurales Otro (NO LEER, ESPECÍFICA) Leer las noticias 3 _________________________ 15. (APLICAR A TODOS) En una semana normal, ¿aproximadamente, cuántos mensajes de texto envía en promedio? (ESPONTÁNEA, RESPUESTA ÚNICA) 1 a 10 1 No envía mensajes de texto 4 11 a 20 2 No sabe como enviar mensajes de texto 5 Más de 20 3 No tiene celular 6 USO ENCUESTADOR GÉNERO NOMBRE DE LA ALIANZA A LA QUE PERTENECE – USO ENCUESTADOR Masculino 1 ESCRIBIR EL NOMBRE DE LA ALIANZA: ___________________________________________________________________________ Femenino 2 DATOS DEL ENTREVISTADO - USO ENCUESTADOR (LLENAR AL FINAL DE LA ENTREVISTA) (LEER) Para terminar con la entrevista, quisiera por favor me proporcione los siguientes datos para que el supervisor verifique la correcta realización de mi trabajo. NOMBRE DEL ENTREVISTADO: ………...………………………………………………………………………………………………………… TELÉFONO FIJO: |__|__|__|__|__|__|__| TELÉFONO CELULAR: |__|__|__|__|__|__|__|__| DIRECCIÓN: …………………………………………………………………………………………………………………………………………… ¡Muchas gracias por su colaboración! DATOS DEL EQUIPO DE IPSOS NOMBRE LOCALIDAD:……………………………………………………………………………………………………… ………………………… NOMBRE DEL COD ENC |__|__|- ENCUESTADOR:……………………………………………………………… |__|__|__|__| ……… NOMBRE DEL SUPERVISOR COD SUP DIF|__|__|- DIF:…………………………………………………………………… |__|__|__|__| TIPO DE DIFERIDA: 1 COINCIDENTAL: 2 SUPERVISIÓN: Survey of PAR beneficiaries - Methodological report (Report prepared by Ipsos Bolivia, hired for the carrying out the sampling and data collection for the survey) In May-June 2014 Ipsos Bolivia carried out the Survey with PAR (Rural Alliances Project) beneficiaries in Santa Cruz, Bolivia. The survey was carried out in 33 Rural Alliances. The main objective of the study was to understand the reasons for using or not a system of feedback from project beneficiaries, through mobile phone. This objective was accomplished by means of face-to-face household interviews among the beneficiaries of the selected rural alliances. Respondents were selected by random contact through listing method. The survey took place from May 24th to June 30th 2014. PROJECT PERSONNEL These were the key personnel in charge of the project:  Alejandra Candia, Director of Ipsos Shopper and Loyalty  Patricia Vargas, Research Analyst  Daniel Loza, Research Executive  Gastón Paredes, IT (Smartphone survey support)  Paula Soria, Head of Operations The questionnaire was provided by the client in Spanish. Ipsos was responsible for a fully formatted into electronic questionnaire for Smartphones. SAMPLING DESIGN DESCRIPTION Universe: The study investigated the main beneficiaries of PAR in Santa Cruz, Bolivia. Sampling frame: The sampling frame was built using data provided by the client; it contains 5992 beneficiaries named primary sampling unit. These beneficiaries are distributed into 141 rural alliances. The sampling frame consisted of a list of rural alliances and their locations in rural areas. It identified: State, Municipal Section and Locality. Number of Number of Number of Rural Total Number of State or Province Municipal Localities Alliances Beneficiaries Sections Andres Ibañez 3 28 33 1.143 Chiquitos 1 2 2 75 Guarayos 3 12 12 461 Ichilo 4 36 43 2.202 Manuel Maria Caballero 2 9 15 630 Ñuflo de Chavez 5 21 21 853 Obispo Santistevan 2 3 3 127 Sara 3 6 6 194 Vallegrande 1 1 5 182 Velasco 1 1 1 125 25 119 141 5.992 Source of above table: Based on list provided by the client. Final sampling frame, once fieldwork started and the provided list was depurated: Number of Number of Number of Rural Total Number of State or Province Municipal Localities Alliances Beneficiaries Sections Andres Ibañez 3 12 14 397 Guarayos 1 1 1 23 Ichilo 3 21 16 817 Manuel Maria Caballero 2 11 12 561 Ñuflo de Chavez 4 10 7 272 Obispo Santistevan 1 1 1 32 Sara 2 5 5 179 Vallegrande 1 1 2 82 Velasco 1 1 1 125 18 63 59 2488 Source of above table: Based on list provided by the client. Units of analysis and sampling: The unit of analysis was the beneficiary in a rural alliance; the sampling units were rural alliances. Sampling technique and selection method: The technique applied corresponds to a three-stage stratified sampling method: Table 4: Summary of Strata, Stages and method of selection Strata/Stage Description Method of Selection Implicit Strata Department Strata Rural Alliances Randomly First Stage Province, Municipal Section and Proportional dispersion Locality Second Stage Beneficiaries at rural Alliance Proportional to universe Last Stage Main beneficiaries Systematic with random start for the first beneficiary Due to the reduced number of Rural Alliances, a random selection within 141 alliances was conducted with a total of 46 selected Rural Alliances. In our experience, while working with a national representative sample; 40 localities are more than enough for national study coverage. According to this, 46 alliances were randomly selected in order to provide a representative sample to a universe of 141 alliances. Also the client’s budget and timeframe were taken into consideration in order to have equal weighting between representativeness, timings and budget. Nevertheless, once fieldwork started, Ipsos had some difficulties due to the lack of updated information on the provided list, 82 alliances were discarded during the telephone contacts and visits to localities. This represents a 58% of the sampling units, leaving a 42% of available alliances to contact. The final sampling frame was built using the remained available data, which contained 2488 available beneficiaries named primary sampling unit. These beneficiaries were distributed into 59 rural alliances. Also a proportional dispersion of provinces and municipal sections were taken into consideration during the random selections of Rural Alliances, in order to have a representative sample among Rural Alliances. The list of Rural Alliances selected for the proposal can be found below: T o tal M UE S T R A C o digo A lia nza N o m bre A lia nza R ura l R ubro A lia nza P ro v inc ia M unic ipio Lo c a lida d B e ne f ic ia rio s A J US T A D A P la n SCZ-1755-009-09 A so ciació n de P ro ducto res de Sésamo Cuatro Cañadas A gríco la (A gricultura) Ñuflo de Chavez Cuatro Cañadas 12 de Octubre y 25 de M ayo 111 30 SCZ-1712-040-07 Co mercializacio n de A rro z "15 de A go sto " A gríco la (A gricultura) Ichilo Yapacaní (Villa B usch) 15 de A go sto 61 16 SCZ-1711-092-07 Co mercializacio n de Carne de Res P ecuario (Ganadería) Ichilo San Carlo s A nto fagasta 76 20 SCZ-1745-136-07 Latco Internatio nal Rio Chico A gríco la (A gricultura) Guarayo s A scenció n (Guarayo s) Rio Chico 56 15 SCZ-1712-044-09 Lecheria Central San German P ecuario (Ganadería) Ichilo Yapacaní (Villa B usch) Central San German Km 27 Faja Central 59 15 SCZ-1742-101-09 P ro ducció n de P apa en Chacarilla A gríco la (A gricultura) M anuel M aria Caballero Saipina Chacarilla 20 5 SCZ-1712-036-09 A so ciació n de M ujeres Emprendedo ras Central San Rafael P ecuario (Ganadería) Ichilo Yapacaní (Villa B usch) Co munidad San Rafael Zo na Sur 25 16 SCZ-1712-003-07 Co mercializacio n de A rro z "Cho re Víbo ra" A gríco la (A gricultura) Ichilo Yapacaní (Villa B usch) Cho re Víbo ra 103 27 SCZ-1741-100-09 A so ciació n Co marapeña de Fruticulto res - A COFRUT A gríco la (A gricultura) M anuel M aria Caballero Co marapa Co marapa 52 14 SCZ-1705-031-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s Cafetal M o nte Verde A gríco la (A gricultura) A ndres Ibañez El To rno Co munidad Cafetal mo nte Verde 33 9 SCZ-1738-105-07 A so ciacio n de P equeño s P ro ducto res Cristo Rey P ecuario (Ganadería) Ñuflo de Chavez San Javier (Ñ.de Chavez) Cristo Rey 38 10 SCZ-1710-003-06 A P ROA SA - A GRICA B V A gríco la (A gricultura) Ichilo B uena Vista Distrito Surutu A ntacawa-San Carlo s 78 20 SCZ-1756-083-09 A so ciació n de P ro ducto s A píco las El Cedro A picultura Sara Co lpa B élgica El Cedro 15 4 SCZ-1738-099-07 P ro ducto res de Leche Jesús Nazareno P ecuario (Ganadería) Ñuflo de Chavez San Javier (Ñ.de Chavez) El Regreso 27 7 SCZ-1705-034-08 A so ciació n de Fruticulto res Espejo A FES A gríco la (A gricultura) A ndres Ibañez El To rno Espejo 30 8 SCZ-1705-117-07 A so ciació n de P ro ducto res de Ganado de Leche P ecuario (Ganadería) A ndres Ibañez El To rno Espejo s 20 5 SCZ-1705-075-07 Leche La Fo restal P ecuario (Ganadería) A ndres Ibañez El To rno Fo restal 43 11 SCZ-1705-066-07 El P o rvenir A gríco la (A gricultura) A ndres Ibañez El To rno Fo restal - San P edro 54 14 SCZ-1737-043-08 A so ciació n de P equeño s P ro ducto res A gro pecuario s 13 de mayo P ecuario (Ganadería) Ñuflo de Chavez Co ncepció n Guayaba, Embo cada, A ltamira 18 5 SCZ-1710-032-08 Co mercializació n de Leche Huaytú - P IL A ndina P ecuario (Ganadería) Ichilo B uena Vista Huaytú 55 14 SCZ-1739-142-07 A so ciació n de A griculto res "21de A go sto " Cultivo de M aíz San Julian A gríco la (A gricultura) Ñuflo de Chavez San Julian Illimani 35 9 SCZ-1705-087-07 "Junta P irai" P ecuario (Ganadería) A ndres Ibañez El To rno Junta P irai 25 7 SCZ-1704-110-07 P ro ducto res de M ani - To to rales A gríco la (A gricultura) A ndres Ibañez La Guardia La Guardia 28 14 SCZ-1704-111-07 A so ciació n de A viculto res 23 de M ayo P ecuario (Ganadería) A ndres Ibañez La Guardia La Guardia 26 SCZ-1739-009-08 P ro ducciò n y Co mercializaciò n de Frejo l Negro A gríco la (A gricultura) Ñuflo de Chavez San Julian Nucleo 23 25 7 SCZ-1705-057-07 A so ciacio n de P equeño s P ro ducto res de Ganado "A SOP EGA " P ecuario (Ganadería) A ndres Ibañez El To rno Nueva Esperanza 54 14 SCZ-1705-088-07 A so ciació n de pequeño s P ro ducto res de ganado "Lo s Ho rno s" P ecuario (Ganadería) A ndres Ibañez El To rno P aso del Chivo , V. San Carlo s, Villa Tumabi 53 14 SCZ-1712-004-07 Co mercializació n de Leche "P uerto A varó a" P ecuario (Ganadería) Ichilo Yapacaní (Villa B usch) P uerto A varó a 62 16 SCZ-1712-020-08 Co mercializacio n de Huevo s de Gallinas Crio llas P uerto A varo a Km 40 P ecuario (Ganadería) Ichilo Yapacaní (Villa B usch) P uerto A varo a Km 40 52 14 SCZ-1717-063-07 KUSCA WIÑA N CHEG P ecuario (Ganadería) Sara Santa Ro sa Santa Ro sa 36 9 SCZ-1745-085-09 A so ciació n de P equeño s P ro ducto res A mazo nas Rio B lanco A gríco la (A gricultura) Guarayo s A scenció n (Guarayo s) Rio B lanco 21 5 SCZ-1754-055-09 A P EM A SCO-A so ciació n de P equeño s y M ediano s A gro pecuario s A gríco la (A gricultura) Obispo Santistevan San P edro (Santistevan) Sagrado Co razó n 32 8 SCZ-1711-076-08 P ro ducto res Lechero s San Carlo s P ecuario (Ganadería) Ichilo San Carlo s San Carlo s 17 4 SCZ-1710-076-07 P ro ducció n de Leche San Isidro y San M iguel P ecuario (Ganadería) Ichilo B uena Vista San Isidro 37 10 SCZ-1742-054-08 Warmis Llankado ras de Runtus P ecuario (Ganadería) M anuel M aria Caballero Saipina San Rafael 23 6 SCZ-1717-002-09 Lecheria Santa Ro sita P ecuario (Ganadería) Sara Santa Ro sa Santa Ro sa del Sara 34 9 SCZ-1725-058-08 A rea P ro ductivas La VID A gríco la (A gricultura) Vallegrande Vallegrande Vallegrande 26 SCZ-1725-059-08 A so ciació n de A piculto res de Vallegrande A picultura Vallegrande Vallegrande Vallegrande 35 22 SCZ-1725-074-08 Co o perativa A gro pecuaria "Sr. De M alta" Ltda. A gríco la (A gricultura) Vallegrande Vallegrande Vallegrande 24 SCZ-1741-059-07 A P ROSEM CO - San Jo se del No rte y M o ntero . A gríco la (A gricultura) M anuel M aria Caballero Co marapa Verdecillo s P ulquina, Co marapa. 38 10 SCZ-1745-137-07 A so ciació n de trabajado res A gro pecuario s Campesino s Villa Fátima A gríco la (A gricultura) Guarayo s A scenció n (Guarayo s) Villa Fatima 43 11 SCZ-1705-068-07 Nuevo A manecer de lo s A piculto res de Villa Flo rida A picultura A ndres Ibañez El To rno Villa Flo rida 20 5 SCZ-1739-017-07 A so ciacio n De pequeño s P ro ducto res A gro pecuario s del Oriente A P P A O A gríco la (A gricultura) Ñuflo de Chavez San Julian villa P araiso - B recha area 5 - Villa Victo ria 40 10 SCZ-1704-112-07 P ro ducció n Lechera "Lo s Co lo no s" P ecuario (Ganadería) A ndres Ibañez La Guardia Villa Ro sario , San Carlo s y San Juan 29 8 SCZ-1705-052-07 A so ciació n de P ro ducto res de M aiz El To rno - A P ROM A A gríco la (A gricultura) A ndres Ibañez El To rno Villa San Carlo s 55 14 SCZ-1712-044-07 Co mpetitividad de La Cadena A pico la Co munitaria Yapacani A picultura Ichilo Yapacaní (Villa B usch) Yapacani 36 9 1.880 500 Women members’ Rural Alliances. Within the universe of beneficiaries of the PAR (5.992) a total number of 362 interviews were estimated to obtain a representative sample with a margin of error of + 5% at a confidence level of 95%. The sample was adjusted into 500 interviews for a lower margin of error: + 4.2% estimated at a confidence level of 95%. Due to the lack of updated information on the provided list, cancellations of appointments by heads of alliances (that discarded 57% of the alliances), and a deadline to complete the project; Ipsos had to recalculate the sample so that it managed to reach the minimum 362 surveys. In order to accomplish this sample, Ipsos increased the number of interviews to be applied in those alliances where an appointment was effective, to reach the minimum of surveys reflecting a margin of error of + 5% at a confidence level of 95%. As a final result, Ipsos was able to achieve 439 effective surveys on 33 visited alliances, this represents a margin of error of + 4,5% at a confidence level of 95%. The final list of Rural Alliances visited and the final sample can be found below: Nº T o tal C ó digo P A R A lia nza N O M B R E D E LA A LIA N Z A R UR A L P R O V IN C IA M UN IC IP IO LO C A LID A D E S B e ne f ic ia rio s M UE S T R A F E C H A D E V IS IT A en B ase P la n A so ciació n Co marapeña de Fruticulto res - M anuel M aria SCZ-1741-100-09 9 Co marapa Co marapa 52 14 Sábado , 24 de M ayo de 2014 A COFRUT Caballero A P EM A SCO-A so ciació n de P equeño s y Obispo San SCZ-1754-055-09 32 Sagrado Co razó n 32 8 Sábado , 24 de M ayo de 2014 M ediano s A gro pecuario s Santistevan P edro (Santistevan) SCZ-1717-002-09 36 Lecheria Santa Ro sita Sara Santa Ro sa Santa Ro sa del Sara 34 9 Lunes, 26 de M ayo de 2014 A so ciació n de P ro ducto res de M aiz El SCZ-1705-052-07 45 A ndres Ibañez El To rno Villa San Carlo s 55 14 M iérco les, 28 de M ayo de 2014 To rno - A P ROM A SCZ-1725-059-08 38 A so ciació n de A piculto res de Vallegrande Vallegrande Vallegrande Vallegrande 35 9 Jueves, 29 de M ayo de 2014 Huaytú a 7 km antes de B uena Vista y Co mercializació n de Leche Huaytú - P IL SCZ-1710-032-08 20 Ichilo B uena Vista lo calidad 55 14 Viernes, 30 de M ayo de 2014 A ndina A rbo lera Yapacaní (Villa SCZ-1712-044-09 5 Lecheria Central San German Ichilo Central San German Km 27 Faja Central 59 15 Sábado , 31de M ayo de 2014 B usch) San Javier (Ñ.de SCZ-1738-099-07 14 P ro ducto res de Leche Jesús Nazareno Ñuflo de Chavez El Regreso 27 7 Sábado , 31de M ayo de 2014 Chavez) SCZ-1705-075-07 17 Leche La Fo restal A ndres Ibañez El To rno Fo restal 43 12 Lunes, 02 de Junio de 2014 Fruta de Nuestra Tierra, de Vallegrande SCZ-1725-062-08 39 (FUNDA CIÓN P A RA EL DESA RROLLO Vallegrande Vallegrande Vallegrande 47 7 M artes, 03 de Junio de 2014 FRUTÍCOLA ) SCZ-1704-058-07 23 A SA P IGUA RDIA -A P IB SA A ndres Ibañez La Guardia La Guardia 27 7 M iérco les, 04 de Junio de 2014 M anuel M aria SCZ-1741-050-09 40 A SOP ROLEC Co marapa Co marapa 29 10 Jueves, 05 de Junio de 2014 Caballero Co mpetitividad de La Cadena A pico la Yapacaní (Villa SCZ-1712-044-07 29 Ichilo Yapacani 36 11 Sábado , 07 de Junio de 2014 Co munitaria Yapacani B usch) Organizació n M ujeres P ro ducto ras SCZ-1755-005-09 19 Ñuflo de Chavez Cuatro Cañadas 4 Cañadas 19 5 Lunes, 09 de Junio de 2014 Co munidad 4 Cañadas SCZ-1704-053-09 24 A so ciació n 25 de Octubre La Guardia A ndres Ibañez La Guardia Labandero 24 10 M iérco les, 11de Junio de 2014 "Las M ujeres Exito sas P ro ducto ras de SCZ-1705-039-07 42 A ndres Ibañez El To rno Jo ro chito 23 10 M iérco les, 11de Junio de 2014 Carne de Cerdo en Jo ro chito " M anuel M aria SCZ-1742-054-08 6 Warmis Llankado ras de Runtus Saipina San Rafael 23 10 Jueves, 12 de Junio de 2014 Caballero A so ciació n de P ro ducto res de Sésamo SCZ-1755-009-09 1 Ñuflo de Chavez Cuatro Cañadas 12 de Octubre y 25 de M ayo 111 30 Viernes, 13 de Junio de 2014 Cuatro Cañadas M anuel M aria SCZ-1742-016-08 35 A SOHFRUT SA IP INA Saipina Saipina 30 15 Viernes, 13 de Junio de 2014 Caballero A so ciació n de P equeño s P ro ducto res A scenció n SCZ-1745-080-08 41 Guarayo s Tacuaral 23 11 Sábado , 14 de Junio de 2014 A gro pecuario s 23 de M arzo "A SOP A GRO" (Guarayo s) SCZ-1705-034-08 10 A so ciació n de Fruticulto res Espejo A FES A ndres Ibañez El To rno Espejo 30 15 Lunes, 16 de Junio de 2014 SCZ-1705-087-07 27 "Junta P irai" A ndres Ibañez El To rno Junta P irai 25 15 M artes, 17 de Junio de 2014 Co mercializacio n de Huevo de Gallina Yapacaní (Villa SCZ-1712-018-08 28 Ichilo Cho re Km 13 Zo na Central 62 25 Viernes, 20 de Junio de 2014 Crio lla Cho re Km 1 3 Zo na Central B usch) A so ciacio n de P ro ducto res de Ganado "10 SCZ-1705-067-07 22 A ndres Ibañez El To rno B elen 24 10 Sábado , 21de Junio de 2014 de A bril" Do mingo , 22 de Junio de 2014 SCZ-1707-071-08 37 M inga - Naturalia "A lmendra Chiquitana" Velasco San Ignacio Co munidades alrededo r de San Ignacio 125 30 Lunes, 23 de Junio de 2014 San Javier (Ñ.de SCZ-1738-096-07 11 Ganado de Engo rde Santa Rita Ñuflo de Chavez Santa Rita 24 10 Lunes, 23 de Junio de 2014 Chavez) A so ciació n de P equeño s P ro ducto res SCZ-1737-043-08 43 Ñuflo de Chavez Co ncepció n Guayaba, Embo cada, A ltamira 18 10 M artes, 24 de Junio de 2014 A gro pecuario s 13 de mayo A so ciació n de A griculto res "21de A go sto " SCZ-1739-142-07 25 Ñuflo de Chavez San Julian Illimani 35 12 M iérco les, 25 de Junio de 2014 Cultivo de M aíz San Julian Co mercializacio n de A rro z co n Equidad _ Yapacaní (Villa SCZ-1712-060-07 34 Ichilo Ichilo 85 23 M iérco les, 25 de Junio de 2014 A P ROGRA I B usch) A so ciació n de P equeño s P ro ducto res SCZ-1717-096-09 30 Sara Santa Ro sa Co munidad La P lanchada 1 54 20 Jueves, 26 de Junio de 2014 A gro pecuario s La P lanchada 1 P o ro ngo SCZ-1703-007-09 18 Granja de Chancho s P o ro ngo A ndres Ibañez El Limó n 24 11 Sábado , 28 de Junio de 2014 (A yacucho ) SCZ-1711-076-08 33 P ro ducto res Lechero s San Carlo s Ichilo San Carlo s San Carlo s 27 10 Sábado , 28 de Junio de 2014 A so ciacio n de P equeño s P ro ducto res San Javier (Ñ.de SCZ-1738-105-07 13 Ñuflo de Chavez Cristo Rey 38 20 Do mingo , 29 de Junio de 2014 Cristo Rey Chavez) 1.3 5 5 439 A lia nza s rura le s de m uje re s For the selection of respondents, initially there was a first telephone contact with the heads of the Rural Alliances. • The Rural Alliance was replaced if the phone number called was busy, and only after 10 missed call attempts at least. • The Rural Alliance was replaced when the contact phone number was not available, and only after 3 attempts at least. • If a phone number did not correspond to the universe, the client was informed in order to remove the number from the base or in order for the client to provide an updated phone number. To replace the discarded Rural Alliance by any of the reasons described above, the non selected Rural Alliances were used. Subsequently, we checked for the province, the municipality and the number of beneficiaries of the discarded Rural Alliance to choose a replacement that met the same features and proceeded to contact the heads of those alliances, using the following methodology: Replacement of Alliances by Listing Contacts marked in blue were the first selected (those who meet the requirements to replace the discarded alliance). If rejected, we continued with the following alliance of the list until effective contact with the head of alliance was achieved. Source of above table: Based on previous studies. When telephone contacts with the heads of alliances were effective, we briefly explained the reason for the call, requiring them to prepare a list of all members of the alliance and we programmed a date for the interviewers to visit them at the locality. Once in the locality, our staff contacted the head of the alliance first, in order to give him or her, a letter (provided by Ipsos and previously approved by the client) explaining the reason of our presence and to ask for the list of all the members. Subsequently, with the list of all partners provided by the head of alliance, our staff firstly selected the potential interviewees, according to the methodology outlined below. Surveys by Listing Contacts marked in blue were the first selected. If rejected, the team continued with the following person on the list until an effective contact interview was achieved. Source of above table: Based on previous studies. Quotas: The fieldwork conducted applied soft gender quotas when possible. The lack of knowledge of the gender distribution within each alliance, could not ensure strict enforcement of quotas. REPRESENTATIVENESS OF THE SAMPLE The final design represents 54% of the total beneficiaries available of the 59 Rural Alliances left (2.488 available beneficiaries) once 82 alliances were discarded due to lack of updated information (wrong numbers, non responses, non existing alliances and cancelled visits). CHARACTERISTICS OF FIELD FORCE In this project, 11 interviewers and 12 supervisors were able to work in the project. These were mostly part-time university students or adults with a technical or university diploma, who were selected based on a two-stage process: a personal interview and an evaluation of logical reasoning. No junior personnel were allowed to participate, due to the interviewee selection methodology. Only interviewers and supervisors that had 1 or more years working at the company were able to participate of the project. Also 80% to 90% of them had experience with prior studies that inclued the selection methodology of interviewees. The list of characteristics of interviewers can be found below: GENDER AGE EDUCATION INTERVIEWERS NON QUANTITY 18 to 30 31 to 55 UNIVERSITY UNIVERSITY CATEGORY MALE FEMALE UNIVERSITY years old years old STUDENTS DEGREE DEGREE Interviewer - Junior 0 0 0 0 0 0 0 0 Interviewer 11 6 5 7 4 6 2 3 The list of characteristics of supervisors can be found below: GENDER AGE EDUCATION SUPERVISORS NON QUANTITY 18 to 30 31 to 55 UNIVERSITY UNIVERSITY CATEGORY MALE FEMALE UNIVERSITY years old years old STUDENTS DEGREE DEGREE Supervisor - Junior 0 0 0 0 0 0 0 0 Supervisor 12 7 5 8 4 5 3 4 Ipsos was responsible for carrying out the local training of interviewers and supervisors. The primary trainer was the Ipsos Research Executive in charge of the project (Daniel Loza); this primary trainer personally conducted the first training to interviewers and to the secondary trainer (the local field coordinator). The training provided to interviewers was two-fold. On the first hand, when interviewers started working with Ipsos on any project, they were trained on how to read and follow the route card instructions in order to conduct their route adequately. When being trained on a specific project, this training is repeated to ensure that all interviewers follow route card instructions correctly. In the second part of the training (the order is indifferent), the Research Executive in charge of the study, or the secondary trainers explained the general objective and nature of the study. The sampling instructions were clarified in terms of the skip to be applied between contacts of the list provided by the head of alliance, interviewee selection, and interviewee replacement. Then a detailed training was conducted on the application of the survey itself. This was done using a projection of the survey on the wall/board or by using Smartphones and asking the interviewers to read and apply the questions out loud to the trainer. This made the training session more interactive and allowed the trainer to “set up possible unexpected or biased answers” for the interviewers, giving incomplete or inaccurate answers to provoke the interviewer to react and ask the “pretend interviewee” (trainer) to clarify his answer. In other cases the trainer gave an accurate answer but difficult to code in the response table, to check whether interviewers understood how to register the answer given by the interviewee. Special emphasis was placed on the application of filters, consistencies and any other complex question. During the training session every interviewer had the smartphone survey, a printed copy of the questionnaire and the showcards in front of them to be able to take notes and ask questions about the survey. At the end of the training session, the trainer asked the interviewers to conduct a “pilot test” amongst themselves, allowing them to practice the application of the survey and allowing the trainer to resolve any doubts that may arise from the application of the survey. If the trainer detected any interviewers who did not fully grasp the correct application method of the survey, the trainer decided to re-train them or to leave them off the team for the specific project. FIELDWORK PROCEDURES Fieldwork was conducted Monday to Sunday with a focus on Monday to Saturday; between 6am or 7am to 8pm. Considering the nature of the study, Ipsos considered it was not necessary to provide incentives to respondents. QUALITY CONTROL 100% of each interviewer’s production was directly observed (accompanied supervisions); additional back-checking was done by telephone where there was telephone service. DATA PROCESSING Data coding: The open-ended responses were prepared for data entry by elaborating a codeframe. Data entry control: A data entry software was developed based on the final questionnaire. This data entry software had controls which allow to minimize errors in the data entry process. Furthermore, during all data entry process, the research team in charge did daily verification the data in order to control progress at field. Data checking: Once the database was complete, the data was thoroughly checked to control any filters and inconsistencies in the data. The data checking was carried out by the programmer in charge of creating the data entry software together with the Executive Analyst and Research Asistent in charge of the study. SAMPLE SIZE The final data base included 439 interviews. INTERVIEW LENGTH The average length of the field survey was of 10 minutes. PROBLEMS ENCOUNTERED All problems encountered during fieldwork were reported to the client. 82 Alliances were discarded due to lack of updated information from May 24th to June 27th. a) Wrong phone numbers b) Non response at phone numbers c) Non available phone numbers d) Incomplete phone numbers e) Blocked or out of service phone numbers f) Non existent or dissolved alliances Also PAR members helped update some of the phone numbers during the first week of June, but it couldn’t prevent 58% of the sampling frame being discarded. See below the list of all 82 alliances discarded: T o tal C o digo A lia nza N o m bre A lia nza R ura l P ro v inc ia M unic ipio Lo c a lida d B e ne f ic ia rio s O B S E R V A C IO N E S P la n SCZ-1753-013-08 A P ROCH-KA NDIRE Obispo Santistevan Fernandez A lo nso Cho re Independencia 70 A P A GA DO SCZ-1712-004-07 Co mercializació n de Leche "P uerto A varó a" Ichilo Yapacaní (Villa B usch) P uerto A varó a 62 A P A GA DO SCZ-1712-003-07 Co mercializacio n de A rro z "Cho re Víbo ra" Ichilo Yapacaní (Villa B usch) Cho re Víbo ra 103 A P A GA DO SCZ-1750-064-08 Flo r del Oriente Ñuflo de Chavez San Ramó n San Ramó n 66 A P A GA DO SCZ-1712-056-09 P ro ducto res de leche fresca Ichilo Yapacaní (Villa B usch) Distrito 7 San Germán 43 A P A GA DO SCZ-1712-002-07 Co mercializacio n de Leche Zo na Central Ichilo Yapacaní (Villa B usch) San German 60 A P A GA DO SCZ-1714-043-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s El P ro greso - A A P Chiquito s P ailó n El Tuná No rte 30 A P A GA DO Y EQUIVOCA DOS SCZ-1747-130-07 A so ciació n de P ro ducto res Surukusi El P uente “ A P ROSUP ” Guarayo s El P uente Surucusi 75 HA CE 1A ÑO QUE YA NO EXITE LA A LIA NZA SCZ-1745-085-08 A so ciació n de P ro ducto res de P iña Guarayo s "A P IGUA " Guarayo s A scenció n (Guarayo s) A scencio n 58 HA CE 2 A ÑOS QUE YA NO EXISTE LA A LIA NZA SCZ-1705-120-07 A so ciació n de P equeño s P ro ducto res de Ganado "Villa P araiso " A ndres Ibañez El To rno Villa P araiso 25 HA CE 2 A ÑOS QUE YA NO TRA B A JA N CON P A R SCZ-1712-020-08 Co mercializacio n de Huevo s de Gallinas Crio llas P uerto A varo a Km 40 Ichilo Yapacaní (Villa B usch) P uerto A varo a Km 40 52 HA CE 2 O 3 A ÑOS QUE YA NO FUNCIONA LA A LIA NZA INA CCESIB LE P OR EL CA UDA L DEL RÍO EN ESTA ÉP OCA DEL SCZ-1711-017-08 A so ciació n de P ro ducto res de M iel Nativa -A P ROM IN Ichilo San Carlo s Santa Fe 35 A ÑO P ODRÍA SER P OR TELÉFONO SCZ-1738-100-07 P ro ducto res de Carne de Cerdo s Turux Napez Ñuflo de Chavez San Javier (Ñ.de Chavez) Turux Napez 15 NO RESP ONDE SCZ-1738-106-07 P ro ducto res de Leche, A so ciacio n El Rancho Ñuflo de Chavez San Javier (Ñ.de Chavez) El Rancho 20 NO RESP ONDE SCZ-1705-088-07 A so ciació n de pequeño s P ro ducto res de ganado "Lo s Ho rno s" A ndres Ibañez El To rno P aso del Chivo , V. San Carlo s, Villa Tumabi 53 NO RESP ONDE SCZ-1739-008-08 A so ciacio n Integral de P ro ducto res A gro pecuario s de San Julian (A IP A S) Ñuflo de Chavez San Julian Nucleo 26 "Villa Co to ca" 65 NO RESP ONDE SCZ-1704-111-07 A so ciació n de A viculto res 23 de M ayo A ndres Ibañez La Guardia La Guardia 26 NO RESP ONDE SCZ-1704-114-07 Lechería "Naranjillo s" A ndres Ibañez La Guardia Naranjillo s 25 NO RESP ONDE SCZ-1705-056-07 A so ciacio n de P equeño s P ro ducto res de Ganado "A P ROGA " El P acay A ndres Ibañez El To rno El P acay 40 NO RESP ONDE SCZ-1705-074-07 A so ciacio n So lidaria La P urita A ndres Ibañez El To rno Villa Esperanza 74 NO RESP ONDE SCZ-1705-108-07 A so ciació n de P o llo s P arrillero s 12 de A bril A ndres Ibañez El To rno Santa Rita 31 NO RESP ONDE SCZ-1710-047-07 A P A FECSA - A GRICA B U Ichilo B uena Vista Carmen Surutu 79 NO RESP ONDE SCZ-1711-083-08 Unió n de Co munidades de Trabajo A nto fagasta Ichilo San Carlo s A nto fagasta 42 NO RESP ONDE SCZ-1712-008-07 Co mercializació n de Leche "Central Lito ral" Ichilo Yapacaní (Villa B usch) Central Lito ral 79 NO RESP ONDE SCZ-1712-012-08 P ro ducto res de San Isidro Ichilo Yapacaní (Villa B usch) San Isidro 37 NO RESP ONDE SCZ-1738-030-08 A so co ació n A gro pecuaria Familias Unidas (A SA FU) Ñuflo de Chavez San Javier (Ñ.de Chavez) B ella Vista 36 NO RESP ONDE SCZ-1739-007-08 A so ciacio n de P ro ducto res A gro pecuario s "Lo s Co ndo res", Ñuflo de Chavez San Julian Nucleo 24 "M o nte Rico " 41 NO RESP ONDE SCZ-1747-048-07 Lechería Do rka Guarayo s El P uente El P uente 40 NO RESP ONDE SCZ-1725-074-08 Co o perativa A gro pecuaria "Sr. De M alta" Ltda. Vallegrande Vallegrande Vallegrande 24 NO RESP ONDE SCZ-1712-001-07 Co mercializació n de So ya Faja No rte Ichilo Yapacaní (Villa B usch) Yapacani 69 NO RESP ONDE SCZ-1711-078-08 A lianza Lechera Divino Niño Ichilo San Carlo s B uen Retiro 32 NÚM ERO B LOQUEA DO SCZ-1705-068-07 Nuevo A manecer de lo s A piculto res de Villa Flo rida A ndres Ibañez El To rno Villa Flo rida 20 NÚM ERO B LOQUEA DO SCZ-1714-079-09 A so ciació n de P equeño s P ro ducto res A ro pecuario s El Ro sal Centro Chiquito s P ailó n El Ro sal Centro 45 NÚM ERO B LOQUEA DO SCZ-1712-037-09 Central M ueler Co ndo r P ro ducto ra de Leche Ichilo Yapacaní (Villa B usch) Central Co ndo r 30 NÚM ERO B LOQUEA DO SCZ-1712-072-09 Co mercializació n de Carne de P escado "B o livar" Ichilo Yapacaní (Villa B usch) B o livar 42 NÚM ERO B LOQUEA DO SCZ-1738-101-07 P ro ducto res de Ganado de Engo rde 20 de Diciembre Ñuflo de Chavez San Javier (Ñ.de Chavez) Zo na Sur 28 NÚM ERO EQUIVOCA DO SCZ-1752-067-09 Carne de Rés Ichilo Co lo nia Japo nesa San Juan San Juan 30 NÚM ERO EQUIVOCA DO SCZ-1703-082-07 Sindicato A grario El Cho rito 2 de A go sto A gua Dulce A ndres Ibañez P o ro ngo (A yacucho ) A gua Dulce 16 NÚM ERO EQUIVOCA DO SCZ-1703-054-07 A gro pecuaria "P atrio ta" A ndres Ibañez P o ro ngo (A yacucho ) A ndrés Ibañez 27 NÚM ERO EQUIVOCA DO SCZ-1739-143-07 P ro ducto res de Sesamo 12 de M ayo (San Julian) Ñuflo de Chavez San Julian El P o rvenir 57 NÚM ERO EQUIVOCA DO SCZ-1739-141-07 A so ciació n de P ro ducto res A gro pecuario s 15 de M ayo Ñuflo de Chavez San Julian Co munidad Huracan 66 NÚM ERO EQUIVOCA DO SCZ-1705-066-07 El P o rvenir A ndres Ibañez El To rno Fo restal - San P edro 54 NÚM ERO EQUIVOCA DO SCZ-1703-075-08 P ro ducció n y Venta de de Leche P o ro ngo A ndres Ibañez P o ro ngo (A yacucho ) P o ro ngo 65 NÚM ERO EQUIVOCA DO SCZ-1712-040-07 Co mercializacio n de A rro z "15 de A go sto " Ichilo Yapacaní (Villa B usch) 15 de A go sto 61 NÚM ERO EQUIVOCA DO SCZ-1746-108-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s Urubicha Guarayo s Urubichá Urubichá 31 NÚM ERO EQUIVOCA DO SCZ-1704-112-07 P ro ducció n Lechera "Lo s Co lo no s" A ndres Ibañez La Guardia Villa Ro sario , San Carlo s y San Juan 29 NÚM ERO EQUIVOCA DO SCZ-1705-053-07 A so c. de P eq. P ro d. de Ganado de Do ble P ro p. "A P GA " A ndres Ibañez El To rno Villa San Carlo s 80 NÚM ERO EQUIVOCA DO SCZ-1710-062-07 A P A FECH - A GRICA B V Ichilo B uena Vista Espejito s 58 NÚM ERO EQUIVOCA DO SCZ-1725-002-08 Desarro llo de la Ganaderia en Vallegrande Vallegrande Vallegrande Vallegrande 50 NÚM ERO EQUIVOCA DO SCZ-1742-052-08 A so ciació n de Ganadero s de Saipina "A SOGA SA " M anuel M aria Caballero Saipina Saipina 30 NÚM ERO EQUIVOCA DO SCZ-1745-042-09 A so ciació n de pequeño s pro ducto res A gro pecuario s Jesus Nazareno Guarayo s A scenció n (Guarayo s) A scensió n de Guarayo s 18 NÚM ERO EQUIVOCA DO SCZ-1739-144-07 A so ciació n de P equeño s P ro ducto res "SINCHIHUA YRA " de M aíz Ñuflo de Chavez San Julian San M artin 44 NÚM ERO EQUIVOCA DO SCZ-1745-006-08 A lianza de A so ciacio n de P equeño s Fruticulto res de Guarayo s "A A P FG" Guarayo s A scenció n (Guarayo s) San Francisco de A sis 34 NÚM ERO EQUIVOCA DO SCZ-1745-115-07 P ro yecto de Ganado de Engo rde po r la A so ciacio n A gro pecuaria 16 de Julio Guarayo s A scenció n (Guarayo s) Cerro Grande 18 NÚM ERO EQUIVOCA DO SCZ-1711-003-08 A so ciacio n de P equeño s P ro ducto res Ganado Vacuno Ichilo San Carlo s A nto fagasta 30 NÚM ERO EQUIVOCA DO SCZ-1711-092-07 Co mercializacio n de Carne de Res Ichilo San Carlo s A nto fagasta 76 NÚM ERO EQUIVOCA DO SCZ-1705-152-07 A lianza de Cítrico s Leó n A ndres Ibañez El To rno Quebrada Leo n 24 NÚM ERO EQUIVOCA DO SCZ-1705-006-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s "El P auro " A ndres Ibañez El To rno El To rno 40 NÚM ERO EQUIVOCA DO SCZ-1705-057-07 A so ciacio n de P equeño s P ro ducto res de Ganado "A SOP EGA " A ndres Ibañez El To rno Nueva Esperanza 54 NÚM ERO EQUIVOCA DO SCZ-1752-024-09 A so ciació n de mujeres las Virtuo sas Ichilo La Enco nada Co lo nia Japo nesa San Juan 14 NÚM ERO INCOM P LETO SCZ-1739-027-07 P ro duccio n Venta de Co o perativistas Ñuflo de Chavez San Julian El Carmen 58 NÚM EROS B LOQUEA DOS/EQUIVOCA DO SCZ-1712-012-07 Co mercializació n de Leche Cho re "San Isidro " Ichilo Yapacaní (Villa B usch) Cho re San Isidro 80 A SOCIA CIÓN ES P A RTE DE LA P URITA HA B LA R CON ELLOS SCZ-1712-064-07 Co mercializado res de Carne de Res Ichilo Yapacaní (Villa B usch) Yapacani 38 YA NO EXISTE LA A SOCIA CIÓN SCZ-1745-136-07 Latco Internatio nal Rio Chico Guarayo s A scenció n (Guarayo s) Rio Chico 56 YA NO TRA B A JA N CON EL P A R SCZ-1705-031-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s Cafetal M o nte Verde A ndres Ibañez El To rno Co munidad Cafetal mo nte Verde 33 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1705-038-09 A so ciació n de P ro ducto res de Cerdo s Espejo s - A P CES A ndres Ibañez El To rno Co munidad Espejo s 30 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1712-029-09 P ro ducto res de carne pura de Cascabel Ichilo Yapacaní (Villa B usch) Central Cascabel 25 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1712-030-09 A so ciació n de M ujeres P ro ducto ras A gro pecuarias "Las Kantutas" Ichilo Yapacaní (Villa B usch) Km 24 FC 22 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1739-009-08 P ro ducciò n y Co mercializaciò n de Frejo l Negro Ñuflo de Chavez San Julian Nucleo 23 25 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1742-063-09 P ro ducció n de Ganado de Carne M anuel M aria Caballero Saipina Chiló n 19 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1742-101-09 P ro ducció n de P apa en Chacarilla M anuel M aria Caballero Saipina Chacarilla 20 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1745-077-09 A so ciació n de P equeño s P ro ducto res A gro pecuario s "Capiata" Guarayo s A scenció n (Guarayo s) Capiata- San Grego rio 44 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1745-085-09 A so ciació n de P equeño s P ro ducto res A mazo nas Rio B lanco Guarayo s A scenció n (Guarayo s) Rio B lanco 21 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1745-137-07 A so ciació n de trabajado res A gro pecuario s Campesino s Villa Fátima Guarayo s A scenció n (Guarayo s) Villa Fatima 43 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1753-003-09 P o ducció n de Cerdo s 19 de A go sto Obispo Santistevan Fernandez A lo nso Chané - M agallanes 25 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1756-083-09 A so ciació n de P ro ducto s A píco las El Cedro Sara Co lpa B élgica El Cedro 15 NÚM ERO EQUIVOCA DO / NO DISP ONIB LE EN P A R SCZ-1712-010-07 Co mercializacio n de Leche Nuevo Ho rizo nte Ichilo Yapacaní (Villa B usch) Nuevo Ho rizo nte 60 NO DISP ONIB LE EN P A R SCZ-1712-028-07 Co mercializació n de So ya Km. 35 Faja No rte Ichilo Yapacaní (Villa B usch) Km. 37 Faja No rte 89 NO DISP ONIB LE EN P A R SCZ-1712-045-09 leche P ura de A yacuchito Ichilo Yapacaní (Villa B usch) Central Lito ral Km 21Faja central 37 NO DISP ONIB LE EN P A R SCZ-1725-058-08 A rea P ro ductivas La VID Vallegrande Vallegrande Vallegrande 26 YA NO EXISTE LA A SOCIA CIÓN SCZ-1739-017-07 A so ciacio n De pequeño s P ro ducto res A gro pecuario s del Oriente A P P A O Ñuflo de Chavez San Julian villa P araiso - B recha area 5 - Villa Victo ria 40 YA NO EXISTE LA A SOCIA CIÓN SCZ-1739-132-07 A briendo Sendas Ñuflo de Chavez San Julian Campesina Flo r del Valle 20 HA CE 3 A ÑOS QUE YA NO EXISTE LA A SOCIA CIÓN During fieldwork 4 head of alliances cancelled the appointment the same day of the visits or asked to re-schedule them, because they were out of the locality, sick or too busy to receive the interviewers. This had an impact on the project’s budget and timings. The following alliances are the ones that re-scheduled the visits one of them in 4 opportunities: T o tal C ÓD . P A R N O M B R E D E LA A LIA N Z A R UR A L P R O V IN C IA M UN IC IP IO LO C A LID A D E S B e ne f ic ia rio s F E C H A D E V IS IT A O B S E R V A C IO N E S P la n Lune s ,2 6 de m a yo de 2 0 14 R EP R OGR A M A D O EN A L M EN OS 4 A so ciació n de M ujeres Emprendedo ras Yapacaní (Villa Co munidad San S á ba do , 3 1 de m a yo de 2 0 14 O P O R T UN ID A D E S Y LA C IT A F UE C A N C E LA D A SCZ-1712-036-09 Ichilo 25 Central San Rafael B usch) Rafael Zo na Sur M ié rc o le s , 0 4 de junio de 2 0 14 P O R LA J E F E D E A LIA N Z A , S O LIC IT Ó S á ba do , 0 7 de junio de 2 0 14 R E P R O G R A M A R P O R Q UIN T A O P O R T UN ID A D Cho re, Naranja., Yapacaní (Villa C A N C E LA D O P O R E L J E F E D E A LIA N Z A , SCZ-1712-089-07 A so ple-La P urita Ichilo P almar, San Germán, 109 V ie rne s , 0 6 de J unio de 2 0 14 B usch) S O LIC IT Ó R E P R O G R A M A R B o livar P uerto Yapacaní (Villa aro ma,Ichilo ,San C A N C E LA D O P O R E L J E F E D E A LIA N Z A , SCZ-1712-002-06 Nasto rr Ltda - A paey Ichilo 50 Lune s , 0 9 de J unio de 2 0 14 B usch) Isidro ,Co ndo r,A varo a S O LIC IT Ó R E P R O G R A M A R ,etc Distrito Surutu C A N C E LA D O P O R E L J E F E D E A LIA N Z A , SCZ-1710-003-06 A P ROA SA - A GRICA B V Ichilo B uena Vista 78 M a rt e s , 10 de J unio de 2 0 14 A ntacawa-San Carlo s S O LIC IT Ó R E P R O G R A M A R T o tal C ÓD . P A R N O M B R E D E LA A LIA N Z A R UR A L P R O V IN C IA M UN IC IP IO LO C A LID A D E S B e ne f ic ia rio s F E C H A D E V IS IT A O B S E R V A C IO N E S P la n SCZ-1725-058-08 Á rea P ro ductiva la VID Vallegrande Vallegrande Vallegrande 26 J ue v e s , 2 9 de M a yo de 2 0 14 LA A LIA N Z A Y A N O E X IS T E A so ciacio n de pequeño s P ro ducto res Villa P araiso - B recha SCZ-1739-017-07 Ñuflo de Chavez San Julian 40 J ue v e s , 0 5 de J unio de 2 0 14 LA A LIA N Z A Y A N O E X IS T E A gro pecuario s del Oriente A P P A O area 5 - Villa Victo ria Campesina Flo r del SCZ-1739-132-07 A briendo Sendas Ñuflo de Chavez San Julian 20 M ié rc o le s , 2 5 de J unio de 2 0 14 S E S E P A R A R O N H A C E 3 A ÑO S Valle At least 3 head of alliances agreed to schedule a visit and to provide the list of members. However, when Ipsos’ interviewers arrived to the localities and asked for the list of members, the head of the alliances said that their asociations dissolved 2 or 3 years ago. This also had an impact on the project’s budget and timings. The following alliances are the ones that agreed to schedule a visit and provide the list of members even when theirs alliance were already dissolved: Further difficulties were faced during fieldwork, that had a direct impact on the project’s timings: a) Bad highways, landslides and rain that involved re-scheduling the visits at the following alliances:  Asociación Comarapeña de Fruticultores – ACOFRUT  Junta Pirai  Asociacion de Productores de Ganado "10 de Abril" b) Long distances between households at localities, that involved extra costs for transportation and a longer timing staying at the locality:  Asociación de Productores de Maíz El Torno – APROMA  Asociación de Apicultores de Vallegrande  Fruta de Nuestra Tierra, de Vallegrande (FUNDACIÓN PARA EL DESARROLLO FRUTÍCOLA)  Competitividad de La Cadena Apicola Comunitaria Yapacani c) Head of alliance and member who live at Santa Cruz capital city an not at the localities. They were interviewed at their households:  Asociación 25 de Octubre La Guardia d) Head of alliance that spoke with a PAR member who did not know about the project and order the head of alliance to not answer or provide any information to Ipsos:  Asople-La Purita GOVERNMENT PERMISSION Government permission was not required for conducting the survey. List of consulted documents List of project documents facilitated by the OnTrack team and partners General project documentation Gigler, B. S. (2013). ICT-Enabled Citizen Feedback Loop [PowerPoint slides]. World Bank Institute, Innovation in Governance Team. World Bank Institute. (2013). OnTrack – Frequently Asked Questions. Innovation in Governance Team. World Bank Institute. (2013). OT finances since FY2012 [Excel spreadsheet]. Innovation in Governance Team. PAR project - Bolivia Endara, A. (2013). Implementation of Citizen Engagement in Operational Projects Replicating the OnTrack Experience [Concept note]. The World Bank. Endara, A. (2013). PAR Deployment Background. The World Bank. Galindo Stenutz, L. (2013). Programa Alianzas Rurales Evaluación y Cierre Fase Piloto Plataforma Empoderar (OnTrack). World Bank Institute. The World Bank. (2005). Project Appraisal Document on a Proposed Credit in the Amount of SDR 18.90 million (US$28.4 Million Equivalent) to the Republic of Bolivia for a Rural Alliances Project. Report No: 32080-BO. The World Bank. (2005). Rural Alliances Project, Bolivia [Project Information Document (PID)]. Report No: 31767. The World Bank. (2011). International Development Association, International Bank for Reconstruction and Development, International Finance Corporation, and Multilateral Investment Guarantee Agency Country Partnership Strategy for the Plurinational State Of Bolivia for the Period FY2012-2015. Report No. 65108-BO. The World Bank. (2012). Documento de Evaluación del Proyecto para un Crédito Propuesto por un Monto de DEG 32,9 Millones (Equivalente a USD 50 Millones) para el Estado Plurinacional de Bolivia para el Proyecto de Alianzas Rurales II [Project Appraisal Document on a Proposed Credit in the Amount of SDR 32.9 Million (US $ 50 Million Equivalent) to the Plurinational State of Bolivia for a Rural Alliances Project II]. Report No.71702-BO. The World Bank. (n.d.). Proyecto Alianzas Rurales (PAR) Executive Summary. PBCV project - Bolivia The World Bank. (n.d.). Programa Barrios y Comunidades de Verdad (PBCV) Executive Summary. The World Bank. (2006). Project Appraisal Document on a Proposed Credit in the Amount of SRD 20.5 million (US$30.0 Million Equivalent) to the Republic of Bolivia for the Urban Infrastructure Project. Report No. 36796-BO. The World Bank. (2012). Bolivia Urban Infrastructure Project (II) Additional Finance [Project Information Document (PID)]. Report No: AB7093. PAF - Nepal The World Bank. (2012). PAF Executive Summary. The World Bank. (2013). Project Appraisal Document on a Proposed Second Additional Grant in the Amount of SDR 53.4 Million (US$80 Million Equivalent) to Nepal for the Poverty Alleviation Fund II Project. Report No: 76677-NP. World Bank Institute. (2013). OnTrack Users Guide Nepal Section I [PowerPoint slides]. Innovation in Governance Team. World Bank Institute. (2013). OnTrack Users Guide Nepal Section II [PowerPoint slides]. Innovation in Governance Team. Zambia Ministry of Local Government and Housing. (2012). Kalingalinga Sanitation Marketing Baseline Survey Report. Ministry of Local Government and Housing of Zambia. The World Bank. (2010). Irrigation Development and Support Project, Zambia [Project Information Document (PID)]. Report No: 59102. The World Bank. (2011). Project Appraisal Document on a Proposed Credit in the Amount of SDR 74.70 million (US$ 115 Million Equivalent) to the Republic of Zambia for the Irrigation Development and Support Project. Report No: 58264- ZM. The World Bank. (2013). Promoting Innovative Approaches to Peri-urban Sanitation Improvement in Zambia [Project Concept Note]. Other documents/Bibliography Custer, S. (n.d.). How-to Notes: ICT-Enabled Citizen Feedback Loops [Draft]. World Bank Institute. Gigler, B.S. (2012). How-to Notes: Valuing Information: A Framework for Evaluating the Impact of ICT Programs [Draft]. World Bank Institute. Gigler, B.S., Bailur, S. (2014). Closing the Feedback Loop: Can Technology Bridge the Accountability Gap? Directions in Development. World Bank. Hunja, R.R., Peixoto, T., & Sjoberg, F. (2013). Digital Engagement Evaluation Framework [PowerPoint slides]. Open Development Technology Alliance. Integrity Action. (2013). The Fix-Rate: A Key Metric for Transparency and Accountability [Working paper]. Nyqvist, M.B., de Walque, D., Svensson, J. (2014) Information is power: experimental evidence on the long-run impact of community based monitoring. World Bank Taylor, B. (2012). Why did Maji Matone fail? 1. Low tech obstacles to high tech solutions? [Blog post]. Retrieved from http://blog.daraja.org/. The World Bank. (n.d.). The Greentree Consensus: Key Principles and Success Factors for Development in the Digital Age [PowerPoint slides.] The World Bank. (2013). Engaging with Citizens for Improved Development Results: Strategic Framework for Mainstreaming Citizen Engagement in WBG Operations [Concept Note]. The World Bank. (2013). Enhancing Transparency and Accountability through Citizen Feedback: Lessons from the E-ISR+ Pilot. Africa Operations Services Series. The World Bank. (2014). Citizen Voices: Global Conference on Citizen Engagement for Enhanced Development Impact. World Bank Institute. (2014). Digital Engagement Evaluation Guidelines (Version 1.4). Open Government Practice Team. List of consulted stakeholders Staff interviewed First Name Last Name Position Country Elena Bertusi OnTrack Country Lead Zambia Public Sector Management Kate Bridges Specialist Zambia Consultant - Demand Side Victoria Cabral Governance Zambia Stephen Davenport ODTA Alexandra Endara OnTrack Country Lead Bolivia Soren Gigler OnTrack TTL Faris Hadad-Zervos Country Director Bolivia Christiaan Heymaans TTL-Peri-urban Sanitation project Zambia Manager, Open Government Robert R. Hunja Practice Frauke Jungbluth TTL-PAF Nepal Keith Leslie PRAN Coordinator Nepal Gabriela Orozco Communications Associate Bolivia Tiago Peixoto Open Government Practice Maria Ponce System Development Consultant Bolivia Bigyan Pradhan Sr Operations officer Nepal Marie Sheppard Practice Manager, Innovation Labs Reena Shrestha OnTrack Country Lead Nepal Ison Simbeye Technical Assistant Zambia Anjalee Thakali PRAN Deputy Coordinator Nepal Trishba Thapa Communications associate Nepal Zoe Trohanis TTL- PBCV Bolivia David Tuchschneider TTL-PAR Bolivia Rajib Upadya Communications officer Nepal Alejandra Velasco Operations Officer Bolivia Aleem Walji Director, Innovation Labs Staff surveyed online First Name Last Name Position Country Natalia Agapitova Senior Program Officer, WBIIN Adam Behrendt Social Development Specialist Bolivia Elena Bertusi OnTrack Country Lead Zambia Kate Bridges Public Sector Management Specialist Zambia Consultant - Demand Side Zambia Victoria Cabral Governance Tambulani Chunga Technical Assistant Zambia Stephen Davenport ODTA Indira Ekanayake TTL-IDSP, Zambia Zambia Claudia Encinas Communications Assistant Bolivia Alexandra Endara OnTrack Country Lead Bolivia Faris Hadad-Zervos Country Director Bolivia Rural Development Specialist - Focal Bolivia Francisco Obreque Point Tiago Peixoto Open Government Practice Maria Ponce System Development Consultant Bolivia Marco Rodriguez Risk Manager Specialist - Focal Point Bolivia Marie Sheppard Practice Manager, Innovation Labs Ison Simbeye Technical Assistant Zambia Zoe Trohanis TTL- PBCV Bolivia Alejandra Velasco Operations officer Bolivia Project Implementing Unit representatives interviewed First Name Last Name Position Country Marco Antonio PBCV - General Coordinator Bolivia Rodolfo Ayala Saavedra PAR - Regional Coordinator Bolivia David Choque PAR - National Coordinator Bolivia Alvaro Flores PAR - Alliances Officer Bolivia Sanjay Jha PAF - Portfolio manager Nepal Kumar Sajada Khaton PAF -Portfolio manager Nepal Ximena Lezcano PBCV - Communications officer Bolivia Miguel Perez Arnez PAR - Systems Development Bolivia Angel Consultant Raj Babu Shrestha PAF - Executive Director Nepal Shree Ram Subedi PAF - Communications Officer Nepal Carlos Zenteno PBCV - Coordinator Bolivia Interviews with Civil Society Representatives First Name Last Name Organisation Country Bidhusan Bista Young Innovations Nepal Pranav Budhathoki Local Interventions Nepal Leon Galindo Ayni Labs, Ex-OnTrack consultant Bolivia Anne Lambert Accountability Lab Nepal Sophie Suman Parajuli Accountability Lab Nepal Focus group discussions carried out Bolivia PAR, Santa Cruz, April 2014: - Focus group with 5 coffee producers (men) - Focus group with 3 diary producers (men) - Focus group with 5 agricultural extension workers (4 men, 1 woman) Nepal, Kapilvastu District, May 2014: - Focus group with 25 people (predominantly women), members of 2 community organizations (Budha CO and Bindabasini CO), near Taulihawa - Focus group with 12 Partner Organisation (Mostly men, 1 woman) in Taulihawa - Focus group with 9 Dalit women, Namouna CO in Tilaurakat - Focus group with 7 women (5 Dalit) with Jyoti CO