TECHNICAL BRIEF Key Facts Statement Testing: Options, Methodologies, and Tools FINANCIAL INCLUSION SUPPORT FRAMEWORK JUNE 2021 FINANCE, COMPETITIVENESS & INNOVATION GLOBAL PRACTICE TECHNICAL BRIEF Key Facts Statement Testing: Options, Methodologies, and Tools FINANCIAL INCLUSION SUPPORT FRAMEWORK JUNE 2021 © 2021 International Bank for Reconstruction and Development / The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org DISCLAIMER This work is a product of the staff of The World Bank Group. The World Bank Group refers to the member institutions of the World Bank Group: The World Bank (Interna- tional Bank for Reconstruction and Development); International Finance Corporation (IFC); and Multilateral Investment Guarantee Agency (MIGA), which are separate and distinct legal entities each organized under its respective Articles of Agreement. We encourage use for educational and non-com- mercial purposes. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Directors or Executive Directors of the respective institutions of the World Bank Group or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. Since the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. CONTENTS Acknowledgments ii Acronyms and Abbreviations iii WHAT IS A KEY FACT STATEMENT? 1 WHY TEST A KEY FACT STATEMENT? 1 WHAT CAN TESTING A KEY FACT STATEMENT ACHIEVE? 1 WHAT METHODOLOGIES CAN BE USED TO TEST A KEY FACT STATEMENT? 2 1. Focus Group Discussions 3 2. Consumer Interviews (Non-rigorous) 4 3. Lab-in-the-field experiments 4 4. Consumer Surveys (Rigorous) 5 CONCLUSION 6 ANNEX A: EXPERIENCES FROM KEY FACT STATEMENT TESTING IN ETHIOPIA 7 ANNEX B. EXAMPLE FOCUS GROUP DISCUSSION GUIDE FOR KFS TEMPLATE FOR TRANSACTION ACCOUNTS 12 ANNEX C. EXAMPLE OF AN EXPERIMENTAL APPROACH TO KFS TESTING FOR TRANSACTION ACCOUNTS 18 REFERENCES 30   i ACKNOWLEDGMENTS This brief is a product of the Financial Inclusion, Infrastructure, and Access Unit in the World Bank Group’s Finance, Competitiveness, and Innovation Global Practice. This brief was prepared by Helen Gradstein (Financial Sector Specialist), Imtiaz Ul Haq (Economist), and Edoardo Totolo (Financial Sector Consultant). The team thanks Charles Hagner for editorial assistance and Maria Lopez for design and layout assistance. Mahesh Uttamchandani (Practice Manager) and Sheirin Iravantchi (Senior Financial Sector Specialist) provided overall guidance to the team. This brief draws from the World Bank Group’s experience testing key facts statements as a technical partner to countries supported by the Financial Inclusion Support Framework program. This report is made possible by the generous support from the Ministry of Foreign Affairs of the Kingdom of the Netherlands and the Bill and Melinda Gates Foundation. The support is provided through the Financial Inclusion Support Framework program. ii ACRONYMS AND ABBREVIATIONS FGD focus group discussion KFS key facts statement   iii WHAT IS A KEY FACT STATEMENT? WHY TEST A KEY FACTS STATEMENT? Disclosure and transparency are cornerstones of financial The design of a KFS is central to its success. Testing a KFS consumer protection and are key drivers of trust in the is an important part of the design process and ensures financial system. Asymmetry of financial information due that a KFS is designed in a way that reduces information to misleading advertising and sales practices coupled with asymmetry between consumers and providers of financial low levels of financial literacy can lead to poor financial services most effectively. decisions and resistance to engage with the formal finan- cial sector. For example, information on fees and charges The purpose of rigorously testing a KFS is to assess how can be incomplete, or terms and conditions can be exces- different versions of a KFS influence the understanding of sively detailed and therefore difficult for the average con- product terms and conditions, as well as the consumer sumer to comprehend, contributing to suboptimal decision-making process. Typically, as part of the tests, the financial decisions. Additionally, without clear information same KFS is outputted twice (but with different sets of provided during the pre-transactional stage, consumers information, simulating different fees, services, interest are unable to comparison shop or to make informed rates, and so forth), and the counterfactual KFS helps test- choices regarding what product is best suited to their ing if participants understand the content and are able to specific needs. select the “better” product. A key facts statement (KFS) is an important tool to improve transparency and disclosure of information for financial WHAT CAN TESTING A KEY FACTS products through the adoption of a standardized format STATEMENT ACHIEVE? that is designed to convey information in a simple and easy-to-understand manner. KFS testing can identify and recommend improvements to the KFS. It does so by testing (1) participants’ design When standardized and easy to understand, KFSs makes and content preferences; (2) the information on which it easier for customers to compare financial products from they typically focus; (3) questions that arise with regard to different providers, therefore promoting the responsible the content and design; (4) the amount of information that delivery of financial products and services and boosting participants can comprehend and process; and (5) how market competition. different presentations of information alter participants’ decision-making.   1 2   Key Facts Statement Testing: Options, Methodologies, and Tools By the end of the testing process, policy makers and prac- offers from different providers? Did consumers know titioners should be able to assess and identify how con- that they can compare offers? sumers perceive and engage with the KFS. Based on the results, the KFS can be changed/improved to yield better 7. Effectiveness of design and language: Is the KFS outcomes. In general, policy makers and practitioners template able to convey the critical information effec- should seek to understand the following: tively? Which components of the KFS (for example, elements highlighted in color, the amount of informa- 1. Consumer preferences: What product-related fea- tion, the font size, types of information in boxes) are tures are most important to consumers in the local more useful than others? Do participants consider the market (for each financial product)? Some aspects, KFS to be excessively long or short? In what language such as opening costs or interest, may be more import- should the KFS be written (English or the national or ant to the decision-making process than others. regional language)? 2. Existing issues in disclosure and transparency: What 8. Comprehension of changes, rights, and disputes: are the major comprehension gaps in consumer under- Do participants understand that the information in the standing? What experiences do consumers have KFS might change before an account is opened? Do opening and interacting with financial services? Did consumers understand how to complain and what the the existing materials address consumers’ informa- dispute-resolution mechanism will be? tional needs? If not, what are the issues with the exist- ing materials with regard to the consumer preferences stated above? WHAT METHODOLOGIES CAN BE USED TO TEST A KEY FACTS STATEMENT? 3. Perception of a KFS: Do participants understand the concept of a KFS? Do respondents see it as a Different approaches can be employed to test a KFS. Prior standardized, comparable summary of key information to selecting a methodology (or combination of methodol- about an account? Or are there different interpreta- ogies), policy makers and practitioners should consider tions? priorities in terms of testing. Quantitative testing (for example, using an experimental approach) can help iden- 4. Value of the information within the KFS: Which tify the best KFS variations and impact of a KFS, while pieces of information are considered most important qualitative methods like semistructured interviews and for a participant’s decision-making process, and which focus groups may be more helpful to build an initial KFS information is considered less relevant? Specific ques- (particularly if resources are constrained). tions may include (but are not limited to) the following: (i) Which items of information do participants think are When used together, both qualitative and quantitative the most important for deciding to sign up for the methodologies can prove to be powerful tools in finalizing product? (ii) What are the terms/ expressions they use KFSs for use in any local market. Mixed approaches allow to describe this information? (iii) Do participants think authorities to understand consumer behavior in depth, that the KFS contains this information? (iv) Is there any identify issues that need to be addressed, and design KFS information the consumer would also wish to see (but templates that are effective in improving disclosure is not available)? (iv) Is any information not needed? and transparency. 5. Understanding of the KFS content, terms, and cal- Selecting a methodology or combination of methodolo- culations: Are the various fees, transaction amounts, gies should depend on a country’s or an institution’s (i) minimum balances, interest rates, or annual percent- available resources for testing; (ii) populations of interest; age rates (for loans), nominal rates, and loan/invest- (iii) need/objective for testing; and (iv) type of KFS being ment periods well understood? Are these concepts tested (that is, credit products, transaction accounts, and still well understood when compounded with various so forth). years, fixed/variable rates, and so on? Do consumers find the example calculations easy to understand? Can Policy makers and practitioners should also note that consumers differentiate between superior and inferior resources intensify depending on the number of types of products based on KFSs? research employed; the breadth of locations, regions, and languages covered; the number of KFS variations tested; 6. Usefulness of the KFS when making comparisons: or the number of participants in the study. Do consumers feel the KFS is useful for comparing Key Facts Statement Testing: Options, Methodologies, and Tools   3 In general, this report recommends prioritizing focus deeper into problem areas, although less so than personal groups for the majority of policy maker and practitioner interviews KFS-testing needs. Should resources permit, combining focus groups with lab-in-the-field experiments helps to Challenges: Focus groups are less helpful in identifying fine-tune KFSs and assess impacts. In general, consumer specific solutions (that is, which KFS version is more effec- interviews and surveys may be less useful and more costly tive than another). While focus groups are most useful as tools for the purposes of KFS testing, although they yield a first approach in developing a KFS and short-listing from their own benefits as well. a wide range of possible solutions, they allow for neither a rigorous empirical test of whether a solution works nor a Appendix A includes a case study and lessons learned comparison of the effectiveness of alternative solutions. It from testing KFSs in Ethiopia. is also difficult to unpack the decision-making process of consumers through focus groups. Additionally, focus 1. Focus Group Discussions groups are not suitable when topics may provoke discus- sion that is deemed private or sensitive (for example, per- Focus group discussions (FGDs) tend to be well suited as sonal financial details). a first step to designing a KFS and contextualizing it the local market. FGDs are guided discussions on relevant Resources: Focus groups can be relatively quick and low topics between six to eight individuals that resemble a cost, as each group takes roughly 60–120 minutes, offers target segment. A number of such group discussions can the opinions of multiple people, and can cover multiple be employed to understand broad customer preferences topics in a short period of time. Costs increase with the and identify existing issues related to disclosure and trans- number of focus groups conducted. Resources required parency. FGDs are also helpful in understanding how con- will increase in cases where the KFS needs to be tested on sumers approach KFSs. FGDs provide a relatively quick multiple different population groups (that is, ethnic and low-cost method of capturing such qualitative infor- groups, regions, languages, and so forth), as more focus mation on a broad range of topics. groups may be required to capture the views of different subgroups. The main resource considerations are hiring/ FGDs can be helpful in developing the initial KFS tem- training competent enumerators to accompany relevant plate by capturing what consumers believe to be the most financial-sector staff, recruiting participants, and finding important content and best design for a particular KFS, an appropriate location to conduct the FGDs. The quality and how to best explain complicated concepts (such as of the moderator is critical to the success of FGDs. terms, overdrafts, consumer protection, and so on). One of the advantages of FGDs is the feedback channel that Methodology: FGDs can be conducted in two stages. allows for identification of gaps in existing or proposed First, participants answer questions without seeing a KFS. KFS templates. This will result in (i) understanding consumer preferences and (ii) identifying existing issues in transparency and dis- Benefits: Focus groups provide a relatively quick and closure. Second, participants provide their reactions to low-resource sample of many consumer opinions cover- the initial KFS template(s) to identify (i) the pieces of infor- ing a broad range of topics, permitting consumer prefer- mation that are considered most valuable and relevant; (ii) ences, perceptions, and areas of confusion to be gaps in the information and consumers’ additional con- understood quickly. One advantage of FGDs is the inter- tent/conceptual needs; (iii) comprehension of contents/ action between participants, which allows for the confir- concepts and possible solutions to explain them better; mation and refutation of the discussion. FGDs are (iv) the usefulness of the KFS to consumers; and (v) gen- particularly valuable as a first approach to understanding eral design/language preferences. This latter is also the market, especially in identifying main issues. They are known as a prototyping session and often contains visual also a useful tool for identifying major topics that should elements associated with the KFS (for example, posters be covered in a KFS and gaps in existing KFS templates, and vision boards) to elicit feedback on early versions of as more quantitative methods often lack flexible feedback the template, with a focus on language, content, and channels. FGDs also provide some flexibility to drill design.1 The objective of a prototyping session is to reveal NOTE 1. If an early version of the KFS template does not exist, an internationally tested KFS can be used as an initial template. The discussions will then provide nuance and help to contextualize the KFS template for the local market. 4   Key Facts Statement Testing: Options, Methodologies, and Tools participants’ actual understanding of terminology and for- 3. Lab-in-the-Field Experiments mat preferences that may otherwise not emerge from ver- bal discussion exclusively. Participants may be asked to Rigorous empirical testing of whether a KFS works and a rate templates if multiple alternatives are to be tested. quantitative comparison of the effectiveness of alternative KFS solutions identify the impact and effectiveness of a Appendix B provides a sample FGD guide on KFS testing KFS or a particular KFS variation. for transaction accounts. Note that this is only for illustra- tive purposes; while the guide includes some sample top- A lab-in-the-field experiment involves testing the impact ics and questions that may be relevant, it is imperative of a KFS by randomly assigning participants to “treat- that the guide be customized to the local context. ment” groups (one group for each KFS version to be tested). Each of the treatment groups can be exposed to 2. Consumer Interviews (Non-rigorous) a different version of the proposed KFS. All participants are then asked the same set of questions to gauge their Consumer interviews generally employ in-depth semi- level of understanding and measure their financial behav- structured interviews with individual consumers. One-on- ior. A comparison of the different groups’ answers sheds one interviews allow for detailed probing of issues, light on the impact of each KFS. The KFS that improves experiences, and perceptions to allow for a nuanced understanding the most and translates to a change in understanding of the related issues and explore possible actual financial behavior is considered the most impactful solutions. It can provide a deep dive into the consumer and should be selected for piloting or launch. experience to help fine-tune (but not to test rigorously) the development of a KFS. The experimental methodology also allows authorities to calculate the impact of introducing a KFS in the local mar- Benefits: Consumer interviews allow for a detailed under- ket more generally. To do this, an additional group (known standing of the consumer’s decision-making process. as the control group) can be introduced to those men- Interviews can be customized according to the individual tioned above. (See footnote 4.) This additional group and can probe areas of interest in greater detail. Inter- would be exposed not to any KFS but only to current mar- views are an appropriate tool to cover sensitive and pri- keting materials for transaction accounts. A comparison of vate topics. They can be used to understand issues as well the most effective KFS template with the control group as explore solutions. Interviews are particularly useful in will capture the impact of introducing the KFS in the identifying gaps in existing KFS templates. They are often market. used to refine findings from FGDs. Given limited resources, in-depth interviews are ideal for shedding light on Through these approaches, lab-in-the-field experiments detailed, various preferences and experiences, particu- can help test the effectiveness of different, finalized KFS larly if representativeness is not needed. versions and their features to see which are most impact- ful, as well as the overall impact of the KFS when com- Challenges: Conducting consumer interviews is costly pared to non-KFS documents. and time consuming. They allow going into greater depth but are limited to smaller samples. Interviews are typically Benefits: Experiments can provide evidence to answer limited to specific areas concerning participants and will questions about the value of introducing a KFS, its con- generate little information on other topics. Consumer tent, and its design. This is a quantitative methodology interviews are often conducted on small samples due to that allows for causal and statistical inference, hence mak- the resources required, making the results unlikely to be ing it a rigorous evaluation tool. Experiments are also representative of the larger population. Such interviews ideal for a comparison of the effectiveness of alternative are typically qualitative in nature and hence do not allow KFSs. This approach can highlight specific aspects of KFSs for rigorous empirical testing of the effectiveness of a KFS. that work better or worse, and hence provide valuable insights when fine-tuning the KFS. Resources: Consumer interviews may require substantial resources and may be time consuming. Generally, inter- Cons: One drawback of experiments is that they require views take 60–120 minutes. Given this constraint, sample prior understanding of consumer preferences and issues. sizes are often restricted, causing results not to be repre- This ex ante knowledge is crucial to construct the initial sentative of the population. The main resource consider- KFS templates to be tested. Hence, the experimental ations are the hiring/training of enumerators and recruiting approach is not well suited to be the first approach to of participants. The quality of the interviewer is crucial to understanding issues in consumer experiences and pref- the success of this methodology. erences (particularly on how best to explain complicated Key Facts Statement Testing: Options, Methodologies, and Tools   5 concepts) in order to construct and finalize an initial set analyze broad views of the KFS that are rigorously repre- KFS. Another shortcoming is that consumer reaction is sentative of particular populations, or even at the measured only for the KFS aspects that are tested. Miss- national level. ing aspects or gaps in the KFS are not highlighted due to the lack of a flexible feedback loop. Benefits: Consumer surveys can capture data about a large sample of consumers, ensuring representativeness Resources: Lab-in-the-field experiments require approxi- for the population. For this reason, surveys are often used mately 30–40 participants per group (control and treat- to understand population-level characteristics, such as the ment groups). The cost to implement the experiment for level or breadth of the incidence of a problem. Surveys each participant is approximately the same as that of a can also be used to identify common issues, including survey of similar length, which places experiments at a those that may involve sensitive or personal information. greater expense than, for instance, focus groups. If the number of KFSs to be tested is high, this approach can Challenges: Representativeness typically comes at the become costly as the number of participants increases expense of depth in the case of large-scale surveys. Such significantly. The main resource consideration is the surveys are usually not designed to collect detailed infor- recruitment of participants. The methodological design of mation about individual experiences and preferences. In the experiment is the crucial factor for success in an exper- addition to cost, this may also not be possible due to the imental approach and may require a technical background fact that such surveys are well structured and not easily in economics. The skills required for implementation are customizable to individual circumstances. While consumer low and similar to those required for implementing a sim- surveys often yield quantitative data, they are not ideal for ple survey. testing the effectiveness of alternative KFSs, especially when fine-tuning is required. 4. Consumer Surveys (Rigorous) Resources: The level of resources required for consumer Large-scale consumer surveys are able to capture the surveys is largely dependent on the sample size. While the views of a large sample of consumers. These surveys are marginal cost of administering an additional survey is low, well structured, with fixed questions and responses, and overall costs can be substantial, as such surveys are typi- can be used to collect both qualitative and quantitative cally employed on a large scale for representativeness. data. Such a method is most suitable when a representa- However, representativeness is usually not required for tive sample is desired. It is also useful for measuring KFS testing, which makes this an expensive method to do change over time and for identifying issues that may be so. The main resource consideration is the hiring of a large sensitive or personal. In general, consumer surveys can number of enumerators. CONCLUSION A well-designed KFS can play an important role in reduc- • Appendix A includes a case study and lessons learned ing informational asymmetries between providers and from testing KFSs in Ethiopia. consumers, allowing individuals to choose more appropri- ate products to meet their financial needs. • Appendix B provides a sample FGD guide on KFS test- ing for transaction accounts. Note that this is only for While different approaches can be employed to test a illustrative purposes; while the guide includes some KFS, policy makers and practitioners should always con- sample topics and questions that may be relevant, it is sider priorities in terms of testing. A combination of both imperative that the guide be customized to the qualitative and quantitative methodologies can prove local context. powerful for finalizing KFSs in any local market. • Appendix C contains more information and examples In general, this report recommends prioritizing focus for conducting an experiment for testing two different groups as a tool that can address the majority of KFS-test- versions of a KFS template. ing needs. Should resources permit, combining focus groups with lab-in-the-field experiments helps to fine- When used together, this technical brief and related tune a KFS and assess impacts. appendixes can help policy makers and practitioners develop initial methodologies for testing KFSs, draft To provide further guidance, this report also provides the terms of references and scopes of work, and provide initial following tools to support policy makers and practitioners resources that can be leveraged during a KFS-testing to design and implement KFS tests: process. 6 APPENDIX A: EXPERIENCES FROM KEY FACT STATEMENT TESTING IN ETHIOPIA To increase financial inclusion, the Government of Ethio- term loans, savings and current accounts, mobile pia launched an ambitious national financial inclusion money wallets, and time-deposit accounts? strategy in 2014, placing high importance on improving ii. What is the best content, design, and layout of KFS consumer protection through various reforms and activi- templates for these products to enhance consumer ties, including the introduction of key facts statements comprehension and decision-making? (KFSs). The Financial Inclusion Support Framework pro- gram supported the government on this front by con- A team of researchers facilitated discussions around par- ducting KFS testing for current accounts, time deposits, ticipants’ decision-making processes when choosing and fixed-term loans. The testing assessed how different financial products, with the aim of understanding what variations of KFS content and layout affected consumer factors are important to them, how they make financial understanding of product terms and conditions, as well decisions, and how they have previously gained the nec- as consumer decision-making processes. essary information. The FGDs were also used to seek early feedback on an initial KFS template, as participants Both focus group discussions (FGDs) and lab-in-the-field were asked direct questions about their understanding experiments were employed to test the KFS variations in and perceptions of different elements of the template. Ethiopia. However, the lab-in-the-field experiments had to be suspended due to the onset of COVID-19, which Prototyping sessions were also conducted as part of the restricted fieldwork. The lab-in-the-field experiments FGD but with different participants. Each session con- were replaced with a phone-based experiment that was sisted of the following four activities: more limited in scope. The sections below describe the methodological approach of all three components and 1. Terminology posters: Participants were asked to the research results of the qualitative research and provide definitions or alternative wording for terms mobile experiments. used in the KFS. A minimum of nine technical terms were selected per product. FOCUS GROUP DISCUSSIONS: IMPLEMENTATION AND FINDINGS 2. Vision boards: Participants were given cut-up ver- sions of the KFS template. They were asked to place The objective of the FGDs was to understand which fac- each section under one of four categories depending tors are important to users when deciding on a financial on their perceived priority for it to be included in the product and how best to present the information to opti- KFS: “Must Include,” “Should Include,” “Could mize decision-making. The research focused on the fol- Include,” and “Would Not Include.” lowing two core research questions: 3. Template rating: Participants were given two more i. What information do consumers value, understand, alternative versions for the KFS template and asked and find most useful in making decisions about fixed- to rate all three according to a series of characteristics.   7 8   Key Facts Statement Testing: Options, Methodologies, and Tools 4. Template feedback: Participants completed a table 1. Difficulty in accessing simple and reliable informa- for each template version giving feedback on what tion: Transparency on financial products is a signifi- they liked, what they did not like, what could be bet- cant challenge for respondents, which affects their ter, and suggestions for improvement. capacity to select appropriate products, especially for fixed-term loans. Respondents do not currently For all activities, the written outputs were collated and receive all the desired information when enquiring translated for analysis. Notes made by the facilitator and about financial products, which creates gaps in their notetaker during and after the session were also collated decision-making process. and translated. 2. Reliance on multiple sources for information: Most The qualitative research consisted of 16 FGDs, evenly respondents relied on a range of sources—including split across the capital Addis Ababa and Adama, a family, friends, employees, and financial service pro- peri-urban location in the Oromia Region. The selection viders—for information when selecting financial of these two research sites enabled the study to be con- products. Respondents valued the benefit of KFSs in ducted in both the Amharic and Oromo languages, presenting comparable information on financial ser- therefore examining how different communities and lan- vices and products. guages might relate to understanding KFS documents. Selected participants were primarily of low to medium 3. Understanding of key terms is a significant chal- socioeconomic status, with the aim of ensuring that the lenge: Some common financial terms—such as KFS templates were designed for an audience as wide as annual percentage cost of a loan, up-front fees and possible. Sessions were gender uniform: half the ses- charges, and transaction limits—were not well under- sions were with men only, and half were with women stood. only. On average, seven participants were in each FGD. The aim was to recruit a balanced number of people who 4. Preference for simplification: Most respondents were already using the financial product and people who expressed the desire for simplified wording and a were not, defined as follows: more illustrative representation of content for better engagement. They also expressed a preference of a 1. Users: People who are currently using the financial more personalized touch over formal text in context product under consideration or have used the product presenting. within the last six months preceding data collection Based on insights gained from the FGDs as well as liter- 2. Potential users: People who do not have the finan- ature reviews, the research team identified key aesthetic cial product under consideration but who are eligible, features, styles of communication, and explanatory in terms of their socioeconomic status and geograph- phrasing of commonly misunderstood terms that were ical location, to apply for the financial product2 then used in the lab-in-the-field experiments in the sec- ond stage. Four people participated in each prototyping session. As before, all sessions were split across the two sites and LAB-IN-THE-FIELD EXPERIMENTS: PILOTING were gender uniform. A mix of users and non-users of AND IMPLEMENTATION DESIGN the products were in most sessions, but the current-ac- count sessions consisted of users only. One fixed-term- The lab-in-the-field experiments were developed and loan session consisted solely of potential users. went through initial piloting in Addis Ababa, but they were suspended shortly before implementation due to The FGDs collected consumer feedback on the content, the onset of COVID-19 pandemic in East Africa. Hence, layout, format, and phrasing of the KFS templates. These this section focuses on the methodology developed for insights were used to inform the second phase of test- the study rather than the study results. ing. The following insights were key takeaways from the first stage: NOTE 2. For the case of current-account FGDs, the potential users are those that are unbanked. Key Facts Statement Testing: Options, Methodologies, and Tools   9 The goal of the experiments was to quantify the impact 4. Recall: Understanding which design elements help of different KFS design elements on consumers’ level of costumers recall the content of the KFS understanding and decision-making. In particular, the experiments were to test new design elements of the The experiment tested five specific KFS design elements KFS, such as infographics, typographical cues, and icons, in isolation as well as three variations of the full KFS for and compare the results against a baseline version of the each product. The five KFS design elements were the KFS, which the World Bank had used in other countries following: and did not include these design elements. Conducting the experiment in the field allowed observation of how 1. Pledges: Literature suggests that pledge framing, participants would react in a natural setting. Tablets, lap- which requires signing an explicit promise statement, tops, and other equipment were to be used to emulate may increase a participant’s level of engagement with the treatment in desired locations. KFS content (Bhanot 2017). To test this, respondents were introduced to an identical offer for a financial The experiment was planned to run in two locations: product that included summary information about Addis Ababa and Adama. A total of 800 participants the costs and other product features. Respondents (400 in each location) were to test KFSs for three prod- were then assigned to three treatments, a plain ucts: (i) fixed-term loans, (ii) saving and current accounts, pledge, a loss framed pledge, and the original default and (iii) time deposits. Participants were evenly distrib- instruction at the top of the KFS (control).3 Partici- uted to the three products so that each product would pants were required to answer a short quiz at the end receive approximately 133 participants in each location. of each experimental sequence that measured com- A treatment arm under a particular sequence of a partic- prehension under the three treatment conditions. ular product (that is, a series of screen displays, and quiz- zes designed to test a particular design element such as 2. Infographic illustrations and iconography: As per infographics, typographical cues, and so on) would the feedback gathered in FGDs, the use of infograph- receive about 45 respondents each. The minimum ics may help respondents better understand product detectable effect for the experiment was estimated to be features. Three treatments were developed to test 11.6 percent, with 86 percent power at 95 percent confi- this: one treatment with no infographics (control) and dence interval. two with infographics with different designs features. All participants were then asked a series of questions The study aimed to understand how each of the design to measure their comprehension of the information elements affected the following four customer contained in the KFS. outcomes: 3. Typographical cues: As per the feedback gathered in 1. Comprehension: Understanding how different FGDs, simple changes to the text format, such as design elements and language used in the KFS affect using bold text in key parts of the KFS, may improve participants’ understanding of the terms and condi- comprehension. To test this, respondents were tested tions of the financial products on their ability to recall parts of the KFS where various typographical cues were used (or none in the 2. Willingness to compare: Understanding if and how control group). different design elements support the consumers’ willingness to compare product offers in terms of 4. Frequently asked questions: Communicating key costs and product features information of the KFS content through questions and answers may be more efficient than using 3. Optimal product choice: Understanding what design descriptions or information in table format. To test elements of the KFS would aid consumers to choose this, respondents were randomly assigned to treat- the product most appropriate to their needs ments with various forms of frequently asked ques- NOTE 3. See appendix D for more details about the pledge experiment. 10   Key Facts Statement Testing: Options, Methodologies, and Tools tions (including a control with no questions) and correct. Here is an example of one of the questions tested for comprehension. administered in this style: 5. Ordering effects (primacy or recency): The experi- “A standing order is an instruction for regular fixed ment also tested the importance of the ordering of payments. Based on this explanation, please pick content, such as with the most relevant information at which of the following is a good example of a stand- the top or bottom of a section or page. Respondents ing order.” were assigned various ordering treatments and tested for comprehension and recall. (i) Sending money to a friend for Christmas shopping PHONE-BASED EXPERIMENT: (ii) Paying for airtime IMPLEMENTATION AND FINDINGS (iii) Sending money from your account to your landlord every month to pay your rent As the lab-in-the-field experiments could not be imple- mented due to the onset of the COVID-19 pandemic, an 2. Preference for terminology: All participants were alternative phone-based experiment was undertaken to introduced to a description or a concept, followed by avoid physical interaction during the fieldwork. As access a list of three or four terms. They were asked to pick to smartphones is limited in Ethiopia, especially among one term from the list that they felt best represented the target groups of the study, a nonvisual approach that the description they heard. Here is an example of one relied on basic phones had to be adopted. Unfortu- of the questions administered in this style: nately, this severely limited the scope of the activity, as most design elements could not be tested. Instead, the “Which of the following do you think best describes focus shifted to understanding the terminology used in an account you can access with your mobile phone to the KFS templates. transfer money and pay for goods and services?” Based on the outcomes of FGDs, the research team (i) Mobile Money identified 20 key terms from the initial KFS templates (ii) Mobile Money Account that either were difficult to understand or required fur- (iii) Mobile Wallet Account ther exploration. For each term, the research tested at least two alternative explanations or alternative terms. Data was captured from the experiment on the compre- The experiment was implemented via phone calls and, hension rates and preferences of all respondents. To hence, was significantly simplified to ensure that it would determine preferences for terminology, a simple descrip- be understood when administered verbally. The length tive analysis was conducted to uncover the most pre- of time required for completion was minimized to avoid ferred terms. Results were also recalculated for key respondent fatigue. The randomization of the original demographic segments, such as women and low-income experimental methodology was retained, and partici- and rural individuals. To test alternative explanation of pants were randomly allocated into one of three treat- financial terms, a comparison was made of the compre- ment groups to ensure a fairly equal balance across hension rates between participants that received differ- gender, location, income level, and educational ent treatments (explanations). A statistically significant attainment. difference implied that one explanation was more effec- tive than the other in enhancing understanding of a par- The phone-based experiment, while significantly limited ticular financial product. A regression analysis was also in scope, allowed for the testing of alternative explana- conducted to identify influential factors that affect an tions and preference for certain terminology, as individual’s comprehension of the terms tested. detailed below: Based on the descriptive and inferential analysis, the 1. Testing of alternative explanations of financial research was able to identify alternative explanations terms: For each identified term, participants were and terms that could help stimulate better comprehen- randomly assigned to a treatment group and tested sion among readers. For example, in the KFS for fixed- on an alternative explanation. In each treatment, field term loans, incorporating the phrase “fee required to officers read out an alternative explanation to respon- get the loan” in explaining “up-front fees and charges” dents, followed by a hypothetical question. Partici- seemed to drive better clarity and improved comprehen- pants were given three or four options for answers sion among the wider population. Similarly, the explana- and asked to choose the one that they thought was tion “full cost of your loan (including interest and fees) at Key Facts Statement Testing: Options, Methodologies, and Tools   11 a yearly rate” boosted comprehension of the term Although the research helped to shed light on trends in “annual percent cost of loans” among women. In the the descriptive statistics, most of the findings were not KFS for current/saving accounts and mobile money, statistically significant, which suggests that most of the adopting the explanation “the amount you can send or alternative explanations tested and the original terms receive from your account” helped improve the compre- are likely to yield the same level of comprehension for hension of the term “transaction limits” among the wider the larger population of Ethiopians. This can arise due to population. The term “standing order” was also difficult limitations if the number of participants is inadequate, as to understand for many respondents. Incorporating the was the case in Ethiopia due to COVID-19 limitations. phrase “instruction for regular payment” or “making The experiment results were further constrained as visual regular payments for your account” seemed to improve content could not be employed over the phone. It can the understanding. be difficult to detect differences in consumer compre- hension due to minor changes, and such restrictions can further limit KFS testing. APPENDIX B: EXAMPLE FOCUS GROUP DISCUSSION GUIDE FOR KFS TEMPLATE FOR TRANSACTION ACCOUNTS WELCOME NOTE These questions should be contextualized and revised based on the needs of the regulator. Introduction of the moderator and recorder—Introduce the objectives of the meeting and the expected out- Be sure to collect the following demographic details are come—Inform the participants that the discussions are collected for analysis purposes: recorded for future reference and analysis—Assure the participants that the information provided will be kept • Age confidential and no names will be quoted while compil- • Prior credit experience ing the findings—Set minimum rules for FGD that include • Gender keeping mobiles silent, one to talk at any time, discuss • Education freely, no right or wrong answer, allow all to speak, and • Family Income so forth—Encourage talking loud and clear for the recorder and moderator and note taker to take notes. 12 Key Facts Statement Testing: Options, Methodologies, and Tools   13 INTRODUCTION Welcome, and thank you for coming today. My name is _________________ and this is ________________. We are from ____. The material we are testing today is part of a consumer study about the information you receive when you go to open or compare a financial transaction account. A transaction account is typically defined as a store-of-value account at a bank, microfinance institu- tion or other regulated financial institution that lets you make/receive payments, save or store money. Specifically, we are testing a summary form of key information for the transaction account—we call such a form a “key facts state- ment” or “KFS.” We are trying to create a form that will summarize all the information consumers need in order to choose the right transaction account and better understand the terms of the account, their rights, and the different features. What you say to us today will help us to better understand what consumers need. With your help, we will be able to prepare an information summary form to help consumers make better decisions about transaction accounts. There are no right or wrong answers, we are just interested in learning from your opinions and experiences. I will be leading today’s session and __________ will be taking notes to help us remember what you say. We will be audio- and /or videotaping this session, based on what you agreed to, to ensure that we collect complete information. The entire session will take no more than 2 hours, and we will take a break about halfway through the session. Over the next 2 hours, I am going to ask you to do a couple of things and to respond to a series of questions. As I go through these questions, and you give me your responses, please remember there are no right or wrong answers. We aren’t testing you. We’re testing the summary form or KFS. We want to learn from you what works best for consumers to help them understand transaction accounts and if these summary forms give you the information you need to decide if you are interested in transaction accounts, and if you are able to use the information to compare amongst providers. Do you have any questions? CONFIDENTIALITY I have a few questions to ask you about the paperwork you filled out when you arrived. Can you please answer either yes or no to each of the questions that I will ask you? (If participants nod or shake head, ask him or her to say the answer out loud for our tape.) Did you fill out a questionnaire when you arrived today? Were you given consent forms to participate in this session for us to audiotape and videotape your session today? Did you give permission for both audiotape and videotape? (If did not give permission for video, ensure video is off.) Did you read and sign those forms? Do you have any questions? All of the information we collect today will be kept confidential to the extent permitted by law, and we will not identify you by name when we compile our results. In addition, we will not use your full name, address, or any other personal identifying information (other than your image if you agreed to videotaping) in reports, papers, or videos based on this research. I want to remind you that you may terminate the interview at any time. Note to moderator: Do not proceed with the session unless the participants answer yes to each of the first three questions. (Participants must agree to audio, but video is optional.) 14   Key Facts Statement Testing: Options, Methodologies, and Tools CONSUMER PREFERENCES AND EXISTING ISSUES Before we show you the KFS, we would like to hear from you. What types of services and products do you use? (MFIs, credit cards from retail stores, pawn shops, informal providers, etc. OTC, nothing?) How do you utilize these products? What are the main criteria you consider when selecting a product? Now, what would be the main criteria you would consider when selecting a transaction account? Please write down up to three features that you consider most important when selecting a transaction account or bank. What information would you like to know when shopping around or considering opening a transaction account? What features of the account are most important to you (for example, costs, fees, and so forth)? What are major gaps in your understanding of a transaction account? What would you like to know but have a hard time understanding? What have been your experiences opening and interacting with financial services? Any positive/negative? For those that have tried to open/opened an account before, did existing materials address your informational needs? What were some of the issues? If there is a common document to be published by every financial institution to help you make decisions about opening accounts, list down the information financial institutions should disclose that would help you to compare and shop around. FOR USERS OF FINANCIAL SERVICES: Why did you select the institutions you got the transaction account from? Do you have more than one account with different institutions? How did you figure out the terms of the transaction account? Who communicated this information to you? Are there specific features of the transaction account that you think could be changed or improved upon (that is, fees, balances, and so on)? Which aspects did you consider when opening an account? How did you obtain this information? Did you have any problems in getting information you wanted? When did you receive this information? (Before or after you received the product?) Did the person offering the product explain the aspects/terms of the product? Did you ask additional questions to resolve any doubts you had? Are there any features that were not clear to you or that you were not aware of when opening the account? If so, what did you do when you found out? HAND OUT KFS I am now going to provide you with the KFS. I would like to ask you to talk out loud about what you are thinking as you read it. I want to hear what you are noticing and thinking about as you look through it as well as your thoughts, questions, and ideas about the KFS. You might be reading out loud and thinking, “Wow! This is great!” or “I don’t like this!” Say those things out loud. Or maybe you think, “What does this word mean?” Say it out loud. Tell me what you like about what you see and what is confusing, so we can improve. Remember we want to hear your thoughts—both positive and negative—so we can make sure consumers have clear infor- mation about transaction accounts. Key Facts Statement Testing: Options, Methodologies, and Tools   15 We want to hear that inner dialogue or voice in your mind that comments on things you read. Talking out loud is the only way we can hear what you are thinking and how you are reacting. This may seem a little odd, but we really want to hear everything that you are thinking as you interact with this summary form for the first time, so we can improve it further. To learn from you, it is important that you: Tell us out loud where you are in the key information document. Tell us what you are reading, skimming, skipping in the form. Tell us any questions, suggestions, or confusion that you might have about any part of the summary form. Here is the KFS. Note to moderator: Allow sufficient time for participants to read through the KFS. VALUE OF KFS What is the purpose of this document? Do you understand the concept of this KFS? Given the scenario, would you be interested in this transaction account? (PROBE: Why or why not?) Can you explain to me what about this summary form helped you to decide? Which pieces of information are most important for your decision, and which ones are less relevant? We want to know what consumers think is the most important and useful information in the summary form. Take a highlighter and highlight parts of the form that you think are most important and useful. • Which 3 items of information do you think are the most important for deciding to sign up for the product? • Does the KFS have all the information you need to make a decision? • Is there any information that you would also wish to see (but is not available)? • Is there any information that is not needed? Any information that you skimmed? Any information that you skipped? • Can you explain to me, in your own words, why you selected the information you did? On what parts or information in the KFS do you focus to take your decision? Does the form contain the information you need to make a decision? If not, what other information should be included? Which are the most difficult parts of the document to understand? Are there any parts that are unclear? UNDERSTANDING THE KFS FORMAT: Do you have any comments on the format of the KFS? What about the elements bolded in color? Should other parts be bolded as well? What parts did you find most important? How was the amount of information? Was it too much? Too little? What about the information in boxes? Can you read and generally understand the English version of the KFS? And the Urdu version? Would you prefer that the KFS was in another language (such as a provincial language—for example, Sindhi)? Do you consider the KFS is too long? Too short? Explain the terminology used in the KFS that is new to you. Is there a better way to present it to make it easier to understand? Is the font or typeface easy to read? Is it large enough? 16   Key Facts Statement Testing: Options, Methodologies, and Tools Is there a specific structure in which you would like the document to be presented? Where do you prefer the placement of the [cover, premiums, fees, and excesses] information? (Moderator: make people vote for each of these features.) TERMS AND FEES: Are the various fees well understood? Do you understand what it means? Are they confusing? How would you prefer for it to be explained? Are the transaction amounts well understood? Do you understand what it means? Are they confusing? How would you prefer for it to be explained? Are the minimum balances well understood? Do you understand what it means? Are they confusing? How would you prefer for it to be explained? Are the example calculations/fees/balances easy to understand? Are you able to calculate correctly the various fees? Do you understand what all the terms mean? Are there any terms you don’t understand? Are there other terms/ways to explain the elements that would make more sense? Easier to understand? UNDERSTANDING THE KFS — DISCLOSURE AND TRANSPARENCY Through the KFS: Do you understand that the information in the KFS might change before an account is opened? Do you understand how to complain? Should complaint and dispute-resolution sections be given more prominence? If you were dissatisfied with the product or had a question to ask, where would you present a complaint or go for help? Describe the different steps that you take/would take to resolve these problems. To whom do you look? What resources, channels, or services would you use to try and resolve them? Do you understand which organizations can help you resolve problems with banks and other financial service providers? UNDERSTANDING THE KFS — PRODUCT COMPARISON NOTE TO MODERATOR: NOW PROVIDE EXAMPLE OF TWO DIFFERENT PRODUCTS USING SAME INFORMATION THROUGH TRANSACTION ACCOUNT Based on the KFS for each of the two products [give sufficient time to read and understand the content of KFS], specify which product you as a customer would choose. Which product would you prefer, transaction account 1 or 2? Explain the reason for why you chose this product. What factors influenced your decision? In your own words, can you summarize how you used the summary forms to compare? Did you have enough information to make a decision? Was there any additional information that you would have needed to make a decision? Key Facts Statement Testing: Options, Methodologies, and Tools   17 Can you explain if any information was not relevant for decision-making? Do you think the availability of this information changed your behavior when selecting a transaction account? Did you feel the KFS was useful for comparing offers from different providers? Did you know that you can compare offers? Did consumers understand how to select the “better” product? Who didn’t understand how to select the “better” product? WRAP UP How can this document be improved to make it more understandable? Would you utilize this information to compare the costs of products? Are there ways in which it could be made easier to compare products with each other? Any other comments or suggestions? Thanks Note — Thanks so much for sharing your thoughts and impressions with us today. Your feedback and suggestions have been very helpful. Do you have any questions I can try and answer? Remind and reassure participants about confidentiality and that no names will be published in any reports. Provide them with contact details in case they do have any further questions or concerns. APPENDIX C: EXAMPLE OF AN EXPERIMENTAL APPROACH TO KFS TESTING FOR TRANSACTION ACCOUNTS An experimental approach to KFS testing allows for a rig- Participants: Participants should be randomly selected in orous empirical investigation of the impact of a KFS on a busy public area that is visited by a variety of people, to consumer perception and eventual decision-making. The ensure sufficient variation in characteristics (such as approach is especially useful to test the effectiveness of a income, education, gender, age, employment, banking KFS template and thus make it an appropriate tool to status, and so forth). Ideally, the entire experiment should compare multiple versions of KFS templates. not be conducted in one slot but distributed over multiple slots across different days of the week and times of day to This appendix demonstrates how an experiment can be ensure this variation. The objective in participant recruit- used to compare the effectiveness of two versions of a ment is to capture individuals that are likely to constitute KFS template, with the objective of identifying the more the potential audience for transactional accounts. appropriate one for the local market.4 The versions used are only for illustrative purposes to outline how an experi- Monetization: The example experiment below is ment in this context should be designed and conducted.5 expected to last 20 minutes on average and can be con- The actual methodology can be borrowed from this guide ducted without any monetary incentives. However, it is but is likely to vary slightly to incorporate specific aspects good practice to monetize an experiment if it lasts longer of the KFS and local market dynamics. than 30 minutes. The monetary award should be propor- tional to the number of correct answers by the participant. A lab-in-the-field experimental methodology is recom- The mean payout should be roughly equivalent to the mended for KFS testing. This involves conducting a highly cost of one meal locally for each hour of the participant’s controlled experiment (similar to a lab experiment) in the time (on a pro rata basis). field with real-world participants who constitute the potential audience for transactional accounts. Details of Logistics: Participants are to be randomly selected in the the experiment are as follows: public area.6 Once an individual agrees to participate in NOTE 4. Alternatively, the objective may be to measure the impact of introducing a KFS, in which case the control group should be given existing marketing materials (if these exist). 5. Two sample versions (along with an accompanying KFS for each) are presented at the end of this appendix. 6. The participants must agree to participate in the experiment. This can be obtained through a consent form. 18 Key Facts Statement Testing: Options, Methodologies, and Tools   19 the experiment, basic demographics of the participant are the second set of questions are to be read, and answers recorded. A script is then read aloud to the participant. A recorded. As before, the participant is allowed to refer to sample script is provided in box C.1. both the KFS templates provided earlier when answering the questions, and there is no time limit to answer. Once Once the script is read, one version of the KFS template is the participant has answered all the questions, the exper- randomly chosen to be presented to the participant, who iment ends, and the participant is free to go.9 is then given time to go through it.7 Once the participant has familiarized himself/herself with the KFS template, the Analysis: The analysis of the data resulting from an exper- first set of questions are to be read, and answers recorded. iment is straightforward. Given random selection of the The participant is allowed to refer to the KFS template participants, only a simple comparison of the means is provided earlier when answering the questions. There is required. A simple statistical test of differences between no time limit, and the participant may take as long as he the number of correct answers between the two will reveal or she wishes to answer the questions. the more effective KFS template. It is also possible, to some extent, to determine which aspects of the KFS tem- Once the participant has answered the first set of ques- plate are better or worse by examining the answers for tions, a second accompanying KFS is to be handed to each individual question. For example, more correct him/her in addition to the first KFS.8 The participant is answers to question 5 (parts a and b) for the second ver- allowed time to go through it. Once the participant has sion will suggest that the warning icon draws attention to familiarized himself/herself with the second KFS template, the warning text and is more effective in communication. BOX C.1 Experiment Script Welcome, and thank you for participating in this experiment. My name is _______________ and I am from ____________. The material we are testing today is part of a consumer study about the information you receive when you go to open or compare a financial transaction account. A transaction account is typically defined as a store-of-value account at a bank, microfinance institution or other regulated financial institution that lets you make/receive payments, save or store money. You will receive two leaflets with information on transaction accounts and will be asked a series of questions. Your answers and personal data will be kept completely confidential and will not be used for any purpose other than this experiment. This experiment will last approximately 20 minutes. We request that you do not use your cell phone or talk to anyone during the experiment. Before we begin the experiment, we would like to collect some information about you. Please tell us your: Age: Gender: Highest education achieved: Current Employment: Family Income (approximately): Banking status (whether you ever had a bank account or not): NOTE 7. It is extremely important that the KFS template version be chosen randomly. This can be done most easily by tossing a coin for each participant to decide between versions 1 and 2. A dice can be used if there are more than two versions. Do not simply alternate between versions systematically. 8. It is important not to take the first KFS away from the participant. 9. In case the experiment is monetized, the participant is to be paid once the experiment ends. 20   Key Facts Statement Testing: Options, Methodologies, and Tools We will now begin the experiment. Here is some information about a transaction account. [Moderator to randomly select one version of KFS template and hand to participant. Record version of KFS template selected here: _______________] This is a sheet that contains some information a bank has provided for its transaction account. Please take your time to review it and let me know when you are ready. I will then ask some questions regarding the transaction account. You may refer to the sheet to answer the questions. Please let me know whenever you are ready to answer the questions. [Wait for participant to inform moderator that they are ready to answer the questions.] I will now ask you some questions. Remember, you can refer to the information sheet to answer the questions. There is also no time limit, so feel free to take as much time as needed to answer the questions. [Read aloud the following questions, followed by the options. Record participant’s answers.] Q1. What type of account is this? a. Current account b. Saving account c. Mobile wallet account d. Other type of account Q2. Can you open this account with no money? a. Yes b. No c. Not specified in sheet d. Don’t know Q3a. Do you need to pay anything if you keep the minimum amount of money required to keep the account? a. Yes b. No c. Not specified in sheet d. Don’t know Q3b. If yes to above: How much do you need to pay every year? a. $50 b. $100 c. $300 d. Not specified in sheet e. Don’t know Q4. How much money will you have in the account after 1 year if you deposit $1,000 today and not use the account again? a. More than $1,000 b. Exactly $1,000 c. Less than $1,000 d. Not specified in sheet e. Don’t know Key Facts Statement Testing: Options, Methodologies, and Tools   21 Q5a. Can the interest payment on this account ever change? a. Yes b. No c. Not specified in sheet d. Don’t know Q5b. If yes to above: Why can the interest payment on this account change? a. Bank may choose not to pay interest after one year. b. Bank may return deposit whenever they want. c. Bank may change the mentioned interest rate. d. Not specified in sheet e. Don’t know Q6. What is the cheapest way to send fund transfers from this account? a. Transfer at a branch b. Transfer through Internet banking c. Transfer through mobile banking d. Standing orders/direct debits e. Not specified in sheet f. Don’t know Q7. What can you do if you are unsatisfied with how the bank has handled your complaint? a. Do nothing b. Report to police c. Contact the Ombudsman’s secretariat d. Not specified in sheet e. Don’t know Q8. How often can the bank change the fees listed in the information sheet? a. Bank cannot change these fees b. Once a year c. Once a month d. Not specified in sheet e. Don’t know Q9. Do you have to open an account if you sign this document? a. Yes b. No c. Not specified in sheet d. Don’t know I will now share another information sheet of a second transaction account. [Moderator to hand to participant the accompanying KFS template for the selected version.] This is a sheet that contains some information a different bank has provided for their transaction account. Please take your time to review it, and let me know when you are ready. I will then ask some questions regarding both accounts. You may refer to both information sheets to answer the questions. Please let me know whenever you are ready to answer the questions. 22   Key Facts Statement Testing: Options, Methodologies, and Tools [Wait for participant to inform moderator that they are ready to answer the questions.] I will now ask you some questions. Remember, you can refer to both the information sheets to answer the questions. There is also no time limit, so feel free to take as much time as needed to answer the questions. [Read aloud the following questions, followed by the options. Record participant’s answers.] Q10. Which account will you open if you have only $100 to deposit? a. Bank Account ABC b. Bank Account XYZ c. Indifferent between the two bank accounts d. Don’t know Q11. Which account will you open if you have $500 to deposit and will not use any of the banking services? a. Bank Account ABC b. Bank Account XYZ c. Indifferent between the two bank accounts d. Don’t know Q12. Which account will you open if you have $500 to deposit and will frequently use some of the listed bank- ing services? a. Bank Account ABC b. Bank Account XYZ c. Indifferent between the two bank accounts d. Depends on which services I use most e. Don’t know Q13. Which account can change the fees presented? a. Bank Account ABC b. Bank Account XYZ c. Both accounts d. Neither account e. Don’t know Q14. Which account offers more services? a. Bank Account ABC b. Bank Account XYZ c. Not specified d. Don’t know This concludes the experiment. Thank you for participating. [Experiment ends.]10 NOTE 10. Correct answers to the questions above are (1) B, (2) B, (3a) A, (3b) B, (4) C (interest minus maintenance fees), (5a) A, (5b) C, (6) A, (7) C, (8) D, (9) B, (10) B, (11) B, (12) D, (13) C, and (14) C. Key Facts Statement Testing: Options, Methodologies, and Tools   23 APPENDIX C.1: Version 1 of KFS Template Read this document carefully if you are considering opening a new account. Use it to compare different accounts and different providers. KEY FACTS STATEMENT FOR ACCOUNT Account provider: Bank ABC Date: Product name: Account ABC Account type: Current Account X Savings Account Mobile Wallet WARNING: This information is accurate as at the date above. Services, fees and interest rates may change. Opening and keeping this account Interest Minimum balance Interest paid on account? Yes To open the account: $ 200 Annual interest rate: 9% To keep the account: $ 300 This rate may change at any time. Maintenance fee $ 100 per year Based on the current rate, this means for every $ 1,000 you keep in the account for 12 months you receive $ 90 in interest. Services Fees ATM/debit card First ATM/debit card $ No fee Replacement ATM/debit card $ 50 Cash withdrawals Branch withdrawal $ No fee Withdrawal from our ATMs $ No fee Withdrawal from other ATMs $ 10 Deposits Branch deposit $ No fee Deposit through our ATMs $ No fee Cheques Cheque book $ 80 Stop cheque $ 30 Sending funds transfers Transfer at a branch $ No fee Transfer through Internet banking $ 75 Transfer through mobile banking $ 75 Standing orders/direct debits $ 30 24   Key Facts Statement Testing: Options, Methodologies, and Tools Services Fees Receiving funds transfers Receiving fund transfers $ No fee Statements Paper statements $ No fee Electronic statements $ No fee Balance inquiries At a branch $ No fee At our ATMs $ No fee At other ATMs $5 On mobile banking $ No fee Internet banking Fees for using Internet banking No fee Mobile banking Fees for using Mobile banking No fee Important: This is a list of the main service fees for this account. It does not include all fees. You can find a full list of fees in our branches and on our website. Calculation and payment of interest Interest is calculated annually on the balance in your account and is credited to your account annually. What happens if you overdraw the account? Overdrawing fees $ 250 Warning: If you dishonestly write a dishonoured cheque, you will commit a criminal offence. Other things you should know • Requirements when opening this account: To open the account you will need to satisfy some identification requirements. These can include providing documents and information to verify your identity. Please ask us for more details. • What happens if you do not use this account for a long period? If you do not make any deposit or payment/withdrawal for 12 months, your account will be treated as dormant. If your account becomes dormant, you will not be able to withdraw/ deposit money. To reactivate your account you must visit the nearest bank branch. • Closing this account: To find out the procedure for closing your account, you should contact the nearest bank branch. [You do not have to pay any extra fee to close your account.] How can you get assistance or make a complaint? Contact us in any of the following ways: Helpdesk at 111 111 111. If you are unsatisfied with how we handle your complaint, you can contact the Ombudsman’s secretariat. I ACKNOWLEDGE RECEIVING THIS KEY FACTS STATEMENT Customer name: ___________________________________ Customer signature: _______________________ Date: __________ Key Facts Statement Testing: Options, Methodologies, and Tools   25 APPENDIX C.2: Accompanying KFS with Version 1 of KFS Template Read this document carefully if you are considering opening a new account. Use it to compare different accounts and different providers. KEY FACTS STATEMENT FOR ACCOUNT Account provider: Bank XYZ- Date: Product name: Account XYZ Account type: Current Account X Savings Account Mobile Wallet WARNING: This information is accurate as at the date above. Services, fees and interest rates may change. Opening and keeping this account Interest Minimum balance Interest paid on account? Yes To open the account: $ 50 Annual interest rate: 12% To keep the account: $ 100 This rate may change at any time. Maintenance fee $ 50 per year Based on the current rate, this means for every $ 1,000 you keep in the account for 12 months you receive $ 120 in interest. Services Fees ATM/debit card First ATM/debit card $ No fee Replacement ATM/debit card $ 70 Cash withdrawals Branch withdrawal $ No fee Withdrawal from our ATMs $ No fee Withdrawal from other ATMs $5 Deposits Branch deposit $ No fee Deposit through our ATMs $ No fee Cheques Cheque book $ 50 Stop cheque $ 50 Sending funds transfers Transfer at a branch $ No fee Transfer through Internet banking $ 50 Transfer through mobile banking $ 50 Standing orders/direct debits $ 100 26   Key Facts Statement Testing: Options, Methodologies, and Tools Services Fees Receiving funds transfers Receiving fund transfers $ No fee Statements Paper statements $ No fee Electronic statements $ No fee Balance inquiries At a branch $ No fee At our ATMs $ No fee At other ATMs $ 10 On mobile banking $ No fee Internet banking Fees for using Internet banking No fee Mobile banking Fees for using Mobile banking No fee Important: This is a list of the main service fees for this account. It does not include all fees. You can find a full list of fees in our branches and on our website. Calculation and payment of interest Interest is calculated annually on the balance in your account and is credited to your account annually. What happens if you overdraw the account? Overdrawing fees $ 200 Warning: If you dishonestly write a dishonoured cheque, you will commit a criminal offence. Other things you should know • Requirements when opening this account: To open the account you will need to satisfy some identification requirements. These can include providing documents and information to verify your identity. Please ask us for more details. • What happens if you do not use this account for a long period? If you do not make any deposit or payment/withdrawal for 12 months, your account will be treated as dormant. If your account becomes dormant, you will not be able to withdraw/ deposit money. To reactivate your account you must visit the nearest bank branch. • Closing this account: To find out the procedure for closing your account, you should contact the nearest bank branch. [You do not have to pay any extra fee to close your account.] How can you get assistance or make a complaint? Contact us in any of the following ways: Helpdesk at 111 111 111. If you are unsatisfied with how we handle your complaint, you can contact the Ombudsman’s secretariat. I ACKNOWLEDGE RECEIVING THIS KEY FACTS STATEMENT Customer name: ___________________________________ Customer signature: _______________________ Date: __________ Key Facts Statement Testing: Options, Methodologies, and Tools   27 APPENDIX C.3: Version 2 of KFS Template11 Read this document carefully if you are considering opening a new account. Use it to compare different accounts and different providers. KEY FACTS STATEMENT FOR ACCOUNT Account provider: Bank ABC Date: Product name: Account ABC Account type: Current Account X Savings Account Mobile Wallet WARNING: This information is accurate as at the date above. Services, fees and interest rates may change. Opening and keeping this account Interest Minimum balance Interest paid on account? Yes To open the account: $ 200 Annual interest rate: 9% To keep the account: $ 300 This rate may change at any time. Service fee $ 100 per year Services Fees ATM/debit card First ATM/debit card $ No fee Replacement ATM/debit card $ 50 Cash withdrawals Branch withdrawal $ No fee Withdrawal from our ATMs $ No fee Withdrawal from other ATMs $ 10 Deposits Branch deposit $ No fee Deposit through our ATMs $ No fee Cheques Cheque book $ 80 Stop cheque $ 30 Sending funds transfers Transfer at a branch $ No fee Transfer through Internet banking $ 75 Transfer through mobile banking $ 75 Standing orders/direct debits $ 30 NOTE 11. The second version of the KFS template is different from the first one in the following ways: (1) The interest-paid example has been removed; (2) an icon has been added to the warning text to draw attention to it; and (3) the second version is shorter and removes the following sections: (a) Calculation and payment of interest, (b) What happens if you overdraw the account?, and (c) Other things you should know. 28   Key Facts Statement Testing: Options, Methodologies, and Tools Services Fees Receiving funds transfers Receiving fund transfers $ No fee Statements Paper statements $ No fee Electronic statements $ No fee Balance inquiries At a branch $ No fee At our ATMs $ No fee At other ATMs $5 On mobile banking $ No fee Internet banking Fees for using Internet banking No fee Mobile banking Fees for using Mobile banking No fee Important: This is a list of the main service fees for this account. It does not include all fees. You can find a full list of fees in our branches and on our website. How can you get assistance or make a complaint? Contact us in any of the following ways: Helpdesk at 111 111 111. If you are unsatisfied with how we handle your complaint, you can contact the Ombudsman’s secretariat I ACKNOWLEDGE RECEIVING THIS KEY FACTS STATEMENT Customer name: ___________________________________ Customer signature: _______________________ Date: __________ Key Facts Statement Testing: Options, Methodologies, and Tools   29 APPENDIX C.3: Version 2 of KFS Template11 Read this document carefully if you are considering opening a new account. Use it to compare different accounts and different providers. KEY FACTS STATEMENT FOR ACCOUNT Account provider: Bank XYZ- Date: Product name: Account XYZ Account type: Current Account X Savings Account Mobile Wallet WARNING: This information is accurate as at the date above. Services, fees and interest rates may change. Opening and keeping this account Interest Minimum balance Interest paid on account? Yes To open the account: $ 50 Annual interest rate: 12% To keep the account: $ 100 This rate may change at any time. Service fee $ 50 per year Services Fees ATM/debit card First ATM/debit card $ No fee Replacement ATM/debit card $ 70 Cash withdrawals Branch withdrawal $ No fee Withdrawal from our ATMs $ No fee Withdrawal from other ATMs $5 Deposits Branch deposit $ No fee Deposit through our ATMs $ No fee Cheques Cheque book $ 50 Stop cheque $ 50 Sending funds transfers Transfer at a branch $ No fee Transfer through Internet banking $ 50 Transfer through mobile banking $ 50 Standing orders/direct debits $ 100 Receiving funds transfers Receiving fund transfers $ No fee 30   Key Facts Statement Testing: Options, Methodologies, and Tools Services Fees Statements Paper statements $ No fee Electronic statements $ No fee Balance inquiries At a branch $ No fee At our ATMs $ No fee At other ATMs $ 10 On mobile banking $ No fee Internet banking Fees for using Internet banking No fee Mobile banking Fees for using Mobile banking No fee Important: This is a list of the main service fees for this account. It does not include all fees. You can find a full list of fees in our branches and on our website. How can you get assistance or make a complaint? Contact us in any of the following ways: Helpdesk at 111 111 111. If you are unsatisfied with how we handle your complaint, you can contact the Ombudsman’s secretariat I ACKNOWLEDGE RECEIVING THIS KEY FACTS STATEMENT Customer name: ___________________________________ Customer signature: _______________________ Date: __________ Key Facts Statement Testing: Options, Methodologies, and Tools   31 APPENDIX D: Pledge Experiment SCREEN 1: INTRODUCTION A new bank in your area, called Zamira, is offering small business loans that you can benefit from. You have been given a task to review a Zamira bank loan and answer a few questions about the loan. We will show your information about the loan in stages, after which you will be asked to answer a few questions about the loan. You will not be able to go back to a previous screen, so use the time you have well. SCREEN 2: LOAN INFORMATION—THREE TREATMENTS Loan Summary Cost of Loan Amount of Your Loan 60,000 Birr Total Interest 6,000 Birr Amount you are borrowing Interest you will be charged for the loan Amount You Will Receive 59,000 Birr Amount you will get after up-front Total Fees and Charges 1,000 Birr fees and charges are removed See below a list of total fees and charges Loan Term 24 months Amount you will get after up-front fees Annual Percent Cost of the Loan 5.93% and charges are removed Total cost of the loan a year in percentage Repayment Amount and Frequency 2,750 Birr The amount you will repay and how per month Total Cost of Your Loan 7,000 Birr often you will repay it All cost for the loan including total interest, fees and charges Treatment 1—Plain Pledge: Please type your first name below and read through the loan information. I ……………………………………………… agree to review the information below carefully. Treatment 2—Loss Frame Pledge: Please type your first name below and read through the loan information. I ……………………………. agree to review the information below carefully before making a choice so that I do not choose a loan that is not right for me. Treatment 3—Original Instruction (Baseline): Carefully review the information below if you are considering a fixed-term loan. SCREEN 3: QUIZ (ALL TREATMENTS) Please answer the following questions to the best of your ability based on the information you have just seen. 1. If you apply for a loan from Zamira bank the loan amount you apply for and the amount you receive will be: a. Different b. The same 32   Key Facts Statement Testing: Options, Methodologies, and Tools 2. What is the total cost of the loan? 3. How often would you have to make repayment - Once a year - Every month - Every day - Every week REFERENCES Bhanot, S. P. 2017. “Cheap Promises: Evidence from Loan Repayment Pledges in an Online Experiment.” Journal of Economic Behavior & Organization 140 (August 2017): 246–66. Krueger, R. A. 2002. Designing and Conducting Focus Group Interviews. Stewart, D. W., and P. N. Shamdasani. 2014. Focus Groups: Theory and Practice, Third Edition. (Sage Publications, 2014).   33