SMARTER SYSTEMS: How Tweaking Your Diligence Process Can Unlock Overlooked Opportunities I N V E S T O R I M P L E M E N TAT I O N G U I D E RESEARCH PARTNERS SPONSORS 1 About the Guide We found that there are discrepancies in how investors evaluate men-led and women-led startups, potentially leading them to overlook promising startups and overestimate less promising startups. This implementation guide outlines three steps we found that institutional and individual investors* can incorporate into their evaluation processes to consistently evaluate all startups more accurately. Specifically, these steps aim to reduce discrepancies and inconsistencies in assessments and increase their objectivity and accuracy when evaluating all startups by adding structure to areas that lack it in the evaluation process, and mitigating investor over-reliance on “gut instinct” when evaluating the founding team. Mitigating inconsistencies in evaluation processes prevents investors from overlooking opportunities with high potential and consequently can help them achieve greater financial returns. For more information on how startups are evaluated inconsistently, why we designed these steps and what we found when testing their effectiveness, visit our Key Insights Report. You can find details of how we tested these steps in our Methodology Report. * E.g. Venture capital firms, angel groups, etc. 2 Contents 5 | Context: Targeting discrepancies in the evaluation process 6 | How to strengthen your evaluation process to evaluate all startups more accurately 7 | Step #1: Collect information on each startup’s risk and growth opportunities to ensure you have a comprehensive understanding of both 8 | Step #2: Assess a team’s potential by evaluating how much they have demonstrated an ability to improve their startup 11 | Question banks and template to incorporate steps 1 and 2 into your evaluation process 15 | Step #3: Pre-define what evaluation criteria will most heavily determine how you assess a company 19 | Endnotes Acknowledgements This toolkit was developed by Village Capital, in partnership with the International Finance Corporation (IFC) and the Women Entrepreneurs Finance Initiative (We-Fi); with support from Visa Foundation, Moody’s, the Aspen Network of Development Entrepreneurs (ANDE) Advancing Women’s Empowerment Fund, Sasakawa Peace Foundation, and the ANDE SGB Evidence Fund; and research design, analysis and support by Amisha Miller of Questrom School of Business, Boston University, Saurabh Lall of the University of Glasgow, and the World Bank Gender Innovation Lab. A special thank you for their contribution to this project to Allie Burns, CEO, Village Capital; William Sonneborn, Global Director, Disruptive Technologies and Funds, IFC, Heather Matranga, Vice-President and Managing Director, Impact Investments, Village Capital, Loretta Foran, Senior Operations Officer, Disruptive Technologies and Funds, IFC, Heather Kipnis, Global Product Lead, Entrepreneurship, Gender and Economic Inclusion, IFC, Shruti Chandrasekhar, Head of Investments, Africa Region Private Equity, IFC, Laurien Field, Investor and Gender Lead, Global Private Equity and Venture Capital Funds, IFC, Stephanie Vak Stephens, former Gender Finance Lead, Global Private Equity and Venture Capital Funds, IFC, Nathaly Botero, Chris Veasey, Ben Wrobel, and Sofía Cándano, Village Capital. The team was advised by Joao Montalvo and Markus Goldstein (formerly) from the World Bank Gender Innovation Lab. The research was guided by conversations with multiple industry experts whose contributions were essential to the report. The team is grateful for their comments, suggestions, and input. 3 Context: Targeting Discrepancies in the Evaluation Process 4 The issue: Evaluating startups inconsistently narrows an investor’s field of vision leading them to overlook startups with high potential. As a result, investors miss out on promising opportunities, and high potential women-led startups do not get the funding they need to scale. Identifying discrepancies in assessments Three steps to increase consistency in the evaluation process Investors ask women-led startups 1) Collect information on each startup’s risk significantly more risk-related questions, and growth opportunities to ensure you have a and men-led startups more growth- comprehensive understanding of both. related questions.1 This imbalance can lead investors to overlook key risks or Prompting investors to think about both risk- and growth opportunities for a startup. growth-related questions helps prevent them from focusing disproportionately on either, and therefore leads to more consistent evaluations. Lack of information on business 2) Assess a team’s potential by evaluating trajectory leads investors to focus how much they have demonstrated an ability heavily on evaluating the founding to improve their startup (e.g. acquiring new team’s potential to grow the startup. customers, identifying and addressing risks in their Yet evaluating a founder’s “potential” business model, securing new partnerships, etc). results in more favorable outcomes for men, likely due to gender bias.2 Investors A startup has potential if it seems likely they will possibly associate men with “potential” be able to grow. To do so, the startup must be able more often than women because: 1) to continually make improvements that allow it to Attributes typically associated with grow. women are perceived as incongruent with those required to be a competent Consequently, evaluating how a founding team entrepreneur who shows potential improves their startup in the short-term helps the for success3 and 2) To replicate past investor make a more accurate, performance-based success, investors may often seek out assessment of the startup’s future potential, by entrepreneurs who are similar to those creating new data to assess how well the team will they have previously invested in and have be able to continue making improvements to their been successful with — which are most company in the future. often men. Research has also found that evaluators 3) Pre-define which evaluation criteria will most adjust the characteristics they initially heavily influence your scoring and decisions. wanted to see in a successful candidate to fit the characteristics displayed by Predefining the weight of each evaluation criterion candidates of their preferred on the final score, by giving it a specific number, gender.4 increases the likelihood that the evaluator will apply the framework consistently.5 This simple strategy prevents the investor from redefining criteria for success. 5 How to Strengthen Your Evaluation Process to Evaluate All Startups More Accurately We recommend that every investor use a set evaluation framework to ensure they are accurately and consistently evaluating all startups. That said, incorporating Steps 1 and 2 does not require having an evaluation framework in place. So if you make a simple change to your evaluation process today, we recommend starting here. Step 3 is designed to be incorporated into an existing evaluation framework. If you do not have an evaluation framework, Step 3 will provide initial guidance on how to set one up. 6 Step #1 Collect information on both risk and growth opportunities for each startup to ensure you have a comprehensive understanding of both Build the following checkpoints into your evaluation process to ensure you have a comprehensive understanding of a startup’s risks AND growth opportunities: 1 Include the following questions in your evaluation framework to remind the investor to seek information on both areas: ▶ Do you have a sound understanding of the startup’s risks? ▶ Do you have a sound understanding of the startup’s growth opportunities? 2 Incorporate a space within the evaluation framework to keep track of the startup’s risks and growth opportunities, and any additional questions regarding each area. This may help you see if you have overlooked one of the two areas. See page 11 for a sample question bank and page 12 for a template you can use to keep track of the information you’ve gathered and any remaining questions you have. In investment committees, it might be helpful to dedicate time to question if the committee has a comprehensive understanding of the startup’s risks and growth opportunities. Investment committee members could ask each other if there are any remaining questions on either of the two areas. This “checkpoint” could serve as an additional layer to prevent under or overestimating a startup. 7 Step #2 Assess a team’s potential by evaluating how much they have demonstrated an ability to improve their startup What Does It Mean to Evaluate Improvement Over Time? By improvement, we mean any meaningful progress the team has made in changing, adapting, strengthening, or adjusting their course of action in a way that keeps them on a path towards continual growth. It also means providing better explanations for why a company made those specific changes. Over time, companies that show improvement should be able to answer some investor questions about strategic decisions made with more data. Importantly, founders can show improvement even if what they have done looks different to what was recommended by others. Startups often receive feedback from multiple channels, and will therefore have to decide on the best course of action for the growth of their company. When evaluating improvement, you should focus on evaluating a team’s demonstrated ability to continually improve. In your own evaluation process, you should evaluate improvement in the key area(s) that you consider most important for a startup’s success and future growth. In our experiment, for example, we evaluated a team’s demonstrated ability to improve and execute its company’s risk mitigation and growth strategies for two reasons: 1) these strategies are key for startups’ future success, and 2) they were broad enough to be applicable to startups from different sectors. As an example, the categories and guiding questions we used in the experiment can be found in the table below: Category: Understanding Demonstrating Understanding risks Demonstrating risk potential for growth potential for growth mitigation Guiding How much has this How much has this How much has this How much has this questions company improved company improved company improved company improved to evaluate in understanding in executing its in understanding in executing on risk category: its path to growth? path to growth? its risks? mitigation? 8 STEP #2 How Can You Measure Improvement Over Time? Identify key area(s) in which to evaluate improvement and incorporate a category(ies) 1 for it in your evaluation framework First, identify which area(s) you consider key for a startup’s success and future growth. Then, add a category(ies) into your evaluation framework to assess improvement in each area. Use a simple scale(s) that reflect(s) how you would define improvement in your 2 selected key area(s) Creating a scale(s) will allow you to both consistently evaluate improvement across all startups and measure how much each team has improved in the given area(s). For example, in our experiment we used the scales below to measure how much a team had improved in understanding and executing their growth strategy: Example of measuring improvement in understanding the company’s path to growth: 1. No Improvement 2. Slight Improvement 3. Moderate Improvement 4. Strong Improvement This company is thinking This company has been This company has explained This company has explained about their business in thinking about how they how they changed their thinking both how they changed their the exact same way from should grow since I last about growth following an inter- thinking about growth and when I first met them. spoke to them. action or gathering more data. why their current choice is the best option. Example of measuring improvement in executing its growth strategy: 1. No Improvement 2. Slight Improvement 3. Moderate Improvement 4. Strong Improvement This company does not This company has made This company has taken several This company has implement- clearly present new data some initial progress actions towards executing their ed substantive improvements or insight on how it will towards executing their improved growth strategy (e.g. in their growth strategy (e.g. grow. growth path (e.g. inter- did market testing and learned new partnerships/channels viewed a few stakeholders whether or not to pursue an to reach more customers and for initial insights). opportunity further). markets; sales growth). Evaluate the founding team’s demonstrated ability to make improvements when 3 conducting due diligence: ▶ When meeting with the founding team, ask: What progress have you made since you first pitched to us/we last met? 9 STEP #2 Other general questions you can ask the founding team include: • Have you made any changes since you first pitched to us/we last met? Why did you make those changes? • Have you reached new milestones? How did you reach them? • Have you changed the way you think about your startup’s future and path to get there? Why and how? Pages 12 and 13 include question banks with sample questions to evaluate improvement in the categories we used in the experiment (understanding and executing company strategies). You can also draft (and consistently use) questions more specific to the key area(s) you are evaluating improvement in. The question banks also include questions you could use to evaluate how the startup has made improvements before meeting you. ▶ Use the scale(s) you created to evaluate their improvement. Ask yourself: Does this team’s progress demonstrate improvement in this area? and/or Does the team demonstrate that they will be able to continually improve in this area? Page 14 includes a template you can use to both track the risk- and growth-related information you are collecting, and evaluate the startup’s demonstrated ability to improve in your key areas. If you meet with the startup more than once, track their improvements over time using the same template. 4 When adding startups to the watchlist: Collect all necessary baseline data on the startup and the team’s proven abilities that will allow you to accurately measure their improvement in the future. You can use the template on page 14 to collect and store this information. When you revisit the startup and assess them in the future, evaluate how much the startup has improved in your key area(s). Added benefits of measuring improvement: A) Assess the team’s ability to listen to feedback, B) It should help the investor determine if the adapt, and make changes when appropriate. team will be able to collaborate with them to Observing and evaluating how a team has improved develop new strategies in the future. can give investors insights on how the team has The information a team shares each time they listened to feedback (from them or other investors make changes and improve their strategies could or mentors), how easily they adapt, and how well give investors a glimpse into how well the team they judge what they believe would or would not be works together, their thought processes, and beneficial for the startup’s growth. early organizational culture. 10 Question Banks and Template to Incorporate Steps 1 and 2 Into Your Evaluation Process Question Banks The table below provides examples of growth- and risk-related questions: Growth Risk What potential regulatory issues or future What makes you unique? government mandates are possible? How long do users stay on the platform before How do you plan to increase sales? leaving? How much can revenue increase by expanding What prevents the competition from starting into new markets? to offer similar services? What is the proof of impact? What actually Does the company have plans to work with changes in the lives of the customers, and how channel partners? do you know? What is the opportunity to upsell or cross-sell What is your timeline to break even? in your customer base? How are you going to acquire new customers? How are you going to protect your IP? What strategy do you have in place to enter How will you ensure quality as you scale? markets with similar offerings? What is the current split between local Are you able to produce your product overseas? customers and those in international markets? 11 The question banks below provide sample questions you can use to evaluate improvement in the categories we used in our experiment (growth and risk mitigation strategies). If you are going to develop questions for other key areas, you can adapt these questions accordingly. Ability to Understand Potential for Growth Main guiding question: How much has this company improved in understanding its path to growth? Guiding questions for Guiding questions for repeat meetings first/singular meetings • How has your growth strategy • Have they made any updates to their growth strategy based on feedback evolved? Why did you choose the you or others at the organization gave them? growth path you are taking? • Has this company presented more data or details explaining their overall venture growth strategy (e.g. the choices they made to pursue specific markets, partners, team members, etc?) • Has this company reworked or restructured the presentation of their company’s strategy to better explain their path to growth, and why they chose the growth path they are taking? Ability to Execute Growth Strategy Main guiding question: How much has this company improved in executing its path to growth? Guiding questions for Guiding questions for repeat meetings first/singular meetings • What is the most material • Have they made progress on executing their goals in any investment change to your business model criteria category? you have made? • Has this company done any new hypothesis testing or market research • What hypothesis testing or to better demonstrate its path to growth (e.g. with new markets, market research have you done products, or partnerships?) to better demonstrate your path • Has this company demonstrated that it has considered multiple paths to to growth i.e., with new markets, growth and chosen to execute in the most relevant path right now? products, or partnerships? • What other paths to growth did you consider? How did you decide executing on your current growth strategy was most relevant right now? 12 Ability to Understand Risks Main guiding question: How much has this company improved in understanding its risks? Guiding questions for Guiding questions for repeat meetings first/singular meetings • How has your understanding of • Have they explicitly listed more and more relevant risks for their your company’s risks evolved? company than previously? • What risks did you identify • Has this company added more nuance or data to their existing risks to and how might they affect the acknowledge the needs their company may have to overcome them? company’s strategy? How did you • Has this company mapped multiple risks, how they might affect the prioritize the most important whole company strategy, and prioritized the most important for them ones to mitigate for right now right now and in the future? and in the future? Ability to Execute Risk Mitigation Strategy Main guiding question: How much has this company improved in executing on risk mitigation? Guiding questions for Guiding questions for repeat meetings first/singular meetings • What is the most material • Have they taken steps to synthesize data on how they might mitigate change you have made to their risks i.e., talking to potential suppliers or different partners for mitigate risk in your business potential market channels? model? • Has this company tested/piloted/launched some risk prevention • What risk prevention strategies strategies, i.e., adding a new supplier, adding a board or team member have you tested/piloted/ to help with risks? launched? (e.g. adding a new • Has this company prioritized dealing with the most concerning risks supplier, adding a board or team that it can right now? member to help with risks?) 13 Template Use this template to help you ensure you have a sound understanding of a startup’s risks and growth opportunities, as well as to evaluate the founding team’s demonstrated ability to improve in the key area(s) you consider the most important for a startup’s success and future growth. Identifying Risks and Growth Opportunities: Growth Opportunities Risks Do you have a sound understanding of their growth Do you have a sound understanding of their risks? opportunities? What additional information would What additional information would you need to have you need to have a comprehensive understanding a comprehensive understanding of their risks? List of their growth opportunities? List questions: questions: List identified growth opportunities: List identified risks: Evaluating Ability to Improve in Understanding [key area]: Timeframe 1 - List how they have improved in understanding [key area]: Improvement Score (1-4) Timeframe 2 - List how they have improved in understanding [key area]: Improvement Score (1-4) Evaluating Ability to Improve in Executing [key area]: Timeframe 1 - List how they have improved in executing [key area]: Improvement Score (1-4) Timeframe 2 - List how they have improved in executing [key area]: Improvement Score (1-4) Note: Duplicate the section on Evaluating Improvement if you are evaluating improvement in more than one area. 14 Step #3 Pre-define what evaluation criteria will most heavily determine how you assess a company Before pre-defining what evaluation criteria will determine how you assess a company, it is important to first ensure your evaluation framework allows you to consistently and comprehensively evaluate ALL of the categories that are important to you when evaluating a startup. Evaluation processes are less objective when they lack comprehensive criteria because investors can easily adapt how they evaluate depending, among other things, on the gender composition of the founding team. If you do not already have set evaluation criteria, check out Village Capital’s VIRAL framework as an example. Having set criteria will help ensure your assessments are accurate and consistent. Look back on your investments over the past six months to identify the most common key factors in your decision-making. In the table on the next page are examples of questions that can help you to determine if your evaluation framework comprehensively captures the elements that influence your decision-making, and identify what is missing. 15 STEP #3 List your current evaluation criteria. Compare it to your Using your answers in the first Guiding questions to identify list in the second column. Are column, list the key evaluation key evaluation criteria your current evaluation criteria criteria you identified: comprehensive? Highlight missing criteria. Think about two or more startups in which you seemed almost equally inclined to invest, but ended up investing in only one. What was the key differentiating factor? What important milestones does a startup need to have met for you to invest in them? What is one key characteristic or milestone shared by all of the startups you have invested in recently? What traits, abilities, and knowledge do you most often look for and value in a team? For example: coachability, grit or perseverance, ability to hold one’s own, or alignment in values. Among the startups you have invested in over the last six months, what characteristic(s) most appealed to you from each? 16 STEP #3 How to Predefine Which Criteria Will Most Heavily Determine the Score or Assessment of a Startup: 1 Add a question at the beginning of the scorecard asking the investor to predefine the weight that they will apply to each criterion when evaluating startups. The template below illustrates how to do this. 2 If you invest in startups in different sectors the weight you apply to each criterion may vary by sector. If so, consistently pre-weigh your criteria for each sector. It is important to evaluate startups within each sector consistently. Criteria Think about how you will make your decisions and weigh each of the criteria below by importance in percentages terms. Make sure it adds up to 100%. Keep this distribution in mind when scoring each startup. Insert Criterion 1 (%) Does the startup [insert guiding question]? Score: Insert Criterion 2 (%) Does the startup [insert guiding question]? Score: Insert Criterion 3 (%) Does the startup [insert guiding question]? Score: Insert Criterion 4 (%) Does the startup [insert guiding question]? Score: Insert Criterion 5 (%) Does the startup [insert guiding question]? Score: The table on the next page provides an example using Village Capital’s standard evaluation categories. 17 STEP #3 Note: We include the percentages below not as a recommended weighting but to illustrate how you can pre-define and weigh the criteria in your own evaluation process. Doing so will ensure that you think ahead of time about which criteria are most important to you. Criteria/Categories Think about how you will make your decisions and weigh each of the criteria below by importance in percentages terms. Make sure it adds up to 100%. Keep this distribution in mind when scoring each startup. Team How confident are you that this team will deliver results and can make the Score: (15%) right hires as it grows? Vision How confident are you that this vision is big enough to continue to scale and Score: (15%) take on new challenges in the next decade? Value Proposition How confident are you that the company’s solution solves a major pain point Score: (20%) and will continue to deliver specific, measurable value to delight customers? Product How confident are you that the product can expand to multiple offerings and Score: (10%) outpace the market on innovation? Market How confident are you that this company’s target market is viable enough to Score: (10%) build a profitable company? Business Model How confident are you that the company’s business model is viable and that Score: (10%) it can make money? Scale As this company scales, are you confident that it can become or remain the Score: (10%) number one or number two in the market? Investor Exit How confident are you that the company will be able to grow large enough to Score: (10%) meet its investor or other capital commitments? If you do not use scorecards you should minimally weigh your criteria so that it is codified and known what is guiding your investment decisions, and by how much. 3 Review predefined criteria every time you evaluate a startup: Before completing an evaluation rubric (or other assessment), review your evaluation criteria and the weight you had assigned to each criterion. Reminding yourself of the defined criteria reduces room for bias to subconsciously enter their evaluation and helps ensure you evaluate all startups consistently6. 4 After selecting startups for investment, cross-check your picks against your pre- defined evaluation criteria: At the end of every investment committee meeting, revisit the criteria. Ask yourself: Are the startups we are putting forward to be invested in still the top startups according to the criteria we set? Are there any inconsistencies? If so, why? Double checking that the startups selected are the best according to your pre defined criteria ensures you are making investment decisions aligned with the “north stars” you had originally defined, and builds in a checkpoint to see if factors such as gender bias may be influencing the final decision. 18 ENDNOTES 1 Dana Kanze, Laura Huang, Mark A. Conley, and E. Tory Higgins, “Male and Female Entrepreneurs Get Asked Different Questions by VCs — and It Affects How Much Funding They Get,” Harvard Business Review, June 27, 2017, https://hbr.org/2017/06/male-and-female-entrepreneurs-get-asked-different-questions-by-vcs-and-it- affects-how-much-funding-they-get 2 Alan Benson, Danielle Li, and Kelly Shue, “Potential” and the Gender Promotion Gap,” June 22, 2022, https:// danielle-li.github.io/assets/docs/PotentialAndTheGenderPromotionGap.pdf 3 Alice A. Eagly and Steven J. Karau, “Role congruity theory of prejudice toward female leaders,” Psychological Review, 2022, https://www.women-unlimited.com/wp-content/uploads/prejudice_against_women.pdf 4 Eric Luis Uhlmann and Geoffrey L. Cohen, “Redefining Merit to Justify Discrimination,” Psychological Science, June 1, 2005, https://doi.org/10.1111/j.0956-7976.2005.01559.x 5 Uhlmann, “Redefining Merit to Justify Discrimination,” https://doi.org/10.1111/j.0956-7976.2005.01559.x 6 Uhlmann, “Redefining Merit to Justify Discrimination”. https://doi.org/10.1111/j.0956-7976.2005.01559.x PHOTO CREDITS: Natalia Jidovanu (PG 6, PG 7, PG 15) Ellaine Rwemiga (PG 11) 19