TECHNICAL NOTE The Next Wave of Suptech Innovation Suptech Solutions for Market Conduct Supervision MARCH 2021 Finance, Competitiveness & Innovation Global Practice © 2021 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of the World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Pub- lisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@ worldbank.org. CONTENTS Acknowledgments  iii Acronyms and Abbreviations  iv EXECUTIVE SUMMARY 1 1. INTRODUCTION 4 2. CATEGORIES OF SUPTECH SOLUTIONS FOR MARKET CONDUCT SUPERVISION 7 3. SUPTECH SOLUTIONS FOR MARKET CONDUCT SUPERVISION 10 3.1. Regulatory Reporting 10 3.1.1. Supervision Information Systems 10 3.1.2. Automated Data Submission via API 13 3.1.3. Web Portal Data Upload with Central Database 14 3.2. Collection and Processing of Complaints Data 15 3.2.1. Complaints Management System 15 3.2.2. Analysis of Unstructured Complaints Data 16 3.3. Non-traditional Market Monitoring 17 3.3.1. Web Scraping 17 3.3.2. Social Media Monitoring 18 3.3.3. Consumer Sentiment Analysis 19 3.3.4. Reputational Analysis 19 3.3.5. Dark Web Monitoring 20 3.4. Document and Business Analysis 20 3.4.1. Document Analysis for Regulatory Compliance 20 3.4.2. Document Analysis for Examination of FSPs 21 3.4.3. Document Analysis for Peer Group Comparison 22 3.4.4. Validation of Terms and Conditions 22 3.4.5. Automated Review of New Provider Registrations 23 3.4.6. Predictive Modeling of Financial Statements 23 3.4.7. Business Intelligence and Geospatial Analysis 24 3.4.8. Managed Data Platform 24   i ii   The Next Wave of Suptech Innovation 4. PEOPLE, PROCESS, AND IT INFRASTRUCTURE: THREE KEY ENABLERS 25 FOR SUPTECH IMPLEMENTATION 4.1. People: Culture and Skillsets 25 4.2. Process: Internal Champions and Strong Governance 26 4.3. Underlying IT Infrastructure 27 5. IMPLEMENTATION CONSIDERATIONS 28 5.1. Key Decisions in Suptech Implementation 28 5.2. Initiatives to Accelerate Suptech Implementation 31 5.3. Additional Challenges Encountered by Regulators 33 6. LOOKING FORWARD 34 REFERENCES 35 FIGURES 1. Suptech Solutions for Market Conduct Supervision and Key Enablers for Implementation 3 2. Suptech Solutions for Market Conduct Supervision and Key Enablers for Implementation 6 3. A Function-Based Suptech Taxonomy with Suptech Use Cases 7 4. Results Framework for Market Conduct Suptech Solutions 9 5. Overview of Suptech Solutions for Market Conduct Supervision 11 6. Dataflow Diagram of SIS Solutions 13 7. CMS Case Workflow and Data Architecture 15 8. Dataflow Diagram for Social Media Monitoring 18 9. Example of Dataflow Diagram in Document Analysis Solutions 21 10. Considerations for In-House Development Versus Using a Third-Party Vendor 29 CASE STUDIES 1. How BNR Designed Its SIS Solution 12 2. How NBR Develops Suptech Solutions 14 3. How Researchers at Princeton University and FSD Kenya Worked with the Central Bank of Kenya 17 to Analyze Complaints Data 4. The FCA’s Development of Sleuth, Its NLP Platform 21 5. How AFM Prioritized People within Its Transformation to Data-Driven Supervisors 25 6. How ASIC’s Innovation Office Collaborates with Industry Stakeholders 32 BOX 1. FinCoNet: SupTech Tools for Market Conduct Supervisors 33 ACKNOWLEDGMENTS This technical note is a product of the Financial Inclusion and Consumer Protection Team in the World Bank Group’s Finance, Competitiveness and Innovation Global Practice. This note was prepared by Ligia Lopes (former Senior Financial Sector Specialist, World Bank), Jennifer Chien (Senior Financial Sector Specialist, World Bank), Mackenzie Wallace (Market Con- duct Supervision Consultant), and Edoardo Totolo (Operations Officer, International Finance Corporation). Mahesh Uttamchandani (Practice Manager, World Bank) provided overall guid- ance. The team is grateful for the substantive feedback received from peer reviewers Douglas Randall (Financial Sector Specialist, World Bank) and Matei Dohotaru (Senior Financial Sector Specialist, World Bank), and from the International Financial Consumer Protection Organisation (FinCoNet). Editorial inputs were provided by Charles Hagner and design and layout assistance was provided by Debra Naylor of Naylor Design, Inc. The team also gratefully acknowledges the generous contributions of time and expertise by financial authorities at the Australian Securities and Investments Commission, the Authority for the Financial Markets (Netherlands), Autorité des Marchés Financiers (Québec, Canada), Banco de Portugal, Bank of England, Bangko Sentral ng Pilipinas (Philippines), the Central Bank of Ireland, the European Insurance and Occupational Pensions Authority, the Financial Conduct Authority (United Kingdom), the National Bank of Rwanda, and Nepal Rastra Bank. Finally, the team gratefully acknowledges the generous financial support of the Ministry of Foreign Affairs of the Kingdom of the Netherlands and the Bill & Melinda Gates Foundation under the Financial Inclusion Support Framework (FISF) program, without which preparation of this paper would not have been possible.    iii ACRONYMS AND ABBREVIATIONS ADF automated dataflow AFM Authority for the Financial Markets (Netherlands) AMF Autorité des Marchés Financiers (Québec, Canada) API application programming interface ASIC Australian Securities and Investments Commission BdP Banco de Portugal BI business intelligence BNR National Bank of Rwanda BOE Bank of England BOL Bank of Lithuania BSP Bangko Sentral ng Pilipinas (Philippines) CBI Central Bank of Ireland CFPB Consumer Financial Protection Bureau (United States) CMS complaints management system CRM customer relationship management EDW Electronic Data Warehouse EIOPA European Insurance and Occupational Pensions Authority EU European Union FCA Financial Conduct Authority (United Kingdom) FSP financial service provider MVP minimum viable product NLP natural language processing NRB Nepal Rastra Bank SIS supervisory information system USD United States dollar iv   EXECUTIVE SUMMARY Around the world, financial sector supervisors are Four key insights for market conduct authorities can experiencing a profound shift to data-driven supervi- be drawn from this note: sion enabled by the next wave of technology and data solutions.1 While technology and data are not new to INSIGHT 1:  Increasing operational efficiency and enhanc- financial oversight, their specific application to financial ing supervisory effectiveness are two of the primary consumer protection and market conduct supervision is a motivations for adopting suptech solutions for market newer and welcome trend. conduct. In implementing suptech, financial authorities are often Supervisory technology, or suptech, refers to the use of driven by two different motivations: (1) increasing oper- technology to facilitate and enhance supervisory pro- ational efficiency and (2) improving hypothesis-driven cesses from the perspective of supervisory authorities. supervision. The former often involves automating busi- As highlighted in the World Bank’s 2018 discussion note ness processes by replacing elements of the supervision on suptech for market conduct supervision (World Bank decision framework with data and algorithms, bringing sig- 2018), examples of suptech for market conduct supervi- nificant efficiencies to the process, while the latter involves sion were initially limited. In recent years, the application helping supervisors to test and prove hypotheses using of suptech for market conduct supervisory purposes has new sources of analyses or data. become more widespread and sophisticated. Recent advancements, particularly in the realm of unstructured Given limited capacity at many financial authorities, and text analysis, present opportunities for market con- implementation of suptech for market conduct often duct supervision where a greater reliance on qualitative focuses on solutions to increase operational efficiency. assessments is required. The rationale is to make existing staff more productive and to enable them to focus on higher-value activities. This technical note draws from a wide set of regulatory Repetitive or time-consuming tasks such as data cleaning experiences to showcase new suptech solutions spe- or document, data, or complaints intake and process- cific to market conduct supervision. The main objective ing are prime candidates for suptech automation. From of this note is to assist market conduct authorities, partic- an initial focus of operational efficiency, some market ularly those in low- and middle-income countries, to build conduct supervisors have since expanded their overall and enhance supervisory capacity and efficiency by pro- approach to include enhancing the effectiveness of their viding concrete examples where supervisory technology supervisory program. can be leveraged. Solution is used in this note to refer to an implementation of people, processes, information, and technology that supports a set of business or 1.  technical capabilities that solve one or more business problems.   1 2   The Next Wave of Suptech Innovation INSIGHT 2:  Suptech solutions for market conduct super- insights in seconds where previously it would have vision can be grouped into four categories. taken supervisors weeks (if even possible at all). Given the more qualitative nature of market conduct super- This technical note explores 18 suptech solutions for vision, advancements in the analysis of text present a market conduct, grouped into the following four cate- potentially significant breakthrough. gories. These categories generally align with their respec- tive supervisory activity, rather than groupings based on In each of the above categories, the suptech solutions technological functionality (which is another approach for described span the data life cycle of a specific supervi- categorizing suptech solutions). sory activity. The solutions within each category present a collection of tools that enable supervisors to collect new Solutions for regulatory reporting by supervised 1.  forms of data or introduce new, more efficient methods for institutions: A primary method for regulators to iden- collecting such data. Suptech solutions can also be used tify market conduct risks and issues is to collect infor- to conduct richer analyses on an exponentially increasing mation directly from supervised institutions, but doing amount of information with limited analytical resources. this can be time consuming and labor intensive. Web These collections of suptech solutions therefore provide portals, application programming interfaces (APIs), market conduct supervisors with both gains in efficiency automated dataflows (ADFs), and comprehensive and the ability to extract new insights to allow for data- supervision information systems (SISs) allow for auto- driven decision making. mated and standardized regulatory reporting that col- lects, validates, transforms, and stores data in real time. INSIGHT 3:  Suptech implementation is about more than just the technology. Solutions for collection and processing of com- 2.  plaints data: Complaints data is one of the most val- Embedding modern technology and data into the ued data sources for market conduct supervisors. A supervisory process is often an ongoing effort. Imple- complaints management system (CMS) is key to the menting suptech solutions requires more than just the efficient processing of these complaints and capturing solution. It requires making investments in three key and managing data to maximize its accuracy and value enablers: people, process, and IT infrastructure. The for supervisory purposes. The application of advanced culmination of broader efforts to implement suptech analytics to complaints data, particularly to unstruc- solutions and underlying enablers is organizational trans- tured text, represents the next step for market conduct formation into a data-driven supervisor. supervisors to deduce new insights in a more efficient • People refers to the talent, mindset, and skills of manner from complaints data. employees and the larger organizational culture toward data and technology. Solutions for non-traditional market monitoring: 3.  The internet provides the opportunity to utilize a • Process refers to how suptech ideas are supported range of new, non-traditional methods for monitoring from ideation to implementation, including how supt- the market, another core activity for market conduct ech is championed and governed. supervision. Monitoring social media, online news, • IT Infrastructure refers to the underlying IT infrastruc- websites, and so on can provide early warning signals ture and capabilities needed to develop and operate of emerging consumer risks. Foundational to these suptech solutions internally. types of solutions is web scraping, which provides the mechanism for collecting and gathering online text for INSIGHT 4:  Various strategies can be used to help accel- analysis. Such text can be used for social media mon- erate the development and implementation of suptech itoring, reputational analysis in the news, consumer solutions. sentiment scoring, and dark web monitoring. Non-tra- ditional market monitoring provides supervisors a use- Some financial authorities have benefited from the ful complement to traditional market monitoring. creation of formal, multiyear suptech or data strate- gies. Innovation offices can also be leveraged to provide 4.  Solutions for document and business analysis: a central place to encourage internal suptech ideation Advances in analytics have been most profound in and learning, as well as improving dialogue with such the realm of unstructured text data. For example, nat- external parties as fintechs or potential suptech solution ural language processing (NLP) solutions can ingest providers. and analyze large quantities of documents, extracting The Next Wave of Suptech Innovation   3 FIGURE 1. Suptech Solutions for Market Conduct Supervision and Key Enablers for Implementation CATEGORIES Collection and Non-traditional Document Regulatory OF SUPTECH Processing of Market and Business Reporting SOLUTIONS Complaints Data Monitoring Analysis • Supervision • Complaints • Web scraping • Document analysis for information systems management • Social media regulatory compliance EXAMPLES • Automated data system monitoring • Document analysis for OF SUPTECH submission via API • Analysis of examination of FSPs SOLUTIONS • Consumer • Web portal data unstructured sentiment • Document analysis for upload with central complaints data analysis peer group comparison database • Reputational • Validation of terms and analysis conditions • Dark web • Automated review of new monitoring provider registrations • Predictive modeling of financial statements • Business intelligence & geo-spatial analysis • Managed data platform People KEY ENABLERS FOR Process IMPLEMENTATION IT Infrastructure In some instances, it is more appropriate to begin The expansion of digital activity prompted by the with an incremental, targeted approach, rather than a COVID-19 pandemic reemphasizes the necessity and broader institutional strategy. Supervisors in low- and value of suptech for financial authorities. This is true for middle-income countries will inevitably face challenges all categories of suptech solutions for market conduct. during implementation. Common challenges include The direct and automated collection of granular regula- underdeveloped supervisory risk frameworks, staffing and tory data from supervised institutions is critical to replac- resource constraints, and technology constraints among ing on-site examinations, as is the ability of supervisors to financial service providers (FSPs). However, successful engage directly with consumers and manage their com- implementation of suptech solutions in these contexts plaints with providers digitally. Meanwhile, both non-tradi- can provide more meaningful gains to efficiency and tional market monitoring and advanced text analysis allow effectiveness in low-capacity countries. These constraints supervisors to monitor fast-moving sentiment remotely favor a targeted approach to suptech implementation and emerging risks to consumers on a more rapid basis. that focuses scarce time, attention, and resources. Such tools that enable supervisors to oversee the Utilizing experimentation and iteration in the financial sector with increased effectiveness and effi- technology-development process can be beneficial. ciency will only become more critical as digital trans- In place of traditional approaches such as “waterfall,”2 formation continues. The initial successes experienced authorities now increasingly use design or tech sprints, by the authorities referenced in this technical note offer a proofs of concept, prototypes, pilots, “minimum via- glimpse of this future—one in which data and technology ble products,” and agile approaches. Such approaches become core to the operations, identity, and culture of all engage and validate capabilities with end users, both supervisors. Such tools hold the promise to help empower ensuring the utility of the solution when delivered and financial authorities to meet the market conduct supervi- condensing the implementation timeline. sory challenges of the next decade. Waterfall software development methodology refers to a linear, sequential approach whereby customer and business requirements are gathered at 2.  the beginning of the project and the technology solution is developed following a sequential project plan to accommodate those requirements. 1. INTRODUCTION Financial sector authorities around the world are expe- A new generation of more advanced suptech solutions riencing a profound shift to data-driven supervision is currently emerging, driven by the latest technological enabled by robust technology and data solutions. While innovations in big data architecture, machine learning technology is obviously not new to financial authorities, (especially NLP), and automated data collection and man- this new wave of digital solutions holds the promise to agement. In this note, the term suptech refers to the use increase the efficiency and effectiveness of supervision in of technology to facilitate and enhance supervisory pro- order to meet key regulatory objectives, including finan- cesses from the perspective of supervisory authorities. cial stability, financial integrity, and, increasingly, financial consumer protection. This technical note showcases new While technology solutions are not new to financial supervisory technology, or suptech, solutions specific to oversight, their specific application to market conduct market conduct supervision that can assist financial sector is a newer and welcome trend. Historically, technol- authorities—including in low- and middle-income coun- ogy solutions for quantitative analysis have been more tries—to enhance and strengthen financial consumer pro- advanced than qualitative ones, with greater application tection and market conduct supervision. for prudential supervision. Recent advancements in data and technology, such as NLP and other machine-learning While regulators have always leveraged data and tech- applications, present new opportunities for market con- nology for supervisory purposes, a marked increase in duct supervisors by enabling greater qualitative analyses. new and ambitious initiatives has occurred in recent As highlighted in the World Bank’s previous discussion years. Examples include the introduction of “TechSprints” note on suptech for market conduct supervision (World at the Financial Conduct Authority (FCA) in the United Bank 2018), examples of suptech for market conduct Kingdom, development of the Electronic Data Ware- supervision were initially limited to complaints data collec- house (EDW) at the National Bank of Rwanda (BNR), and tion and analyses. In the past few years, the application of the launch of “Step 1” technology transformation at the suptech for purposes of market conduct supervision has Authority for the Financial Markets (AFM) in the Nether- become more widespread and sophisticated, as explored lands,3 among many other technological developments at in this note. financial authorities worldwide. Suptech solutions are increasingly critical given the This latest wave of new suptech solutions builds on digital transformation of the financial services industry earlier generations of technology solutions. Most in recent years. Supervisors have often lagged behind in supervisory technology to date has focused primarily on their capacity to monitor these growing and increasingly data-management workflows and descriptive analytics. complex markets. However, supervisors can leverage the However, many of these solutions involve a certain degree technological advances behind digital transformation to of manual processing or had other limitations (BIS 2019). overcome resource constraints and make processes and AFM (Netherlands) developed a multiyear, three-phase suptech transformation program: Build, Pilot, and Transform. The “Build” phase began 3.  with an assessment of AFM’s own data and analytics capacity. 4   The Next Wave of Suptech Innovation   5 procedures more effective and efficient. In the face of lim- for market conduct supervision, and (2) practical consid- ited capacity and resources, a particular concern in low- erations for successful implementation of a data-driven er-income economies, suptech can be used to leverage supervision program, such as investments and organiza- data and technology to supervise financial services more tional changes required to support implementation. efficiently and effectively for market conduct. The main audience for this note is market conduct While adoption has been most pronounced in high- authorities and other stakeholders in low- and mid- income countries, suptech solutions are relevant and dle-income countries. Considering that the potential for translatable to lower-capacity countries. The uneven gains in supervisory efficiency and effectiveness is high global uptake of suptech solutions can be partly attributed in lower-capacity countries, this note highlights solutions to the additional logistical barriers that supervisors in low- that can be adapted to these contexts and practical con- and middle-income countries often face. However, the siderations in doing so. In addition, the note should bene- broadening landscape of suptech solutions presents such fit development practitioners assisting financial authorities authorities with the opportunity to learn from technology by informing the development and design of technology examples in other countries. Many of these examples of support programs. technology and data solutions can still be translated and adapted to countries with lower capacity, where their Information Sources potential for positive impacts on supervisory efficiency and effectiveness may be even more powerful. This note draws from a wide set of regulatory expe- riences and is the result of primary and secondary research with 14 financial authorities. These financial Research Objectives and Key Audience authorities represent a diverse cross-section with var- The main objective of this note is to assist market con- ied levels of financial market development and internal duct authorities in their efforts to build and enhance capacity. Each authority was selected on account of its supervisory capacity and efficiency by providing con- successful track record in developing suptech solutions for crete examples of situations in which supervisory tech- market conduct supervision. Research methods included nology can be leveraged. An efficient market conduct interviews, demonstrations, questionnaires, and reviews supervision framework requires the collection of a wide of internal materials, external publications, and public-fac- range of data from disparate sources; doing this is chal- ing websites. lenging in many jurisdictions. Market conduct supervi- sors must also undertake complex qualitative analyses to The following financial authorities contributed critical determine compliance with legislation or regulation that is inputs to this note: often principles-based or composed of judgement-based • Australian Securities and Investments Commission rules. These challenges are compounded when supervi- • Authority for the Financial Markets (Netherlands) sors have under their jurisdiction a large diverse range of • Autorité des Marchés Financiers (Québec, Canada) FSPs with unique or unfamiliar risk profiles. Consequently, • Banco de Portugal market conduct supervision continues to be manual and • Bangko Sentral ng Pilipinas (Philippines) labor-intensive in many countries. Suptech presents the • Bank of England opportunity to enhance both supervisory capacity and • Bank of Lithuania efficiency to tackle these inherent operational challenges, • Central Bank of Brazil particularly important in light of growing and rapidly digi- • Central Bank of Ireland tizing financial markets. • Consumer Financial Protection Bureau (United States) •  European Insurance and Occupational Pensions While collective knowledge on suptech has grown in Authority recent years,4 the literature specific to market conduct • Financial Conduct Authority (United Kingdom) supervision is limited. This note seeks to address this • National Bank of Rwanda gap by providing financial authorities with (1) an array of • Nepal Rastra Bank5 concrete examples of suptech solutions that can be used Since the World Bank published its note on suptech for market conduct supervision (World Bank 2018), organizations such as the Bank of Inter- 4.  national Settlements, International Financial Consumer Protection Organization, Toronto Center, Milken Institute, R2A, Consultative Group to Assist the Poor, Columbia University, and others have published on suptech. The Central Bank of Brazil, Bank of Lithuania, and Consumer Financial Protection Bureau (United States) contributed to the 2018 World Bank 5.  discussion note on suptech (World Bank 2018). 6   The Next Wave of Suptech Innovation Structure of Technical Note CHAPTER 4: People, Process, and IT Infrastructure: Three Key Enablers for Suptech Implementation. The technical note is structured into the following Successful implementation of a suptech solution goes chapters: beyond the technology itself. Three suptech enablers are critical for implementation: (1) people, (2) process, and (3) CHAPTER 2: Categories of Suptech Solutions for Mar- underlying IT infrastructure. ket Conduct Supervision. Four main categories of supt- ech solutions are introduced: (1) solutions for regulatory CHAPTER 5: Implementation Considerations. Common reporting by supervised institutions, (2) solutions for col- considerations when implementing suptech solutions lection and processing of complaints data, (3) solutions emerged across country examples. Authorities often for non-traditional market monitoring, and (4) solutions face key decisions related to prioritization, determining for document and business analysis, especially of unstruc- whether to build a solution in-house or to buy from a ven- tured data. dor, and deciding how to organize data and technology staff. It is also useful to consider whether to undertake CHAPTER 3: Suptech Solutions for Market Conduct efforts to accelerate suptech adoption through formal Supervision. Individual suptech solutions for market con- suptech or data strategies, adaptive technology develop- duct are identified for each of the four main categories ment, and internal innovation offices to liaise with external noted above, and a total of 18 solutions are presented. stakeholders. For each solution, there is a description of how the solu- tion works, its benefits, and considerations for implemen- CHAPTER 6: Looking Forward. This section includes brief tation, drawing from country experience and including final thoughts on the value of suptech solutions for market detailed case studies. conduct supervisors operating in an increasingly complex environment. FIGURE 2. Suptech Solutions for Market Conduct Supervision and Key Enablers for Implementation CATEGORIES Collection and Non-traditional Document Regulatory OF SUPTECH Processing of Market and Business Reporting SOLUTIONS Complaints Data Monitoring Analysis • Supervision • Complaints • Web scraping • Document analysis for information systems management • Social media regulatory compliance EXAMPLES • Automated data system monitoring • Document analysis for OF SUPTECH submission via API • Analysis of examination of FSPs SOLUTIONS • Consumer • Web portal data unstructured sentiment • Document analysis for upload with central complaints data analysis peer group comparison database • Reputational • Validation of terms and analysis conditions • Dark web • Automated review of new monitoring provider registrations • Predictive modeling of financial statements • Business intelligence & geo-spatial analysis • Managed data platform People KEY ENABLERS FOR Process IMPLEMENTATION IT Infrastructure The Next Wave of Suptech Innovation   7  ATEGORIES OF SUPTECH 2. C The four main categories of suptech solutions for mar- ket conduct supervision are as follows: SOLUTIONS FOR MARKET CONDUCT SUPERVISION 1. Solutions for regulatory reporting by supervised insti- tutions No taxonomy of suptech solutions is widely accepted 2. Solutions for the collection and processing of com- globally. To date, most existing taxonomies have taken a plaints data function-based approach toward describing suptech eco- systems (see Figure 3). Existing suptech taxonomies tend 3. Solutions for non-traditional market monitoring to categorize suptech solutions based on the flow of data 4. Solutions for document and business analysis from collection to validation, consolidation, and analysis. This technical note takes a slightly different approach, Solutions for regulatory reporting by supervised 1.  categorizing suptech solutions for market conduct institutions supervision by supervisory activity. Unlike the other tax- A primary method for regulators to identify market con- onomies of suptech solutions, the categories employed duct risks and issues is to collect information directly from in this note extend beyond dataflow to include engaging supervised institutions. Historically, such submissions have with supervised institutions and consumers as well as new been collected manually through reporting templates types of non-traditional data collection and analysis. This submitted by mail, email, or fax, resulting in a slower, inef- categorization is not meant to be exhaustive for all possi- ficient, and more error-prone process. ble suptech solutions for market conduct but reflective of the solutions described in this note. FIGURE 3. A Function-Based Suptech Taxonomy with Suptech Use Cases Mac ro-p n rud sio ervi en tia p su Detection of Early warning l su FT networks indicators pe /C Stress testing L rv AM isi Risk scoring Network Forecasting on Big Data analysis AI/ML Market NLP surveillance AML NLP compliance Policy assessment Big Data GIS evaluations DATA Credit risk Big Data Text Electronic mining record AI/ML keeping Liquidity risk NLP NLP Dynamic Electronic Governance risk visualization document on management Centralized isi Cyber risk repositories v er Li up ce Automated data ns ls Case management tia reporting ing en prud o- Micr Supervision phases Use cases Technologies Source: World Bank (2020). 8   The Next Wave of Suptech Innovation Today, web portals, APIs, ADFs, and comprehensive 3. Solutions for non-traditional market monitoring SIS allow for automated and standardized regulatory The internet provides the opportunity to utilize a range of reporting that collects, validates, transforms, and stores new, non-traditional methods for conducting market mon- data in real time. The most sophisticated solutions rely itoring, another core activity for market conduct supervi- on machine-readable taxonomies, customer relationship sion. Monitoring social media, online news, websites, and management (CRM) systems, and data warehousing with so on can provide early warning signals of emerging con- permission-based datamarts. However, a solution need sumer and reputational risks. By keeping a pulse on con- not be overly complex to deliver immense regulatory ben- sumer sentiment in social media and web forums, these efits for market conduct supervisors, including enhanced solutions provide the potential for more uninhibited, real- efficiency and increased analytical capability. Further, time access to the “voice of the consumer” and consum- applying automated data analytics allows market con- ers’ experiences with FSPs. Overall, these web monitoring duct supervisors to support their supervisory framework solutions provide a useful, low-cost complement to tra- and prioritize scarce supervisory resources toward areas ditional market monitoring to gather regulatory insights. of greatest risk. Solutions for non-traditional market monitoring include Suptech solutions for regulatory reporting include the fol- the following: lowing: • Web scraping • SIS6 • Social media monitoring • Automated data submission via API • Consumer sentiment analysis • Web portal data upload with central database • Reputational analysis • Dark web monitoring  olutions for the collection and processing of 2. S complaints data 4. Solutions for document and business analysis Complaints data is one of the most valued data sources for market conduct supervisors. Suptech solutions in Advances in analytics have been most profound in the complaints handling alleviate the operational burden realm of unstructured text data. For example, NLP solu- through greater automation. Such solutions can also tions can ingest and analyze large quantities of docu- introduce new front-end digital channels to engage with ments, extracting insights in seconds where previously it consumers regarding their complaints and inquiries, would have taken supervisors weeks (if even possible at such as via websites, mobile apps, text messaging, and all). Given the more qualitative nature of market conduct chatbots. After initial setup, digital channels tend to be supervision, advancements in the analysis of text present lower in cost to operate, expanding regulators’ reach a potentially significant breakthrough. beyond urban areas. In addition, such solutions enhance the quality of the information collected about consumer Suptech solutions for analysis also leverage automation complaints. Advancements in database management and combine data sets together to produce a more holis- and analysis allow for supervisors to extract more under- tic view. Some suptech tools also bring in new types of standing and insight from consumer-submitted com- external data sets that were traditionally difficult to com- plaints via CMSs, providing a critical resource for market bine for analysis, such as geospatial data. Solutions for conduct supervision. advanced analytics provide market conduct supervisors with both significant gains in efficiency and the ability to Solutions for the collection and processing of consumer extract new insights from data to allow for data-driven data include the following: decision making. • CMS7 • Analysis of unstructured complaints data Solutions for regulatory reporting also include machine-readable taxonomies, data validation systems, and ad hoc transmission systems. 6.  These solutions often include both case management interfaces for supervisory staff and digital user interfaces for consumers. 7.  The Next Wave of Suptech Innovation   9 FIGURE 4: Results Framework for Market Conduct Suptech Solutions Potential Suptech use cases Automated data Advanced data validation, Platform and Data management collection processes analysis, visualization database integration and storage (use of data-pull or (cleaning and analysis (examiner dashboards, (use of cloud computing data-input systems; of unstructured data; workflow tools, merging to store big data) machine readable and identification of spikes disparate data sets) executable regulation) and trends) Potential Suptech supervisor-level outcomes Improved scope, Enabling/enhancing More efficient use More efficient information accuracy, consistency, risk-based supervision of resources flows between providers and timeliness of (better identification and (reallocation of staff away and supervisors, between collected information measurement of risk) from manual tasks) consumers and supervisors, and across supervisors Potential Suptech impacts Larger share of financial Improved consumer Improved conduct Better value for limited sector under outcomes (better of providers government resources supervision protection, increased confidence in market) Solutions for document and business analysis include the The four main categories of suptech solutions for mar- following: ket conduct supervision represent an update from the • Document analysis for regulatory compliance Suptech Conceptual Framework first introduced in the 2018 World Bank discussion note (World Bank 2018). • Document analysis for examination of FSPs As noted above, these suptech solutions drive both effi- • Document analysis for peer group comparison ciency and effectiveness at the supervisor level and ulti- • Validation of terms and conditions mately lead to potential beneficial impacts in the broader • Automated review of new provider registrations market, such as via improved consumer outcomes. • Predictive modeling of financial statements • Business intelligence and geospatial analysis • Managed data platform 10   The Next Wave of Suptech Innovation SUPTECH SOLUTIONS FOR 3.  to analyze text and speech data. This includes the ability to infer topics in text, classify and categorize documents, MARKET CONDUCT SUPERVISION and measure other text characteristics, such as sentiment. Common types of NLP algorithms found within suptech Within the four main categories of suptech solutions solutions include topic modeling, sentiment analysis, and for market conduct supervision, 18 individual solutions text summarization. NLP has the advantage of being rep- are described in this chapter. These suptech solutions licable, systematic, and more transparent, but challenges are currently operational, in pilot, or were expected to be remain. NLP requires continuous fine-tuning and interpre- operational in 2020. For each solution, information is pro- tation for its outputs to be accurate and regularly usable. vided on what the solution is, the benefits it provides, and considerations for implementing the solution. Solutions are accompanied by country examples and select case studies. 3.1. Regulatory Reporting Data and reports submitted by supervised institutions are It is worth noting that suptech solutions need not among the sources of information used most widely by always be particularly “high-tech” or the most complex market conduct supervisors to inform supervisory activi- to have real, significant supervisory benefits. The com- ties. In addition to market conduct, financial authorities plexity of suptech solutions varies within each category. regularly use technology solutions for regulatory report- What this means practically for financial authorities, espe- ing to support prudential, financial inclusion, or other cially in low- or middle-income countries, is that authorities goals. Solutions for regulatory reporting vary in their level have options. Financial authorities can focus on the solu- of complexity and are presented here beginning from the tion(s) that best matches their needs, available resources, most complex (3.1.1 “Supervision Information Systems”) and existing capabilities. Figure 5 summarizes the level of to less complex (3.1.2 “Automated Data Submission via implementation complexity across solutions. Supervisors API”) to least complex (3.1.3 “Web Portal Data Upload in lower-capacity countries evaluating potential solutions with Central Database”). from this list should first consider adding capabilities in a category(s) for which the authority does not currently 3.1.1 Supervision Information Systems have a solution. Once the authority has baseline capabil- SIS represent a comprehensive IT upgrade to the collec- ities within a category, authorities can opportunistically tion, validation, and analytics of reported information from enhance their capabilities by implementing more sophisti- supervised institutions. While the exact technical deploy- cated solutions, depending on supervisory need and avail- ment can vary among authorities, SIS solutions share the able resources. As with any investment, authorities should following technical elements: ADFs to retrieve data from evaluate the solution’s business case in context of supervi- supervised institutions; a central data warehouse with a sory goals and available resources. CRM system to store, manage, and secure documents and data; “datamarts” to facilitate permission-based access to Solutions within each category are interrelated and different teams and departments within the authority; and complementary. When viewed together, suptech solu- business intelligence (BI) tools that equip supervisory staff tions within each category span the data life cycle related to analyze and monitor data for trends and risks. to the specified supervisory activity. Individual solutions may allow authorities to collect new forms of data, intro- The solution’s high complexity requires a significant duce new methods for its collection, or conduct new or investment of organizational time and resources. This richer analyses of this information. This is particularly true often includes external consultants and software vendors, as it relates to new types of analytics, whose functionality in addition to in-house technology staff. Involvement of is common across all four categories of suptech solutions supervisory staff is also crucial to ensure the solution is but can be employed to serve specific supervisory use designed appropriately to support an authority’s specific cases requiring domain expertise. supervisory framework. This includes considering defini- tions of standards and reporting guidelines to supervised In particular, the latest wave of advanced analytical entities and the solution’s data validations. solutions in multiple categories is enabled by NLP. NLP refers broadly to the ability of computers and algorithms The Next Wave of Suptech Innovation   11 FIGURE 5: Overview of Suptech Solutions for Market Conduct Supervision SUPERVISOR IMPLEMENTATION CATEGORY SOLUTION DESCRIPTION EXAMPLES COMPLEXITY & COST8 3.1 Regulatory Supervision information 3.1.1  Comprehensive IT upgrade to the BNR, AMF Most sophisticated Reporting systems (SIS) collection, submission, and analytics of FSP reported data Automated data submission 3.1.2  FSPs prepare database extracts and share BSP Moderate sophistication via API data via consolidated API transmission Web portal data upload with 3.1.3  Low-complexity data sharing solution to NRB Foundational capability, central database replace manual data sharing over email, inexpensive fax, or not at all. Collection & 3.2  Complaints management 3.2.1  Automates complaints handling BOL, CFPB, Moderate sophistication Processing of systems (CMS) processes, improves data quality, BSP Complaints and introduces digital interfaces for Data consumers and case workers Analysis of unstructured 3.2.2  Identifies topic, sentiment, and thematic FSD Kenya Inexpensive, but requires complaints data patterns in consumer complaint text analytics staff 3.3 Non-traditional 3.3.1 Web scraping Gathers text data from online sources FCA, AMF, Foundational capability, Market (e.g., FSP website, social media, web CBI inexpensive Monitoring forms, blogs, news) 3.3.2 Social media monitoring Topical analysis of consumer posts on FCA, AMF, Inexpensive, but requires social media related to FSPs or financial CBI, EIOPA analytics staff products 3.3.3 Consumer sentiment analysis Analysis of consumers’ tone and emotions BOE, AMF, Inexpensive, but requires in their interactions with FSPs online CBI analytics staff 3.3.4 Reputational analysis Analysis of news media’s view of specified AMF Inexpensive, but requires FSPs analytics staff 3.3.5 Dark web monitoring Identify fraud, scam, etc. risks on the BOE Moderate dark web sophistication 3.4 Document Document analysis for 3.4.1  Inspects FSP-provided documents to FCA Inexpensive, but requires and Business regulatory compliance determine compliance with specified analytics staff Analysis regulations Document analysis for 3.4.2  Topical analysis of FSP-provided AMF Inexpensive, but requires examination of FSPs documents to scope and support analytics staff supervisory examinations Document analysis for peer 3.4.3  Analysis of FSP-provided documents to BOE Inexpensive, but requires group comparison spot risks and trends across a peer group analytics staff Validation of terms and 3.4.4  Automation of the review of product BdP Inexpensive, but requires conditions terms and conditions to identify analytics staff compliance risks Automated review of new 3.4.5  Evaluates and identifies new provider or AFM Inexpensive, but requires provider registrations product registrations that are higher-risk analytics staff Predictive modeling of 3.4.6  Evaluates financial statements for AFM Inexpensive, but requires financial statements misstatement or other risks analytics staff Business intelligence (BI) & 3.4.7  Supports analysis and interpretation AMF, BOE, Ranges from low to high geo-spatial analysis of data, often a complement to other FCA, NRB, complexity suptech solutions AFM 3.4.8 Managed data platform Standardizes, centralizes, and makes AFM Most sophisticated accessible internal data from a multitude of sources Implementation costs are from the authors’ interpretation of anecdotal information. 8.  12   The Next Wave of Suptech Innovation The solution’s benefits can be substantial. BNR designed a direct connection to the IT systems of supervised insti- its solution, called the Electronic Data Warehouse (EDW), tutions or, more commonly, through “middleware” that to centralize data from across the authority into a single serves as an intermediary between the SIS and the IT internal data store for comprehensive analysis, including systems of supervised institutions. An advantage of mid- data from the national payments system, credit refer- dleware is its interoperability with the various types of ence bureaus, and the statistics department. Autorité des databases used by supervised institutions (for example, Marchés Financiers (AMF) in Québec, Canada designed Oracle, SQL, MySQL, and so on). This interoperability its solution to serve as an Offsite Supervision System allows supervised institutions to continue to use their which streamlines many of the operational, cybersecurity, same provider and connect via the middleware using sim- and data integrity challenges associated with collecting ple data-transfer protocols. The middleware also adapts granular data from supervised institutions. Such granu- data from different types of databases into a common lar data is typically contained in requests for supervisory readable format for the SIS. Finally, the middleware also information. Like BNR, AMF’s solution also centralizes and provides supervised institutions with a buffer, as the SIS compiles data sets from across the authority to create a accesses only data that the supervised institution inten- richer, more holistic view to generate insights for data- tionally makes accessible. In this way, SIS solutions using driven decision making. The supervisory infrastructure to middleware do not require access to the full database or conduct off-site examinations has become increasingly core banking systems of supervised institutions. important in 2020, as the logistics of on-site examinations are made more complex (or infeasible) due to the COVID- Data analytics and reporting through datamarts 19 pandemic. While central warehouses and CRM systems store, man- age, and protect the data retrieved from supervised insti- Pulling data directly from supervised institutions tutions, datamarts are used by supervisory staff to access An innovation of SIS solutions, ADFs allow supervisors to and analyze the data. Datamarts are typically user-permis- “pull” data directly from supervised institutions, rather sioned and facilitate access to the subsets of data within than having supervised institutions “push” data to the the central repository deemed appropriate based on job authority. This data pull can be facilitated either through role, function, department, or other distinction of the user CASE STUDY 1 How BNR Designed Its SIS Solution BNR’s EDW is an end-to-end regulatory reporting data plat- system, credit reference bureaus, and the statistics department, form with both prudential and market conduct applications. among others. It was the culmination of a three-year IT effort from proof of The EDW imposed relatively little additional burden on FSPs. concept to deployment and cost approximately USD 1M to This is a result of its technical design for software interoperabil- implement. It overhauled previous data-management sys- ity. FSPs can continue using the same database provider (for tems, requiring investments not only in hardware and software example, Oracle, SQL, MySQL, and so on) and connect to the at BNR but also (and more importantly) in upgrading staff skills EDW using simple data-transfer protocols. Further, manage- and coordination among the more than 600 institutions that ment at BNR reports that frequent engagement with FSPs, par- it supervises. ticularly relating to providers’ concerns about the level, nature, The EDW solution introduced three new dimensions to and frequency of supervisor’s access to their data, was key to its BNR’s regulatory reporting infrastructure: (i) data-pull tech- ultimate widespread adoption. nology that allows supervisors to connect directly to the Throughout the three-year initiative, management at BNR databases of FSPs and collect data from the source, rather indicated the importance of managing change within the finan- than sharing data via Excel spreadsheets; (ii) the collection of cial authority. Supervisory staff accustomed to BNR’s data-man- account-level data that provides more granular data, provided agement processes initially met the changes introduced by the daily, rather than aggregated by institution on a monthly or EDW with skepticism. Staff who performed manual data-cleaning quarterly basis; and (iii) data analytics and reporting that are or data-consolidation processes had to learn new skills to interact now automated and linked to interactive dashboards. Within with the more sophisticated system. Many also were retrained to BNR, the EDW was also designed to break down internal data perform business analysis, focusing on the analysis and interpre- siloes. As a central data warehouse, it integrates with other tation of the data (with greater value-add) rather than on such internal data sources, such as data from the national payments mechanical processes as consolidation and cleaning. The Next Wave of Suptech Innovation   13 FIGURE 6. Data Flow Diagram of SIS Solutions Data Collection & Validation Data Storage & Management Data Analysis & Reporting Sandbox Environment Supervised Institutions share data via automated data flows (ADFs), APIs, System performs Data consumer or other secured data validations and Enterprise Data Supervisors, risk transmission transformations Warehouse experts, and other The validated data is regulatory staff use archived and stored in the data and insights a central data warehouse Analysis Tools for oversight Additional datasets are Datamarts Dashboards, alerts, merged and test data is Datamarts manage statistical tables and available in the Sandbox permission-based graphs are created Environment access of data to to understand specific departments trends and risks or teams within the data Source: Figure adapted from materials provided by the Autorité des Marchés Financiers (Québec, Canada) requesting access. Datamart interfaces can also help users by institutions. Further, this data can be validated in real to link data sets together and produce automated reports. time, as upward of thousands of validation rules are run in parallel. Together, the process typically averages 10 In designing a SIS solution, supervisors should consider seconds per submission in the Philippines—a substantial the nature of their supervisory framework. How the data is improvement from the 30 minutes or more a submission collected and maintained over time will partly depend on via web portal upload might take for a supervisor to pro- whether the authority takes a risk-based or institution-type cess and validate (di Castri, Grasser, and Kulenkampff focus to oversight. 2020b). For supervisors, the solution reduces staff time spent on processing and managing data. This is especially 3.1.2 Automated Data Submission via API true of the time spent on cross-validations, which grows An API acts as a software intermediary that enables two or as the number of items requiring reconciliation grows with more systems to talk to each other. For regulatory report- every new report. ing, supervised institutions can prepare database extracts and share their data with supervisors via API transmission. This solution provides benefits for supervised institutions These data and report transmissions are most valuable in as well, reducing reporting burden and compliance costs. a machine-readable format to minimize the operational In the case of Bangko Sentral ng Pilipinas (BSP) in collabo- burden on supervisory authorities associated with manual ration with the RegTech for Regulators Accelerator (R2A),9 processing, data cleaning, and validation and making the the number of reporting fields required of FSPs was cut data readily available for market conduct analysis. in half, from 107,000 to 50,000, as duplicated or calcu- lation fields were eliminated. Further, this consolidation Direct machine-to-machine transmission via API has sev- allowed for the retirement of older reporting templates in eral benefits. The raw data extracted from supervised the move to automated database extracts (in XSD format). institutions’ core banking systems is converted into a single encrypted XML file that is pushed directly to the The desirability and feasibility of this solution is likely to vary supervisor. This single unified reporting scheme can among market conduct supervisors in low- and middle-in- replace dozens of previous reports submitted separately come economies. Countries with larger digital finance The RegTech for Regulators Accelerator, launched with support from the Bill and Melinda Gates Foundation, the Omidyar Network, and 9.  USAID, partners with financial sector authorities and technology firms to accelerate innovation in financial sector supervision, regulation, and policy analysis. See https://www.r2accelerator.org/about. 14   The Next Wave of Suptech Innovation ecosystems are likely to have supervised institutions with fields, formats, and frequency of reporting are prescribed relatively advanced levels of technical capability to support by the supervisor depending on their needs. this solution. Countries with a varied ecosystem of super- vised institutions, including those with more advanced and Regulatory reporting via web portal provides many bene- more limited technical capability, will likely need to main- fits for market conduct, though it is also helpful to under- tain simpler means of regulatory reporting, such as report- stand its limits. The solution increases the efficiency of ing via a web portal, alongside more advanced solutions, data collection, particularly beneficial where market con- such as API submission where feasible. duct supervisors have limited capacity and must oversee a vastly greater number of institutions than prudential 3.1.3 Web Portal Data Upload with Central Database supervisors. In addition, web portals typically collect stan- Regulatory reporting via web portal represents a low-com- dardized reports at regular intervals. This data provides plexity solution that can replace manual data sharing by supervisors with a regularly updated, landscape-wide email, fax, or mail. This solution can be developed rel- view that is crucial for market intelligence and informing a atively quickly and inexpensively because of the many risk-based supervisory framework. However, these reports off-the-shelf software solutions that have become com- tend to be less helpful when supervisors need to investi- mercially available. gate specific questions or risks requiring more detailed data in a follow-up information request. A secure web portal allows for manual data entry, data upload via widely accepted formats like XBRL, CVS, or This solution is used by financial authorities such as Nepal XML, or server-to-server transmission in more advanced Rastra Bank (NRB), which standardized its reporting tem- instances between supervised institutions and the finan- plates and introduced regulatory reporting via web portal cial authority. The range of data-sharing methods is and data upload in 2016. NRB now has plans to upgrade designed to accommodate supervised institutions that to a more comprehensive SIS that incorporates API-based are likely to vary in their technical capabilities. The data submissions via XRB in 2021. CASE STUDY 2 How NRB Develops Suptech Solutions Financial authorities without a large IT department or ded- The web portal acts as a single point of data submission. icated innovation office may benefit from starting small, To achieve this, the solution’s design required up-front work by proving the concept, and making continuous technology NRB staff to consolidate its many different report types from improvements. At NRB, reporting involves uploading data via throughout the authority into standard reporting templates with a web portal. This solution has been part of a natural technol- a consistent format. Consequently, a single set of validations ogy evolution that has gradually leveraged increasing levels (some of which are automated in the portal itself) are performed of technology and data to regulatory oversight. What started at the point of submission before the data is stored centrally out with hard-copy submissions by FSPs prior to 2009 has and made available on a permissioned basis throughout the evolved with the availability of new technologies. Over the authority. past 10 years, fax, email, and a first version of its web portal Digitizing its regulatory reporting infrastructure has stream- were each deployed and retired. In 2016, NRB launched its lined how and what data NRB makes publicly available, includ- current web portal. ing granular data on credit, deposits, branches, financial access, NRB’s web portal was designed to support the varied tech- and financial inclusion. Sharing raw data and reports publicly has nical abilities of the country’s FSPs. A web portal does not a multitude of benefits, including market intelligence for FSPs require supervised institutions to install or manage any new and overall intelligence on the state of financial stability, finan- reporting or software in their systems (or to have any special- cial inclusion, and market conduct in Nepal. ized systems). This avoids complex system integrations, the For NRB, its technological evolution continues today. In need to have in-house technical talent, or even a Microsoft 2021, NRB intends to launch the next generation of regulatory Excel license, as users can upload or enter data directly into the reporting—a SIS that will introduce enhancements such as API- web portal. Web portals, however, do require digital connec- based data collection and a stronger capability for business tivity, which is a challenge for FSPs operating in rural regions. intelligence (BI) analysis. The Next Wave of Suptech Innovation   15 3.2. Collection and Processing of Complaints Data A CMS expands visibility and access to the public for submitting a complaint, introduces automation into com- Consumer complaints data is one of the most valued data plaints-handling processes, and optimizes data man- sources for market conduct supervisors. A CMS is key to agement for supervision. Complaints submission can be the efficient processing of these complaints and capturing made further accessible to consumers through the intro- and managing complaints data to maximize its accuracy duction of new digital user interfaces such as SMS. More and value for supervisory purposes (see section 3.2.1). critically for market conduct supervisory purposes, cen- The application of advanced analytics to complaints data tralizing manual submissions through a single “case man- (see section 3.2.2), particularly to unstructured text, rep- ager” digital interface streamlines complaints submitted resents the next step for market conduct supervisors to through other channels and helps to structure and cate- deduce useful insights from complaints data in a more gorize complaints data for supervisory analysis. efficient manner. In 2020, BSP is expected to deploy its CMS solution in 3.2.1 Complaints Management System coordination with R2A and Sinitic, a software vendor. BSP’s Traditional complaints handling often relies on manual CMS will include new digital interfaces for staff (called a processing and offers limited ability to interact with con- case manager) and consumers (digital SMS submissions via sumers, a result of outdated communication channels and API and chatbot) and a more robust complaints database. accessibility issues. Complaints data may also suffer from The CMSs at the US Consumer Financial Protection Bureau the entry of incomplete, inaccurate, or inconsistent infor- (CFPB) and Bank of Lithuania (BOL) were previously high- mation that limits its value for supervisory purposes. This lighted in the 2018 World Bank report From Spreadsheets heavy reliance on manual complaints processing is both to Suptech. The CFPB goes further than typical in sharing operationally intensive and prone to data-entry error. As its complaints data publicly through an external interface a result, complaints departments at financial authorities that redacts personal information and is accessible to all. are often overburdened and are not equipped to provide The CFPB also makes its complaint database available to valuable data that can inform supervisory activities. other government agencies through a private portal. FIGURE 7: CMS Case Workflow and Data Architecture Case information captured Case resolved and information used Customer complains . . . in database for supervisory activities . . . via online platform Resolution decisions (sanctions or recommendations) Consumer application recorded in database routed directly to electronic database Market Conduct Complaint Unit Consumer application E CAS TION review . . . via phone, post FO RMA Dispute Dispute Dispute IN or email Resolution Resolution Unit Committee DAT A Risk assessment FSP risk rating DAT exercise A Product or Quarterly or annual service rating statistics publication Consumer application scanned or transcribed and uploaded to the electronic database Source: World Bank (2018) 16   The Next Wave of Suptech Innovation Standardized complaints capture solutions improve the quality of complaints data, aiding Modern CMS solutions introduce new digital channels supervisory analyses and increasing the effectiveness of for submitting complaints. For example, BSP’s CMS market monitoring. Analyses of complaints data allow is enabled by API and an NLP-powered text engine supervisors to understand consumers’ experiences with through which consumers can submit complaints using financial providers and identify emerging trends and risks. either smart or feature phones on a variety of messag- ing platforms, including Facebook Messenger, SMS, In the case of BSP, the supervisor expects that new types or a chatbot embedded on an FSP’s website. Through of complaints analyses, such as topic modeling, will each of these digital channels, an NLP-powered engine reveal previously hidden patterns of consumer and firm interprets and responds to consumer messages, lead- behavior to add to their market monitoring. This is par- ing consumers through a pre-defined conversation and ticularly true as the CMS begins adding more complaints complaints submission experience. In the background, a submissions from outside of the capital region of Manila, supervised machine-learning model identifies opportuni- where the bulk of previous consumer complaints came ties to improve the model’s understanding and ability to from. New digital submission methods are expected to interpret consumer messages accurately. CMS solutions help the regulator reach harder-to-reach cities and rural employed at BSP, CFPB, and BOL also consolidate man- regions of the country. ual channels that require higher-touch human interaction, such as phone calls, mail, or emails, into a single stan- 3.2.2 Analysis of Unstructured Complaints Data dardized complaint-management interface. This provides As noted above, consumer complaints represent a rich operational benefits in the form of more streamlined data source for market conduct supervisors. However, human interaction, as well as the data integrity benefits of such data is often unstructured, making it difficult for indi- a single interface for intake. vidual supervisors to identify patterns or emerging risks. This is particularly true where a consumer’s description of Database storage a complaint comes in the form of a free-form narrative Incoming complaints data from new digital channels flows and in countries where complaints reporting is not stan- into a centralized complaints database. Past complaints dardized. Due to the large volume of data that results, data is also imported into the centralized database to supervisors often decide to sample random complaints or deepen the historical record for analytics and to train purposely select those that seem to pose higher market machine-learning models for purposes of topical model- conduct risk. However, this can be a time-consuming pro- ing and sentiment analysis. cess and result in missing key risks. Other common meth- ods, such as keyword searches, can become biased if a Complaints processing via case manager interface systematic method is not followed. Supervisory staff can view analytics, configure complaints intake (such as chatbot logic in the case of BSP), and enter NLP algorithms for topic, sentiment, and risk identification and process individual complaints, including complaint can help financial authorities automate analysis of unstruc- notes or attachments, from a central user interface. Fur- tured complaints data and conduct such analyses on high ther, the CMS automates the routing of complaints to the volumes of data with greater accuracy. “Topic modeling” appropriate supervised entity for response with a portal infers topics from raw text by analyzing word co-occur- for financial providers to manage their complaints. rence in the text of each complaint to identify the topics and proportion of topics within each. Co-occurrence is a The introduction of new, easily accessible digital chan- measure of the frequency with which two topics or phrases nels also encourages more consumers to contribute their are both present in a complaint. Topic models apply an voice and enables the regulator to provide timely infor- inductive approach of inferring, rather than assuming, pat- mation on financial services to consumers, contributing terns in the text. It can also include metadata in the anal- to the enhancement of consumers’ ability to make more ysis and thereby link external variables such as industry, informed choices. This ultimately reinforces the solution’s timing, or demographics to fine-tune the analysis. effectiveness and relevance. The overall implementation complexity and investment Analysis of complaints data level of these types of suptech solutions are generally The process automation introduced by CMS solutions low. However, these solutions require supervisors to have enables financial authorities to shift staff resources from in-house expertise to conduct and maintain these analy- time-consuming complaints processing to complaints ses; this requirement may pose a limiting factor for finan- analysis for supervisory purposes. Furthermore, CMS cial authorities in lower-capacity countries. The Next Wave of Suptech Innovation   17 CASE STUDY 3 How Researchers at Princeton University and FSD Kenya Worked with the Central Bank of Kenya to Analyze Complaints Data Researchers at Princeton University in partnership with FSD  nterpretation: The most crucial step comes toward the end iv. I Kenya and the Central Bank of Kenya analyzed 37,000 con- of the analytics process. Interpreting the findings requires a sumer complaints, demonstrating how supervisors can apply qualitative reading of the topics identified by the algorithm, these analytical tools to enhance their understanding of their often aided by visual tools and dashboards available in the own consumer complaints data. software. Here, analysts identify the findings most pertinent The analysis consisted of the following five steps: to supervision and dig deeper where needed.  ata cleaning: Ensures high-quality data for analysis. It can i. D  nalysis and documentation: As the NLP model and its out- v. A be time consuming, especially if the raw data is untidy and puts are finalized, the last step is to document patterns and has not been used for quantitative analysis before. relationships in the complaints data.  lgorithm selection: Once the data and metadata are ii. A Once the final model is specified and analysts are confident with structured in tabular form, they can be fed into the algo- its results, the model can be used repeatedly for daily or weekly rithm. For this analysis, researchers used Structural Topic monitoring of incoming complaints from consumers. In Kenya, Modeling, a specific kind of topic-modeling algorithm that researchers identified three key learnings from the interpretation is open source and available in R—one of the most well- of the model results. known free software environments for statistical computing First, not all topics identified by the algorithm are useful. and graphics—in a freely available package called “stm.” In fact, of the 60 most common topics, only 14 provided clear In order to run the algorithm, data scientists must make added value to the regulators, while an additional 12 topics pro- a number of initial arbitrary choices, the most important vided potential added value. being the selection of k—the number of topics that the Second, topic modeling is particularly well suited to identify algorithm will identify in its first iteration. emerging risks and unknown problems in the market. Confirm- iii. Model optimization: Optimizing the algorithm to max- ing the capacity to discover topics without prior specifications imize accuracy requires an iterative process to ensure or assumptions is one of the key strengths of topic modeling. that the topics are relevant and not skewed by irrelevant Lastly, topic modeling can help identify patterns in risks factors. that are already known. By associating topics to the metadata, analysts discovered how complaints varied by bank or by time period, and whether there was a difference between complaints categorized as “open” or “closed.” 3.3. Non-traditional Market Monitoring a timely, useful, and low-cost complement to traditional market monitoring, such as complaints handling, particu- The internet provides the opportunity for supervisors to larly for resource-constrained financial authorities. expand market monitoring activities to new, important pools of data in a more efficient and effective manner. 3.3.1 Web Scraping The solutions described in this category span the data Information on websites presents market conduct supervi- life cycle—enabling supervisors to collect the data con- sors with a valuable, non-traditional data source. The pri- tained within new, non-traditional text data, such as social mary method to gather information on websites is via web media, blogs, forums, and news media, and apply new, scraping, which pulls text, metadata, and other informa- advanced analytics methods. Foundational to all solutions tion from the websites of FSPs, or consumer posts shared in this category is web scraping (section 3.3.1), which on social media websites, online forums, and review provides the mechanism to collect online text data for websites. Web scraping allows supervisors to view and analysis. Collecting data then allows for various forms of aggregate this information to identify emerging risks to market monitoring analyses, such as social media mon- consumers, such as how financial products are marketed itoring (section 3.3.2) and consumer sentiment analysis or sold. Note that the actual analysis of web-scraped data (section 3.3.3), reputational analysis (section 3.3.4), and is described in sections 3.3.2–3.3.5. dark web monitoring (section 3.3.5). Overall, these solu- tions strengthen market conduct supervision and provide 18   The Next Wave of Suptech Innovation Prior to the advent of web scraping, market conduct Social media monitoring solutions analyze consumer supervisors would have to undertake this type of analysis posts collected via web scraping (see section 3.3.1) from manually via web-related search tools, such as Google, web forums, social media websites such as Facebook, or manual review of individual websites. Besides being Twitter, and Instagram, and consumer-review websites. time intensive and inefficient, manual procedures proved Consumer posts are categorized via keyword mining or to be quite ineffective for supervisory purposes given the topic modeling, tagging posts by such criteria as the asso- practical difficulty of evaluating the whole market and ciated FSP’s name, financial product (for example, insur- spotting outliers. ance, mortgage, and so on), or stage in the product life cycle (opening an account, closing an account, and so on). Variations of this solution are currently in use or expected Posts can also be categorized with a consumer sentiment to be deployed by such authorities as the FCA, AMF, and score (see section 3.3.3). the Central Bank of Ireland (CBI). While some authori- ties have built this tool in-house using APIs provided by Developing an appropriate topical categorization requires a Google and other third-party providers including social taxonomy, which authorities indicated requires supervisory media companies, the predominant approach among expertise and often takes three to six months to develop, authorities has been to engage an external software or refine, and measure. Even with additional refinement, some data provider. Financial authorities that collect and store level of categorization errors will remain. Human review is consumer data, such as social media data, cite ano- recommended to correct for errors and make refinements nymizing or redacting personal and sensitive information to the solution to further enhance its accuracy. when web scraping as a best practice. Social media monitoring can provide supervisors with 3.3.2 Social Media Monitoring an early warning system. For example, the CBI discov- Social media monitoring is the most common solution ered through social media that a new retail credit firm employed for non-traditional market monitoring. Super- appeared to be operating without authorization in the visors use this solution to attempt to listen directly to Irish market. Investigation by the CBI determined that the “voice of the consumer” and understand consum- the firm was operating using disposable email addresses ers’ interactions with financial providers on a nearly real- and fake names and addresses. The CBI moved to protect time basis. This enables supervisors to identify and act consumers by issuing a warning and publishing the name on emerging consumer risks more quickly. As more and of the firm. Following these actions, no further activity more commercial solutions have become available at involving this firm was seen in the market. competitive prices, more financial authorities have piloted or adopted such solutions. FIGURE 8: Dataflow Diagram for Social Media Monitoring Social Media & News Web Scraping & Data Analysis & Monitoring Data Preparation Webscraping transforms online content into unstructured text and metadata for analysis Topic modeling and sentiment analyses allow for trend and risk spotting Source: Graphic developed by the World Bank based on research with the Autorité des Marchés Financiers, the Financial Conduct Authority, and the Central Bank of Ireland The Next Wave of Suptech Innovation   19 In another example, the CBI became aware through social Lastly, supervisors should be aware of effects of language media monitoring of a sudden and substantial increase differences. Algorithms tend to be trained to analyze web in the number of customer comments to one retail bank. content in a single language at a time. This represents Further investigation revealed that the complaints related a challenge in jurisdictions with multiple languages. For to the availability of customer support services, with cus- example, the European Insurance and Occupational Pen- tomers expressing their concern with having to wait up to sions Authority (EIOPA) oversees insurance regulation 40 minutes to get through to a customer service represen- across the 27-member European Union (EU). In its own tative. Detailed market intelligence and underlying data social media pilot, the authority found it difficult to draw about call waiting times were used by CBI supervisors to conclusions due to variations in cultural norms and lan- confirm indications from social media monitoring, and guages across the EU. CBI’s concern was then relayed to the firm’s senior man- agement. Based on this information, the firm recruited 3.3.3 Consumer Sentiment Analysis additional customer service staff to address the lengthy As more people have taken to expressing their opinions call waiting times and also agreed to revisit staff schedul- through social media, blogs, and online forums, market ing to increase coverage at peak times. conduct supervisors now have timely access to consumer sentiment toward FSPs. This solution is often used to Social media data can also be triangulated with the mar- complement the trend and topical analyses described ket conduct supervisor’s own complaints data. For exam- in section 3.3.2. Whereas complaints generally focus on ple, the CFPB began hearing of consumer issues with negative experiences, this solution includes a broader transaction failures at a prepaid card provider, RushCard, window into both positive and negative experiences on social media in 2014, which was subsequently backed with FSPs. Further, this solution allows supervisors to up by complaints reported directly to the regulator. The hear from a wider universe of people than complaints CFPB used social media data in tandem with complaints data alone. This enables market conduct supervisors to data as an early warning, allowing the regulator to position enhance their understanding of consumers’ experiences itself for action even before the company itself reported with financial providers. the outage issue to its customers or regulators. Consumer sentiment analysis solutions monitor language However, though social media monitoring is a powerful sentiment toward FSPs across social media mentions, new addition to the supervisory toolbox, it is important to blogs, and online forums in a rapid, automated manner be aware of its inherent limitations as a data source. First, by using algorithms. The solution benefits from the use social media data is self-reported by consumers. People of open-source NLP libraries that detect and assign either on various social media platforms may differ from mem- a positive or negative tonal score to online posts. More bers of the general population in their education, wealth, advanced NLP libraries can be applied to identify more age, financial literacy, and so on. Social media posts nuanced emotional cues. A consolidated report then should not be taken as representative of the full popula- benchmarks institutions over time and can compare peer tion, particularly the most vulnerable segments. institutions against each other. Second, social media data is not statistically representa- Financial authorities such as the Bank of England (BOE) tive. For example, there may be significantly more social use such solutions to complement current supervisory media posts regarding consumers’ experiences with their analyses, rather than as a substitute for other data. It is banks than with their insurance providers because con- also important to note that consumer sentiment analysis sumers interact with their banks regularly but may engage solutions share similar challenges as other NLP-derived with their insurance providers only infrequently. A qualita- solutions—principally, the misidentification of emotional tive lens is needed to interpret and understand the voice tone, such as sarcasm. To date, sentiment algorithms are and nature of consumer comments to generate useful more often trained on an individual word basis, whereas insights for supervisory purposes. jokes, humor, and sarcasm are embedded in sentence structure. As a result, human oversight and some level of Third, consumers provide varying details on social media, manual review is recommended to correct for any miscat- as there is no standard reporting template. Though a egorizations. supervisor may want to understand consumer posts spe- cific to a type of product, consumers tend to be quite 3.3.4 Reputational Analysis general. For example, a consumer may complain gener- Given limited staff, financial authorities do not have ally about “mortgages,” rather than about a specific mort- the capacity to monitor the entire online ecosystem for gage product, such as interest-only mortgages, that may changes in the financial marketplace. At the same time, be of particular supervisory concern. 20   The Next Wave of Suptech Innovation early detection of reputational or consumer risks in the records and contact information. “Closed source” refers marketplace would indicate where market conduct super- to information with restricted or private access. The solu- visors should focus their efforts. Reputational analysis tion presents topical information and sets up real-time tools are beneficial in providing real-time market intelli- alerts when new, relevant information appears. gence on public perceptions of FSPs in the news media. Supervisors may look at insights from reputational analysis While monitoring the dark web can help supervisors to in combination with other market intelligence data sets, identify emerging consumer risks proactively, it is import- such as complaints data, to understand overall percep- ant to note that the dark web is not the only place online tions of a specific FSP. where criminal activity occurs. Criminal activity can also happen through private transactions in closed networks Reputational analysis solutions analyze text from press that do not appear on the dark web or can occur outright releases and mentions in the financial news. Topic mod- in the public eye. This solution simply helps supervisors to eling or keyword mining is employed to categorize top- monitor another potential source of harm to consumers. ical trends and assign sentiment scores to the text data (see section 3.3.3). Aggregated reports enable supervi- While services for dark web monitoring are commercially sors to understand changes in the topics discussed by available, many of these service providers are focused financial institutions over time and compared to peer on servicing consumers and FSPs, rather than financial institutions, as well as to track shifts in market sentiment authorities. Partly as a result, BOE is piloting its own supt- toward FSPs overall. ech solution for dark web monitoring. The reputational analysis solution is powered through 3.4. Document and Business Analysis topic modeling, which requires both up-front and ongoing refinement to maximize its accuracy. As noted previously, Market conduct supervisors have introduced a range of financial authorities report that setup can be a labor-in- cutting-edge suptech solutions that focus specifically on tensive process lasting between three and six months to leveraging NLP for document analysis. Many of these enable the tool to classify and analyze media mentions solutions share common NLP capabilities. For financial properly. The use of primarily open-source libraries in R authorities, this means that the development of one of and Python, such as the “Ida” package for topic model- these solutions can assist in the development of others. ing, makes this solution more accessible to authorities and NLP applied to document analysis can be used for a vari- positions authorities to benefit from future developments ety of market conduct use cases, including for regulatory and improvements in analysis software. This solution is compliance (section 3.4.1), institution monitoring (sec- typically used together with sentiment analysis. (See sec- tion 3.4.2), peer institution analysis (section 3.4.3), credit tion 3.3.3.) agreement reviews (section 3.4.4), new provider regis- trations (section 3.4.5), and public financial statements 3.3.5 Dark Web Monitoring and investor report filings (section 3.4.6). In addition, Monitoring tools focused on the dark web can detect business intelligence (section 3.4.7) and managed data suspicious activity, identify risks, and enable proactive platforms (section 3.4.8) represent suptech solutions that defense of the financial authority, FSPs, or consumers help supervisors manage and combine the data sets cre- against threats posed by bad actors. For market conduct ated by other solutions and to use the combined data for supervisors specifically, dark web monitoring can be ben- advanced supervisory analyses. eficial in detecting identity theft, fraud, scams, or other activities that can lead to consumer harm. 3.4.1 Document Analysis for Regulatory Compliance Many financial authorities receive thousands of documents The dark web refers broadly to an area of the internet collected from FSPs, such as minutes of board meetings, where websites, networks, and content exist on private internal audit reports, and other management reports. encrypted networks called “darknets” that are acces- These documents contain key information for market con- sible only with specific browser software, such as Tor or duct supervisors related to audit and compliance risks. I2P. Users can communicate and conduct business on the dark web anonymously. Its anonymous nature makes it With this solution, supervisors can upload a set of doc- attractive for use by bad actors. This particular suptech uments and run a series of core analyses on document solution allows supervisors to curate intelligence reports text. This solution does not aim to automate analyses and and search both “open” and “closed” sources. “Open conclusions. Rather, it allows triaging of a large number of source” refers to information or derived information that documents in a much faster manner in order to find rele- is available to the general public and includes public vant information, which can then be analyzed in greater The Next Wave of Suptech Innovation   21 FIGURE 9: Example of Dataflow Diagram in Document Analysis Solutions Upload Documents Calibration of NLP Models Reports & Analysis Common examples include topic modeling, bag of words, sentiment scoring Use cases for document analysis that use NLP: 3.4.1 Document analysis for regulatory 3.4.4 Validation of terms and conditions compliance 3.4.5 Automated review of new provider 3.4.2 Document analysis for examination of FSPs registrations 3.4.3 Document analysis for peer group 3.4.6 Predictive modeling of financial comparison statements Source: Graphic developed by the World Bank based on research across multiple financial authorities CASE STUDY 4 The FCA’s Development of Sleuth, Its NLP Platform With the increasing power and accessibility of NLP to analyze agement, and NLP. Supervisors can upload a set of documents text and speech data, NLP-enabled solutions are growing in and run a series of core analyses. These analyses help with both popularity among market conduct supervisors. NLP platforms understanding and navigation. Supervisors can understand represent a new piece of IT infrastructure that is useful in common themes across and within documents, as well as where developing such suptech solutions. to navigate within the documents to learn more. The FCA’s RegTech and Advanced Analytics team has Sleuth was designed to scale and streamline subsequent received an increasing number of internal requests to assist investments in new, NLP-enabled suptech solutions by develop- supervisors in their analysis of the large volumes of docu- ing its platform to be able to support and be interoperable with ments received from supervised institutions and from publicly future solutions. As a result, the resources required to imple- available data sources. ment subsequent NLP-enabled suptech solutions tend to be To meet the increasing supervisory need to analyze doc- lower, given economies of scale, and faster to develop. The NLP uments, the FCA developed its NLP platform, called Sleuth. platform also equips internal technology staff with the tools to This platform represents a common set of advanced IT and develop their own tests and pilots of future suptech solutions. analytics infrastructure in cloud computing, database man- detail for regulatory compliance purposes. The solution terms directly from the EU General Data Protection Regu- provides supervisors with the ability to determine the lation (GDPR)10 and EU regulations. frequency and location of key search terms, as well as to make topical comparisons across documents. 3.4.2 Document Analysis for Examination of FSPs Financial authorities receive a broad array of documents While this solution facilitates the inspection of documents from supervised institutions, covering topics ranging to determine compliance with specified regulations, it from business strategy and financial performance to risk requires both legal and supervisory skills to determine management. However, capacity is limited for staff to be the appropriate search terms to use in order to utilize able to review this high volume of documents in their the solution effectively. For example, in its use of a supt- entirety and to identify the most pertinent institution- ech solution for document analysis, the FCA drew search specific patterns. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to 10.  the processing of personal data and on the free movement of such data (General Data Protection Regulation). 22   The Next Wave of Suptech Innovation This solution can be conceptualized as a powerful “Ctrl-F” comparisons across FSPs are challenging. Often, making tool to find certain words and phrases, powered by two comparisons requires an intensive process in which super- types of NLP algorithms: (1) “bag of words” and (2) “topic visory staff look through hundreds of pages of documents models.” “Bags of words” allow for “smart searches” to belonging to multiple institutions to spot risks and trends identify the location and frequency of words, phrases, across a peer group. or groupings of words within the document text. “Topic models” go beyond this and use pre-defined topic filters The peer group comparison solution specifically helps to detect patterns and prevalent themes within or across market conduct supervisors analyze and compare doc- documents. This solution is designed to be both inductive umentary information across FSPs more efficiently. and deductive—that is, supervisors can both inspect doc- The solution’s core functionality is similar to solutions uments for a pre-determined theme, while the solution is described in “Document Analysis for Regulatory Compli- also able to review and indicate back to the supervisor the ance” (section 3.4.1) and “Document Analysis for Exam- key themes contained within a document. ination of FSPs” (section 3.4.2), as it also uses NLP to analyze documentary text. While similar in functionality to the solution described in “Document Analysis for Regulatory Compliance” (sec- However, this solution’s capabilities go further in three tion 3.4.1), this particular solution is broader in its scope areas. First, through topic modeling, the solution allows and capabilities. Where the previous solution is focused for thematic comparisons across peer groups and the abil- on identifying regulatory compliance with a specific reg- ity to confirm the presence or absence of key concepts. ulation, this solution inspects a broader range of docu- This includes the ability to classify documents according ments to help identify institution-specific trends beyond to type, language, and other criteria. Second, the solu- those related to regulatory compliance. This is helpful for tion can detect unstructured text that is numerical or supervisors as they receive documents asked for as part of financial, such as tables, graphs, charts, or images, and information requests sent to supervised institutions and to export this data in a single machine-readable CSV or XLS develop the scope of these examinations. When applied file for enhanced quantitative analysis. Third, the solution across institutions in a methodical way, this solution pro- can apply sentiment analysis to the documents and assign vides helpful benchmarking as described in the following a sentiment score to the document text. These insights section, “Document Analysis for Peer Group Compari- are presented through an interactive dashboard that per- son” (section 3.4.3). mits supervisors to understand emerging themes, identify trends at the firm level, and conduct a cross-sectoral com- For example, AMF implemented its document analysis parison across a peer group. solution to support the monitoring and scoping of exam- inations. Manual review of documents received from FSPs For example, the Data Innovation Team within BOE devel- would consume considerable staff time. Further, due to oped its solution to help supervisors analyze and compare the multitude of people involved in the document review unstructured information more efficiently. The new supt- process, comparisons across documents reviewed by dif- ech solution has allowed supervisors at BOE to shift to ferent people were challenging. As a result of implement- more strategic analysis of the data and is currently being ing their document analysis solution, supervisors can now used by approximately 400 supervisors. The solution was efficiently extract insights and data that had previously the culmination of a four- to six-month proof of concept remained contained within these documents. followed by a year of supplier selection and contracting. A key challenge faced by BOE was navigating legal con- This solution is inexpensive to develop using open-source straints on sharing data with third parties when testing R or Python software and libraries. However, it does and selecting the right solution and supplier. require underlying IT infrastructure in database manage- ment for the documents and text data. It is also recom- 3.4.4 Validation of Terms and Conditions mended that NLP algorithms be updated periodically, at The timely review of credit agreements, including terms least once per year. and conditions in particular, can be challenging given limited supervisory staff and the volume of lengthy and 3.4.3 Document Analysis for Peer Group Comparison ever-changing agreements to review. This solution intro- Much of the documentary information that supervisors duces automation to the validation of credit agreements receive is unstructured, contained in board presentations, and their terms and conditions to identify compliance risks financial reports, internal audit reports, meeting minutes, by leveraging NLP algorithms to analyze the text of the and other reports and documents. Due to its unstructured credit agreements and compare the text to legal and reg- nature and the high volume of documents received, timely ulatory requirements. This solution could also be applied The Next Wave of Suptech Innovation   23 to verify whether other consumer disclosure documents In its deployment of this solution, AFM estimates super- comply with requirements, as well as for other types of visory time savings of 25 to 50 percent in the process of financial products and services. reviewing fund registrations. It has also led to more robust assessment of new fund registrations given that supervi- For example, Banco de Portugal (BdP) receives over 300 sors can now focus on the review of higher-risk registra- credit agreement templates annually related to new prod- tions more than on the manual review of each registration. ucts introduced in the market, and over 1,000 changes These efficiencies were made possible through the imple- to credit agreement templates for existing products. Such mentation of over 16 decision rules that replaced checks a large volume of credit agreements poses challenges in previously done manually. AFM designed its solution as a the allocation of supervisory resources sufficient to man- central data platform built as part of its CRM, which pro- ually review the agreements for compliance with market vides a central place to view and configure decision rules conduct rules. In 2019, BdP piloted the use of suptech and to make insights available through interactive dash- solutions focused on the validation of 12 compliance boards to end users (that is, supervisors). rules of relatively low complexity for 20 types of credit agreement templates. The subset of rules covered by the 3.4.6 Predictive Modeling of Financial Statements solution represent around 20 percent of the total number Ensuring the accuracy of public financial statements, of compliance rules for these products. BdP initially part- investor report filings, or other financial statements is nered with an external vendor that made its platform avail- important to protect investors from errors, omissions, or able, where the rules to be validated were designated and mistakes. Solutions that allow for predictive modeling of then applied to the sample of credit agreements either via financial statements are intended to facilitate supervision the platform or by exposing APIs to be invoked by other of listed public firms for purposes of consumer and inves- applications. tor protection. This solution to validate terms and conditions comes with The solution is best described as a “bot report” that certain challenges. A lack of standardization in the lan- checks for the percentage of mistakes in financial state- guage or format of agreements used by credit providers ments of listed firms. First, the solution imports the finan- makes the solution more difficult to apply. Similarly, the cial statements via XBRL11 format. (No manual input is addition of more complex legal and compliance checks required to prevent data quality issues.) Quality checks are on credit agreements will requires higher levels of invest- then carried out to verify, for example, if liabilities match ment in both time and budget to define and refine the assets, negative values that should have been positive, NLP algorithms appropriately. check for extreme differences, and so on. The next step is scanning all imported financial statements with a validat- Automated Review of New Provider 3.4.5  ing algorithm to estimate the risk of misstatements. From Registrations there, qualitative risk scores are assigned to each financial Financial authorities, and market conduct supervisors statement and a logistic regression model is used to pre- specifically, commonly license or register new financial dict the chance of misstatement per financial statement. institutions, such as investment funds. The registration or Finally, a decision tree algorithm is applied to identify licensing process is often characterized by several manual the greatest possible chance of misstatement. Financial steps. statements with a high risk of error, based on the limits set by supervisors, are given higher priority and are fully This solution reviews and signals any risks associated reviewed by supervisors. with fund registrations. Automation is enabled through detection and classification functions, both types of NLP By employing this solution, AFM enabled its supervisory algorithms. A detection function reviews documents and staff to increase the percentage of financial statements it information received from the fund registration. Classifi- reviewed from 10 percent to nearly 100 percent of the cation functions then undertake an automated check and approximately 350 financial statements received each provide either approval or specific advice for further test- year. These efficiency gains have allowed their market ing. These checks are based on predetermined decision conduct supervisory to scale up, enabling risk identifica- and legal rules established by the user, in this case, the tion in a faster, more comprehensive, and higher-quality market conduct supervisor. manner. eXtensible Business Reporting Language (XBRL) is a global framework and standards-based way to communicate and exchange business 11.  information. The reporting language is XML-based. XBRL is commonly used to define and exchange financial information, such as financial statements. 24   The Next Wave of Suptech Innovation 3.4.7 Business Intelligence and Geospatial Analysis financial data based on geography, region, municipality, and branch (Gurung and Perlman 2018). A supervisor may be data rich but information poor if the financial authority lacks the ability to evaluate data BI tools benefit users through faster and more accurate for insights. Business intelligence (BI) refers to a tech- reporting, analysis, or planning. At the same time, these nology-driven process for analyzing data and present- solutions also come with challenges for their effective use. ing actionable insights to help executives, managers, BI tools require training and successful socialization to be and supervisory teams make informed evidence-based used within and across an organization. Many require ini- decisions. tial setup and connections to the regulator’s underlying databases. In addition, any BI or analysis tool is only as Whereas NLP and its associated solutions help supervi- useful and accurate as the integrity of the underlying data. sors make sense of unstructured text, BI solutions help supervisors make sense of large quantities of structured 3.4.8 Managed Data Platform data. BI tools can be used on a single data source or can For market conduct supervisors that receive a large combine data sources together for a more holistic view. quantity of data regularly from varying sources, a com- An enterprise-wide BI platform typically supports live data prehensive, integrated data and analytics platform can connections to a variety of underlying databases (either support ongoing supervisory functions such as identify- directly or through datamarts that act as an intermediary ing, monitoring, assessing, and analyzing risks. A man- layer). A BI tool can also operate without a live data con- aged data platform standardizes, centralizes, and makes nection and use CSV or comparable data exports from a accessible data from across the financial authority for database instead. supervisory staff. Financial authorities typically use BI tools in combination While a managed data platform can be designed in a with other suptech solutions to interpret information and multitude of ways, AFM designed an integrated plat- data. For example, interactive, self-service dashboards form, structured around three areas: (1) Source data is can be created using BI visualization tools such as Tableau collected and stored in a persistent staging area. (2) Data or Microsoft PowerBI, which are often used with solutions is then processed and stored in an integration area and for regulatory reporting to interpret findings for non-tech- transformed (for example, validated, classified, orga- nical supervisory staff. R or Python workbenches are used nized, structured). (3) After applying specific rules or by technical users to synthesize and analyze large data algorithms, the resulting data is finally made available sets, especially as they relate to the aforementioned doc- through datamarts for use by staff via dashboards and ument analysis suptech solutions with NLP. Meanwhile, reports. The platform collects and stores different types relational database management and query tools, such as and formats of semi-structured data, including CSV, MySQL, and larger BI platforms from large tech providers, JSON, XML, databases, and so on. like SAP, Oracle, and IBM, support the retrieval and man- agement of data contained within large databases. AFM uses its managed data platform solution for all of its regulatory and supervisory activities of retail and BI can also bring in new data sources. For example, mar- wholesale financial markets, providing supervisors with ket conduct analysis can be enhanced through the addi- data for around 90 percent of risk monitoring and 50 tion of geospatial data to understand geographic-level percent of risk assessments undertaken. This coverage trends. NRB uses geospatial data to understand access across multiple markets was possible given AFM’s deci- points to FSPs in rural Nepal and other trends in rural sion to design the platform using a modular approach financial inclusion. Its solution uses an “E-Map Portal” that with cloud-based infrastructure. Risk-mitigation activities relies upon a GIS system to map financial access points continue to be conducted manually by supervisors, as across Nepal. NRB analytics use this solution to combine they require very technical judgments and a strong legal submitted data from supervised institutions to understand background. The Next Wave of Suptech Innovation   25  EOPLE, PROCESS, AND IT 4. P in people’s capacity and skills so they can identify data needs and use data insights in their supervisory work to INFRASTRUCTURE: THREE identify, assess, and mitigate supervisory risks. KEY ENABLERS FOR SUPTECH IMPLEMENTATION Supervisory authorities are perennially resource con- strained, resulting in the need to invest in internal The authorities who contributed to this note indicated capacity. Investments in internal capacity can involve two that they view suptech adoption as an evolutionary elements: (1) investing in people directly to add new skill- process, rather than an end goal. Embedding modern sets either through hiring, retraining existing employees, technology and data into the supervisory process is an or leveraging external parties; and (2) investing in technol- ongoing effort, and the desired end state is organizational ogy to make employees more productive. transformation into a data-driven supervisor. Achieving this transformation requires making investments in three Supervisory authorities need new technology and data key enablers: (1) people, (2) process, and (3) IT infrastruc- skillsets within their workforce, including data scien- ture. These implementation enablers are broadly applica- tists, analysts, and programmers. These new skillsets ble across all types of suptech solutions, for both market may be added through training of existing employees or conduct as well as other regulatory objectives. external hiring. As an example of the former, the FCA pro- • People: The talent, mindset, and skills of employees motes internal analytics communities (to which a quarter and the larger organizational culture toward data and of the staff belong) and leads a Community of Practice technology for analytics skills development and augmentation, which includes frequent training through internal workshops • Process: How suptech ideas are supported from ide- and seminars, internal publications and demonstrations, ation to implementation, including how suptech is and seminars and interviews with external subject matter championed and governed experts. • IT infrastructure: The underlying IT infrastructure and capabilities needed to develop and operate suptech As a financial authority implements suptech solutions, solutions internally it can be difficult to know in advance which types of talent and skills may be needed and in what quan- tity. In fact, all authorities who contributed to this note 4.1 People: Culture and Skillsets indicated that they employed the assistance of external Financial authorities often cite investment in their staff parties, such as consultants, technology vendors, or tech- as the greatest contributor to success in suptech adop- nical assistance providers before building up in-house tion. Skilled employees are important not just to process expertise. For example, the FCA, BOE, AFM, and CFPB data and build solutions, but to ensure that the right ques- all heavily relied on consulting firms during initial stages of tions are asked in the first place. It is important to invest suptech implementation. By hiring consultants with data, CASE STUDY 5 How AFM Prioritized People within Its Transformation to Data-Driven Supervisors In the case of AFM, its suptech journey can be nal know-how to leverage data to enhance supervi- grouped into three distinct steps: (1) build, (2) pilot, sory activities. Now underway, AFM’s step 2, “Pilot” and (3) transform. Step 1, “Build” (2016–19), began (2019–21), seeks to amplify these cultural changes with an assessment of its own in-house capacity to further with new processes that support the piloting deal with and analyze data to support supervisory of new technologies. Lastly, AFM’s step 3 “Transform” activities. Since most supervisors have a stronger (from 2022 onward), seeks to transform the organiza- legal background than a technology background, tion both culturally and operationally to become a AFM realized that new investments would need to data-driven regulator through fully distributed and be made in its staff and culture through trainings and decentralized responsibility for data teams through- new hires, alongside data and technology solutions. out the financial authority. In each phase, AFM’s focus Step 1 efforts concluded with a successful shift in the is on its people and on cultivating the skills, culture, internal cultural mindset and developing the inter- and structures to support its suptech strategy. 26   The Next Wave of Suptech Innovation technology, and business analysis expertise, these regu- of their suptech solutions, business units were motivated lators were able to move quickly to establish innovation, to identify the supervisory problem they sought to solve data-science, or other technology offices. Similarly, BSP and to own ultimate deployment of the suptech solution. worked with R2A to implement its suptech solution, while NRB worked with the United Nations Capital Develop- Other offices can also play the role of internal cham- ment Fund. pion. In several authorities, IT departments led supt- ech initiatives. A specific business office may champion These initial successes created the rationale for further the adoption of specific suptech solutions, such as the investment. Regulators were also able to understand supervisory department, complaints department, or the through firsthand experience which skillsets were needed office responsible for reporting by FSPs. For example, internally to support desired supervisory outcomes. Only the complaints-handling department at BSP led invest- after learning this did regulators begin hiring or retraining ments in increased automation for complaints handling as staff to reflect the skillsets now recognized as needed to described above, supported by external software vendors be suptech- and data-driven regulators. In both the Philip- and technical assistance. pines and Nepal, regulators started with a small core team (supported with external technical assistance) and began In addition to an identifiable champion for suptech, expanding the data and technology skillsets of their own buy-in from top management is also important. Top staff only after working with the external party to develop management support is helpful both in setting the stra- its suptech solution. tegic, high-level goals for data and technology and in the tactical implementation of specific solutions. Top However, while external parties can help develop supt- management also provides crucial advisors to the iden- ech solutions, regulators need to plan to have inter- tification and implementation of solutions. Senior man- nal capacity to manage, use, and maintain the solution agement can identify data and skills gaps and serve as immediately at its launch. BNR experienced a gap of advocates for suptech generally and for broader invest- nearly one year between the completion of its infrastruc- ments or changes to people, process, or technology that ture upgrade and the development of its internal analytics represent the key implementation enablers for suptech. team. During this time, the authority was not able to make Sometimes, this support takes the form of a public state- full use of the data available. This gap shows the need ment. For example, BOE committed in 2019 to develop- to have a clear plan for appropriate capacity to leverage ing a “world-class regtech and data strategy” with the aim suptech solutions to process and analyze large quanti- of making data collection more efficient for firms while ties of data over both the near and long term in order to also improving BOE’s technology and data capabilities reap the full benefits from such technologies. The same is (BOE 2019). Support of senior management is also key true of the solution’s maintenance. As suptech is a rapidly to secure adequate funding and to cultivate relationships evolving area, continued maintenance and support are with industry players, including both providers of poten- critical to keep the solution relevant. If an external ven- tial suptech solutions and supervised institutions. dor is used to build the solution, the authority may want to consider negotiating ongoing maintenance or for the Financial authorities also indicated that strong gov- vendor to train internal staff to manage the solution. Such ernance processes are necessary to complement the training can present a capacity-building moment for staff. introduction of suptech solutions, in particular regard- ing access to the new data collected through these solutions. Governance processes create mechanisms for  rocess: Internal Champions and Strong Gov- 4.2 P the authority to manage the suptech solution and align it ernance with evolving regulatory objectives. The insights derived Across all country examples, an important factor for from these solutions are important inputs for data-driven success was having an internal champion for suptech. policy making. For example, AMF’s supervision informa- The most frequent internal champions for suptech were tion system is managed by a collection of internal reg- innovation offices or a specific business office. In the case ulatory and IT teams to support its off-site supervision of both AMF and the FCA, dedicated innovation offices framework. Solution governance includes a supervisory were created to support business units to develop proofs team focused on standards and reporting guidelines to of concept for suptech solutions. Via a hub-and-spoke supervised institutions and a second group of risk experts model, innovation offices centralized technical and data focused on determining and maintaining data validations. expertise across each authority, making the recruiting and Together, these two teams govern the solution’s data col- retaining of technical talent easier. In addition, by making lection, analysis, and reporting to support AMF’s supervi- individual business units responsible for the deployment sion framework. The Next Wave of Suptech Innovation   27 It is also common for authorities to develop formal IT infrastructure may be a particular challenge for low-in- procedures to govern data access to minimize data come countries. These authorities may need to devote mismanagement, cybersecurity, and information secu- dedicated attention and resources to IT infrastructure, rity risks. Each department has specific permission-based especially to implement solutions of higher complexity. access to view data that is proportional to the need of When upgraded IT infrastructure is available and acces- the role or department. For example, the research depart- sible across a financial authority, any internal “entrepre- ment at BNR has access to data determined to support its neur” can run experiments, test concepts, and propose policy-making objectives, whereas supervisors reviewing new suptech solutions. a specific FSP have a more granular view. The CFPB and Australian Securities and Investments Commission (ASIC) Successful implementation of suptech solutions, par- go further and are among the authorities that have intro- ticularly for lower-income countries, also depends on duced a central department with a chief data officer to the technical capacity, readiness, and involvement govern data access. ASIC’s Chief Data Office has devel- of external users, such as supervised institutions. It is oped a Data and Information Governance Framework that important to assess if FSPs will be able to implement and has introduced dedicated forums, such as a digital gover- use certain suptech solutions, such as with respect to reg- nance board, data governance council, and data-analyst ulatory reporting. This consideration is particularly rele- network, to govern data access and management. vant for market conduct supervisors who may be required to oversee a large number of smaller and less sophisti- Good governance also helps ensure that the insights cated institutions. For example, BNR suffered operational derived from these solutions are fed as inputs into delays due to the complex nature of its EDW. Through data-driven policy making. For example, clear processes active collaboration and dialogue, BNR refined its solu- are put in place to ensure that the outputs of the CBI’s tion to accommodate the varied technical capabilities social media monitoring solution are distributed and of its supervised institutions. FSPs with strong technical used across the authority. This is facilitated by the Mar- capabilities report regulatory data through ADFs,12 while ket Monitoring Working Group, which convenes people FSPs with lower technical capabilities can report data via from across the authority to understand developments in simpler methods, such as by means of APIs or web portal the marketplace and discuss implications across internal uploads, depending on their capabilities. departments. Lastly, when considering a large investment in under- lying IT infrastructure, it is important to consider the 4.3 Underlying IT Infrastructure near-term and long-term suptech application it sup- All suptech solutions rely to varying degrees on the ports. IT infrastructure investments vary in their complex- financial authority’s underlying IT infrastructure. Supt- ity and required timeline for implementation. For example, ech solutions are often built as applications on top of AFM refers to different “horizons” when considering supt- existing IT infrastructure. Where modern IT infrastructure ech and IT infrastructure investments—near term (one to exists, the time, cost, and effort to implement the supt- two years), medium term (three to five years), and long ech solution is likely to be lessened. Relevant underlying term (up to 10 years out). Through this lens, authorities technology and competencies include database ware- can take an approach that considers and sequences antic- housing and management, big-data computing, CRM ipated projects in a portfolio and balances the delivery of systems, information security protocols, and licenses to BI both near-term wins and planning for large, longer-term and analysis software. The absence of modern or existing investments. As of 2020, 16 banks, 16 insurance companies, and several forex bureaus and microfinance institutions participate in this manner. 12.  28   The Next Wave of Suptech Innovation 5. IMPLEMENTATION Suptech can be driven by “pull” factors like supervisory demand. This is especially the case where supervisors CONSIDERATIONS face limited resources and seek increases in employee productivity. For example, regulators in Nepal, Rwanda, Common considerations when implementing supt- and the Philippines developed interactive dashboards for ech solutions emerged across the country examples supervisors to view and query supervisor data in real time. included in this note. In the process of implementing Interactive dashboards are not a new technological inno- suptech solutions, supervisors often face key decisions vation; they have existed for several years. The new dash- related to determining the business case for a suptech boards were developed to enable employees to analyze solution, whether to build a solution in-house or buy from the increasing volumes of data faster. a vendor, and how to organize their data and technology staff. Financial authorities that have implemented suptech Industry can also lead to demand for suptech adoption. solutions for market conduct have also benefited from ini- Pressure exists to make market conduct supervision more tiatives that accelerate adoption, including formal supt- efficient for both financial authorities and supervised insti- ech or data strategies, innovation offices, and liaising with tutions. For example, solutions for regulatory reporting, stakeholders. such as the APIs in the Philippines and ADFs in Rwanda, were driven by promises made to industry by regulators to 5.1 Key Decisions in Suptech Implementation reduce the regulatory burden on supervised institutions. Building the Business Case for Suptech Adoption Other suptech solutions are driven by “push” factors Investment in any suptech solution should have a clear such as the availability of new technologies in data rationale and business case. Suptech solutions should analysis, IT infrastructure, and computer science. These be supervisor-centric—that is, they should represent clear innovations provide supervisors new applications to per- enhancements to support established supervisory objec- form new functions or enhance existing ones. In this con- tives. Value is delivered through a combination of greater text, supervisors are rarely driving the primary research operational efficiency and enhanced supervisory effec- and development of new algorithms or technologies. tiveness. While seemingly obvious, authorities who partic- More often, they leverage cutting-edge innovations, new ipated in this technical note emphasized the importance technologies, and advancements in IT infrastructure and of beginning with defining a specific supervisory problem, configure them to their own needs and use cases. This rather than elevating the technology solution before the is particularly true regarding advancements in machine problem. learning and NLP. Advanced algorithms were until recently restricted to small teams of highly specialized computer Suptech solutions can serve multiple types of super- scientists, often in university departments or high-tech vision. In pursuing a suptech solution, market conduct firms. Now, they are in reach for financial sector regula- supervisors can partner with other departments to build a tors, due to the popularization of data-science software stronger business case for a solution that supports multiple and platforms. These are enabled by widely used pro- departments and supervisory objectives. Many suptech gramming languages, such as R and Python, and made solutions used for prudential or financial inclusion supervi- accessible with open-source machine-learning libraries sion can be adapted to serve market conduct supervisors and APIs. Though programmers still need to know how or vice versa. For example, the data and insights gathered these algorithms work and their underlying statistical pro- directly from supervised institutions and consumers, or cesses and parameters, this knowledge has become far from non-traditional market monitoring sources, can be more accessible and easier to learn. used to support market conduct as well as prudential or financial inclusion supervision. Solutions for document and Addressing Legacy IT Systems business analysis can also be adapted to support other As supervisors seek to develop new suptech solutions, types of supervision beyond market conduct. authorities must decide what to do with existing, older IT systems. Supervisors must choose whether to The motivation to explore suptech solutions is often focus their energies and budget on replacing older sys- driven by two major forces: (1) supervisory demand tems with new infrastructure or on building new solutions and (2) technological innovation. On the one hand, using existing IT systems, and what the appropriate bal- demands on supervisory staff with limited resources drives ance between these two competing demands is. Across efforts to boost operational efficiency using technology. authorities who contributed to this note, the typical On the other hand, the availability of new technologies approach was to choose aspects of both, depending on present authorities with the opportunity to enhance the the regulatory need and the solution’s context. supervisory process. The Next Wave of Suptech Innovation   29 For new suptech areas with comparatively limited leg- or analytics. For example, ASIC has partnered with the acy IT systems, such as NLP analysis, regulators can Sydney-based intelligence, analytics, and cybersecurity focus on building new systems and solutions. For exam- firm Nuix since 2005 in order to extend ASIC’s capacity to ple, the FCA developed a new in-house NLP platform that become a more data-driven, intelligence-led law enforce- supports several suptech applications. ment agency (ASIC 2017b). In Rwanda, BNR partnered with Sunoida Solutions to develop an Electronic Data For suptech areas with legacy IT systems, such as regu- Warehouse to automate and streamline data submission latory reporting solutions or complaints management, and analytics. authorities often focus on improving the existing IT infrastructure. For example, NRB is maintaining its legacy Sourcing suptech from industry vendors can be faster regulatory reporting database and data-submission infra- and usually benefits from economies of scale and the structure while introducing a next-generation solution in availability of dedicated staff with specialized technical 2021. This solution will replace and eventually sunset its expertise. Widely available commercial solutions benefit web portal with data upload with a new API-based report- from economies of scale, leading to demonstrated track ing method. Likewise, AFM developed a suptech road- records for quality and efficient timelines for implemen- map that prioritizes both near-term suptech development tation. External sourcing also reduces the challenge of with longer-term infrastructure needs, as exemplified by hiring and retaining technical staff, at least initially. Both its managed data platform that will be expanded across CBI and AMF contract with external vendors for specific its supervisory department as the solution expands to solutions for non-traditional market monitoring. These include additional markets under its jurisdiction. arrangements are particularly efficient when the needs of regulators can be readily adapted from commercial solu- Deciding Between In-House Solutions Versus tions and they do not change frequently over time. Third-Party Vendors Supervision departments must make the strategic deci- However, there are limits to relying on external parties; sion on whether to invest the staff time and resources the market for specialized private sector suptech solu- to build in-house tech solutions for market conduct or tions is growing but not yet mature. Few firms specialize to rely on commercially available solutions. This deci- in serving financial authorities, though this is evolving with sion depends primarily on the complexity of the solution the emergence of innovation offices within authorities to and the optimal allocation of financial resources and staff collaborate with vendors on solutions. Most vendors have skills. At the same time, it is also a decision that reflects predominately developed tech solutions predominately the business culture of an authority and the available solu- for private sector firms, which is the larger market. For tions in the market. example, in the context of non-traditional market monitor- ing, banks or insurance firms contract vendors for brand Buying solutions from software vendors is often best protection on social media, whereas supervisors aim to for complex investments in IT infrastructure, reporting, detect market conduct risks such as product mis-selling FIGURE 10: Considerations for In-House Development Versus Using a Third-Party Vendor Building a Custom Suptech Buying a Suptech Solution from a Vendor Solution In-House PROS • Absolute control over the solution • Brings external technical and data expertise and • Ability to customize or adapt to perspective authority’s needs • Stability and quality of widely available tools are typically high, known • Build internal tech capacity and technology skillsets CONS • Requires ongoing resources to • Vendor lock-in (for example, a long-term contract is typically required) maintain or upgrade solution • Difficult to discern how readily available a solution is out of the box • Challenge of finding, hiring, and (for example, navigating a vendor’s marketing of the solution) retaining skilled staff • Solution may require adaptation from an industry use case to regulatory use case • Procurement process may be burdensome, or process may preclude hiring the preferred vendor with a cheaper, less preferred alternative • Requires a strong vendor oversight process • Constraints on data sharing may exist 30   The Next Wave of Suptech Innovation and customer complaints. While the two are related, they collaboration between supervisors and data scientists require customization and adaption, nonetheless. and enables data scientists to be more aware of the day-to-day challenges that business units face. How- Depending on the legal and regulatory framework, lim- ever, this often creates silos between departments, its on the collection, sharing, and management of data reducing the opportunity for spillovers and cross-fer- with external parties can also be a barrier. In some coun- tilization of ideas. Moreover, a decentralized function tries, privacy or data-sharing laws limit regulators’ ability requires highly talented managers capable of manag- to collect data or share data with others. There may also ing both supervisory and data-science staff of different be restrictions on how data can be housed. Legal require- backgrounds within the same team. ments may limit viable solutions available commercially. 3. Hub-and-spoke model: This increasingly popular model combines elements of the two previous approaches. The “build vs. buy” decision should be made on a solu- In this approach, a centralized data office coordinates tion-by-solution basis after regulatory goals, benefits, with data-analytics staff within the business units, main- and costs are considered properly. Examples of the fac- taining a constant flow of communication and spillover tors to consider are included in figure 10. within the teams. This approach is particularly effec- tive at avoiding silos as well as isolation of data-sci- Organizing Internal Staff Working on Data and Technology ence teams. However, it requires higher coordination Management of internal talent on data science and and managerial capacity to implement properly. This analysis is essential for a data-driven supervisory strat- approach normally makes sense in larger organiza- egy to take effect fully. Authorities such as the FCA, tions. ASIC, and AFM view investments in building data-science awareness and capacity among internal staff as crucial to Utilizing Adaptive Approaches for Suptech Development supporting their suptech strategy. In low- or middle-in- The purpose of adaptive approaches is to engage and come countries such as Rwanda, the Philippines, and validate a suptech solution’s design and functionality Nepal, recruiting to increase data-science capacity is by involving end users, which maximizes the utility of often cited as a key challenge, making effective manage- the solution when delivered. People are the ultimate end ment of employees even more important. users of these solutions, and solutions must be designed to serve their specific needs. A lack of involvement by Across the county experiences included in this note, end users in developing these solutions is likely to ham- authorities faced a common decision point regarding per the solutions’ ability to meet organizational needs. how to integrate supervisors and data scientists. Three For example, “Agile,” a common adaptive approach main approaches to organizing data-science and analysis to software development, prioritizes delivery of smaller, teams were observed, as described below. Most author- more frequent pieces of functionality for users. Earlier ities adopt hybrid approaches, establishing centralized user interaction with the solution results in more engage- teams but encouraging the formation of interdisciplinary ment with end users and the opportunity for those users working groups to tackle specific business questions. As to provide more timely feedback. Adaptative approaches authorities expand data-science capacity overall, some can replace more traditional software-development pro- envision transitioning to decentralized data-science teams cesses, such as “waterfall” methods. Under waterfall, or hub-and-spoke models. extensive definition of requirements and development 1. Centralized data-science team: All data scientists sit of the proposed solution happen before interaction with together in the same department and manage ana- end users. This results in limited flexibility for changing lytics across the authority. This is also often called requirements once final, or expensive changes that may the “center of excellence” model, as data-science come up only at the end stages. teams are expected to maintain a channel of com- munication with business units but maintain a degree Authorities can utilize experimentation and iteration in of independence. The main benefit of centralization the technology-development process. These processes is that it helps data-science teams focus on their own validate a solution’s concept, establish a solution’s feasi- initiatives. However, this independence from business bility, and build the investment case internally for a larger units limits their visibility into business needs. outlay of resources to build and deploy a full solution. Iteration is key to the development of appropriate supt- 2. Decentralized data-science team: Data scientists are ech solutions and is present in many of the experimental embedded across different business units. The main approaches of financial authorities. Possible approaches benefit of decentralization is that it promotes strong include the following: The Next Wave of Suptech Innovation   31 • Design or tech sprints, including hackathons: These of initiatives to accelerate data-driven supervision. This occur early in the technology-development process, includes creating a Chief Data Office to develop and typically in the ideation phase. A small project team manage internal data policies, establishing a Data Gov- comes together to develop a product idea in a con- ernance Council and Digital Governance Board as ded- fined amount of time, most frequently over a few days icated forums for data governance, and setting up a to a week. During the “sprint,” the team develops a data-analyst network. In the United Kingdom, the FCA potential solution, typically presented as an initial developed its first data strategy in 2013 (FCA 2013) and solution concept or mockup. This process allows for updated it in 2020 to include new technologies and capa- ambiguities to be identified and addressed before pro- bilities. Similarly, BOE published the 2020 report Trans- ceeding with the idea further. Dummy data and mock forming Data Collection from the UK Financial Sector, by visualizations are common at this stage. which it initiated a dialogue with supervised institutions and solution vendors to shape data reporting over the • Proof of concept: A small application, experiment, or next 5–10 years (BOE 2020). In each instance, the devel- exercise to test the feasibility of a suptech solution. It is opment of a formal institutional strategy on data served as usually conducted to demonstrate that a solution can a catalyst to align internal departments around a multiyear be implemented, but without exploring the full imple- suptech program to foster technology development and mentation details. inform data-driven supervision. • Prototype: An approach to developing a tangible model to test the desirability of a suptech solution. The While a formal suptech or data strategy can be helpful, objective of a prototype is to share the solution with such a formal plan is not mandatory in order to begin users for feedback to identify errors, inefficiencies, or implementing suptech solutions. Successful regulators other issues. were just as likely to employ an incremental approach that focused on introducing targeted solutions to address spe- • Pilot: As the prototype of a suptech solution becomes cific supervisory problems, as opposed to developing a higher fidelity (that is, more closely resembling the comprehensive institutional strategy. final product), it is common to pilot the solution with a limited population or subset of the final users. The Innovation Offices and Liaising with Stakeholders goal is to understand how the solution works in a real Innovation offices can be helpful for financial author- context and what changes may be needed to scale to ities implementing formal data or suptech strategies. full production. Also referred to as “labs” or “hubs,” innovation offices • Minimum viable product (MVP): The suptech solution is provide a dedicated forum and often dedicated IT, implemented with a core set of capabilities or features. data-science, and regulatory staff to test and develop a It is important to note that when introduced, an MVP variety of data and suptech solutions. Innovation offices solution will not be fully featured and will have aspects allow for an experimental mindset and a data-driven cul- of its end-state functionality missing. The objective is ture of suptech innovation within the constraints of a regu- to minimize time to implementation and delivery of latory authority. For example, BdP launched its innovation value to the authority and for end users. From this core lab (called inov#) in 2019 to streamline current supervisory set of features, additional capabilities are then added processes by exploring new capabilities, with a focus on to improve the solution over time. NLP use cases. Innovation offices also create a central place for learn- Initiatives to Accelerate Suptech 5.2  ing and development opportunities for employees to Implementation gain stronger data and technology skills. For example, Formal Suptech or Data Strategies the FCA’s innovation office has organized analytics com- The creation of a formal suptech or data strategy can munities. It has also designed a comprehensive Data Train- accelerate data capacities and suptech adoption within ing Programme tailored to different roles, ranging from a a financial authority. Formal strategies allow for a struc- member of the FCA Executive Committee to supervisory tured approach to increasing data capacities or suptech analysts. An informal innovation community has devel- adoption that has institution-wide buy-in, and coordinate oped around its innovation office through an organized action and the support of an authority’s senior manage- calendar of events and seminars including “Data Week,” ment. For example, ASIC developed a three-year data a week-long program of data-focused events with over strategy (2017–20) outlining a vision for suptech and its 50 sessions; a “reverse mentoring” scheme that matches approach to capture, share, and use data (ASIC 2017a). senior leaders with data scientists; and competitions to ASIC’s data strategy has been accompanied by a series expand appetite for analytics. 32   The Next Wave of Suptech Innovation Liaising with stakeholders is also key to accelerating fintech or regtech14 firms more closely. Through collabo- successful suptech implementation. Authorities will ration, the authorities intend to build awareness about the need to engage with industry, both as a source of supt- regulatory landscape and encourage industry innovation ech solutions and as supervised institutions. Innovative in regtech and suptech solutions. By opening a dedicated ideas can often come from unexpected areas. Financial office to collaborate with industry, authorities can signal authorities can benefit from bringing together regulators, greater priority and demand by the regulator for such academics, vendors, and developers to promote an envi- solutions and encourage solution providers to bring new ronment for innovation, share experiences and informa- suptech and regtech solutions to market. tion, and enable new developments that can benefit the varied participants. Lastly, international networks also promote financial innovation and provide regulators with platforms for Innovation offices can help improve dialogue among interaction. For example, the Global Financial Innovation diverse types of experts, industry, and fintech startups. Network, formally launched in January 2019, comprises For example, CBI, ASIC, BdP, and AFM each established an international group of financial sector regulators and an innovation hub13 whose primary purpose is to engage related organizations committed to supporting financial CASE STUDY 6 How ASIC’s Innovation Office Collaborates with Industry Stakeholders ASIC launched its innovation hub15 in 2015 to promote the cases. ASIC began by releasing a set of problem statements development of fintech and regtech solutions that improve out- to solution providers and invited applications on how NLP comes for consumers and market integrity. could solve for each of them.16 The trials explored potential As part of ASIC’s research efforts, the innovation hub aims to efficiencies in supervision, including through automation and keep up to date on the latest developments in regtech and supt- prediction. The results of these trials gave ASIC insights into ech. ASIC often convenes roundtables, liaison forums, national data-availability issues, data-annotation work, and areas on and international network events, and regulator meet-ups, and which to focus internal capacity in the future. Subsequent it promotes training events for its proofs of concept. Through pilots focused on monitoring financial promotions, financial these activities, ASIC has learned about trends related to con- advice, voice analytics and voice to text, and technology-as- sumer and industry demand, identified appropriate technology sisted guidance tools (ASIC 2019). With respect to financial use cases in the market, and clarified roles of industry and regu- promotions, trial demonstrations accurately detected poten- lators regarding the use of regtech and suptech. tial breaches of mandatory disclosure requirements in over 90 ASIC is keen to monitor and understand market develop- percent of specific cases. Similarly, demonstrations identified ments related to technology innovation for purposes of its own compliance issues in financial advice files at accuracy rates of internal use. Since 2018, ASIC has led a series of regtech and around 90 percent in the sample data set. suptech research-and-development initiatives, including research The innovation hub also provides informal assistance to to understand commercially available solutions and promoting start up and scale up businesses to navigate the regulatory internal trials of new, emerging technologies. In order to partic- framework and share how regulation may affect them. Fintech ipate, ASIC generally requires suptech and regtech providers and regtech providers receive access to (1) practical regula- to demonstrate how their technology solution(s) can potentially tory support and informal assistance from senior ASIC staff, promote better outcomes for investors, financial consumers, or (2) options relating to ASIC’s relief powers, such as the regula- markets, or how their technology solution(s) promotes ASIC’s reg- tory sandbox, and (3) events focused on fostering cross-sector ulatory objectives, improved risk management, or compliance. engagement, including regtech quarterly liaisons, demonstra- For example, ASIC trialed NLP solutions in 2018 to under- tions, and a role as an observer in trials. stand the relevancy of the technology to key supervisory use 13. As an example, see www.centralbank.ie/regulation/innovation-hub. 14. Regtech refers to the use of new technologies by FSPs to meet their regulatory requirements. See FSB (2020). 15. https://asic.gov.au/for-business/innovation-hub/. 16. The initial pilots focused on (1) identifying promotions of concern for financial and credit services, (2) phone sales practices of insurance providers, (3) review of managed- fund product-disclosure statements, (4) review of financial advice files, (5) review of financial reporting in company announcements, and (6) review of prospectuses. The Next Wave of Suptech Innovation   33 innovation in the interests of consumers. It seeks to provide Additional Challenges Encountered by 5.3  a more efficient way for innovative firms to interact with Regulators regulators, enabling pilots for firms wishing to test innova- Where regulators still need to develop their basic tive products, services, or business models across jurisdic- supervisory processes or framework for market con- tions. It also aims to create a framework for cooperation duct, suptech implementation should likely take between financial sector regulators on innovation-related secondary priority. As noted previously, supervisors topics and to facilitate the sharing of different experiences commonly encounter the challenge of how to design and and approaches.17 The International Financial Consumer implement suptech solutions that support the authori- Protection Organisation (FinCoNet), an international orga- ty’s supervisory processes and are tailored to meet their nization of market conduct supervisory authorities, similarly supervisory needs and capacity. However, in some juris- provides a platform for market conduct supervisors to share dictions, market conduct supervision may be a brand-new knowledge and experiences and learn from one another. or just-emerging function. Implementing suptech solu- Financial sector authorities have also developed bilateral tions is particularly challenging in jurisdictions that have relationships to learn from one another, share data, and not developed their core market conduct supervisory work more closely together on suptech solutions and mar- framework yet. In such cases, there are several caveats ket conduct supervision. For example, the FCA and AFM announced a bilateral partnership in 2019. BOX 1 FinCoNet: SupTech Tools for Market Conduct Supervisors FinCoNet is an international organization of financial tools to the challenges of digital financial product and consumer protection supervisory authorities. The goal of services, and it highlights a series of useful takeaways to FinCoNet is to promote sound market conduct and to be considered by the supervisory community. enhance financial consumer protection through efficient and effective financial market conduct supervision, with a Following the release of this report, the standing com- focus on banking and credit. mittee then initiated a project to review the most inno- vative tools carried out by the supervisors’ community, In recent years, the effects of digital transformation on summarized in the report SupTech Tools for Market financial consumer protection have become a priority item Conduct Supervisors (FinCoNet 2020), published in on the agenda of supervisory authorities. In this vein, Fin- November 2020. A workshop on this topic took place CoNet has stated that the shift from traditional financial at FinCoNet’s annual general meeting in November sector delivery channels to online and mobile technology 2019, with the participation of the World Bank, the has important implications. These include supervisory Organisation for Economic Co-operation and Develop- authorities’ ability to identify emerging consumer risks ment, and the Bank for International Settlement, and arising from digitization and to have appropriate tools other organizations. to mitigate such risks. Consequently, the FinCoNet Gov- erning Council decided to include a standing committee This new report, based on a survey of a wide range to develop further work in these areas in FinCoNet’s Pro- of authorities, aims at capturing the general framework gramme of Work for 2017/2018. for the development of suptech, detailing strategies, supervisory needs, levels of use, operational readi- This work led to the report Practices and Tools Required ness and areas of application, thus complementing the to Support Risk-Based Supervision in the Digital Age (Fin- research work hereby presented by the World Bank. CoNet 2018), published in November 2018. This report Moreover, FinCoNet’s report contains a description reflects the experiences of various supervisory authorities of the most relevant SupTech tools, how such tools as they tackled the challenges stemming from the need are applied in market conduct supervision, how they to ensure proper consumer protection in the framework enhance supervisory processes, and the main chal- of digitization and the ways they are adapting supervisory lenges encountered. 17. https://www.thegfin.com/. 34   The Next Wave of Suptech Innovation to consider in focusing on technology and data, without Already in 2020, financial markets and authorities first putting in place the fundamentals of a framework for faced a huge new test due to the COVID-19 pandemic. proper risk indicators and strategic metrics for market The crisis has prompted a significant increase and reliance conduct supervision. For such jurisdictions, supervisors on digital financial services, further accelerating the dig- likely will first need to dedicate more effort to developing ital transformation. It is likely that many consumers will supervisory frameworks for market conduct before begin- become comfortable with digital financial services during ning to explore suptech solutions to automate processes this crisis and will stick to these new behaviors even as the and procedures and collect the types of data that support crisis lessens. What started over a decade ago in small their supervisory needs. pockets may soon become the default way of banking around the world. Regulators may also open themselves to new opera- tional and reputational risks through suptech. The col- Within this landscape of digital transformation, supt- lection of more, and new, types of data requires regulators ech becomes an invaluable tool for financial authori- to assume a new role as responsible data managers and ties. This is true of every category of suptech for market to act appropriately to protect this data from accidental conduct supervision. The direct and automated collection disclosure or from bad actors. Data privacy, information of granular regulatory data from supervised institutions security, and cybersecurity represent new competencies obviates the need for on-site examinations, while digital for regulators, who will need to mitigate these risks. In a interfaces for complaints handling enable regulators to rush to collect more and more data without a commensu- engage directly with consumers online while automating rate investment in internal skills and capacity, regulators complaints data collection and analyses. Non-traditional also open themselves to the risk of having more infor- market monitoring, meanwhile, provides a real-time pulse mation than can be processed or analyzed in a proper on fast-moving sentiment and consumer risks with FSPs, or timely manner. This presents a potential reputational while solutions for text analysis can extract insights from risk, whereby the regulator has had access to the proper documents in seconds where previously it would have information to identify the market conduct issues but ulti- taken weeks or longer. The real-world solutions presented mately lacked the proper capacity to do so in a timely in this note provide authorities with the tools to oversee manner. increasingly complex markets with increased effectiveness and efficiency. 6. LOOKING FORWARD To be sure, suptech for market conduct is not a silver bullet. It certainly does not replace having the fundamen- Over the last decade, consumer finance has been trans- tals of market conduct supervision in place, and there are formed due to digital technology. Traditional players many pre-conditions and challenges to successful imple- have undergone digital transformations, while a range mentation of suptech solutions. However, when com- of new, non-traditional players have entered the market, bined with the right mix of smart, competent staff and a including fintechs, mobile network operators, and tech- comprehensive market conduct supervision framework, nology firms. While this transformation has led to bene- suptech solutions can better position market conduct ficial innovations for financial consumers, it also presents supervisors for the challenges and opportunities ahead. market conduct supervisors with new challenges. They The initial successes of regulators highlighted in this note must safeguard markets burgeoning with new users, new offer a glimpse of the future—one in which data and tech- institutions, new technologies, and new business models, nology become core to the operations, identity, and cul- balancing protection of consumers with innovation. ture of financial authorities to enable them to achieve their regulatory mandates. It may be tempting for regulators to step on the brakes to hold back these forces. The solution, of course, is not to limit their development but to ensure that regulators have the tools they need to oversee these rapidly evolv- ing markets efficiently and effectively. Greater regulatory confidence and capacity reinforce healthier, inclusive financial markets. This is precisely where suptech can play such a critical role. REFERENCES ASIC (Australian Securities and Investments Commission). 2017a. ASIC’s Data Strategy 2017–2020. https:// download.asic.gov.au/media/4511295/asic-data-strategy-2017-20-published-11-october-2017.pdf. ———. 2017b. “ASIC Extends Partnership for Digital Investigation and Analytics Software.” https://asic. gov.au/about-asic/news-centre/find-a-media-release/2017-releases/17-262mr-asic-extends-partnership-for- digital-investigation-and-analytics-software/. ———. 2019. ASIC’s Regtech Initiatives 2018–19 (Report 653). https://download.asic.gov.au/ media/5424092/rep653- published-20-december-2019.pdf. BCBS (Basel Committee on Banking Supervision). 2012. Core Principles for Effective Banking Supervision. Washington, DC: Bank for International Settlements. https://www.bis.org/publ/ bcbs230.pdf. ———. 2017. Sound Practices: Implications of Fintech Developments for Banks and Bank Supervisors. Washington, DC: Bank for International Settlements. https://www.bis.org/bcbs/ publ/d415.pdf. BIS (Bank of International Settlements). 2018. Innovative Technology in Financial Supervision (Suptech)—the Experience of Early Users. https://www.bis.org/fsi/publ/insights9.htm. ———. 2019. The Suptech Generations. https://www.bis.org/fsi/publ/insights19.pdf. BOE (Bank of England). 2019. Future of Finance. https://www.bankofengland.co.uk/report/2019/future-of- finance. ———. 2020. Transforming Data Collection from the UK Financial Sector. https://www.bankofengland. co.uk/-/media/boe/files/paper/2020/transforming-data-collection-from-the-uk-financial-sector. pdf?la=en&hash=6E6132B4F7AF681CCB425B0171B4CF43D82E7779. CGAP (Consultative Group to Assist the Poor). 2017. Data Collection by Supervisors of Digital Financial Services. https://www.cgap.org/research/publication/data-collection-supervisors-digital-financial-services. di Castri, Simone, Matt Grasser, and Arend Kulenkampff. 2018. Financial Authorities in the Era of Data Abundance: Regtech for Regulators and SupTech Solutions. https://papers.ssrn.com/sol3/papers. cfm?abstract_id=3249283. ———. 2020a. A Chatbot Application and Complaints Management System for the Bangko Sentral ng Pilipinas (BSP): R2A Project Retrospective and Lessons Learned. https://papers.ssrn.com/sol3/papers. cfm?abstract_id=3596268. ———. 2020b. An API-Based Prudential Reporting System for Bangko Sentral ng Pilipinas (BSP): R2A Project Retrospective and Lessons Learned. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3596276. FCA (Financial Conduct Authority). 2020. “Data Strategy” (web page). https://www.fca.org.uk/publications/ corporate-documents/data-strategy. ———. 2013. The FCA Data Strategy—How We Will Manage and Use the Data We Collect. https://www.fca. org.uk/publication/corporate/fca-data-strategy.pdf. FSB (Financial Stability Board). 2020. The Use of Supervisory and Regulatory Technology by Authorities and Regulated Institutions: Market Developments and Financial Stability Implications. https://www.fsb. org/2020/10/the-use-of-supervisory-and-regulatory-technology-by-authorities-and-regulated-institutions- market-developments-and-financial-stability-implications/.   35 36   The Next Wave of Suptech Innovation FinCoNet (International Financial Consumer Protection Organisation). 2018. Practices and Tools Required to Support Risk-Based Supervision in the Digital Age. http://www.finconet.org/Finconet_Report_Practices- tools-for-risk-based-supervision- digital-age_November_2018.pdf. ———. 2020. SupTech Tools for Market Conduct Supervisors. http://www.finconet.org/FinCoNet-Report-SupTech-Tools_Final.pdf Gurung, Nora, and Leon Perlman. 2018. “Use of Regtech by Central Banks and Its Impact on Financial Inclusion.” https://dfsobservatory.com/publication/use-regtech-central-banks-and-its-impact-financial- inclusion. IMF (International Monetary Fund). 2015. Can Financial Inclusion Meet Multiple Macroeconomic Goals? https://www.imf.org/external/pubs/ft/sdn/2015/sdn1517.pdf. Murphy, Dan, and Jackson Mueller. 2018. RegTech: Opportunities for More Efficient and Effective Regulatory Supervision and Compliance. https://milkeninstitute.org/reports/regtech-opportunities-more-efficient-and- effective-regulatory-supervision- and-compliance. Toronto Center. 2018. SupTech: Leveraging Technology for Better Supervision. https://res.torontocentre.org/ guidedocs/ SupTech%20-%20Leveraging%20Technology%20for%20Better%20Supervision%20FINAL.pdf. World Bank. 2018. From Spreadsheets to Suptech: Technology Solutions for Market Conduct Supervision. https:// openknowledge.worldbank.org/handle/10986/29952. ———. 2020. A Roadmap to Suptech Solutions for Low Income (IDA) Countries. https://documents. worldbank.org/en/ publication/documents-reports/documentdetail/108411602047902677/a-roadmap-to-suptech-solutions-for- low-income-ida-countries. Annex II: Examples of Key Facts Statements   37