PROCUREMENT IN WORLD BANK INVESTMENT PROJECT FINANCING REVIEW OF RESULTS AFTER FIVE YEARS OF IMPLEMENTATION OF THE PROCUREMENT FRAMEWORK FY17 TO FY21 The Vision: “Procurement in IPF operations supports Borrowers to achieve value for money with integrity in delivering sustainable development� April 20, 2022 ABBREVIATIONS AND ACRONYMS AFR Africa Region ALB Abnormally Low Bid APA Alternative Procurement Arrangements ASA Advisory Services and Analytics BFP Bank-Facilitated Procurement COVAX COVID-19 Vaccines Global Access COVID-19 Coronavirus Disease 2019 DPF Development Policy Financing EAP East Asia Pacific Region ECA Europe and Central Asia Region EFI Equitable Growth, Finance and Institutions E-GP Electronic Government Procurement ESF Environmental and Social Framework FCV Fragile Conflict Violence GBV Gender-Based Violence GPP Green Public Procurement HEIS Hands-on Expanded Implementation Support IDA International Development Association IEP Industry Engagement Program INT Integrity Vice Presidency IPF Investment Project Financing ISR Implementation Status and Results IT Information Technology KPI Key Performance Indicator LCC Life Cycle Costing LCR Latin America and Caribbean Region M/Mi Million MAPS Methodology for Assessing Procurement Systems MDB Multilateral Development Bank MDPP Mandatory Direct Payment Pilot MNA Middle East North Africa Region N/A Not Applicable NGO Non-Governmental Organization OECD Organization for Economic Co-operation and Development OPCS Operations Policy and Country Services OPRC Operations Procurement Review Committee PFM Public Financial Management PforR Program for Results PPE Personal Protective Equipment PPM Procurement Practice Manager PPR Post Procurement Review PPSD Project Procurement Strategy for Development P-RAMS Procurement Risk Assessment and Management System RfB Request for Bids RfP Request for Proposals SAR South Asia Region SEA-SH Sexual Exploitation and Abuse-Sexual Harassment SME Small and Medium Enterprises SPD Standard Procurement Document SPP Sustainable Public Procurement STEP Systematic Tracking of Exchanges in Procurement UN United Nations UNDB United Nations Development Business UNICEF United Nations Children’s Fund UNOPS United Nations Office for Project Services USD United States Dollars VfM Value for Money WHO World Health Organization CONTENTS EXECUTIVE SUMMARY .........................................................................................................................................1 I. INTRODUCTION ....................................................................................................................................................7 A. CHANGING INSTITUTIONAL ROLES AND DECISION-MAKING .........................................................................9 B. SHIFTING CULTURE, CHANGING BEHAVIORS AND IMPLEMENTING NEW WORK PRACTICES ........................9 C. INDEPENDENT TESTING, SKILLS ASSESSMENT AND CERTIFICATION OF BANK STAFF ................................ 11 D. IT SYSTEM ENHANCEMENTS ........................................................................................................................ 11 II. METHODOLOGY AND DATASET .................................................................................................................. 13 III. FINDINGS ........................................................................................................................................................... 16 A. BUILDING BORROWER PROCUREMENT CAPACITY ...................................................................................... 16 B. IMPROVING VALUE FOR MONEY ................................................................................................................. 21 C. PROJECT PROCUREMENT STRATEGY FOR DEVELOPMENT ........................................................................ 23 D. STREAMLINING PRIOR REVIEW TO MANAGE FIDUCIARY RISK MORE EFFECTIVELY .................................. 25 E. STRENGTHENED CONTRACT MANAGEMENT ................................................................................................ 27 F. IMPROVING PROCUREMENT OUTCOMES BY REJECTING ABNORMALLY LOW BIDS ................................... 29 G. IMPROVED PROCUREMENT OUTCOMES IN FCV SITUATIONS ...................................................................... 30 H. IMPACTS OF THE COVID-19 PANDEMIC ON SUPPLY MARKETS .................................................................. 33 IV. ENHANCEMENTS AND INITIATIVES ......................................................................................................... 34 A. BENEFICIAL OWNERSHIP ............................................................................................................................. 34 B. MANDATORY DIRECT PAYMENT.................................................................................................................. 35 C. SEA-SH PREVENTION AND RESPONSE, AND CONTRACTOR DISQUALIFICATION ....................................... 35 D. CASCADING THE ENVIRONMENTAL AND SOCIAL FRAMEWORK TO CONTRACTORS .................................. 36 E. BANK-FACILITATED PROCUREMENT........................................................................................................... 37 F. INDUSTRY ENGAGEMENT PROGRAM ........................................................................................................... 37 G. CYBERSECURITY........................................................................................................................................... 38 H. APPLICATION OF DISRUPTIVE TECHNOLOGIES TO PUBLIC PROCUREMENT ............................................. 39 I. ELECTRONIC GOVERNMENT PROCUREMENT ............................................................................................. 39 J. GREEN PUBLIC PROCUREMENT ................................................................................................................... 40 V. CONCLUSIONS ................................................................................................................................................... 41 ANNEX 1 – CASE STUDIES (BASED ON ACTUAL PROJECTS AND INITIATIVES IN REGIONS) ........ 44 A. ABNORMALLY LOW BIDS ............................................................................................................................. 44 B. BORROWER CAPACITY BUILDING ............................................................................................................... 44 C. E-GOVERNMENT PROCUREMENT ................................................................................................................ 52 D. GENDER/SMALL MEDIUM ENTERPRISE PARTICIPATION ............................................................................ 55 E. SUSTAINABLE PUBLIC PROCUREMENT ........................................................................................................ 56 F. PROJECT PROCUREMENT STRATEGY FOR DEVELOPMENT ........................................................................ 57 G. HANDS-ON EXPANDED IMPLEMENTATION SUPPORT .................................................................................. 59 H. ALTERNATIVE PROCUREMENT ARRANGEMENTS........................................................................................ 61 PROCUREMENT IN WORLD BANK INVESTMENT PROJECT FINANCING REVIEW OF RESULTS AFTER FIVE YEARS OF IMPLEMENTATION OF THE PROCUREMENT FRAMEWORK EXECUTIVE SUMMARY 1. Management agreed, when seeking Board approval of the new Procurement Policy in 2015, that “a formal, detailed review of the entire Framework…�1 should be undertaken after five years of implementation to assess whether the Procurement Framework (Framework) has been implemented successfully and has improved procurement results. This report details Management’s review of implementation experience from FY17 to FY212 including results and trends and provides an update on recent enhancements and procurement initiatives in progress. 2. Since it became effective on July 1, 2016, the Framework has had a notably positive impact on many key aspects of procurement performance under Bank-financed investment operations, by increasing competition among bidders, enabling Borrowers to adapt fit-for-purpose procurement approaches that achieve improved Value for Money (VfM), building Borrowers’ procurement capacity, speeding up procurement actions and managing procurement risks more effectively. 3. For over 50 years, the Bank’s Procurement Guidelines had served as an international model in public procurement rules among Multilateral Development Banks (MDBs). They provided an example for both other MDBs’ procurement policies and many client countries’ national procurement legislation establishing the Bank’s position as a global leader in public procurement regulation. By the early 2000s, however, global practice included innovative procurement approaches not allowed by the Guidelines and the procurement profession had matured in an environment of rapid globalization and modern information technologies. As a result, it became apparent that Bank procurement policy and procedures were in need of modernization3. As compared to new approaches that had emerged, the Procurement Guidelines4 often led to suboptimal results, too much staff time was spent prior reviewing low-value or low- risk transactions, and innovation was inhibited by the need for formal waivers for even minor deviations. Procurement data were sparse, with disjointed IT systems, which made monitoring procurement performance difficult. These concerns prompted Management to initiate5 a reform of Bank procurement which, following two rounds of global consultations, led to a very significant change program. 1 Procurement in World Bank IPF – Phase II: The New Framework (R2015-0120), June 21, 2015, para 122. 2 Data is sourced from the Bank’s STEP system and includes both post and prior review contracts entered by the Borrower, some FY22 data is detailed where June 30, 2021 figures are not available, this is referenced accordingly. 3 Independent Evaluation Group report The World Bank and Public Procurement (CODE2013-005), August 9, 2013. 4 Procurement rules applied to Bank financed procurements with Project Concept Notes approved up to end FY16. 5 The World Bank’s Procurement Policies and Procedures: Initiating Discussion and Approach Paper (AC2012- 0010, CODE2012-0007), February 10, 2012. 1 4. Implementing the Framework required sustained focus. Changing behaviors and culture has required sustained Management effort over the past several years. To launch the scale of cultural and behavioral changes needed, Management first focused on realigning procurement resources with Bank strategy, independently testing staff’s technical procurement knowledge, increasing their procurement skills, and ramping up the Bank’s efforts to build Borrower procurement capacity. Management also addressed the need to better understand and manage a wide range of risks affecting procurement outcomes, increase direct support to Borrowers and expand business and industry engagement in order to incentivize well-performing bidders to participate in Bank-financed procurement. Significant investments were made to modernize controls, IT systems, data intelligence and reporting and to create a foundation for the Framework to succeed through more agile, fit-for-purpose, and innovative procurement practices. 5. Introduction of the Framework paved the way for the Bank to develop innovative ways to proactively support Borrowers in conducting procurement. Hands-On Expanded Implementation Support (HEIS), one of the innovative modalities introduced by the Framework, has enabled Bank procurement staff to provide much-needed direct support to Borrowers, especially in Fragile Conflict Violence (FCV) environments. Responding to the COVID-19 pandemic, HEIS was extended further by offering Bank-Facilitated Procurement (BFP) which enabled the Bank to improve consolidation of demand and negotiate contracts for individual Borrowers’ consideration. BFP has been critical in helping to open up health markets for Bank Borrowers while supply chains and logistics were significantly disrupted and continues to be a highly effective way to assist Borrowers in rapidly procuring the essential medical supplies they need to respond to the ongoing COVID-19 pandemic. 6. The Framework’s innovations helped to significantly increase competition for Bank - financed contracts by attracting new bidders, as evidenced by data showing that, between FY17 and FY21, 77% percent of awarded contracts by value and 78% of the number of suppliers were new market entrants (i.e., bidders who bid for a Bank-financed contract for the first time). This sizable influx of new bidders indicates that the Framework has increased the private sector’s confidence in the Bank’s procurement procedures. The increase in competition in FCV countries is particularly encouraging. 7. Key innovations introduced by the Framework have resulted in the widespread application of fit-for-purpose procurement approaches by Borrowers. The Project Procurement Strategy for Development (PPSD) requires Borrowers to adopt a strategic approach to procurement planning including an analysis of the target supply market, which better enables Borrowers to identify and attract the right bidders, achieve agreed project development objectives (e.g., climate change mitigation, resilience, sustainability, gender inclusion, economic equality, participation of Small and Medium Enterprises (SME) or disadvantaged groups, etc.) and to maximize VfM. While producing highly effective PPSDs requires advanced procurement skills and, in some cases, significant support from Bank staff, it directly leads to more effective VfM approaches and improved procurement outcomes. These approaches may include, on a fit-for- purpose basis, the use of the Request for Proposals (RfP) selection method which facilitates the application of rated-type assessment criteria to evaluate non-price aspects such as quality or sustainability, life-cycle costing in bid evaluation, value engineering, Alternative Procurement 2 Arrangements (APA) with other MDBs and United Nations (UN) agencies, Best and Final Offers (BAFO), and electronic reverse auctions. In particular, the increased use of HEIS, as recommended in the PPSD, has enabled the Bank to better support Borrowers with capacity constraints in situations of fragility, conflict, or violence (see Annex 1 for case studies). 8. The introduction of the Framework and its supporting Information Technology (IT) systems, particularly the Systematic Tracking of Exchanges in Procurement (STEP), has significantly improved efficiency of procurement actions and Management’s ability to monitor this and other intended objectives of the reform. As to be expected, implementing a major IT system with thousands of users was not without its challenges and has required a great deal of troubleshooting. Due to extensive Borrower training and implementation support provided by the Bank, the benefits of STEP are now evident. For example, data generated by STEP show that the median turnaround time to complete procurement transactions (defined as the time elapsed between the first action taken by the Borrower to start a procurement activity and the date of contract signature) has decreased at an annual compound rate of 7% since FY17. Median procurement turnaround time in projects under the Framework decreased by almost two-thirds, from 172 days6 in FY17 to 61 days in FY21. Over half of all procurements are now completed in 63 days or less. 9. The Framework has helped to increase Borrower procurement capacity. Borrower capacity is a measure of a range of administrative and professional implementation capabilities that combine to make procurement successful. It can be measured indirectly through output indicators such as procurement implementation times (as reported above) and handling of complaints from bidders and suppliers7. Improved complaint handling has resulted in half of all procurement complaints lodged in FY21 being resolved in fewer than 11 days. 10. The Bank’s approach to prior review is now balanced towards high -risk, high-value contracts, where Bank review adds more value. The prior review process itself has become more efficient, freeing up staff resources to focus on more value-added activities, including paying closer attention to critical or innovative procurements, HEIS, BFP, building Borrower capacity, monitoring complaints resolution and supporting Borrowers in reforming their own public procurement systems. Post review8 has been improved with all award decisions publicized. Detailed procurement data on both prior and post review contracts is included in STEP, is now available in the Procurement App9 and also on the World Bank website to further improve transparency. To optimize prior and post review, Management is testing use of Artificial Intelligence (AI) to increase efficiency and target staff resources for review of higher risk procurements flagged by AI software. In Management’s view, in addition to increasing efficiency AI technology also has the potential to further mitigate procurement integrity risks. 11. While use of the RfP selection method for complex procurements is increasing, uptake has proven slower than expected. Although RfPs are normally used for the procurement of 6 All days quoted in this paper are calendar days. 7 Throughout this paper, the term “supplier� refers the different types of suppliers of goods, contractors performing civil works and non-consulting services and consultants providing consulting services. 8 Procurements that are not prior reviewed, are post reviewed on a sample [circa 10 percent] basis by Bank staff 9 Link to World Bank Project Procurement on the App Store (also available for Android) 3 consulting services, they are increasingly used in complex works procurement in order to give bidders the opportunity to offer innovative technical solutions that meet more effectively the Borrower’s performance requirements. Award decisions in these cases are based on a combination of price, quality and other non-price factors. Up to November 2021, 146 contracts for goods, works and non-consulting services worth a total of $1.5 billion had been awarded using the RfP selection method. Use of RfP has accelerated recently and continues to increase, with another $1.75 billion worth of procurements for 199 contracts currently planned or in progress. Going forward, Management will strengthen Borrowers application of quality based, rated evaluation criteria (i.e., non-price criteria) and Life Cycle Costing (LCC) to optimize award decisions based on overall VfM. 12. FCV Borrowers have especially gained from the introduction of the Framework. Notable improvements in procurement outcomes in FCV situations have, in many important respects, exceeded gains made elsewhere. For example, turnaround times for procurement in FCV countries have decreased at an average compound rate of 9% since FY17, exceeding results overall, and the competitiveness of procurement has also improved faster in FCV than in other countries. 13. Current supply chain disruptions, market distortions, logistics problems, increased delivery lead times and inflationary cost pressures further highlight the need for procurement to remain a strategic tool for the Bank to help manage risks and achieve development objectives. Procurement-related challenges facing Borrowers are not expected to ease in the short term. In some major supply markets, stimulus programs and major investments in infrastructure are increasing domestic demand at the same time as global supply remains constrained, thereby potentially reducing competition for Bank-financed contracts awarded by Borrower countries, and further distorting supply and demand balance. With Borrowers facing such challenging supply market conditions, the flexibilities and innovative approaches built into the Framework are proving even more valuable. Ongoing supply chain disruptions and other emerging risks will continue to require close Management attention and adequate resources to help Borrowers achieve their development objectives while managing risks. 14. Notwithstanding these ongoing challenges, this comprehensive retrospective on the results from the first five years of the Framework’s implementation highlights significant benefits for both Borrowers and the Bank, as evidenced by the improvements in many key metrics of procurement performance. As Management originally committed, this success is measured by progress in building Borrower capacity, improved VfM, streamlined processes that optimize use of resources, and agile responses to new challenges and opportunities whether at the global level, or in a specific operating environment. This paper details these improvements, identifies ongoing challenges, and is underpinned by data covering the first five years of the Framework’s implementation from July 1, 2016 to June 30, 2021. 15. Management has identified actions to further enhance effectiveness of the Framework. Building on experience discussed in this five-year retrospective, Management will implement the following actions during the current and next fiscal year: 4  Strengthening Borrower application of VfM award decisions. Accelerate Borrower application of quality based rated evaluation criteria and LCC to determine contract awards that optimize overall VfM. Bank procurement staff will scale-up training and HEIS to support Borrowers carrying out procurements using rated criteria in both Request for Bids (RfB) and RfPs, advise them on quality assurance and risk management decisions, and help them engage with domestic control agencies to ensure that evidence-based judgment inherent in non-price evaluation is not discouraged. STEP will be updated to enable tracking of Borrower use of rated evaluation criteria in RfBs and RfPs;  Evaluating overall VfM under bank financed procurement. Management will develop a methodology that includes clear indicators to inform future assessment as to whether the Framework is helping to achieve overall VfM;  Improving management of cybersecurity risks. Require Borrowers procuring activities with cybersecurity risks to utilize the RfP selection method and/or use non-price rated evaluation criteria to determine contract awards that include optimal management of cybersecurity risks;  Enhancing contract management support and integrity. More systematic tracking of Key Performance Indicators (KPI) for large contracts, deep dives on high risk/complex projects to help reduce time and cost overruns and HEIS for contract implementation (beyond more traditional hands-on support for procurement) will help maintain VfM and support mitigation of integrity risks during this critical phase for development effectiveness. A new contract management system will support Borrowers to better track contract implementation (i.e., track milestones including timely completion, payments, KPIs and contractual disputes, and producing reports to support contract management). Additional contract monitoring information will be used to support further mitigation of integrity risks during implementation;  Applying the SEA/SH disqualification mechanism. Closely monitor the implementation of the SEA/SH disqualification mechanism and update on progress with implementation and its impact;  Strengthening supply chain management and security. Leveraging recent experience with BFP, market intelligence, industry engagements, and fit-for-purpose approaches to mitigate emerging global risks, the Framework’s toolbox will be expanded to help countries and Bank teams build supply chain resilience and manage associated risks. This will include technical guidance and capacity building on supply chain mapping, risk analysis, traceability protocols, quality inspection/testing, cost-benchmarking, and managing logistics. Among others, the Health Sector will be a particular area of focus to help develop procurement solutions that support stronger country health systems;  Continuing to support public procurement reforms and capacity building in Borrower countries. Policy dialogue and technical assistance will continue to include broad systemic reforms and discrete interventions to strengthen individual institutions. These efforts will include a deeper focus on the areas included in this action plan such as, for example, 5 green/sustainable procurement, working with Supreme Audit Institutions to modernize fiduciary control frameworks in ways that support VfM decision-making, and developing effective emergency procurement approaches;  Operationalizing mandatory direct payment. Based on the successful pilot, this requirement will be mainstreamed for all procurements in FCV environments except when Borrowers prefer to pay national suppliers from dedicated project accounts; and  Operationalizing Beneficial Ownership. Consistent with good international practice and the pilot’s success, this requirement will be mainstreamed and extended to all internationally advertised contracts (for which the specific procurement notice is published on or after July 1, 2022). 16. Management will continue to closely monitor the Framework’s performance, and make needed enhancements, regularly updating the Board on progress. Every year, many more projects using the Framework are added to the Bank’s portfolio creating a richer data set available in STEP to analyze, compare, and inform the Framework’s ongoing evolution. This will ensure that the Framework continues to meet its overarching objectives of supporting projects’ development objectives and the Bank’s key agendas through strategic, fit-for-purpose, and innovative procurement approaches with integrity and helping to protect the environment and the communities we serve. 6 PROCUREMENT IN WORLD BANK INVESTMENT PROJECT FINANCING REVIEW OF RESULTS AFTER FIVE YEARS OF IMPLEMENTATION OF THE PROCUREMENT FRAMEWORK I. INTRODUCTION 1. This paper provides a five-year retrospective on the results of implementation of the Framework, which became operational on July 1, 2016 10. Management agreed in 2015, when seeking Board approval of the new Procurement Policy11, that “a formal, detailed review of the entire Framework…� should be undertaken after 5 years of implementation to assess whether the Framework has been implemented successfully and has improved procurement results. 2. Over the past five years, Management has prioritized successful implementation of the specific commitments made to the Board, as summarized in Figure 1, below. These include delivering on four long-term commitments - Borrower capacity building, VfM decision-making, development of Bank staff skills, and industry engagement - which, by their iterative nature, are ongoing activities which require Management attention to continue to build on the progress made to date in each of these areas. As during the last five years, two detailed progress updates have been already provided12 to the Board, this paper includes a summary of Management’s overall implementation experience and focuses primarily on results achieved and trends observed. Figure 1: Management’s Procurement Reform Commitments13: No. Management’s Commitments: Status: 1. The Bank will increase support to Borrower capacity building and institutional strengthening. Ongoing 2. VfM decision-making will be applied in contract award decisions to determine the most Ongoing advantageous bid/proposal. 3. Management proposed that the Bank provide options for tailored procurement approaches, and Actioned make hands-on expanded implementation support available as an option for operations covered under paragraph 12 of Operations Policy 10.00 (now IPF Policy) 10 For all projects with a concept note on or after July 1, 2016. Projects dated prior apply the Procurement Guidelines. 11 Link to Procurement Policy 12 Procurement in World Bank Investment Project Financing, Procurement Framework - Implementation Update July 1, 2016 to June 30, 2017 (AC2017-0025, CODE2017-0025). Procurement in World Bank Investment Project Financing – Procurement Framework – Implementation Update - July 1, 2017 to June 30, 2018 (AC2018- 0055/CODE2018-0048). 13 Procurement in World Bank Investment Project Financing – Phase II: The New Framework (R2015-0120), June 21, 2015. 7 4. Integrity will be strengthened in the new Framework; the Bank’s Anti-Corruption Guidelines Ongoing and sanctions procedures will continue to apply. No. Management’s Commitments (continued): Status: 5. If the Borrower requests, sustainable procurement criteria may be used in procurement under Actioned IPF. 6. Alternative Procurement Arrangements (APA) may be used in clearly defined circumstances. Actioned 7. The Bank will significantly enhance its approach to procurement-related complaints. Actioned 8. Procurements above identified thresholds will continue to be internationally advertised and Actioned will use the Bank’s standard selection documents unless an APA has been agreed. 9. The Bank will streamline its requirements for the use of a country’s own national procurement Actioned system. 10. The Bank will focus its prior review activity to optimize performance. Actioned 11. The Bank will provide more support to contract management in the most significant Actioned procurements. 12. The Bank will implement a planning and tracking system called STEP to provide data on Actioned procurement activities, establish benchmarks, set performance metrics, and inform reporting to CODE/AC on this Framework. 13. Management will begin a program of external sectoral engagement with industry called the Ongoing Industry Engagement Program (IEP). 14. An independent skills assessment of Bank procurement staff has been completed and will be Actioned used to create a training and development program in support of the Framework. 3. In addition to implementing the commitments made to the Board in 2016, Management has initiated further enhancements to the Framework’ s toolkit in response to emerging risks, evolving operating environments, and changing global supply chain dynamics. As later explained in this paper, Management’s enhancements include, inter alia, improved transparency on beneficial ownership, reducing suppliers’ payment risks, measures to prevent Sexual Exploitation and Abused/Sexual Harassment (SEA/SH) in infrastructure projects, issuing sector-specific procurement guidance, providing Borrowers with “ready-to-go� contracts for the COVID-19 health response, supporting many actions needed to implement the new 8 Environment and Social Framework (ESF)14 by cascading requirements to contractors, as appropriate, and a range of global knowledge initiatives. Over the five years since its launch, the Framework had continually evolved to remain at the forefront of international public procurement practice and to set a standard for other development partners and countries. A. Changing institutional roles and decision-making 4. In FY17, a new Procurement Directive and Procedure were published. The Procurement Directive15 details the specific roles, responsibilities and accountabilities of Bank staff operating within the Framework, while the Procurement Procedure16 includes a comprehensive Accountability and Decision-Making Framework for the Bank’s procurement operations. From FY17 to FY20, key responsibilities for leading implementation of the Framework were assigned to the Governance Global Practice in the Equitable Growth, Finance and Institutions (EFI) Vice-Presidency and for policy making and oversight to the Operations Policy and Country Services (OPCS) Vice-Presidency17. 5. In FY21, the Procurement Directive and Procedure were updated to align with the FY20 Bank-wide changes in reporting lines. Reflecting the updated roles of Regions and Global Practices, Framework implementation and operations procurement matters are no longer EFI’s responsibility. Regions are now responsible for implementation of the Framework. Procurement Practice Managers (PPMs) manage procurement staff and exercise technical leadership in their respective regions, while EFI Regional Directors are responsible for managerial oversight. The Chief Procurement Officer (CPO) in OPCS continues to lead on policy and oversight of the procurement function and the Framework and represents the Bank externally on procurement matters. The CPO coordinates closely with the PPMs and the EFI Reginal Directors with enhanced oversight and reporting responsibilities. B. Shifting culture, changing behaviors and implementing new work practices 6. Implementing the Bank’s procurement reform required sustained effort and leadership to fundamentally transform the operating culture and behaviors of Bank procurement staff. In FY17, Management initiated a major change program on multiple fronts, including, inter alia, testing the procurement knowledge and technical skills of Bank staff, requiring more strategic thinking in procurement arrangements under investment projects, greater application of evidence-based professional judgment, investing more in building Borrower procurement capacity, and in promoting innovation and proactive market engagement. 7. Efforts to promote behavioral changes included establishing a working environment that facilitated the free exchanges of ideas, knowledge, and experiences among procurement 14 Launched in 2017, one year after the new Procurement Framework. 15 Link to Procurement Directive 16 Link to Procurement Procedure 17 Procurement in World Bank Investment Project Financing – Phase II: The New Framework (R2015-0120), June 21, 2015, paragraphs 104 to 106 9 staff. To that end, several Communities of Practices were launched on themes as, for example, e- Government Procurement (e-GP), and Sustainable Public Procurement (SPP). Communities of Practice provide staff with intensive, continuous learning about specialist topics and have proven instrumental in accelerating staff’s ability to apply the many novel approaches designed into the Framework and in ensuring their full ownership of its successful implementation. 8. The Bank’s procurement processes, regulations, bidding documents, contract conditions and library of procurement guidance have been expanded to reflect and articulate the Framework’s fit for purpose approach. On July 1, 2016, revised Procurement Regulations for Borrowers18 were published replacing the Procurement Guidelines. The new procurement guidance collection19 was extensively researched, written and published on such matters as, inter alia, Value for Money20, Procurement Strategy21, Abnormally Low Bids22, Sustainable Procurement23 and Competitive Dialogue24. Specific Guidance was also published for bidders on “Finding business opportunities and winning contracts financed by the World Bank�25, “Procurement related complaints - How to complain�26. Borrower Guidance was also supplemented with a comprehensive training program, including (pre-pandemic) face-to-face training events, on-line training, and an online certificate program in public procurement27 for Borrower staff. 9. The Framework is reorienting Bank procurement culture from a one-size-fits-all to a fit-for-purpose, strategic approach that is designed to meet specific development objectives in each country and projects context and considering the relevant markets’ dynamics. The Framework aligns Bank selection methods with modern best practices by providing a broad range of options and tools to support VfM decision-making (e.g. by using rated criteria to evaluate quality, sustainability and innovative aspects of proposals). The newly introduced processes and procedures also fostered more effective risk management by refocusing staff resources on more value-added reviews of large, complex, and high-risk transactions, hands-on support and capacity building for Borrowers. 10. Procurement strategies for projects are now informed by a comprehensive assessment of risk and opportunity (e.g. integrity, constrained capacity, supply chain challenges vis a vis emerging market innovation, stronger local institutions, collaboration with development partners etc.). Changes aimed at increasing efficiency such as streamlined prior review have been applied to projects under both the Procurement Regulations and the Procurement Guidelines benefitting the whole portfolio. 18 Link to Procurement Regulations for IPF Borrowers updated in November 2020 19 Link to Procurement Library of Policy, Directive, Procedure, Regulations, Guidance and bidding documents 20 Link to Procurement Guidance, Value for Money 21 Link to Procurement Guidance, Project Procurement Strategy for Development (PPSD) 22 Link to Procurement Guidance, Abnormally low bids and proposals 23 Link to Procurement Guidance, Sustainable Procurement 24 Link to Procurement Guidance, Competitive Dialogue 25 Link to Supplier Guidance, Finding Business Opportunities and Winning Contracts 26 Link to Supplier Guidance, Procurement related complaints - How to Complain 27 Link to Certificate Program in Public Procurement (CPPP) | Public Procurement Training Courses | World Bank 10 11. Greater efficiency also supported the transition from a transactional, mainly compliance-checking approach to more complex decision-making, hands-on support and capacity building. Prior reviews have been reduced from a pre-Framework, FY1428 level of 10,917 prior reviews per annum, to 1,376 in FY21 allowing PPMs to redeploy the equivalent of 74 full-time procurement staff from low value-adding activities to riskier projects/contracts and other higher-order tasks. C. Independent testing, skills assessment and certification of Bank Staff 12. Technical skills of Bank procurement staff have progressively increased during the last five years. Following an initial independent skills assessment, in FY19 Management commissioned a follow-up assessment, including an online test to measure the progress made by staff in enhancing the key technical skills required to ensure effective use of the Procurement Framework. The FY19 independent testing demonstrated that the skills of staff have increased since 2015, when the first skills assessment was undertaken. The Bank’s staff average test score compares favorably with global public and private-sector benchmarks, with a comparative performance of 104.3%. Improvements were recorded in eleven of the sixteen procurement competence areas tested, while the other five areas require further attention to achieve the target benchmark. Enhanced training and mentoring, with targeted attention to these competence areas, has been provided to staff, and themes have included: data analytics, financial analysis, application of disruptive technologies to procurement, and design of procurement capacity building programs. 13. Following mandatory training, all Bank procurement staff were also tested on their specific knowledge of the Framework and are now certified to apply it. In FY21, Management launched a new Procurement Accreditation system to process, record, and track accreditation of procurement staff. The CPO is responsible for all procurement accreditation decisions. Each staff member’s “people page� profile on the intranet now shows their accreditation status, which makes it easier for Task Leaders to identify accredited staff for their projects. The accreditation system is being integrated with other procurement IT tracking systems to ensure that only accredited staff and consultants perform procurement clearance functions within their accreditation thresholds. D. IT System Enhancements 14. Management made a major investment in IT systems to better monitor and report on the performance of procurement. STEP is a comprehensive online contract-level tracking tool, including end-to-end transaction monitoring, populating the data needed for risk management, planning, and market intelligence purposes. All Borrowers and Bank staff are required to use STEP to plan and record procurement activities under IPF projects. 15. Using STEP has proven a major change for Bank staff and Borrowers alike and continues to require close management attention and ongoing investment. A total of 48,459 users are registered to use STEP29, including Bank and Borrower staff. Each of these 48,459 users 28 In FY14, the Bank prior reviewed 10,917 contracts ($13.5 Bn), of this 9,849 contracts (89.8%) were below the Bank’s mandatory prior review thresholds. 869 contracts were prior reviewed over the Bank’s thresholds, valued at $10 Bn, representing 74% by value of all procurements prior reviewed. In FCV countries, 2,070 contracts were prior reviewed below the Bank’s thresholds, valued at $433 Million (3% by value). 29 November 2021, it is not possible to retrospectively identify the number of STEP users at end June 2021. 11 has been trained in person. Due to the high turnover of Borrower staff, the Borrower’s ability to use STEP requires continuing user training and support to maintain system effectiveness and data integrity. Early challenges in implementing STEP, which are to be expected for a complex, Bank- wide IT system with a marked borrower interface, have progressively been addressed through change management, user training and continuous technical support. Bank staff invested significant time in supporting users in various ways. For example, as Indonesian counterpart staff were not comfortable with the Bank’s training videos in English, local Bank staff quickly developed training videos in Bahasa, which are now regularly used by implementing agencies. 16. Approximately 300,000 procurement transactions are currently recorded in STEP30. Management continued to upgrade the system including, for example, with a new module, launched in FY18, to track handling of procurement-related complaints and a major system improvement in FY19 to introduce new functionality, improve data integrity and enhance user experience. STEP enables Bank and Borrower staff to closely monitor procurement progress and consistency with the agreed procurement plan, and identify bottlenecks and their causes (i.e., who/what is delaying procurement implementation). It also provides a broad range of information supporting risk management and market intelligence such as procurement documents used, actions taken by the Bank and Borrowers, bidders registered, bids submitted, cost estimates, and evaluation results. For example, most of the data underpinning the analysis in this report is produced by STEP. The introduction of STEP also helped Borrowers do better procurement planning and contributed to significant improvements in turnaround times, as later discussed in this paper. 17. The procurement risk and management IT system (P-RAMS) has been significantly enhanced. Procurement risks associated with each project are updated regularly in P-RAMS and cleared by the PPM in each region. The project-level risk ratings in P-RAMS automatically populate the STEP dataset allowing to monitor, manage, and update risks for each contract, as appropriate. Performance and risk ratings from P-RAMS also inform the fiduciary risk rating and procurement performance ratings in each project’s Implementation Status Results (ISR). 18. Management also introduced a new Procurement Post Review (PPR) IT system in FY21. As the number of prior reviews decreased, the number of procurements sampled in post review increased correspondingly (typically, 10% of procurements not prior reviewed are post reviewed). The PPR system, together with STEP, allows staff to oversee these procurements remotely, which is proving especially useful to access relevant information during COVID travel restrictions. The system also enables additional analytics and reports supporting risk management decisions. 19. The Bank’s procurement website has been redesigned31 and a Project Procurement Application (App) for mobile devices launched. The Bank’s website now offers a comprehensive public library for Borrowers, bidders, Bank staff and others interested in modern procurement practices. Bank guidance and SPDs have been used by several MDBs to update their own guidance and documents further increasing MDBs harmonization. An App was also launched 30 November 2021, it is not possible to retrospectively identify the number of transactions in STEP at end June 2021. 31 Link to Bank Procurement website www.worldbank.org/procurement 12 for mobile devices that allows users to access the Bank’s historic procurement data over the last 22 years32 and to check current procurement notices published by all Borrowers. The App was updated to include contract award information on post review contracts and includes an additional �review type� filter to allow deeper data interrogation by users. Finally, a procurement notices e- mail alerts system was launched to allow subscribers to receive e-mail alerts for contracts in their area of interest whenever the notices are published by Borrowers. II. METHODOLOGY AND DATASET 20. This report utilized both quantitative and qualitative data with a main focus on results rather than inputs. Twenty-one different internal datasets, encompassing almost 100,000 procurement activities and 2,187 IPF projects, were analyzed to assess the impact of the Framework across countries, sectors, and levels of project complexity. Approximately 70% of observations are associated with projects implemented under the Procurement Guidelines while Framework projects accounted for about 30% of the overall dataset33. 21. The data span five fiscal years, from FY17 to FY21 (July 1, 2016 to June 30, 2021). The first 36 to 42 months of this reporting period are more representative of normal Framework trends, whereas data from the last 18 months (January 2020 to June 2021) were impacted by the COVID- 19 pandemic and the procurement challenges it created globally. Consequently, when reviewing tables, graphs and analysis, it is important to consider both pre-COVID-19 trends (FY17 to FY19) and the impacts the pandemic had on the FY20 and FY21 figures. 22. Comparing projects under the Procurement Guidelines with projects under the Procurement Regulations offered an opportunity to assess broad trends affecting the whole portfolio, which provided a key insight into improved procurement productivity across both policies and set of rules34. 23. A small number of Bank-financed procurement contracts accounts for a large share of the awarded contract value. In the five-year period covered, 59% of awarded value accounted for only 1.1% of the total number of contracts. Of note, contracts of over $100 million in value made up 23.5% of the value and only 0.07% of the total number of contracts (Figure 2). Figure 2. FY17 – FY21, Profile of Bank-financed procurement35 32 App includes data from 1995 to 2022, but data prior to 2000 is limited due to disparate IT systems at that time. 33 Majority of the Dataset are procurement activities recorded in STEP. 34 Internal inter alia organizational, procedural, decision-making, IT system, STEP, skills development changes needed to implement the Framework were also applied to projects under the Guidelines (last issued 2014), by default these changes made to support Framework implementation also had a spillover effect on application of the Guidelines. 35 Almost 73% of the awarded value in FY17-FY21 are in complex, higher risk procurement. Just 1.1% of the volume of contracts – 1,135 contracts in five fiscal years - represented 59% of the total awarded value in the period. 13 Contract size USD % value % volume Awarded value Volume # Projects # Countries # Agencies Up to $25k 0.5% 42.2% $385M 42,139 1,767 141 2,169 $25k - $52k 0.8% 15.2% $558M 15,120 1,676 141 1,965 $52k - $100k 1.2% 12.2% $888M 12,125 1,561 137 1,866 $100k - $210k 2.0% 9.8% $1,404M 9,730 1,477 138 1,778 $210k - $500k 3.7% 8.2% $2,672M 8,199 1,417 138 1,627 $500k - $1M 4.5% 4.6% $3,261M 4,611 1,053 129 1,198 $1M - $2M 6.0% 3.1% $4,325M 3,079 908 130 1,001 $2M - $4M 7.9% 2.0% $5,663M 2,011 687 114 780 $4M - $10M 14.3% 1.6% $10,316M 1,642 610 103 675 $10M - $100M 35.6% 1.1% $25,612M 1,068 452 98 459 > $100M 23.5% 0.1% $16,885M 67 51 32 52 Grand Total 100.0% 100.0% $71,967M 99,791 N/A* N/A* N/A* * Grand Totals are not applicable for projects, countries and agencies as each has multiple associated contract values. 24. The concentration of value in a small number of contracts is a key dynamic in this analysis, as larger, more complex contracts require stronger Borrower capacity, more Bank support for successful implementation and are more likely to attract complaints. These large contracts are, therefore, particularly relevant in assessing whether the Bank has met its key reform goals of increasing Borrower capacity and improving fit-for-purpose procurement outcomes. 25. Bank-financed procurement has a risk and complexity profile that often requires expert support. Data show that, during FY17-FY21, 74% of the value of contracts awarded consisted of high-risk or complex contracts. The Bank’s procurement portfolio is typically more challenging than the contracts financed by Borrowers with their own resources, with a much higher proportion of complex, high-value contracts and specialist consultancy services. While this is consistent with the Bank’s mission, it also further underscores the need for providing implementation support to Borrowers in addition to building long-term capacity. 26. Data were also concentrated in ten project categories that accounted for 71% of the total awarded value between FY17 and FY21. Heavy construction works accounted for over $15 billion in awarded value, followed by professional engineering services at $6.5 billion and permanent buildings and structures at $5.3 billion. These complex projects are not only among the highest-value projects the Bank finances, but also involve high risk. Of the projects analyzed in this report, 74% of awarded value was for complex, high-risk procurements. The Framework was designed to equip staff with the tools needed to manage these increased risks more effectively. 27. As expected, data shows a steady increase in use of the Framework over time. As the Framework applies to projects with a concept note on or after July 1, 2016, projects under the Framework accounted for only about 22% of awarded contract value between FY16 and FY2136. However, in FY21 alone, coverage of the Framework grew to 35% of awarded value and saturation will continue to increase faster as more older projects close and newer are approved. 36 This is happening because saturation of the Framework is increasing, older projects under the Procurement Guidelines are increasingly closing and are being replaced by new projects under the Framework. 14 15 III. FINDINGS A. Building Borrower procurement capacity 28. Data show good progress in Borrower procurement capacity, suggesting that change management efforts and the new tools and approaches positively impacted procurement performance. Data indicate improvements in turnaround times (i.e., the time taken to complete a procurement, from advertisement to contract signing), with notable gains in high-value, complex contracts, indicating that the Framework has been most effective in those areas where it was designed to have greatest impact. It is clear that STEP and improvements in both team-level and individual-level performance helped to drive overall improved efficiency. 29. Turnaround times have decreased overall in the five years since FY17 (Figure 3), most notably, in FCV environments. This improvement in procurement efficiency is especially pronounced among Framework projects, but also present in projects under the Procurement Guidelines. Projects under the Framework experienced a decrease in median lead time from 172 days in FY17 to 61 days in FY21. This trend was also mirrored in projects implemented under the Procurement Guidelines, which improved from a median turnaround time of 80 days in FY17 to 63 days in FY21. These significant decreases in turnaround times are due to improvements at all stages of the procurement process (planning, advertising, bid preparation, bidding process, bid evaluation, approvals, contract award) and are indicative of overall stronger capacity among both Borrower implementing agencies and Bank staff. Figure 3. FY17 – FY21, Turnaround times in Procurement, number of days from start to contract award (median) 30. More than half of all procurements are now completed in 63 days or less. For example, in Africa the median turnaround time decreased significantly, falling from 114 days to 63 days between FY17 and FY21 (Figure 3) and in East Asia and Pacific Region, median turnaround times fell from 111 days to 62 days during the same period. 16 31. Turnaround times improved for almost all contract values (Figure 4). For the very largest contracts (above $100 million), median turnaround time decreased from 407 days to 218 days from FY17 to FY21, a decrease of 47%. High-value contracts (between $10 million and $100 million and representing the largest share of awarded values), decreased from 251 days in FY17 to 131 days in FY21. A small increase in turnaround time for contracts under $25k is being examined further to assess how these lower-value procurements can be accelerated. Figure 4. FY17 to FY21, Median turnaround times in Procurement by Contract Value (USD), number of days 32. There is a clear link between turnaround time and Borrower capacity. Once turnaround time data are controlled for complexity, size, and type of procurement, significant dispersion remains. For example, in construction contracts valued in the range of $4-10 million, median turnaround time by implementing agency varied from 101 days at the first quartile to 290 days at the fourth quartile. Such dispersion can be explained by variations in implementing agencies’ capacity, even within the same country, and by operational environments in different countries, further supporting the observation that turnaround time is related to Borrower capacity. 33. The Framework has made it easier for bidders to submit complaints and receive faster resolution. Framework projects show a statistically significant improvement in the number of complaints resolved by comparison with projects under the Procurement Guidelines. The Framework has also improved the timeliness of Borrowers’ handling of procurement complaints. The 2015 Board paper37 recommended improving complaints resolution in order to increase bidders’ confidence in Bank-financed procurement processes and encourage greater participation 37 Procurement in World Bank IPF – Phase II: The New Framework (R2015-0120), June 21, 2015. 17 by bidders. This was highlighted as particularly relevant in low-capacity environments where suppliers’ confidence tended to be lower. In FY17, the Bank published specific guidance for bidders on “How to Complain� 38 and introduced a standstill period which entails a ten-day pause from the Borrower’s notification of its intention to award the contract until contract signing. Furthermore, STEP’s new functionality was launched in FY19 to improve recording and tracking of complaints handling. Data available since FY19 (Figure 5) 39 show that most complaints are submitted during bid evaluation or refer to this stage of the procurement process. Many of these complaints were raised and resolved during the standstill period and 19% were substantiated with an appropriate action taken by the Borrower in all cases to address the issues raised by the bidders. Figure 5. FY19 – FY21, Procurement stage when a complaint is made Procurement Notices and Invitations Shortlisting / initial selection Procurement Documents and ammendments Contracts and ammendments Bid / Proposal submission and opening Standstill/Debriefing Bid / Proposal Evaluation 0% 5% 10% 15% 20% 25% 30% 35% 40% 34. In FY21, approximately half of all procurement complaints were resolved in 11 days (Figure 6) even while the number of complaints increased (Figure 7) as a result of the system’s improvements and the Bank’s efforts to inform bidders on how to use it. Figure 6. FY17 – FY21, Median Time to resolve procurement-related complaints (calendar days) 38 Link to Supplier Guidance, Procurement related complaints - How to Complain 39 Procurement stage complaint data is only available from FY19, following Management’s upgrade of STEP 18 Figure 7. FY17 – FY21, Number of Complaints40 35. Procurements over $10 million have been significantly more likely to receive a complaint when compared to smaller activities (Figure 8) as bidders make substantial investments to prepare their bids and processes are more complex and require greater capacity. Given the overall high risk/high value profile of Bank-financed procurement, hands-on support to Borrowers in planning and implementing procurements is key to reducing both the number of complaints and those that are found to be substantiated. Proactive support is also very important to guide Borrowers in resolving the complaints to the Bank’s satisfaction (i.e., ensuring that all procurement actions are consistent with Bank policy, the Procurement Regulations, and the requirements of the specific procurement process). Figure 8. FY17 – FY21, Incidence of procurement-related complaints by contract value (USD), as % of the number of activities for each group of contract values 40 Decrease in the number of complaints made in FY21 is likely linked to the impacts of the COVID-19 pandemic. 19 36. While a direct correlation cannot be made, better handling of complaints including the introduction of the standstill period may have contributed to increased market confidence. This feedback has been frequently expressed by participants in the Bank’s numerous market outreach events and may help to explain the significant expansion in the range of suppliers participating in Bank-financed procurement as well as the increased foreign supplier participation in FCV environments, as discussed later in this paper. 37. Borrower capacity-building tools introduced by the Framework are being applied to a growing number of projects and procurements. The number of projects using HEIS grew twelvefold between FY17 and FY21, from 6 projects to 75. Management introduced HEIS to formalize, mainstream and expand the procurement implementation support Bank staff have always provided to Borrowers. HEIS significantly expanded the range of actions41 Bank staff can take to help Borrowers complete key stages of the procurement process. HEIS is particularly beneficial to Borrowers in small states, FCV situations or during emergencies such as natural disasters and health emergencies (see Annex 1 for case studies). 38. A similarly positive trend applies to Alternative Procurement Arrangements (APA) with other development partners, which grew almost fourfold from 7 in FY18, to 27 in FY21. APAs allow the Bank to rely on the procurement arrangements of other international organizations such as UN agencies and other MDBs. This approach helps avoid overlapping requirements reducing the burden on Borrower implementing agencies and maximizing the benefits of co- financing. While the Framework also allows the Bank to rely on procurement procedures and capacity of well-performing Borrowers, no Borrower has to date used an APA. However, Borrowers have increasingly used aspects of national procurement procedures for low to medium- value activities due to the increase in thresholds for nationally advertised contracts and 41 Compared to the Procurement Guidelines, under the Framework Bank staff providing HEIS are now empowered to provide additional support to Borrowers to: (i) draft procurement documents, (ii) attend pre-bid meetings/bid openings, (iii) identify strengths and weaknesses in bids, (iv) advise Borrowers on areas of bids to clarify, (v) attend contract negotiations, (vi) draft the contract award notice; and (vii) draft the contract. 20 simplification of the requirements for their use42. Notwithstanding advances in domestic procurement reforms, Borrowers still prefer to use Bank procurement policy, procedures, and fiduciary management to implement Bank projects and consider them among the most valuable benefits of Bank financing. APAs with Borrower agencies remain available and send an important signal on the Bank commitment to build well-performing country procurement systems. It is anticipated that Borrowers will request more use of their own procurement arrangements as their capacity continues to increase. 39. Strengthening Borrowers’ capacity is an explicit objective of the Bank’s procurement policy which is pursued also by helping to reform national procurement systems, often within the context of a wider public-sector reform (see Annex 1 for case studies). For example, a leading-edge public procurement spend analytical tool has been applied in several countries using big data to identify key areas of expenditure and possible areas for improved performance and to prioritize related reform actions. In addition, a particular focus in the Bank’s policy dialogue with Borrowers on procurement reforms will be placed on strengthening domestic green/sustainable procurement approaches. 40. The Bank has been instrumental in developing the Methodology for Assessing Procurement Systems (MAPS). The Bank led a multi-stakeholder group including MDBs, many bilateral donors and several countries ranging from fragile to middle income countries which developed a universal tool for identifying procurement systems’ strength and weakness es and guide reforms, known as MAPS. The tool is being applied consistently by development partners and countries including some Organization for Economic Co-operation and Development (OECD) members. The Bank also currently leads the MAPS Technical Advisory Group, providing thought leadership and technical advice to practitioners and is a permanent member of the MAPS Steering Committee convened periodically by a Secretariat established withing the OECD. The Bank uses MAPS as its key procurement analytical tool to guide policy dialogue with Borrowers on procurement modernization. 41. The COVID-19 pandemic highlighted an increasing need for procurement capacity- building to help Borrowers respond to global emergencies, supply chain challenges and global market disruptions. The largest increase in Borrower requests for HEIS was during the height of the COVID-19 pandemic which exposed procurement capacity weaknesses in most countries, especially with respect to emergency procurement procedures. While every country struggled to respond to the rapidly changing market conditions created by the pandemic, the effect was most acute in low-capacity countries. To address this emerging need, the Bank is including in its capacity strengthening programs a particular focus on building greater resilience in Borrowers’ supply chains. B. Improving Value for Money 42 Prior to the Framework, use of National Competitive Bidding (NCB) typically included a bespoke side-letter instructing the Borrower to implement many changes to better fit their s ystems with the Bank’s Procurement Guidelines; under the Framework National Procurement requirements were simplified to nine key issues including inter alia open advertising, eligibility, application of Anti-Corruption Guidelines, ESF, publicized award notice etc. 21 42. Improving VfM is a core objective of the Framework. To this end, procurement methods and evaluation criteria have been expanded to support fit-for-purpose procurement approaches. Borrowers may now use non-price quality criteria and adapt their market approach to best suit the needs of a particular procurement. These enhancements aimed to attract the right contractors/suppliers incentivized by the knowledge that their added value will be part of the evaluation and they will be competing on a level playing field with other well-qualified bidders. 43. Data show an encouraging number of new market entrants who did not previously bid for Bank-financed procurement. New entrants won 77% percent of contracts by value between FY17 and FY21 representing 78% of all bidders receiving a contract during this period and pointing to a dynamic, motivated supplier base (Figure 9). Diversity of suppliers is important to achieving VfM as it helps Borrowers to award contracts that best suit their needs. This trend was most pronounced among high-value contracts as 85% of contracts above $100 million were awarded to new entrants even though these supply markets are generally more consolidated43. In addition to suggesting increased confidence in Bank-financed procurement, these figures indicate the absence of market concentration as repeat bidders do not enjoy a significant advantage. The notable participation of new market entrants also suggests that fit-for-purpose procurement strategies and industry outreach are effectively tailored to specific market segments relevant for the project’s objectives. In all cases, new market entrants are required to meet the same rigorous qualification requirements under the Bank’s SPD. Figure 9. FY17 – FY21, Market entry rate in procurement FY17-FY21, awarded value in USD billion FY17-FY21, % number of suppliers 44. The Framework supports the use of the RfP selection method and rated criteria to assess non-price factors. Most procurement award decisions are not based on lowest price alone but consider many other factors, such as consumables, after-sales services, maintenance, and life- cycle cost of operation and disposal. To help Borrowers in making better VfM procurement decisions, the Bank has published specific guidance and provides ongoing training on VfM 43 Consistently, 6% of winning bidders are the same under the Procurement Regulations and the Procurement Guidelines and the percentage increases to 35% for contracts valued over $100 million. 22 including all new procurement methods introduced by the Framework (Competitive Dialogue, Negotiations, BAFO, and use of RfPs with rated Evaluation Criteria for goods and works). While RfPs are regularly used for consulting services with quality being the main factor (typically 80, or 90 and even 100 percent in some cases), they can also be an appropriate procurement method for complex works, goods and non-consulting services as a way to encourage bidders to innovate by specifically rewarding quality and sustainability (Figure 10). Figure 10. FY17 – FY21, Use of RfP selection method and rated criteria for goods, works and non-consulting services Value in USD Volume of activities Procurement Method: Status: Total Total Request for Proposals Signed & completed (FY17 to FY21) $826.54M 88 Under Implementation $1,580.11M 43 Planned/Pending Implementation $125.38M 4 TOTAL $2,687.54M 135 45. Over the past five years, there has been a slow, but increasing use of RfPs with rated criteria for works, goods and non-consulting services. As discussed, the Framework includes an extensive toolbox of procurement approaches to address a wide range of procurement needs. Some of these methods are more advanced and appropriate for complex contracts on a fit-for- purpose basis. While several Borrowers are ready to take full advantage of more sophisticated procurement approaches, others still require significant support to implement them. For example, no Borrower has used yet the competitive dialogue selection method44. Some Borrowers are not keen on using non-price quality criteria or negotiating contracts as they are concerned that their auditors will object to procurement award decisions based on factors other than the lowest price and involving some subjective judgment. This issue is exacerbated in some countries by the possibility of criminal charges stemming from qualified audit opinions, alleged breaches of local laws, or allegations of malfeasance. Management will strengthen Borrower application of VfM award decisions, accelerating Borrower implementation of rated evaluation criteria, LCC, RfPs and/or RfB with rated evaluation criteria to assess non-price attributes during bid evaluation (e.g. quality, environmental, social) to determine the optimal VfM award decision. Management will also develop a methodology that includes clear indicators to inform future assessment as to whether the Framework is helping to achieve overall VfM. C. Project Procurement Strategy for Development 46. The PPSD is being used as a strategic tool that enhances VfM. The PPSD requires Borrowers to complete market research and identify the optimal procurement strategy for the project. This includes needed technical support/HEIS, market constraints, relative buyer-supplier dynamics, appropriate mitigation measures for a broad range of risks (e.g., market, social and 44 A sophisticated procurement method first introduced by the European Union in 2004 to better support complex multi-stage procurements which is also available under the Framework. 23 environmental, cybersecurity, integrity, etc.), and a contract management strategy to deliver the desired outcomes through the contracts awarded under the project. 47. While there has been consistent progress with the use of PPSDs, Management will continue to build capacity and provide HEIS on this key aspect of the Framework, especially as data needed to inform robust market analysis are often unavailable to Borrower implementing agencies. Some Borrowers don’t have the capacity to conduct effective market assessment and even less to proactively engage with relevant industries. As a result, they are not well equipped to identify or address challenging supply chain constraints. These Borrowers require significant support to prepare PPSDs which may reduce their ownership of the analysis and commitment to implement the resulting strategy. The COVID-19 pandemic further highlighted the importance of market analysis, understanding supply chains and designing procurement strategies to engage suppliers effectively. Indeed, our analysis found a strong correlation between a well-prepared PPSD and successful procurement results (see Annex 1 for case studies). 48. While particularly resource-intensive, hands-on support in preparing PPSDs has contributed to improved procurement outcomes in FCV environments and small states. In particular, our review of PPSDs in FCV countries shows increased use of RfP with rated criteria for works and goods, APAs with other MDBs and UN agencies, LCC, BAFO, and electronic reverse auctions demonstrating that new procurement approaches can be implemented successfully in all client segments if appropriate resources are deployed to support strategic procurement planning and implementation. For example, in the Central African Republic Emergency Power Project, the PPSD recommended the use of HEIS to support Borrower in conducting a complex procurement of a 25MW solar power plant using innovative VfM features, including initial selection and rated criteria. This successful procurement approach resulted in cost savings of $15 million on a $35 million contract45. The plant is expected to double the power generation capacity of the Central African Republic (see Annex 1 for other case studies). 49. PPSDs have been essential in supporting implementation of the ESF and related social sustainability and gender initiatives. Among several examples of socially sustainable, gender- responsive procurement, the PPSD for the West Bank Gaza Digital Project promoted gender equity in recruitment and retention by ensuring inclusion of women in all decision-making bodies under the project. PPSDs have also been used to support racial justice in the face of longstanding discrimination: in the South Africa ESKOM Investment Support Project (ESIP) and Renewables Support Project (ERSP), use of PPSDs helped redress the impacts of apartheid-era discrimination through a preferential procurement program targeting job creation and increasing the skill level of the workforce in enterprises owned or managed by historically discriminated individuals, thus further supporting South Africa’s Black Empowerment Program. Another example is the Senegal Water Supply Project, which used an RfP approach to reward bidders’ strategies for community engagement and sensitization (see Annex 1 for other case studies). 45 Initially a direct award contract was negotiated for $50M and submitted for Bank no objection. Informed by the market analysis work in the PPSD, Bank Procurement staff challenged the direct award and proposed a competitive approach, after some negotiation with the Borrower this was agreed and following a competitive procurement the contract was awarded for $35M. 24 50. PPSDs have helped to strengthen risk assessment and management in IPF projects. The PPSDs use modern risk analysis tools to drive evidence-based decision making, consistent with the level of complexity of the project. PPSD assessments identify a complete range of market, institutional, integrity, environmental, social, health and safety risks, including SEA/SH that impact or can be managed through procurement. Drawing from expert inputs in these areas, PPSDs recommend appropriate measures, designed to mitigate each type of risk including the most appropriate procurement method/documents as well as other risk mitigation measure. For example, in the Niger Trans Saharan Corridor Project, the PPSD promoted the design of robust actions to monitor GBV by involving independent monitoring by a non-governmental organization. D. Streamlining prior review to manage fiduciary risk more effectively 51. The significant reduction in non-value adding prior reviews helped improve the Bank’s fiduciary assurance role. The 2015 Board paper recommended streamlining prior review to use the Bank’s procurement resources more efficiently and enhance risk management. Management committed to focus prior reviews on high-risk/high-value procurements, where it adds more value46. Risk remains a key factor in setting prior review thresholds in FCVs, small states and for highly complex contracts where prior reviews add real value. By contrast, prior reviews divert limited staff resources from those more challenging environments if conducted where Borrower capacity is strong, procurement processes straightforward and risk moderate-to- low. 52. Streamlined prior review help to free up procurement staff time to advance key priorities of the Framework. Reducing the number of low-value/low-risk prior reviews that were previously conducted under the Procurement Guidelines has made more staff time available for building the procurement capacity of Borrowers, supporting PPSDs, up-front procurement planning, data collection and analytics, providing HEIS to Borrowers and focusing on the most critical contracts, where more attention is required to ensure that procurement can effectively support the project’s development objectives. FCV countries and small states have especially benefited from the increased time invested in providing HEIS. In addition, Management’s drive to streamline prior reviews during the first years of implementation of the Framework has helped engender a culture of productivity and efficient use of resources by focusing on risk and outcomes rather than processes. 53. Prior review focus is now closely aligned to the risk level of each contract. Most attention is focused on high-risk contracts valued over $10 million, with a median staff time spent prior reviewing of 21 days, compared to 14 days for low-risk contracts. Similarly, contracts in the $10 million - $100 million value band (which are 35.6% of total awarded contract value, but only 1.1% by total number) benefitted from the highest percentage (18.6%) of staff time spent on prior reviews (Figure 11). By tailoring reviews to contract risk levels, procurement staff are now able to exercise their core fiduciary assurance responsibilities more effectively. Figure 11. FY17 – FY21, Prior review 46 Procurement in World Bank IPF – Phase II: The New Framework (R2015-0120), June 21, 2015, para 74 to 79. 25 FY17 – FY21, volume of prior reviews by project risk level FY17 – FY21, % of total prior review time and median review time by risk-rating of contracts (days) by contract size (USD) 54. FCV countries are benefitting from a greater investment of procurement staff time. As to be expected, a significantly higher share of procurement in FCV countries is rated high or substantial risk. While contracts awarded by FCV Borrowers make up 28% of activity volume, they account for 40% of prior review time by procurement staff. Conversely, non-FCV contracts make up 72% of activities but account for 60% of staff review time. The risk-based approach to prior review ushered in by the Framework has been key to ensuring that Bank procurement staff dedicate significantly more of their time and attention to supporting FCV Borrowers, which has contributed to improved procurement performance in these countries as also evidenced by shorter turnaround times and faster procurement implementation. 55. Procurement staff conducting prior reviews are more productive. This trend is reflected in the period of staff-time spent on reviewing key stages of the procurement process. In FY17, the median review time to complete prior review and issue a Bank no-objection exceeded 16 days (i.e., the total review time for all steps of a procurement). By FY21, the median review time decreased to 9 days. Notably, this improvement occurred in procurements under both the Procurement Regulations and the Procurement Guidelines confirming the Framework’s spill-over effect in terms of increased productivity across the entire portfolio. 56. Streamlined prior review helped release the equivalent of 74 full-time procurement staff to work on the Framework’s new activities. Prior reviews have been reduced from a pre- Framework, FY1447 level of 10,917 contract prior reviews per annum, to 1,376 in FY21. This represents the equivalent of 16,220 days48 of staff time that have been released from low value- added prior review work to support higher added-value tasks that are necessary for the effective implementation of the Framework. The time released has enabled Bank procurement staff to 47 In FY14, the Bank prior reviewed 10,917 contracts ($13.5 Bn), of this 9,849 contracts (90.2%) were below the Bank’s mandatory prior review thresholds. 869 contracts (8%), valued at $10 Bn, were prior reviewed over the Bank’s thresholds; 74% by value of all procurements were prior reviewed. In FCS countries, 2,070 contracts were prior reviewed below the Bank’s thresholds, valued at $433 Million (3% by value). 48 In FY15, Management advised that each prior review action took on average 1.7 days, Procurement in World Bank Investment Project Financing – Phase II: The New Framework (R2015-0120), June 21, 2015 (para 78). 26 increase their investment in Borrower capacity building including, inter alia, HEIS, BFP, supporting MAPS, conducting procurement Advisory Services and Analytics (ASA), reviewing and advising on procurement laws, assessing and advising on e-GP systems, conducting post reviews, and by training Borrowers on the use of the Bank’s Framework (see Annex 1 for case studies). 57. As the number of prior reviews undertaken by the Bank declines, post reviews assume greater importance in the Bank’s fiduciary oversight of its investment portfolio. Post reviews currently sample at least 10 percent of contracts that have not been prior reviewed. Detailed procurement data on both prior and post review contracts is now included in the Procurement App and World Bank website to improve public transparency. In FY21, the newly launched PPR system was used to review 710 projects. The review sampled 12% (3,933) of the contracts under these projects. Of these, 126 contracts worth $88 million, were found to have red flags for potential fraud and corruption and have been shared with INT for its follow-up, as appropriate. Management will continue to use post reviews to improve compliance, monitor integrity, and build Borrower capacity. Considering supply chain challenges and increased procurement risks in FY21, Bank staff post reviewed 100 percent of the contracts under COVID-19 emergency response projects financed within the Multiphase Programmatic Approach. 58. In order to optimize risk management, the Bank takes a fit-for-purpose approach to its prior and post review approach tailored to the specific risks. In higher risk situations such as with FCV, more prior reviews are conducted to ensure greater oversight. Likewise, given the heightened risk, all COVID-19 vaccine contracts were prior reviewed irrespective of value. 59. To further enhance prior and post reviews, Management is testing use of Artificial Intelligence (AI). Leveraging AI technology has the potential to further increase review efficiency by better targeting staff resources to possible non-compliant procurements flagged by the AI software (i.e., rather than relying on sampling for post review). Using AI technology to review all procurements has high potential to further increase fiduciary assurance and mitigate integrity risks. E. Strengthened contract management 60. The Framework recognizes that effective contract management by the Borrower, supported by the Bank, is essential to achieving VfM. Overruns in contract prices or completion times, as well as other contract implementation issues, can undermine development effectiveness and reduce VfM. Prior to the Framework, Bank procurement staff paid minimal attention to this phase of the procurement process as fiduciary supervision was mostly focused on the planning, bidding, and contract award phases. The Framework requires the inclusion of contract management considerations into the earlier stages of procurement planning and on-going decision- making process based on KPIs for more complex contracts. Contract management risks must be factored into the PPSD so they may be effectively mitigated throughout the implementation and supervision phase of the project. Data on overruns in contract prices and completion times suggest that this proactive approach is working, with both decreasing significantly over the last five years. 27 61. The frequency of contract price overruns has decreased since the Framework was introduced. In FY17, price overruns occurred in 15% of contracts awarded under the Procurement Regulations and 22% of contracts awarded under the Procurement Guidelines. By FY21, those figures had fallen to 9% and 12% respectively. The FY21 rates are well below the five-year average of overruns in 17% of contracts (Figure 12). Greater attention to high-risk, complex contracts is an important factor in this positive trend as they are the most likely to encounter implementation challenges leading to costs escalation and implementation delays. Additional staff time allocated to HEIS for contract management has helped mitigate these risks especially during the COVID-19 pandemic. As with several other metrics, this positive impact of the Bank’s procurement reform has benefited contracts awarded under the Procurement Guidelines as well. Nevertheless, contract price overruns have still been significant in several cases and increasing Borrower capacity to manage contracts, especially complex, high-value contracts is a key priority requiring continuous attention going forward. Figure 12. Percentage of contracts with contract price overruns FY17 – FY21, Incidence of contracts with price overruns averaged 17% over 5 years and is trending sharply downwards since FY20. 28 62. Contract completion time overruns have decreased significantly. The average contract completion time overrun decreased by a remarkable 77% from 227 days overrun in FY17 to 51 days in FY21. As with contract price overruns, this improvement was experienced by both Framework and Procurement Guidelines projects. The incidence of completion time overruns also decreased from 46% percent to 34% of all contracts (i.e., the percentage of contracts affected). However, time overruns remain an issue in some specialized and complex consulting services and high-value construction works which will continue to require significant staff resources to help Borrowers address implementation bottlenecks. 63. Despite these improvements, contract management remains a challenge, requiring significant attention to mitigate project impacts and further investment to strengthen Borrowers’ capacity. A relatively small number of contracts incurred price overruns which doubled their initial contract price. Also, while completion time overruns decreased overall in both frequency and size, this trend was less positive for certain types of procurement. For example, the average time overrun in individual consultant contracts was almost twice the average overruns elsewhere. These issues will be addressed by helping clients through specialized training on planning and contract management and by increasing HEIS where most critical. In addition, a new contract management module is being finalized within STEP and will be launched in FY23. The module is expected to provide an effective IT tool to help manage contracts and better enable the Bank to monitor implementation performance across the portfolio. F. Improving procurement outcomes by rejecting Abnormally Low Bids 64. As an additional action to improve VfM, the Framework requires Borrowers to reject ALBs. The Bank’s SPDs now include a methodology to identify and analyze ALBs and guidance for Borrowers on its application has been issued. Bids are rejected when bidders fail to demonstrate their ability to perform a contract at an apparently low price, as identified by applying the methodology (see Annex 1 for a case study). The Bank’s ALB methodology, which is used consistently in all internationally advertised contracts, contributes to creating a level playing field for reputable contractors providing an incentive to all bidders to price their bids realistically while remaining competitive. The introduction of the ALB methodology is likely to have incentivized new market entrants (see Figure 9 above) increasing competition since the introduction of the Framework. 65. A sample of 35 high-value procurement in FY21 and overall experience with reviews of the largest, most complex contracts by the Operations Procurement Review Committee (OPRC) suggest a correct and helpful application of ALB provisions by Borrowers. The lowest evaluated bid was rejected by the Borrower in 4 cases in the sample through correct application of ALB provisions. In all sampled cases, the Borrowers correctly defined the ALB risk zone49 and the process resulted in closer scrutiny of the bids, enhanced due diligence, and better understanding of market dynamics and risks. Risk factors associated with the in-depth analysis of the bids included unbalanced item pricing, front loading of bid prices, under-resourcing of works, and potential lack of knowledge on the part of bidders of site-specific conditions such as the 49 ALB risk is measured in one of two ways, if 5 or more bids are received then the ALB risk zone are bids that are one standard deviation below the average of the other bids, whereas when 4 or fewer bids are received the ALB risk zone are bids that are ≥20% below the Borrower’s cost estimate. 29 availability and proximity of sources of required construction materials. Additional lessons learned with the application of the ALB provisions thus far include:  Reliability of Borrower’s cost estimate: The “relative� approach (i.e., the standard deviation- based identification of the ALB risk zone) proved to be an accurate indicator. When fewer than 5 substantially responsive bids are received, the “absolute� approach (i.e., the comparison with the Borrower’s cost estimate) is more reliable, as envisioned when the requirement was developed. However, reliability of the Borrower’s cost estimate is a very important factor in all cases not only to always benchmark market prices, but also to set qualification requirements correctly. For this reason, greater attention is now being placed during procurement planning on the quality and timeliness of the Borrowers’ cost estimates as important factors contributing to successful contract performance;  Quality of analysis following identification of ALB risk zone: While already very useful, the quality of ALB analysis should continue to be enhanced to enable a closer understanding of bidders’ technical approach, methodology, quality, resourcing and productivity in relation to their bid prices; and  The ALB risk zone is a red flag but not an absolute indicator: The risk zone identification is not just a mathematical exercise and does not replace a comprehensive review of the bids applying all relevant criteria which requires adequate capacity to avoid unnecessary and unfair rejections (not identified in the sample, but a risk in low-capacity environments which is addressed with more prior review and HEIS). G. Improved procurement outcomes in FCV situations 66. FCV countries are among the most challenging procurement environments in which the Bank operates. Weak Borrower capacity and bidders’ reluctance to participate significantly increase operational risk and make VfM decision-making more challenging. Risks affecting procurement in FCV countries are not only higher, but also more diverse than those we face in other operating environments requiring even closer coordination with other disciplines within the project team and across the Bank. 67. The Framework’s fit-for purpose approach and extensive toolbox including specific guidance have been instrumental in addressing these risks50. In addition, staff training has included a significant focus on procurement in FCV situations and 120 information sessions for Borrowers have been delivered and recorded to provide continuous access by their staff. Efforts to improve procurement outcomes in FCV situations also include ongoing outreach to task teams and country management units by OPCS in collaboration with the FCV Unit to explain and support on tailored procurement approaches and risk management in FCV settings which provides additional opportunities for cross-regional learning. 68. The median time to complete procurement processes in FCVs decreased from 92 days in FY17 to 62 days in FY21 decreasing at an annual compound rate of 9% (Figure 13). This improvement is likely a result of increased resource allocation, including for HEIS, more efficient Bank prior reviews, and better capacity in implementing entities. While FCV countries accounted for 28% of the total awarded volume of Bank-financed contracts during the period covered by this 50 Link to Procurement Guidance, Procurement in Situations of Urgent need of Assistance or Capacity Constraints 30 retrospective, they received 40% of the total time Bank Procurement Specialists spent on prior reviewing procurement. Figure 13. FY17 – FY21, Median turnaround times by FCV and other countries, in days 69. Suppliers have been more willing to participate in FCV countries increasing competition and enabling better VfM for these Borrowers (Figure 14). International suppliers are now bidding more frequently on procurements in FCV environments. The mandatory direct payment pilot is among several initiatives launched with the Framework (in FY17) which has been particularly appreciated by international suppliers as it is perceived as an additional incentive by reducing payment risk. It is worth noting that the share of new market entrants in Bank-financed procurement in FCV countries has been higher than in non-FCV countries every year between FY17 and FY21. Figure 14. FY17 – FY21, % of new market entrants in FCVs and other countries 31 70. APAs have been used increasingly in FCV countries to support the Borrower’s capacity to implement procurement. As of the end of FY21, there were 27 Bank projects implemented with a UN agency APA. For example, the project portfolios of South Sudan and Yemen have relied mostly on APAs using UN procurement arrangements. An APA with the United Nations Office for Project Services (UNOPS) under the Yemen Emergency Electricity Access Project has supported electrification of 122 schools, 102 clinics and 12 water wells as of December 2020 and provision of 28,979 solar systems for households, including 15% female beneficiaries. Under the Yemen Integrated Urban Services Emergency Project, the APA with UNOPS has financed the installation of 3,696 solar powered streetlights, the rehabilitation of 236km of urban roads, the provision of solar photovoltaic systems, water pumps and spare parts to 104 water wells and pumping stations, and the generation of 1.4 million person-days of temporary employment as of December 2020. In South Sudan, APAs with the United Nations Children’s Fund (UNICEF) and the International Committee of the Red Cross under the Provision of Health Services Project has enabled 2.6 million people to receive essential health, nutrition, and population services and supported immunization of more than 300,000 children. In another example, in Zimbabwe, an APA involving several UN Agencies enabled cash transfers for food to over 239,000 people, provision of agricultural inputs and livestock support to 7,473 people and rehabilitation of community infrastructure benefiting over 200,000 people. 71. Streamlined contracting arrangements helped FCV Borrowers engage flexibly with UN Agencies. The total value of Borrowers’ engagements with UN Agencies over the past five years using Standard Agreements developed by the Bank exceeded $3.17 billion (635 contracts). Agreements with UN agencies under the COVID-19 Multiphase Programmatic Approach amounted to $1,141 million in value51 including implementation support in several FCV countries. For example, in Haiti, UNOPS delivered 200,000 N95 medical masks, 750 Oxygen Concentrators, 60,000 Oxygen Masks and Pulse Oximeters, in addition to supporting logistics and distribution in ten departmental hospitals. Other examples of implementation support by several UN agencies 51 Data at December 2021 in order to include COVID-19 vaccine related contracts. 32 include the Bank’s response to the Ebola crisis in West Africa and an emergency slope stabilization and remediation in Freetown, Sierra Leone, following the 2017 landslide and flooding. H. Impacts of the COVID-19 pandemic on supply markets 72. The following trends have characterized supply markets since FY20, especially for medical goods, since the beginning of the COVID-19 pandemic and are expected to continue over the short to medium term52:  Supply chain disruptions;  Re-shoring agendas (buyers changing suppliers, shortening and securing supply chains);  Manufacturing and delivery lead times longer than pre-pandemic levels;  More complex logistics arrangements and higher logistics costs;  Commodity price increases and inflation;  Professional and skilled labor supply shortages (leading to wage pressures in some economies);  Erratic employee turnout (unpredictable need for individuals to self-isolate following a positive Covid-19 test impacting inter alia customer service levels, manufacturing output and logistics);  Increased stimulus and infrastructure investments in developed economies, increasing buyer demands and further tightening key supply markets from which Borrowers normally source internationally;  Panic buying exacerbating shortages; and  Pent-up consumer demand, fueling increased inflation. 73. Supporting Borrowers to respond to these trends will continue to require significant Management attention and direct hands-on support from the Bank procurement staff. However, the intense technical assistance provided during the health crisis has also helped to ensure greater Borrower focus on supply chain resilience and is expected to spur further use of modern procurement practices and mainstreaming of sound emergency procurement approaches into Bank projects and the Borrowers’ own procurement systems. 52 Reports of supply chain disruption inter alia World Bank Report, Global Economic Prospects January 2022 33 IV. ENHANCEMENTS AND INITIATIVES Policy and operations-related enhancements and initiatives53 A. Beneficial Ownership 74. A Beneficial Ownership procurement pilot was launched in FYI8. The pilot, the first of this kind in any international organization or any national government, is part of the Bank’s overall strategy to enhance transparency and open contracting. As with other Framework enhancements, an important objective of this pilot was to create a level playing field for bidders with sound corporate governance structures and high integrity standards. With respect to contracts above the OPRC threshold, the Bank’s SPDs now require winning bidders to identify beneficial owners, using thresholds for determining ownership and other definitions of control developed by the Financial Action Task Force. For contracts covered by the Beneficial Ownership pilot54, the Bank requires Borrowers to publish beneficial ownership data, along with other contract award information, of the winning bidders of procurements valued over the Bank’s OPRC review thresholds. 75. Procurements subject to the beneficial ownership pilot represent approximately 40 percent of Bank-financed procurements by value. However, the number of contracts included so far is relatively small, given the high value threshold that applies and because of the normal lag between the project’s approval dates (the pilot began to be applied to contracts awarded in projects approved after November 1, 2017) and the award of major contracts financed by these projects. By the end of FY21, beneficial ownership information has been disclosed for 9 eligible, awarded contracts with a combined award value of approximately $1.876 billion. Beneficial ownership information for these contracts was published on the Bank website and in United Nations Development Business55. The Bank’s STEP system is programed to publish Beneficial Ownership data and OPCS monitors publication. Currently in FY22, there are 38 ongoing additional procurements estimated at $5.8 billion that are subject to the pilot. Beneficial ownership information will be published when the processes are completed. 76. During the beneficial ownership pilot, the Bank continued to monitor implementation closely and actively shares information with other MDBs who already have or are considering launching a similar program, the International Monetary Fund and a broad range of stakeholders, including donors, Borrower governments and civil society organizations. The high threshold for the pilot’s application was decided following consultations with stakeholders, due to the novelty of the requirement and still unknown unintended consequences for Borrowers, bidders and natural persons involved. Following implementation, the response to the pilot has been consistently positive and no negative consequences or complaints by any of the involved parties have been registered to date. 53 Policy and operations-related enhancements and initiatives led by OPCS 54 Procurements over the Operations Procurement Review Committee thresholds 55 United Nations Development Business, and World Bank Web site, link to Procurement Notices 34 77. Management will extend the Beneficial Ownership pilot to all internationally advertised contracts for which the specific procurement notice is published on or after July 1, 2022. In addition to piloting this requirement for all international contracts financed by Bank projects, the Bank will continue to advocate for its inclusion in domestic procurement laws as part of the Bank’s policy dialogue on procurement modernization with Borrowers. B. Mandatory Direct Payment 78. A Mandatory Direct Payment Pilot (MDPP) was launched in FY17. This pilot aimed to reduce supplier payment risk as part of a broader effort to foster bidder participation in FCV environments, as discussed in this report above. The MDPP applies to operations56 financed under International Development Association (IDA) with Decision Review held on or after November 1, 2017 and requires mandatory use of a direct payment disbursement method for contracts procured by open or limited competition or direct selection, except where a Special Commitment (letter of credit) is used. While the direct payment disbursement method has always been available, this pilot makes it mandatory in these cases. By the end of FY21, over 125 projects were eligible for MDPP (overall number of projects subject to direct payment accelerated in FY20/21 due to the rapid increase in COVID-19 projects) and no particular issues have been flagged since its inception. Exceptions were granted for only three eligible projects and were related to payments made under contracts awarded to local contractors, where the procurement was conducted through international competition. In these cases, it proved more efficient to pay these national contractors in local currency from project advances made by the Bank to the Borrower. Having successfully piloted the approach and acknowledging that payment risk continues to be a major source of concern for bidders, from July 1, 2022, Management will operationalize mandatory direct payment for procurements in difficult environments (including FCV) with an exception for Borrowers to pay from project advances if the winning bidder is a national supplier. C. SEA-SH prevention and response, and Contractor Disqualification 79. Management has implemented strong procurement measures to help prevent, mitigate and manage SEA-SH risks in Bank-financed projects. The approach is underpinned by the recommendations of the Global GBV Task Force of independent experts, convened by the Bank in October 2016. The recommendations of the Task Force are set out in the July 2017 GBV Task Force Report, which informed the Bank’s comprehensive Action Plan to Address SEA- SH/GBV in Bank-financed operations (November 2017)57. 80. Among several complementary measures, the Action Plan included two important procurement-related objectives: i) issue revised SPDs for Works, with specific requirements relating to management of SEA-SH risks in the form of qualification and contractual requirements and remedies, technical specifications, and other relevant appendices; and ii) hold contractors 56 The conditions for the MDPP pilot are: (i) the project is Investment Project Financing (IPF) (including Additional Financing); (i) the project is financed by IDA; (ii) the project triggers paragraph 12 of Section III of the IPF Policy (Projects in Situations of Urgent Need of Assistance or Capacity Constraints ); and (iii) the Decision Review for the project is held on or after November 1, 2017. 57 Link to Global GBV Task Force: Action Plan for Implementation 35 responsible for fulfilling these requirements during contract implementation, including through stricter remedies for non-compliance. 81. All works-related SPDs have been revised to add specific qualifications, bidder declarations and contractual requirements relating to prevention of SEA-SH by contractor’s personnel. These qualifications and requirements guide an assessment of bidders’ responsiveness and competence to comply with specific SEA-SH-related conditions during the bid evaluation process. They also set clear expectations in contract conditions by stipulating obligations to manage SEA-SH risks within the contractor’s control. 82. Building on the implementation experience with the strengthened SEA-SH requirements, Management enhanced its approach to holding contractors accountable for fulfilling their SEA-SH obligations during contract implementation. The contractor (and subcontractor) disqualification remedy was launched on November 24, 2020 and is applicable to procurement processes for works contracts using the Bank’s SPDs, initiated after January 1, 2021 in projects assessed as high-SEA-SH risk58. This enables the Bank to disqualify contractors that fail to comply with their SEA/SH obligations from being award Bank-financed contracts for two years. The Bank’s Task Leader, Environmental and Social Specialists, and Accredited Procurement Specialists all coordinate within the task team to ensure that SEA-SH risks are controlled and, during supervision, monitor contract compliance. The new remedy is supported by a clear and transparent process, which embeds important due process rights for all parties, including the contractors. It relies and builds on tested contractual mechanisms that are consistent with good industry practices and Bank policies and is fully consistent with the survivor-centric approach adopted by the Bank’s GBV Task Force Action Plan. The mechanism was discussed extensively with industry and civil society worldwide, as well as with development partners both while it was being developed and after its launch. In addition, a major training program was implemented by the Bank and collaboration with relevant professional associations continues to help mainstream GBV prevention and mitigation into industry practices. Due to a normal time lag between the beginning of a procurement process and contract implementation, there are no lessons learned to report to date. The mechanism is currently being applied to 23 ongoing and planned procurements for an estimated $750 million under 10 projects and 1 contract worth $24 million in Burkina Faso has been awarded to date59. Stakeholders engaged so far consistently supported its use and provided very positive feedback on the Bank’s initiative. Management will continue to monitor the implementation of the SEA/SH disqualification mechanism and will update on progress with implementation and its impact. D. Cascading the Environmental and Social Framework to contractors 83. Procurement is a key delivery mechanism that supports implementation of the ESF. Over fifty different ESF requirements that apply to contractors, consultants, and suppliers, as appropriate, were added to procurement documents to set clear expectations and obligations. For example, the SPD for works has been significantly enhanced to include ESF requirements on, inter alia, protection of the environment, health and safety, inclusion of experts for environmental and social aspects, immediate notification and progress reporting obligations on environmental and 58 Link to Strengthening Prevention of Gender-Based Violence (GBV) 59 As at end April 2022 36 social matters, code of conduct (including on SEA-SH aspects), social benefits and workers’ rights, nondiscrimination, workers’ organizations, child labor, forced labor, remedies, etc. E. Bank-Facilitated Procurement 84. The COVID-19 pandemic continues to disrupt global supply chains. While the world was coming to terms with the scale and impact of COVID-19, it became clear from the onset of the pandemic that global stocks of medical supplies and equipment were being quickly depleted and some Borrowers were in danger of being left behind as all countries raced to procure urgently- need medical supplies and equipment. Rapidly changing market conditions also led to price increases, uncertainty in delivery times, heightened risk of predatory practices, lack of cargo space, and escalating freight charges. 85. Management launched BFP as an enhancement of HEIS to further help Borrowers respond to these procurement challenges, while remaining fully within the boundaries of IPF policy. The Bank leveraged its convening power to help Borrowers consolidate demand and gain better market access with a stronger contracting position. BFP support included Bank staff sourcing and negotiating contracts for medical supplies, preparing contracts ready for Borrowers to sign, helping with logistics, as needed, and supporting hands-on contract administration and payment. With multiple suppliers available for a wide range of products, Borrowers can now also source much-needed medical equipment and Personal Protective Equipment (PPE) through an electronic BFP catalogue (including, for example, vaccine consumables, oxygen concentrators, ventilators, X-Rays, Ultrasounds, PPE, logistics providers etc.). All products offered are required to meet or exceed World Health Organization (WHO) standards and the Bank conducts extensive price benchmarking and commercial and integrity due diligence on suppliers before including them in the BFP program. The Bank does not enter into contracts or assume any financial or contractual obligation and Borrowers remain solely responsible for signing the contracts and arranging logistics. However, the Bank has also helped hands-on to outsource logistics (e.g., transportation, and inspection services) in several cases, as requested by Borrowers and with the same BFP modality. Bank staff and consultants working on BFP also provide regular market intelligence on emerging procurement and market trends to anticipate and address urgent Borrower needs for medical products and logistics. Last, but not least, the BFP team regularly engages with COVID-19 vaccine manufacturers to monitor the availability of doses and production forecasts, resolve emerging issues affecting supply to Borrowers (e.g. delays in delivering doses), or the Bank’s ability to finance (e.g., with respect to policy red li nes such as the applicability of the Bank’s Anti-Corruption Guidelines and the obligation for Borrowers to publish basic contract information in accordance with the Procurement Regulations). At the Borrowers’ request, the BFP team is also available to support hands-on with vaccine contract negotiations and outsource of vaccines deployment services to help scale-up vaccination programs as more vaccine are arriving in countries (e.g., end-to-end logistics, workforce solutions, and administration/monitoring services). F. Industry Engagement Program (IEP) 86. In FY17, Management initiated a pilot engaging with key industry sectors to seek their input to improve the Bank’s approach to complex procurements. IEP began by targeting 37 power transmission projects, using the CASA1000 power project as a pilot to test the benefits of deeper industry engagement. The Bank, Borrowers and industry representatives worked through the project’s risk profile and allocation to ensure a balanced approach, placing risks with the party best able to manage them and these engagements resulted in compliant bids submitted from leading contractors, who had declined to participate in a first round of bidding. 87. In FY18, the IEP focused on procurement of Medical Diagnostic Imaging equipment (e.g., x-rays, Magnetic Resonance Imaging scanners, ultrasounds etc.). Extensive market research was conducted on this sub-sector of the medical industry and a series of engagements was held with suppliers and industry associations globally to seek feedback on the Bank’s procurement processes and invite industry suggestions on which aspects of the Bank’s procurement processes could be improved and how. This IEP with the medical sector subsequently proved critical at the onset of the COVID-19 pandemic (FY20/21) as the industry’s earlier input helped Management to shape the BFP program and accelerate procurement support to the COVID-19 emergency response. 88. In FY19, the IEP targeted the water sector. Through a series of industry workshops, the Bank engaged with 180 water industry participants to identify procurement issues of concern and debate possible solutions to improve Bank-financed water sector procurements. These issues included the need to use more performance-based specifications and quality criteria for evaluation to incentivize innovation, provide more hands-on support to Borrowers and encourage early market engagement. 89. Responding to the water industry’s suggestions and views expressed by Bank staff, a new performance-based SPD and Guidance were published for projects involving water loss reduction. This innovative contract is designed to help achieve and maintain a certain service level for loss reduction in water networks within a specified service area (e.g., by creating incentives to find major leaks, or to ensure that remedies are sustainable) and is mentioned here as an example of how the Procurement Framework’s toolbox continues to be updated to address both em erging risks and lessons learned from relevant markets and Bank projects. After a pause due to the need to focus more on the health sector market during the pandemic, the IEP will continue its proactive engagement with key industry sectors to identify procurement-related challenges and develop demand-driven solutions which can benefit both prospective bidders and Borrowers by facilitating innovation and better fit-for-purpose solutions. G. Cybersecurity 90. The Bank is working on strengthening its operational response to cybersecurity in IPF projects. Emerging standards for cyber-security are not universally applied and there is no single global regulatory authority that monitors or manages their application. Comprehensive guidance on cybersecurity is being prepared under the leadership of the Digital Development Unit in the Infrastructure Vice-Presidency with wide participation from relevant units across the World Bank Group and will include guidance on preparing appropriate procurement strategies to help mitigate this risk. To further support cybersecurity risk mitigation, Borrowers procuring activities with cybersecurity risks are now required to utilize the RfP selection method and/or use rated 38 evaluation criteria to determine contract awards that include optimal management of cybersecurity risks. 91. The Framework includes a comprehensive suite of tools and techniques to support Borrowers in applying cyber-secure solutions in IT procurement that meet best practice standard. Proportionate security measures are considered for all procurements that involve an IT component (a far broader set than just IT projects), based on a structured assessment of the threats, risks, and vulnerabilities to which key information assets are exposed. While most aspects required to design and implement secure systems are part of a broader IT strategy, procurement approaches can be tailored to support implementation of a country’s cyber security policy and practices at different stages in the procurement and project cycles. For procurement, the main focus is to ensure that:  IT risk assessments are in place to ensure cyber security threats are identified;  Cyber security policy and standards are included in specifications;  Supplier selection considers a supplier’s cyber security credentials, practices, track record, governance, and jurisdiction in which a supplier’s activities are controlled;  Conditions of contract have specific clauses that address cyber security requirements and resulting enforcement actions for any breaches;  Contract management is resourced, takes place, and focuses on cyber security threats; and  Borrowers have the necessary capacity to draft and manage IT contracts. Examples of global knowledge initiatives60 H. Application of Disruptive Technologies to Public Procurement 92. Disruptive technologies have the potential to fundamentally change the way in which public procurement value is realized over the next decade. These technologies will help support accuracy of decision-making by providing accurate data, increase compliance with contractual agreements, and help buyers increase direct savings, adding value to their procurement strategy through access to contracting tools that provide information on previous contracts within a procurement category and method. The process of harmonizing new technologies and defining common principles can be very time-consuming and costly for Borrowers. In FY21, the Bank published a paper on “Disruptive Technologies in Public Procurement�61 with the objective to promote procurement reform and innovation by providing guidance to Borrowers on how to prepare themselves for the implementation of disruptive technologies. I. Electronic Government Procurement 93. E-GP systems are key drivers of efficient public procurement. However, e-GP implementation worldwide is still uneven, with developing countries often lagging. E-GP systems help reduce transaction costs and enhance competition in public procurement processes, generating 60 Global knowledge initiatives led by the Governance Global Unit in the EFI Vice-Presidency. 61 Link to paper Disruptive Technologies in Public Procurement 39 savings and opening procurement markets to SMEs. E-GP also increases the availability of data to help inform procurement policies and management decisions. Developing countries, particularly in Sub-Saharan Africa, have yet to achieve the technical capacity to drive e-GP reforms independently. E-GP reforms, therefore, are long-term and comprehensive and require sustained support. As a part of the IDA19 policy commitments, the Bank is expected to support at least 40 percent of IDA countries to implement e-GP and conduct detailed procurement data analytics, in order to increase efficiency of public spending and mitigate corruption risks. See Annex1 for case studies. J. Green Public Procurement 94. Green Public Procurement can drive market innovation to help combat climate change, and promote sustainable agriculture and forestry, energy efficiency, the move to a circular economy, and smart cities development. A global engagement is underway to help mainstream green procurement into countries’ public procurement reforms which includes the FY21 report “An overview of Green Reforms in Country Procurement Systems� 62. This report provides an overview of international experience on green procurement’s implementation focusing on the institutional framework needed to support mainstreaming of good practices across government agencies. It draws from a wide range of country examples and provides links to handbooks and tools for practitioners. 62 Link to Report, Green Public Procurement: An Overview of Green Reforms in Country Procurement Systems 40 V. CONCLUSIONS 95. The analysis presented in this five-year retrospective suggests that the Procurement Framework is being implemented in a way that is generally successful and has significantly improved procurement results. Following the introduction of the Framework, procurement outcomes have improved across a wide range of metrics, countries and circumstances, both in newer projects that fall under the Framework and in older projects being implemented under the Procurement Guidelines. Borrowers are now better equipped and supported to achieve VfM. Increased bidder participation across regions suggests greater market confidence in the Bank’s procurement processes. Performance gains in procurement have been even more positive in FCV countries than in non-FCV environments. 96. Streamlined prior reviews have allowed the Bank to redeploy procurement resources from low-value prior reviews to more value-adding tasks including building Borrowers’ procurement capacity, which has overall improved. Bank staff now provide much-needed direct procurement support to Borrowers, are empowered to innovate and can better address emerging risks and opportunities. In addition, projects may rely on procurement rules of development partners, as most appropriate to achieve the project’s development objectives in a specific country operating environment. 97. The Framework continues to evolve and adapt to better meet the needs of stakeholders. As intended, the Framework has continued to evolve with enhancements and new initiatives (see Section IV) which strengthened the Bank’s ability to help Borrowers respond quickly and effectively to emerging challenges such as, for example, environmental and social risks and the COVID-19 pandemic. 98. Implementing the Framework will continue to require close Management attention and adequate resources to ensure that the many opportunities it provides are fully harnessed. For example, continued attention is required to meet supply chain challenges, improving contract management and address emerging risks which affect procurement outcomes. Further deepening the Bank’s procurement reform will continue to be a heavy lift over the coming years. However, with sustained support from the Bank’s leadership, including the Board, and adequate resources, the Bank is well positioned to continue to be in the forefront of global procurement innovation. 99. Management has identified actions to further enhance effectiveness of the Framework. Building on experience discussed in this five-year retrospective, Management will implement the following actions during the current and next fiscal year:  Strengthening Borrower application of VfM award decisions. Accelerate Borrower application of quality-based, rated evaluation criteria (i.e., non-price criteria) and LCC to determine contract awards that optimize overall VfM. Bank procurement staff will scale- up training and HEIS to support Borrowers carrying out procurements using rated criteria in both RfBs and RfPs, s advise them on quality assurance and risk management decisions, and help them engage with domestic control agencies to ensure that evidence-based judgment inherent in non-price evaluation is not discouraged. STEP will be updated to support tracking Borrower use of rated evaluation criteria in RfBs and RfPs; 41  Evaluating overall VfM under bank financed procurement. Management will develop a methodology that includes clear indicators to inform future assessment as to whether the Framework is helping to achieve overall VfM;  Improving management of cybersecurity risks. Require Borrowers procuring activities with cybersecurity risks to utilize the RfP selection method using rated evaluation criteria to determine contract awards that include optimal management of cybersecurity risks;  Enhancing contract management support and integrity. More systematic tracking of Key Performance Indicators (KPI) for large contracts, deep dives on high risk/complex projects to help reduce time and cost overruns and HEIS for contract implementation (beyond more traditional hands-on support for procurement) will help maintain VfM and support mitigation of integrity risks during this critical phase for development effectiveness. A new contract management system will support Borrowers to better track contract implementation (i.e., track milestones including timely completion, payments, KPIs and contractual disputes, and producing reports to support contract management). Additional contract monitoring information will be used to support further mitigation of integrity risks during implementation;  Applying the SEA/SH disqualification mechanism. Closely monitor the implementation of the SEA/SH disqualification mechanism and update on progress with implementation and its impact;  Strengthening supply chain management and security. Leveraging recent experience with BFP, market intelligence, industry engagements, and fit-for-purpose approaches to mitigate emerging global risks, the Framework’s toolbox will be expanded to help countries and Bank teams build supply chain resilience and manage associated risks. This will include technical guidance and capacity building on supply chain mapping, risk analysis, traceability protocols, quality inspection/testing, cost-benchmarking, and managing logistics. Among others, the Health Sector will be a particular area of focus to help develop procurement solutions that support stronger country health systems;  Continuing to support public procurement reforms and capacity building in Borrower countries. Policy dialogue and technical assistance will continue to include broad systemic reforms and discrete interventions to strengthen individual institutions. These efforts will include a deeper focus on the areas included in this action plan such as, for example, green/sustainable procurement, working with Supreme Audit Institutions to modernize fiduciary control frameworks in ways that support VfM decision-making, and developing effective emergency procurement approaches;  Operationalizing mandatory direct payment. Based on the successful pilot, this requirement will be mainstreamed for all procurements in FCV environments except when Borrowers prefer to pay national suppliers from dedicated project accounts; and 42  Operationalizing Beneficial Ownership. Consistent with good international practice and the pilot’s success, this requirement will be mainstreamed and extended to all internationally advertised contracts (for which the specific procurement notice is published on or after July 1, 2022). 100. Management will continue to closely monitor the Framework’s performance, and make needed enhancements, regularly updating the Board on progress. Every year, many more projects using the Framework are added to the Bank’s portfolio creating a richer data set available in STEP to analyze, compare, and inform the Framework’s ongoing evolution. This will ensure that the Framework continues to meet its overarching objectives of supporting projects’ development objectives and the Bank’s key agendas through strategic, fit-for-purpose, and innovative procurement approaches with integrity and helping to protect the environment and the communities we serve. 43 ANNEX 1 – CASE STUDIES (BASED ON ACTUAL PROJECTS AND INITIATIVES IN REGIONS) A. Abnormally Low Bids A 2013 works procurement process63 under the previous Procurement Guidelines (which did not include ALB provisions) was awarded to the lowest evaluated substantially responsive bidder for 936 million in local currency while the contract estimate was 1.2 billion (i.e., the winning bid was 22% lower than the estimate). The contract was subsequently terminated due to multiple failures and the Borrower initiated a second process to complete the works. Had this procurement been undertaken under the 2016 Framework, the lowest as well as the second lowest priced bidders would have been flagged as ALB risk and may have been rejected. The procurement was re-bid including the ALB provisions, as agreed with the Borrower even if not required under the applicable rules, and one bid was rejected because the bidder could not demonstrate its ability to deliver at the price proposed in its bid. The contract was awarded to the second lowest bidder and the project has progressed well since. B. Borrower Capacity Building Cameroon Cameroon’s Public Procurement system was characterized by the centralization of all procurement decisions at the level of Ministry of Public Contracts (MINMAP) created in December 2011, this created some challenges between institutions. The reforms supported by the Development Policy Financing (DPF) series (2017-2021) aimed to improve the public procurement process by increasing accountability, ensuring value for money, and enhancing project quality. Through DPF2 two important texts were adopted: The Public Procurement Code decree and the “Arrêté� introducing a performance-based payment of procurement commissions. The new Public Procurement Code defines the roles and responsibilities of various stakeholders involved in public procurement, aiming to reduce the time required to complete the process while enhancing transparency. The new Public Procurement Code separates the contracting, supervisory, and regulatory functions. The code gives sector ministries greater responsibility over contract awards and execution while limiting MINMAP (the Ministry of Public Contract) ’s role to prior review above given thresholds. The body tasked with addressing complaints has been transferred from Ministry of Public Contracts (MINMAP) to the Public Contracts Regulatory Agency (ARMP). The DPF2 also supported the introduction of a performance-based payment system of procurement committees, designed to reduce processing time. The DPF3 strengthened and consolidated the actions achieved during DPF2. the Government has adopted an implementing text on the Dispute Resolution Committee (DRC), in order to enhance its independence on its organization and operation. This new regulation on DRC is a major step forward to improve transparency in Cameroon’s Public Procurement System. It materializes the transfer of the decision authority from the Ministry of Public Contracts (MINMAP) to the newly established committee for disputes 63 The project and contract are not identified as the information is from deliberative Bank documents. 44 resolution. The DRC is a multipartite committee involving the civil society (3 members), private sector (3), Government (2) and Regulatory Agency (ARMP). Conflict of interest matters are specifically addressed. Eswatini The Government of Eswatini requested the Bank to provide medium-term support to the country’s recovery and reform program. As a result, a programmatic series of two DPF operations combines critical measures under three pillars to respond to the COVID-19 pandemic with structural reforms. Pillar 2 of the DPF - Strengthening Transparency and Public Financial Management - supports inclusive and sustainable economic recovery by addressing key structural bottlenecks including through reforming public procurement processes. As a prior action of the first DPF operation (P174447), new procurement regulations were approved by the Cabinet for submission to the Parliament and adopted by the same in October 2020. The new regulations are holistic in nature, prompt open competition, and enable the rationalization of public procurement expenditures through modern practices and techniques, such as: (i) annual procurement planning, (ii) use of framework agreements for common-use items, (iii) promotion of local SMEs and sustainable development objectives including innovation, social welfare, and environmental conservation. This prior action is followed by the trigger of the second DPF operation (P175317), which facilitates transparency, accountability, and efficiency through the introduction of electronic e- Government Procurement (e-GP). For this, the Ministry of Finance has approved and launched the implementation of the e-GP program based on an e-GP strategy which was developed with technical assistance by the World Bank. The procurement reform measures of the programmatic DFP series will result in 100% of public tenders awarded in line with the new regulations in all agencies and procured electronically in ten selected pilot agencies by December 2022. Together with the roll-out to all procuring agencies, the introduction of e-GP based on the new regulations can contribute to reducing the fiscal gap by generating savings of up to 5% of Gross Domestic Product. The Gambia The Bank supported public procurement reforms in the Gambia through a range of instruments, starting with policy reforms through a DPF series and complemented with operational reforms through an IPF operation. The Fiscal Management, Energy and Telecom Reform DPF series, supported the revision of the Procurement Act followed by the preparation of the related Regulation. The law introduced new concepts such as sustainable procurement, framework agreements and e-GP. The Act was approved by the Cabinet and is pending adoption by the Parliament, after which the Regulation could also be adopted by the Government. The IPF operation (Fiscal Management Development Project, P166695) aims at supporting two major reforms: the professionalization of the public procurement function and the implementation of an e-GP system in the country. The readiness assessment for the e-GP has been concluded and the process of implementation is underway. In order to create awareness on relevant topics, the Bank 45 invited national authorities, through the Gambia Public Procurement Authority (GPPA), in some presentations made by the Bank, namely the e-GP Implementation/Solution models and overview of countries’ Green Procurement practices. The Bank is collaborating with other partners for the public procurement reforms in the Gambia. The Act and the Regulation were both prepared with technical assistance support from a Public Financial Management (PFM) project funded by the European Union, which is also supporting the preparation of a guideline (procedures manual) for the practitioners. This collaboration is expected to continue in the future on topics such as sustainable procurement. India The Framework identifies contract management as one of the focus areas. India, one of the largest Borrowers, can have challenges with speed of implementation and disbursement in infrastructure projects and contractual disputes. To address these challenges, the Bank team designed a multi- pronged capacity building strategy for contract management. In view of India’s size and the portfolio, partner institutions who may be able to deliver these courses on a cost-recovery basis, while the Bank team provides inputs for curriculum design, training of trainers, and monitoring quality of the delivery. All India Management Association (AIMA), with over 38,000 members and close to 6,000 corporate/ institutional members through 67 Local Management Associations affiliated to it, was identified as the first partner institution, which launched a 6-month Certificate in Contractual Dispute Resolution (click here for link) in FY17. This blended learning program aims at avoidance of contractual disputes through better contract management practices and early resolution through alternate dispute resolution (ADR) mechanisms. Construction Project Management was also identified as a challenge and the team supported two institutions namely National Institute of Construction Management and Research (NICMAR) (https://nicmar.ac.in), and L&T Institute of Project Management (L&T IPM) (www.lntipm.org) in designing and launching a Short-term training program in Construction Project Management during FY19. NICMAR, which is a non- profit organization, is engaged in promotion of training, research, professionalism, and skill formation at all levels of the construction and other related industries. The L&T IPM, on the other hand, is a private institution, established for creating excellence in project management. In addition to the above programs, in FY21 the Bank’s India team has also supported the launch of a 6-month Certificate in Procurement and Contract Management for Public Private Partnership (PPP) Projects; and an 18-month Post Graduate Diploma in Management Program with a specialization in Procurement and Contract Management. To date these courses have been attended by circa 600 officials that handle various projects in India. As student intakes for these programs are not only limited to Bank-financed Projects, these initiatives are also likely to support broader capacity building in India to enable the country to achieve its planned infrastructure growth. Jordan Public procurement in Jordan was governed by more than fifty by-laws. In addition to the large legal framework, there was no central policy and oversight unit for the procurement system nor a complaint handling mechanism. 46 The Government of Jordan enacted a consolidated legal framework for public procurement through the issue of a new By-Law (No.28), published in the Official Gazette in May 2019. The consolidated By-Law with comprehensive coverage and application to all procuring entities in Jordan created a single regulatory framework for public procurement, which will assist significantly in maximizing both efficiency and bidders’ confidence. In addition, the new By- Law provided for the establishment of a central policy and oversight unit and an independent complaint handling mechanism, and contains provisions on Framework Agreements, electronic Government Procurement (e-GP). In addition; the Government of Jordan had developed e-GP system (JONEPS) and made its use mandatory through a circular issued in February 2019 (Prior action to DPF-II). The Bank is supporting the Government of Jordan in leading multifaceted reform initiatives to operationalize the new legal and institutional set-up in Jordan through;  development of SPDs;  comprehensive capacity building activities to sensitize public and private sectors;  operationalization of the Procurement Policy Committee;  support to roll out of e-GP system (in Jordan, called JONEPS) and further enhancement to the system;  development of the communication strategy; and  professionalization of Procurement. Mauritania The government of Mauritania requested the Bank's support to improve the country’s public procurement law and regulations. Following this request, the Bank’s procurement team quickly deployed a team of experts to assist the country in designing its new procurement law and regulations. The work required close collaboration with the Ministry of Economic Affairs and with the other entities in charge of procurement policy, and procurement control in Mauritania. The new law has now been approved by the Council of Ministers and the preparation of the implementation decrees are well advanced. This reform work includes close coordination with the International Monetary Fund to introduce beneficial ownership regulations in the country’s new Procurement Framework. Nigeria Staffing the public procurement system has been one of the key challenges that constrain sector evolution in Nigeria. Building on federal multi-stakeholder consensus, the Bank, jointly with core agencies, rolled out a Nigeria-unique program dedicated to capacity building not only in procurement, but also in environmental and social standards, in an integrated and sustainable manner in partnership with six Federal Universities that will offer various blended and hybrid educational tracks. Participatory preparatory work culminating in 2020, resulted in setting up and digitally interconnecting six Centers of Excellence at six Universities, completing curriculum design and developing ten advanced courses on procurement. In February 2021, the Training Of Trainers (TOT) for sixty academic and industry trainers was launched. In May 2021, the process 47 of enrolling 12,000 practitioners on a multitude of short courses started. Furthermore, 9,000 students will graduate from other educational tracks to fill the competency gap in both public and private sectors, with Federal entities certifying 4,000 practitioners. Through this program extending till 2024, linkages between procurement, environment and social standards will be fortified leading to more socially and environmentally responsive procurement and project execution. Philippines The Bank Procurement team provided the technical assistance to the Philippines Department of Budget and Management (DBM) through the Reimbursable Advisory Services for Supporting Public Financial Management Reforms in the Philippines. The Bank team engaged DBM and the Procurement Service (PS) in working on the Philippines Government Electronic Procurement Service (PhilGEPS) data, one of the activities under the Reimbursable Advisory Services is PhilGEPS Data Analytics. Based on the available procurement data, the Bank team provided insights and ideas for procurement policies and strategies that generate financial savings. The report includes the key findings of data analytics and recommended strategies for availing potential savings in the government procurement. Changes to laws or regulations are not required to materialize the savings, but rather an enhanced strategic approach to government procurement. With the use of data analytics, variables directly and indirectly impacted by procurement policies and strategies were identified together with the estimated savings. Strategies relating to purchase quantity, advertisement period, and product bundling, among others, need to be revisited as these would allow potentially major savings for the Government. The detailed findings and recommendations were presented to PS, DBM and to the Government Procurement Policy Board, consisting of heads of at least 14-line departments, and its Technical Support Office. Aside from the report, a URL link providing open access to the interactive dashboards with visualizations and codes was provided to the Government. Based on the remarkable result of the study, the DBM is planning to incorporate data analytics features in its modernized PhilGEPS. Romania Since 2016 the World Bank has been supporting Romanian authorities, in particular the Public Procurement Agency, in building institutional capacity and implementing reform activities in the area of public procurement, in line with the adopted national strategy for public procurement. One of key areas of assistance includes specific support in the area of capacity development of procuring entities. Through its interventions, the Bank aims at:  supporting the professional development of the procurement function by creating a general framework of competencies and revising the existing occupational standard for “procurement expert�;  analyzing the opportunity to introduce the procurement function as a profession and a system of incentives for public procurement staff; and  supporting better performance in public procurement of selected procuring entities by optimizing internal procurement processes and increasing the capacity of the procurement professionals. 48 Some of the key results in the area of professionalization and capacity development in public procurement, achieved in close collaboration with the Public Procurement Agency in Romania, include a competency framework for procurement professionals fully in line with the European recommendations (ProcurCompEU), a revised occupational standard setting, a general training framework in public procurement, on-the-job support to circa 140 procuring entities at the country level which included support for planning 170 procurement processes, organizing around 20 workshops on the planning, evaluation and contract management stages, daily guidance and support for different procurement activities. Uzbekistan The Government has made substantial and consistent efforts over 2018-2020 to enhance its Public Procurement System (PPS). The new applicable Public Procurement Law (adopted in July 2021) provides a legal framework following good international practices. The regulatory and normative roles are assigned to Department of Public Procurement (DPP) within the Ministry of Finance (Public Procurement Authorized Body). The legal framework provides a broad description of permissible procurement methods and their procedural requirements. The main challenge for the Government is to ensure its practical implementation along with development of the country tailored “public procurement development strategy� for next five years to ensure sustainability of the reform work. The Uzbekistan Ministry of Finance is now implementing the project (financed by European Bank for Reconstruction and Development (EBRD)) to launch the integrated public procurement web-portal (a full-fledged single-window e-GP system) - this portal will significantly improve a performance of the entire PPS. Building on the reforms already implemented, the next phase of procurement reforms offers an opportunity to:  introduce more comprehensive Monitoring and Evaluation (M&E) tool for measuring the performance effectiveness of public procurement system with increasing system openness, improving integrity, providing for efficient public oversight and citizen’s engagement; and  incorporate SPP and GPP into the PPS by bringing the best international practice in this area. Encouraging the emergence of a private sector is essential for country growth and the Enterprise Survey of SMEs will identify approaches, incentives and bottlenecks for SMEs promotion to better access the public procurement markets. The abovementioned activities have been intensively discussed with the Uzbekistan Ministry of Finance and supported by other Development Partners (Asian Development Bank (ADB), Islamic Development Bank (IsDB), EBRD) and Agence Française de Développement (AFD). Vietnam Following the introduction of the Framework, the Bank’s Vietnam office enhanced its support to capacity building and public procurement reforms. During 2017-2021, the Bank carried out a series of Programmatic ASAs (PASAs), in collaboration with other major Development Partners (ADB, Japan International Cooperation Agency (JICA)) to help the Government of Vietnam (GOV) strengthen capacity and performance of various public procurement institutions and systems as well as further development of procurement legislation. These PASAs also supported the 49 sustainable use of country systems in Bank-financed procurement and the enhancement of integrity and accountability in procurement at national and project levels. The Bank’s support in these PASAs has yielded significant results and outcomes. At the national level, the Bank successfully helped the GOV to update the Anti-Corruption Law in 2018; develop and issue the first- ever PPP Law; and to issue a Decree guiding the implementation of government procurement commitments under two major Free Trade Agreements, the Comprehensive and Progressive Agreement for Trans Pacific Partnership (CPTPP) and the EU-Vietnam Free Trade Agreement (EVFTA) in 2020. At the institutional level, the Bank supported major government agencies to strengthen their procurement capacity and performance. Specifically, the Bank provided technical support to the Ministry of Planning and Investment and Government Inspectorate to strengthen the procurement audit and inspection functions; the Ministry of Health to enhance and improve its centralized procurement system for drugs and pharmaceuticals; and the Academy for Policy Development (APD) a leading national procurement training institution to build-up and strengthen its procurement education and training capacity to international standards. The Bank also supported the Ministry of Planning and Investment and its Public Procurement Agency develop an SPD for GPP of works and pilot the application of GPP in goods contracts under CPTPP and EVFTA. Overall, the Bank’s strategies following the launch of the Framework have made significant contribution to the improved public procurement systems in Vietnam. West Bank & Gaza The Bank has been supporting the public procurement reform in West Bank and Gaza for more than ten years. A combination of financial instruments are used to support the reform process:  the procurement prior actions which are a central element of the annual Development Policy Grants (DPGs);  the Technical Assistance (TA) on procurement through the ASA; and  the IPF operations through the ongoing Public Financial Management Improvement project (PFMI) and Digital West and Gaza Project. As background to this reform, a new Public Procurement Law (PPL) and its implementing regulation were enacted in 2014 and the High Council for Public Procurement Policies (HCPPP) was established with support from the Bank. The creation of HCPPP is one of the most important building blocks in the construction of a modern, best-practice public procurement system that is an enabler of development. HCPPP is the entity mandated by the PPL for oversight of all public procurement activities, as well as for the development of the procurement system, including policy setting, institution building, procurement documentation, guidelines and manuals, training, and public awareness. Under the PFMI Project and the ongoing ASA, the public procurement has been strengthened through:  issuing the National Standard Bidding Documents (SBDs) for works, goods, non-consulting services, consultants’ services, and frameworks agreements;  developing and implementing the Single Procurement Portal; 50  training of the procurement workforce on use of SBDs and the Single Procurement Portal;  operationalizing the Dispute Review Unit to act as independent mechanism for administrative review of bidders’ appeals;  conducting a functional review for the two central procurement departments;  preparing a baseline study on SME access to public procurement;  preparing e-GP implementation strategy;  preparing of a national procurement capacity building and professionalization strategy; and  establishing a framework for performance monitoring of the procurement system. Building on the progress made under the PFMI Project towards establishing a modern public procurement system, the ongoing Digital WB&G project is also financing the first phase of an e- GP system by supporting the development of priority functionalities and their implementation in selected high spending agencies. The Terms of Reference for the design, development and implementation of e-GP system have been drafted and the selection process is expected to be launched in FY22. A second phase of the ongoing PFMI Project is under preparation. The procurement component under the new phase will aim to improve procurement management through scaling up activities implemented by HCPPP and supported by the current PFMI Project (phase I) as well as supporting new interventions in priority areas. Activities under PFMI Project (Phase II) will include:  implementation of the Procurement Capacity Building and Professionalization Strategy;  enhancing procurement practices in specific sectors to achieve value for money and efficiency in public spending;  strengthening HCPPP capacity to perform its policymaking, monitoring, and oversight role; and  development of additional sector-specific standard bidding document (SBDs). Improving COVID-19 vaccine procurement capacity through HEIS and the Framework In 2020, the World Bank’s Board approved a $12 billion Additional Financing to the COVID-19 Health Multiphase Programmatic Approach for Borrowers to finance the purchase and distribution of vaccines, tests and treatments through multilateral efforts and close collaboration with WHO, COVID-19 Vaccines Global Access (COVAX), African Vaccine Acquisition Trust (AVAT) and UNICEF, among others, to equitably distribute vaccines and help protect the most at-risk groups globally. Management is coordinating closely with UNICEF and Pan American Health Organization in their capacity as procurement agents for COVAX Advance Market Commitment (AMC) countries to help operationalize contracts that COVAX has agreed with manufacturers. Management has also worked closely with the African Union, the African Exim Bank, UNICEF and Johnson and Johnson to help develop and launch the AVAT (African Vaccines Acquisition Trust) initiative to complement COVAX and other vaccines supply options for Africa Region, Middle east North Africa Region and Caribbean Community (CARICOM) countries. As part of its support to countries, Management has undertaken extensive global vaccine market research in critical areas such as commercial, contractual, regulatory, liability, cold chain, and logistics to support countries’ decision making on how to approach this unique, highly complex marketplace. 51 Management’s efforts includes outreach to the major vaccine manufacturers to better understand key issues, explain Bank policy including red lines on the application of the Anti-corruption Guidelines and requirement to publish basic contract information, and help Borrowers address complex contractual issues with HEIS, as requested. C. E-Government Procurement Bangladesh Technology driven public procurement reform through e-GP. Pursuant to the vision of the Bank’s Procurement Policy to support Borrowers to achieve value for money with integrity in delivering sustainable development, the Bank supports Borrower countries in enhancing and implementing sound procurement systems and institutions. The Bank is supporting the third phase of procurement reform in Bangladesh and includes as a core component the further enhancement and full implementation of an end-to-end e-GP system. The Bank is also supporting other important areas to enable holistic and sustainable procurement reform. These include enhancing the authority and autonomy of the public procurement regulatory agency, extensive institutionalized capacity development program, data-driven procurement and contract implementation monitoring, citizens engagement, strengthening emergency procurement procedures, and developing a Sustainable Procurement for Green, Resilient, and Inclusive procurement including increasing participation of women-led businesses and SMEs, which have been worst affected by the pandemic. The Project has supported bringing the entire procurement process online through e-GP, 70% of public procurement expenditure ($17.5 billion) in FY21 was processed through e-GP, compared to 2% at the start of Project in FY17. Some of the notable impacts of e-GP so far include:  With the total Project cost of $70 million, the Bangladesh e-GP system has already contributed an average annual savings of $1.1 billion.  During the COVID-19 pandemic, the e-GP system, which operates 24x7, served as a backbone for ensuring uninterrupted delivery of services by enabling over 1,300 public sector organizations to process all procurement activities online, with 32,000 procurement contracts processed during the lockdown in March to June 2020 compared to 37,000 procurement contracts processed during the same period in 2019.  Improved market access and competitive bidding environment compared to manual bidding - number of registered bidders increased to 90763 on June 2021, from 23,000 on Dec 2017.  Average procurement time decreased to 57 days in FY21, from 100 days in FY17.  Increased transparency by publication of 100 percent of procurement notices and contract award information.  Increased transparency and public trust with the launching of Citizen Portal (https://citizen.cptu.gov.bd) which provides public access to nation-wide procurement and contract implementation data with facilities for data analytics. 52 Comoros The Government of the Union of Comoros has identified the development of the digital economy as a key sector in its "Accelerated Growth and Sustainable Development Strategy" (SCADD) document. As part of its e-GP program, the Government has therefore identified the Digitalization of public procurement as one of the key components among other projects to be implemented. In this regard, the World Bank has provided technical support for the preparatory works for the acquisition and for the acquisition of the e-GP System. Under Comoros RCIP project, Project Implementation Unit has already issued tender for procurement of e-GP solution and the selection process is now underway. Madagascar The current public procurement information system in Madagascar called SIGMP was introduced in 2010 and is scheduled for replacement. The Government began conceptualizing and developing seven modules of an e-GP. To assist the Government, the World Bank has been supporting Madagascar for the development of e-GP Strategy and a roadmap for the implementation under the PRODIGY project. A roadmap for the implementation of e-GP until the end of the year 2025 years has established taking into account the work that was underway for e-GP. Funded by the World Bank, PRODIGY is a project that focuses on e-government initiatives, the modernization of identification systems and the digital transformation of the public sector in order to bring efficiency in public services for the satisfaction of Malagasy citizens. In this context, PRODIGY is funding this current mission and has set aside funds for subsequent e-GP initiatives. Mozambique Mozambique is modernizing its procurement system. The e-GP was created as a module of the e- SISTAFE /MEP. The Bank has extended its support for the development of an e-GP strategy and to prepare a project roadmap. As the e-GP project unfolds, it is common that the e-GP strategy becomes "the public procurement strategy" as in the future all procurement is intended to be electronic. It is therefore natural that the purpose of e-GP coincides with the purpose of public procurement as a whole. The Government of Mozambique (GoM) has decided to follow the “Ownership and management of the e-GP System by the GoM and implementation through in- house technological development� modality and will follow the international standards and best practices of e-GP implementation to develop and implement an end-to-end e-GP System in the country. Nigeria The Bank engaged with federal and state governments to drive forward e-GP by utilizing existing funds in the Fiscal Governance and Institution Project (FGIP) and States Fiscal Transparency, Accountability and Sustainability Project (SFTAS). The drive has been enabled by fully capitalizing on Federal and State champions, massive networking, lobbying and strategic communication, and leveraging the PforR approach which incentivizes improvement in Governance. As a result, Kaduna State, after signing a Framework Agreement, has successfully launched its first cloud-based (SaaS) e-GP system in January 2021. In a yet another achievement, fifteen other States have signed Call-up Contracts following the same system, to be rolled out soon, 53 thus averting the critical risk of either not onboarding at all or adopting completely different systems and technologies. At the Federal level, having contracted the same Service Provider, the e-GP system will be launched by June 2021 in six high spending Ministries representing 20% of its annual budget ($29 billion and $27 billion in 2020 and 2021 respectively). Tanzania Public Procurement Regulatory Authority (PPRA) is managing the Tanzanian National e-GP System (TANePS) which is web-based, collaborative system, developed in accordance with the requirement of public procurement laws, to facilitate public procurement processes in Tanzania. TANePS supports processes of procuring Goods, Works, Consultancy, Non-Consultancy and Disposal of assets. The system supports various public procurement procedures including user registration, tender notification, tender preparation and submission, online tender evaluation, contract awarding, creation and management of catalogue, creation and management of framework agreements and auctions and payments. The Bank conducted a number of assessments to guide PPRA to make TANePS compliant to the requirements of the Multilateral Development Banks (MDBs). At present 49 implementing agencies under 20 Bank funded projects are using the TANePS following national procurement procedures. Use of e-GP has already shown indication of improved access to public procurement information, improved efficiency in procurement processing, reduction in implementation delays in the Bank portfolio. The e-GP system has also provided electronic interface to private sector bidders during the COVID-19 pandemic to submit their bids and to the procuring entities to process numerous day-to-day procurement activities. Zambia Zambia Public Procurement Authority (ZPPA) is progressively introducing an end-to-end e-GP system with the support of the Bank. Zambia e-GP system supports the procurement of goods, works and services using all procurement processes, and practices through all procurement methods leveraging the innovative functions and tools to address all aspects of good governance through electronic environment. The e-GP system has already been rolled out in thirty high spending implementing agencies and is in the process of country-wide roll-out. In December 2021, the Public Procurement Act was amended to make use of e-GP mandatory. The Bank funded projects in 23 implementing agencies are contributing towards operations and maintenance of the e-GP system. The Bank has also provided technical and financial assistance to develop and operationalize an online e-GP training platform to train the procurement practitioners and bidding communities. In addition, the Bank has recently introduced a project to facilitate full roll-out of the e-GP system across the country covering capacity development program, strategic communication and behavioral change program, data analytics to improve procurement performance, enhance internet penetration etc. The rollout of e-GP has already shown indication of improved access to public procurement information, improved efficiency in procurement processing, reduction in implementation delays, improved data collection and data analytics. The e-GP system has also provided electronic interface to private sector bidders during the COVID-19 pandemic to submit their bids and to the procuring entities to process numerous day-to-day procurement activities. 54 D. Gender/Small Medium Enterprise participation Colombia At the Borrowers’ request, the Bank included a gender criterion in project contracts to promote the participation of women in civil works which offered equal pay for men and women. The contract requirements were written to include compliance indicators for hiring females and penalties, including fines, if the goals were not met. All contractors met these stipulations. On the construction sites women initially made up only 4% to 6% of workers. Following support from the Bank, their presence practically tripled to reach 15% of the work force. Officials drafted a labor code with a gender perspective and designed an awareness campaign to prevent sexual abuse and harassment. These are now a reference for replication in other projects. Women who were hired have given positive feedback on the initiative. Morocco The Bank’s public procurement reform portfolio in Morocco focuses on data and data transparency and facilitating SMEs in the public procurement market. These two issues are the top priorities for Government of Morocco (GoM), as evidenced by the GoM’s recently issued New Development Model (NDM) that outlines Morocco’s economic strategy and includes public procurement as a key priority. In the past five years, the Bank’s Morocco procurement team has utilized a range of instruments to support the GoM to strengthen public procurement in Morocco. Achievements include:  the establishment of a Procurement Observatory and the piloting of Open Data Contracting Standard (DLI, GovTech ENNAJAA PforR);  the public disclosure of contracts awarded to SMEs, mandatory electronic submission of bids, and the introduction of electronic bid security features (Prior Actions, Digital and Financial Inclusion DPFs). In FY20, the Advisory Services and Analytics (ASA) supported the development of a strategy for a Supplier Development Program aimed at building the capacity of SMEs, self-employed, and cooperatives to compete for government contracts; and the assessment of Morocco’s e-GP system. Senegal The law on promotion of SMEs in Senegal reserves certain public contracts to SME, but those owned by women have not fully taken advantage of this opportunity. Within the WeFi initiative, the Bank is supporting the efforts in addressing the challenge through an ongoing ASA which main outcomes are:  concrete provisions in the national procurement regulations/manuals needed for enforcement of gender affirmative procurement legislation;  capacity building and women-owned SMEs' awareness of and skills to utilize public procurement opportunities; and 55  capacity of civil servants to implement procurement regulations as well as to engage and support women-owned SMEs applying for public procurement bids. This effort is further supported by a prior action in the Development Policy Operation (DPO) regarding the inclusion in the Senegal procurement law of a provision regarding preferences for women-led business and a guide note on how women can access public contracts. E. Sustainable Public Procurement Argentina An estimated 145,000 Argentinian households lack access to electricity service. To provide these rural communities with electric energy, the government was looking for a “plug and play� technology, as to say, easy-installation solar panels kits for its implementation. These kinds of kits can permit targeted households to have access and significantly eliminate the expense associated with batteries, candles, and kerosene. The main implementation challenge was to find a supplier who could deliver at the optimum price and meet the technical requirements, without knowing exactly how many units to purchase and where and when to install them. The Framework Agreement provided a market-led solution, attracting the best vendors (the offer) and addressing many beneficiaries’ needs spread in remote rural areas (the demand). Since the terms were already spelt out in this umbrella agreement, the government of Argentina benefited from economies of scale, gained access to the latest technology in the market, and reduced administrative work when awarding contracts. This project is expected to help approximately 193,000 people in Argentina’s remote rural areas, providing them with an essential service to build up resilience in these COVID- 19 challenging times. Grenada In 2018, a public procurement spend analysis led by the Bank’s Procurement team in Grenada showed that government could play a major part in reducing water and land pollution because more sustainable goods last longer, consume fewer resources, and cause less harm to the environment. This analysis helped spark the government’s drafting of a new policy on sustainable public procurement, enacted in 2020. The policy identifies which categories of products are to be included under this new sustainable approach. Grenada has continued developing a more robust public procurement process by introducing sustainability criteria for its purchases that include giving priority to purchasing from local industry, nurturing sustainable development, and championing protection of the environment. Poland The Institute of Meteorology and Water Management in Poland is procuring high performance computers with modernization of the buildings for their location by installing solar power supply systems and decided to use smart design including the capital and operational cost in the evaluation of bid. To evaluate and compare bids the Institute included the following life cycle cost criteria: 56  Cost of service support for computer clusters at a level analogous to the warranty from the manufacturer of the delivered devices for 2 years after the end of the warranty. As part of this support, the Purchaser will have the right to: replacement / repair of the device in case of failure; firmware and system software updates, etc. of delivered devices.  Savings calculated by multiplying the amount of kWh that the proposed photovoltaic installation is capable of producing during 10 years by the unit price of kWh. To ensure that the Institute receives the promised benefits, the contract will define the functional guarantees and liquidated damages to be applied if the promised parameters are not met. The contract shall be awarded to the proposal with the highest score and therefore the Most Advantageous Proposal with the best overall VFM. This will allow reducing future costs and risks. Turkey Energy Saving Company (ESCO) Contracts under Energy Efficiency in Public Buildings (EEPB) Project. One of the objectives of the EEPB Project is to develop the local ESCO market so eventually the vast public building stock could be renovated using revolving schemes and more commercial financing resources. Under traditional building renovation contracts, the offer with the lowest evaluated cost which is substantially responsive to the bidding document is awarded. Also, the financing is provided by the investors. In order to achieve economy, sustainability, efficiency and develop a market, under ESCO contracts, rather than lowest evaluated cost, a fit for purpose approach leading to sustainability through Net Present Value (NPV), Minimum Energy Saving per Year and Simple Payback Period as three parameters to be considered in the evaluation of the bids. In ESCO contracts, the highest NPV value, where any offer with negative NPV is rejected, is the parameter of which the decision is given. Minimum energy saving and simple payback period are values determined in the bidding documents to be achieved. The first ESCO contract was signed on March 2021 and will be completed in January 2022. Upon completion, after confirmation of success with studies by means of energy consumption, the other bids will be launched. Improvements if needed will be done in the bidding documents to increase the participation. The aim is to develop an ESCO market with the support of procurement arrangements that capital cost of energy saving investments in buildings are paid by the ECSOs which will be repaid by the building owners upon confirmation of the energy saving. F. Project Procurement Strategy for Development Argentina In 2018, the World Bank approved a loan to support the dredging of parts of the Salado Integrated River Basin to mitigate the impact of recurring floods in the province of Buenos Aires in Argentina at an estimated contract value of US $290 million. Building on the Bank’s previous experience on this type of works, the assessment included a thorough market analysis, the availability of required equipment and lessons learned from previous stages. In addition, based on the contract execution of previous phases, in which there was no participation of foreign companies and the average number of bids was three, the technical capacity of international and local companies in terms of the availability of specific equipment (dredges) was studied in detail. This was intended to determine the maximum amount of work that each local company could qualify for and thus define 57 the lowest number of sections of work (lots) possible, to encourage competition at international level. The market approach chosen on the basis of the PPSD led to a sizeable increase in the average number of bids and of international bidders. The PPSD concluded that the most appropriate procurement approach was an open international competitive process through a Request for Bids divided into four lots (being this the optimal number that arose from the PPSD’s market analysis and the technical requirements). When the bidding process was carried out based on the parameters identified during the development of the strategy, eight bids were received, four of which were international. Five bidders offered discounts for the combined award of two or more lots. The robust competition and positive VfM outcomes emerged directly out of the contextual sensitivity of the bidding process made facilitated by the Borrower’s PPSD analysis. A comprehensive Management Plan with Key Performance Indicators (KPIs) were developed, critical milestones requiring intervention of third parties identified, and a dashboard developed based on the PPSD. The PPSD analysis facilitated consensus on KPIs between the parties, enabling effective contract management over the course of the project. Azerbaijan To have a better understanding of the market, foster competition, increase bidder awareness, a detailed PPSD market study was conducted. The PPSD market study approach included reaching out, identifying potential local vendors, collecting feedbacks, and raising awareness. These were done through desk review, market knocking, written questionaries and meetings (virtual due to COVID-19 pandemic). Several vendors were identified in the fields of:  infrastructure modernization;  digitalization;  IT equipment; and  software development. Bidders discussed and identified challenges that may prevent them from participating in the procurement. The Borrower’s PPSD was revised to incorporate the findings. The PPSD’s enhanced market Assessment showed that local market is well suited, not only to participate in the competition for most types of IT requirements, but also to complement any bidder, including international ones for after sales services. Pakistan Using Market Outreach Event for Complex Enterprise Solutions to Socialize Transparency and Enhance Participation in High-end Procurement Approach. With the Framework, the Bank has ushered into an era of strategizing the procurement to meet project outcomes by undertaking a PPSD. Under the Higher Education Development Project (P161386) in Pakistan, there were two key high value procurements to select suites of software solutions for the automation of universities, viz.: Student Lifecycle Solution, Faculty Lifecycle Solution and their associated ERP (core financials, HR, Procurement, Inventory, etc.). These procurements would involve various systems and solutions for university management. 58 HEC with technical support successfully organized a market outreach event on November 10, 2020. The main idea of the event was active engagement of market before the actual procurement activity to have feedback on the broad outline of functional and technical requirements and pre- disclosure on the eligibility, qualification and experience criteria so as to enhance transparency and wider participation. An Overview Document for the Market Outreach event was prepared which broadly explained the technical requirements, bidding process and expectation of qualification and was made accessible to all concerned and interested parties/firms. The event was well structured with an overview of technical and commercial requirements explained by HEC and finally representatives of the firms given the opportunity to discuss queries and exchange views on the process of acquiring the large enterprise solutions. The event was conducted virtually via video link due to COVID-19 Pandemic. The participants included implementation/procurement team of HEC and representatives of about 35 firms which are potential suppliers of these procurements. Additionally, the information of the event was disseminated through electronic as well as print media on various forums to ensure that all interested parties are given the opportunity to exchange the ideas with HEC on the large enterprise solution planned to be acquired. Impact of Market Sounding as part of the PPSD, notwithstanding a highly complex procurement which generally attracts fewer bidders, presumably a full disclosure through market outreach has resulted in a very encouraging response for each of six lots with 10-14 applicants on each lot for Initial Selection. The IS has been completed and the procurement has proceeded to next stage where issuance of RfP will be followed by discovery meetings overseen by probity assurance provider. The success of HEC did not stop -- a similar outreach event, for the Khyber Pakhtunkhwa Workflow and Document Management System, was held on June 25, 2021. The ensuing procurement saw over 14 bidders submitting their bids. G. Hands-on Expanded Implementation Support Afghanistan The Afghanistan Sehatmandi Project provides an illustrative example of the ways in which the Framework’s features improve procurement outcomes in FCV situations. The Sehatmandi Project was a grant of $600 million aimed at providing “Basic Package for Health Services� (BPHS) and “Essential Package of Hospital Services� (EPHS). A capacity assessment by Bank staff found gaps in implementing agency capacity, including a lack of familiarity with recent changes in Consultant Services Selection Procedures. The PPSD revealed further risks, indicating serious concerns about the ability of the designated implementing agency to ensure effective procurement. HEIS empowered Bank staff to work with Borrower officials to ensure effective procurement in the face of FCV risks. The use of HEIS enabled the project to proceed without the delays and costs that are common in situations of low procurement capacity. HEIS to the Ministry of Public Health, brought clarity to evaluation and selection processes, while leaving decision making to the implementing agency. In addition, awareness of the Bank’s support bolstered their confidence among bidders, thus increasing their participation in the procurement process: average of eight bids were received for all non-consulting contracts. 59 Nepal HEIS has been adopted first time in Nepal for use in Nepal COVID-19 Emergency Response and Health Systems Preparedness Project in March 2020. The Project is the Emergency COVID-19 Project in Nepal and was the first emergency project in the country. HEIS commenced at preparation stage and continues through implementation of parent project, first and second additional financing of the emergency health sector project of USD122 million. As an emergency operation, the support was focused on helping the Ministry of Health and Population to prepare the PPSD, accept and process BFP to secure scarce medical equipment supplies and Personal Protection Equipment (PPE) as well as to select United Nations Agencies like United Nations Office for Project Services (UNOPS) and United Nations Children’s Fund (UNICEF) for ensuring the supplies. This active engagement under the HEIS arrangement contributed to exceptionally efficient procurement implementation of almost all activities of the project. As a measure of success, Nepal is first signatory for COVID-19 Moderna vaccines supplies agreement with GAVI/UNICEF under COVAX cost sharing arrangement, and all the major project procurement including agreements with UNOPS and UNICEF are efficiently implemented and closed. Learning from the experience of Nepal COVID-19 project, HEIS may play a pivotal role and is a suitable approach to support efficient project preparation and implementation of large/ complex/ emergency nature projects in Nepal. Papua New Guinea HEIS was first approved for use in a Bank financed project in the Pacific in October 2016. The Project was the Emergency Tuberculosis Project in Papua New Guinea (PNG). As an emergency operation, the support was focused on helping the client to select the consultants for the Project Management Unit (PMU) as well as select United Nations Agencies. The Bank’s active engagement under the HEIS arrangement contributed to significantly improved procurement outcomes. The procurement process was much quicker than that observed in another project in PNG. As a measure of success, PNG are now implementing four (4) Bank financed projects including three (3) emergency operations (Tuberculosis, COVID 19 Emergency Response, and Additional Financing for the COVID 19 Emergency Response). In addition, they are implementing another project Improving Access to and Value from Health Services in PNG: Financing the Frontlines. The Covid 19 Emergency Response Project also used the HEIS under which the contract with UNOPS was signed three weeks ahead of the planned schedule. In PNG, there is an energy sector project where HEIS was requested and has been agreed. Another two (2) projects currently under preparation have also requested HEIS. 60 H. Alternative Procurement Arrangements Kiribati Water supply issues in Kiribati are among the most complex in the world. To address the issues related to water and sanitation sector, the World Bank and the ADB agreed to provide a joint financing to support the Kiribati South Tarawa Water Supply Project. In response to the request from the Government of Kiribati for a joint operation, rather than two separate projects, to develop a harmonized approach, both during preparation and throughout implementation is critical to minimize duplication, transaction costs and complexity, which are essential in a context of limited implementation capacity and market conditions. An APA has been agreed to allow the project to follow the ADB’s procurement procedures. In another Bank and ADB co-financed project in Kiribati, Kiribati Outer Islands Transport Infrastructure Investment Project, a similar APA arrangement was agreed and is being implemented which allows the project to follow the Bank’s procurement procedures. 61