Document of The World Bank Report No: ICR00001000 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-40820 TF-54017 TF-55170 TF094801) ON A CREDIT IN THE AMOUNT OF SDR 5.7 MILLION (US$ 8.54 MILLION EQUIVALENT) TO THE REPUBLIC OF MOLDOVA FOR THE PUBLIC FINANCIAL MANAGEMENT TECHNICAL ASSISTANCE PROJECT MAY 29, 2014 Poverty Reduction and Economic Management Belarus, Moldova and Ukraine Country Management Unit Europe and Central Asia CURRENCY EQUIVALENTS (Exchange Rate Effective May 29, 2014) Currency Unit = Moldovan Leu (MDL) MDL 13.77 = US$1 US$0.073 = MDL 1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ACC Accounting IMF International Monetary Fund APA Academy of Public Administration ISPPIA International Standards for the Professional BES Budget Execution System Practice of Internal Auditing BPS Budget Preparation System IT Information Technology BW Business Warehouse MDM Master Data Management CAS Country Assistance Strategy M&E Monitoring and Evaluation CD Country Director MOF Ministry of Finance CFAA Country Financial Accountability Assessment MTEF Medium Term Expenditure Framework CIS Commonwealth of Independent States NIAS National Internal Audit Standards CLF Cash Liquidity Forecasting NGO Non-Government Organization CMU Country Management Unit PDO Project Development Objective CNFD Collection of Non-Financial Data PEFA Public Expenditure and Financial COM Council of Ministers Accountability Assessment CPAR Country Procurement Assessment Report PEMR Public Economic Management Review CQ Consultant Qualification PEO Project Executive Officer DFC Department of Financial Control and Revision PER Public Expenditure Review DFID Department For International Development, PFM Public Financial Management United Kingdom PFMTA Public Financial Management Technical ECA Europe and Central Asia Assistance Project EGPRSP Economic Growth and Poverty Reduction PHRD Policy and Human Resources Development Strategic Paper PIFC Public Internal Financial Control FMIS Financial Management Information System PIU Project Implementation Unit FCD Internal auditors in central IA Unit PO Project Office FM Financial Management PRSP Poverty Reduction Strategy Paper FMR Financial Monitoring Report QALP Quality Assessment of the Lending Portfolio ESW Economic and Sector Work QCBS Quality and Cost Based Selection EU European Union QER Quality Enhancement Review GFS Government Financial Statistics SIDA Swedish International Development GOM Government of Moldova Cooperation IA Internal Audit SDP Standard Disbursement Percentage ICB International Competitive Bidding TA Technical Assistance ICT Information and Communication Technology TTL Task Team Leader IDA International Development Agency TF Trust Fund IFMIS Integrated Financial Management Information UNDP United Nations Development Programme System USAID United States Agency for International IIA Institute of Internal Auditors Development Vice President: Laura Tuck Country Director: Qimiao Fan Sector Manager: Adrian Fozzard Task Team Leader: Shilpa Pradhan ICR Team Leader: Ivor Beazley CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph   1  Project Context, Development Objectives and Design 1  1.1  Context at Appraisal...................................................................................................................... 1  1.2  Original Project Development Objectives (PDO) and Key Indicators.......................................... 2  1.3  Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification. ................................................................................................................... 5  1.4  Original Components .................................................................................................................... 5  1.5  Revised Components..................................................................................................................... 5  1.6  Other Significant Changes ............................................................................................................ 5  1.7  Original Policy Areas Supported by the Program ......................................................................... 6  1.8  Revised Policy Areas .................................................................................................................... 6  1.9  Main Beneficiaries ........................................................................................................................ 6  2  Key Factors Affecting Implementation and Outcomes 7  2.1  Project Preparation, Design and Quality at Entry ......................................................................... 7  1. 2.2...........................................................................................................................  Implementation ..................................................................................................................................................... 8  2.3  Monitoring and Evaluation ......................................................................................................... 13  3  Assessment of Outcomes 14  3.1  Relevance of Objectives, Design and Implementation ............................................................... 14  3.2  Achievement of Program Development Objectives .................................................................... 14  3.3  Efficiency .................................................................................................................................... 15  3.4  Justification of Overall Outcome Rating..................................................................................... 16  3.5  Overarching Themes, Other Outcomes and Impacts .................................................................. 16  3.6  Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ........................... 17  4  Assessment of Risk to Development Outcomes 17  4.1  Bank Performance ....................................................................................................................... 17  4.2  Borrower Performance ................................................................................................................ 19  5  Lessons Learned 20  6  Comments on Issues Raised by Borrower/Implementing Agencies/Partners 22  Annex 1: Project Cost and Financing 23  Annex 2: Bank Lending and Implementation Support/Supervision Processes 24 Annex 3: Beneficiary Survey Results 26 Annex 4: Stakeholder Workshop Report and Results 27  Annex 5: List of Supporting Documents 28 Annex 6: Country Map 29  iv DATA SHEET Moldova Public Financial Management Project (P082916) EUROPE AND CENTRAL ASIA ECSP4 A. Basic Information Public Financial Country: Moldova Project Name: Management Technical Assistance Project IDA-40820,TF-54017, Project ID: P082916 L/C/TF Number(s): TF-55170,TF-94801 ICR Date: 05/29/2014 ICR Type: Core ICR REPUBLIC OF Lending Instrument: TAL Borrower: MOLDOVA Original Total 15.3M Disbursed Amount: 12.4M Commitment: Revised Amount: 12.4M Environmental Category: C Implementing Agencies: Ministry of Finance Academy of Public Administration Cofinanciers and Other External Partners: International Development Association (IDA) NETHERLANDS: Min. of Foreign Affairs / Min. of Dev. Coop. SWEDEN: Swedish Intl. Dev. Cooperation Agency (SIDA) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 03/04/2004 Effectiveness: 01/03/2006 03/06/2009 12/28/2011 Appraisal: 03/29/2005 Restructuring(s): 12/28/2012 12/16/2013 Approval: 06/16/2005 Mid-term Review: 10/24/2008 2/12/2009 Closing: 06/30/2010 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Moderately Quality of Supervision: Moderately Satisfactory Agency/Agencies: Unsatisfactory Overall Bank Overall Borrower Moderately Moderately Satisfactory Performance: Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No Satisfactory at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision Yes Satisfactory (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 80 80 Sub-national government administration 20 20 Theme Code (as % of total Bank financing) Other public sector governance 33 33 Public expenditure, financial management and procurement 67 67 vi E. Bank Staff Positions At ICR At Approval Vice President: Laura Tuck Shigeo Katsu Country Director: Qimiao Fan Paul G. Bermingham Sector Manager: Adrian Fozzard Deborah L. Wetzel Project Team Leader: Shilpa B. Pradhan Svetlana I. Proskurovska ICR Team Leader: John Ivor Beazley ICR Primary Author: David Waigwa Wachira John Ivor Beazley F. Results Framework Analysis vii Project Development Objectives (from Project Appraisal Document) Achieving effective and transparent management of public finances as an enabler of poverty reducing programs and economic growth. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Indicator Baseline Value Original Target Formally Actual Value Achieved Values (from Revised Target at Completion or approval documents) Values Target Years Indicator 1 : Overall fiscal discipline and strategic resource allocation improved (as evidenced by the IMF monitoring assessment) Value The actual fiscal surplus Fiscal deficit is less The fiscal deficit for quantitative or during 2003-05 is better than than 0.5% of GDP by 2013 was 1.8 percent of Qualitative) in the approved budgets. The the end of 2008. No GDP. expenditure arrears in 2005 at new arrears 3% of total expenditure. accumulated, existing The arrears problem Budget has little correlation arrears fully cleared. was addressed. with EGPRSP priorities. Medium-term and annual sector allocations tend to match government policy priorities. Date achieved 01/03/2006 12/31/2009 12/31/2013 Comments Partially achieved. The medium term budget process has delivered closer alignment of the (incl. % budget with policy priorities and largely eliminated extra budgetary funding. The arrears achievement) problem appears to have been addressed but the project did not systematically monitor this. The fiscal deficit indicator is too high level for the result to be directly attributable to the project. Indicator 2 : Public financial decisions in the government entities follow sound PEM rules and procedures and are based on reliable financial data Value Limited availability of timely Useful and timely data Budget classification quantitative or and accurate expenditure data. generated by FMIS and chart of accounts Qualitative) Budget classification and chart system. Budget have been modernized. of accounts incompatible with classification and The budget preparation international standards. chart of accounts module of the system Decision-makers lack aligned with GFS was accepted but it is consolidated financial data 2001. Decision- not being utilized. makers have access to reliable financial data Date achieved 01/03/2006 12/31/2009 12/31/2013 Comments Partially achieved. The budget classification and Chart of Accounts have been developed and (incl. % aligned to GFS 2001 (Order of the Minister No.40 of 02 April 2012). If the Government decides achievement) to complete project implementation using other resources, the FMIS could generate the reports at a later date. Indicator 3 : Public managers of central government bodies issue annual reports on the use of public funds and achievement of policy outcomes Value Financial reporting is divorced All budget entities Budget now includes quantitative or from accountability for results. produce mandatory results information. Qualitative) Performance reports are not performance reports Budget execution mandatory and not linking resources reports do not yet standardized. spent with the include performance outputs/outcomes information. vii achieved. Date achieved 01/03/2006 12/31/2009 12/31/2013 Comments Partially achieved. Program based budgeting methodology was improved and the scope of (incl. % program based budgeting was extended, so that the draft state budget for 2014 is fully based on achievement) programs. However the FMIS budget execution module was not implemented. Indicator 4 : Expenditure management improved through a Treasury Single Account Value National budget is executed Aggregate control of TSA system is fully quantitative or through separate and non- spending and operational since March Qualitative) consolidated accounts for commitment is 2007. central government, local established. authorities and social spending Predictability of cash agencies release by Treasury improved. Date achieved 01/03/2006 06/30/2010 05/31/2013 Comments Partially achieved. Although the Budget Execution module of the FMIS was dropped the TSA (incl. % functionality is managed through existing treasury system. Predictability of funding has not been achievement) systematically monitored. (b) Intermediate Outcome Indicator(s) Indicator Baseline Value Original Target Formally Actual Value Values (from Revised Target Achieved at approval documents) Values Completion or Target Years Indicator 1 : Medium-term budgeting provides a credible framework for annual budget planning Value Medium-term budget planning MTEF incorporates Target indicator for (quantitative is limited to three sectors. 80% of sector MTEF coverage was or Qualitative) expenditure. New achieve by 2011 (84% Budget classification of sector expenditure). and chart of accounts replaced fragmented system. FMIS uses new budget classification and chart of accounts. Date achieved 01/03/2006 01/01/2009 12/31/2012 Comments Fully achieved. The MTEF is established as part of the budget process. (incl. % achievement) Indicator 2 : FMIS system implemented Value Fragmented IT solutions FMIS operating to the A partial FMIS system (quantitative support budget execution designed specification was delivered in 2013 or Qualitative) management. A concept of a for budget planning (44% of the future FMIS designed. and execution by functionality) which Telecommunication network January 2009. will include the BPS, for FMIS absent. BES Master Data Management (MDM), CNFD, CLF, and some of the Business Warehouse (BW) modules are implemented. Date achieved 03/01/2006 01/01/2009 12/31/2013 Comments Partially achieved. Only the Budget preparation module has been accepted from the core ix (incl. % software modules. Business warehouse and other features that were “accepted” are not being achievement) utilized in the absence of an integrated solution. All transaction processing is still being handled by the old system. Indicator 3 : Internal Audit methodology, standards and legislation developed and approved. Internal Audits are conducted according to the new, EU compliant, legal framework. Value Internal financial control and The legal basis, All targets achieved. (quantitative revision service under MoF standards and or Qualitative) conducts compliance audits. A methodology for concept of a modern internal internal audit are audit is being designed. 25 implemented. Internal internal auditors received audit units established initial awareness training. in ministries and selected rayons. Date achieved 06/01/2006 12/31/2009 12/31/2013 Comments Fully achieved. Moldova has made a successful transition from a soviet model inspection (incl. % system to a modern internal audit, consistent with the EU PIFC concept. The IA function has achievement) benefitted from EU financed twinning with Sweden and the Netherlands. Indicator 4 : The Academy of Public Administration has capacity to train civil servants in conducting training in 18 new training modules. Value The Academy has established The Academy delivers The project supported (quantitative a Department of Civil Service training to civil the development of 12 or Qualitative) Training. The first training in servants in 18 training modules, which the management and internal modules related to Academy is delivering control systems is being organizational and on a regular basis. prepared, local trainers are financial management. FMIS related training trained for this module. Trainers trained. will be scaled down to BPS and basic ICT support needs. Date achieved 06/01/2006 12/31/2009 12/31/2013 Comments Partly achieved. Training courses were successfully established with the exception of the FMIS (incl. % training, which was substantially scaled down. achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 06/26/2006 Satisfactory Satisfactory 0.00 2 04/09/2007 Satisfactory Moderately Satisfactory 0.25 3 12/06/2007 Satisfactory Moderately Satisfactory 0.00 4 06/24/2008 Satisfactory Moderately Satisfactory 0.00 5 03/17/2009 Satisfactory Moderately Satisfactory 0.00 6 12/30/2009 Satisfactory Moderately Satisfactory 0.00 7 06/23/2010 Satisfactory Moderately Satisfactory 0.83 8 02/28/2011 Satisfactory Moderately Satisfactory 1.04 9 09/21/2011 Moderately Unsatisfactory Moderately Unsatisfactory 1.11 10 05/19/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.19 11 12/01/2012 Moderately Unsatisfactory Moderately Satisfactory 1.22 12 05/27/2013 Unsatisfactory Unsatisfactory 1.28 13 12/30/2013 Moderately Unsatisfactory Moderately Unsatisfactory 1.28 x H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions Project was extended following delays in procurement of the FMIS 3/6/2009 N S MS 0.0 system (Not considered restructuring under procedures at time of extension). Project was extended because the implementation and testing of FMIS could not be completed by 12/28/2011 N MU MU 1.19 the end of 2011 (Not considered restructuring under procedures at time of extension). The project was extended to accommodate delays in the implementation of the FMIS activities, due to difficulties in 12/28/2012 N MU MS 1.22 defining the scope of the remaining work, reaching agreement on a revised implementation schedule and the contract amendment. Part of credit was cancelled due to 12/16/2013 N U U 3.3 the reduction in the scope of the FMIS as finally delivered. xi I. Disbursement Profile xii 1 Project Context, Development Objectives and Design 1.1 Context at Appraisal At the time of project approval in 2005, after twelve years of transition, Moldova remained one of the poorest countries of the Europe and Central Asia (ECA) region with a fragile economy, partially implemented market reforms, a high level of debt and growing inflation. Entrenched corruption had become a serious obstacle to economic growth and economic restructuring. Little progress had been made in modernizing the public administration or improving transparency and accountability. Notwithstanding the efforts of the Ministry of Finance (MOF) to maintain sound fiscal position, a collapse of government revenues between 1997 and 2001 resulted in ad hoc expenditure adjustments with a severe impact on the social sectors. The 2002 Country Assistance Strategy (CAS) update argued that public expenditure reforms were essential for poverty reduction, growth and stability. In spite of ambitious budget management reforms promoted by the MOF the efficiency of public budget management remained low, with particular concerns related to the Government’s ability to allocate resources based on strategic priorities and to ensure that public spending was efficient and cost-effective During 2002-2003, the Bank completed a comprehensive analysis of Moldova’s public financial management and fiduciary practices. The Country Financial Accountability Assessment (CFAA) concluded that the financial accountability framework was weak, particularly in the audit and controls systems across the government. The Country Procurement Assessment Report (CPAR) highlighted the impact of poor procurement on the quality of public services. In 2003 the MOF developed a “Strategy for Strengthening the Management of Public Finances”. This aimed to improve the process of budget preparation and execution, increase budget transparency and strengthen the financial control framework. The strategy built on the successes of the earlier reforms that included the establishment of the State Treasury, introduction of elements of computerized budget execution system, gradual introduction of a Medium Term Expenditure Framework (MTEF), and piloting of program budgeting in social protection, health and education sectors. In May 2004 Moldova adopted the Economic Growth and Poverty Reduction Strategy (EGPRSP)1 which laid out three basic priorities: sustained and inclusive economic growth; reduction of poverty and inequality; and human resource development. To achieve stable conditions for continuing sustainable economic growth and poverty reduction, the Government planned to stabilize public debt (through improved debt management), tighten fiscal policies and ensure balanced budgeting. In the area of public financial management, the stated goal was “an efficient and sustainable financial management system, based on effective instruments and mechanisms, designed and operating to European standards”. 1 Approved by the Government in May 2004 and by the Parliament in December 2004. 1 Moldova already had the basic elements of a public financial management system in place: the institutional structure, including distinct Treasury and financial control functions; legislation; some automation of budget execution, and hardworking, reform minded personnel at the MOF. Reforms undertaken by the MOF laid the foundation for more strategic resource planning through a medium term expenditure framework. However, the budget preparation and execution methodologies needed improvement to better guide ministries. The budget classification did not meet the needs of program budgeting. The chart of accounts differed from the International Monetary Fund (IMF) GFS 2001 standards, and accounting and reporting needed streamlining to provide meaningful information on the state of the public finances. The Treasury was not able to resolve cash rationing issues, creating uncertainty for budget managers. The automation of management of public finance information was incomplete, difficulties in consolidation of financial statements for the whole of the government and a lot of manual work leading to errors and delays in information flows. The Government and the Parliament did not receive regular, timely information on budget management since the system could not automatically generate information on request. The internal control and internal audit systems did not meet international standards specified in International Standards for the Professional Practice of Internal Auditing. The national system of in-service training was not capable of providing relevant training in organizational and financial management, auditing, and strategic management. This constrained the possibilities to implement and sustain reforms leading to more effective and strategic management of public resources. The Public Financial Management project, financed by the International Development Association (IDA) and development partners, was designed to address these shortcomings. 1.2 Original Project Development Objectives (PDO) and Key Indicators The project’s development objective was to achieve effective and transparent management of public finances, as an enabler of poverty-reducing programs and economic growth. 2 Table 1. Results Framework   PDO Outcome Indicator Effective and transparent public 1. Overall fiscal discipline and strategic resource allocation improved (as financial management evidenced by the IMF monitoring and assessment) 2. Public financial decisions in the government entities follow sound PEM rules and procedures are based on reliable financial data 3. Public managers of central government bodies (ministries and big spending agencies) issue annual reports on the use of public funds and achievement of policy outcomes Intermediate Results One Results Indicators for Each Component per Component Component 1: Budget Planning (i) Consistency between the annual budget laws and approved medium- and Execution System term expenditure frameworks (existing deviations are fully explained   and determined by objective circumstances) Subcomponent 1.A. Increased (ii) MTEF incorporates a growing number of sector expenditure plans usefulness of the national public every year; at least 80% of allocations covered through MTEF sector budget as a strategic policy expenditure plans by the end of the project instrument resulting from improvements in budget (iii) Comprehensiveness of the annual budget laws coverage, further formulation methodology improvement in the degree of integration of off-budget activities into the annual budget laws presentation. No new off budget funds established over the project life outside the annual budget law framework. (iv) Improved reliability of approved budgets: The need for within the year budget revisions under normal (no shock) circumstances steadily reduced to maximum two technical revisions per year by the end of the project. Component 1: (i) Aggregate control of spending and commitment is established as   evidenced by the availability of functioning mechanisms for prevention of Subcomponent 1.B. Budget accumulation of new expenditure arrears, elimination of the existing stock execution methodology aligned of past expenditure arrears by the end of the project with international standards, ensuring compliance with the (ii) Availability of accurate, timely, complete, and consistent information adopted budget and enabling on budget execution to decision-makers, including the legislature operational efficiency is developed (iii) Predictability of cash release by the treasury improved, systematic cash rationing practices eliminated Internal controls to prevent irregularities are established in public institutions Component 1: (i) The functional design of the FMIS is coherent with the business   processes of budget planning, and FMIS execution is competed and Subcomponent 1.C. Financial implemented Management Information (ii) All central government entities, all rayon administrations and selected System established and functioning municipalities are equipped, have access, trained personnel and use FMIS for budget planning and budget execution (iii) The system records financial transactions and produces adequate financial information and is protected from IT fraud (iv ) Adequate number of staff is trained in the use of the new FMIS 3 Component 2: Internal Audit Adequately staffed with trained auditors and methodologically and A functionally independent internal technically equipped Internal Audit (IA) units are set in all ministries and audit mechanisms with adequate major spending agencies authority and scope is established and it is using internationally recognized auditing standards Component 2 (i) National Internal Audit Standards compatible with the   International IA standards (ISPPIA) developed Subcomponent 2.A. Normative and Legal Framework for (ii) A normative and methodological framework for internal control Internal Control and Audit is developed developed (iii) A Methodology and a Hand-book for internal auditors designed (iv) Basic IA legislation is developed and submitted for parliamentary adoption Component 2 (i) Job descriptions, competences and relevant training prepared   (ii) Internal auditors in central IA unit (FCD), Treasury and 2 pilot 2.B. Transformation of Financial ministries trained in IA methodology; Control into Internal Audit Function (in the Center of (iii) Transition to new IA practice performed in the FCD, Treasury and Government) completed two pilot ministries Component 2 (i) Internal Audit Units in line ministries and central agencies   established(budgeted), staffed and equipped Subcomponent 2.C. Expanding (ii) 5 Internal Audit trainers prepared; the Internal Audit System to Line Ministries and Sub- (iii) 60 Internal auditors in line ministries trained; national Entities implemented (iv) Central Coordination of IA established; (v) IA Resource Center established; (vi) Methodology and training program for special audits developed; (vii) Legislation on IA completed and adopted Component 3. Training capacity (i) Training management capacity developed at the civil service training and training. department at the Academy of Public Administration and its budgeting is Sustainable training capacity is resolved established for in-service training in (ii) Curriculum and training materials for training in areas related to the Academy of Public organizational management, budget management, internal audit, risk Administration and civil servants management, accounting, impact assessment developed trained (iii) Local trainers trained in new training disciplines (iv) Training conducted for ministries, central agencies, and local governments 4 4. Project Management, (i) Decisions on project implementation (or trouble- shooting) made in a Monitoring and Evaluation timely manner (ii) Day- to-day project management conducted (iii) Timely and proper execution of the procedures on resource management performed (iv) Coordination among components’ activities facilitated (v) Monitoring and reporting the achievement of project results conducted in a timely manner (vi) Information on the project contents and project achievements is provided to the highest decision- makers, international community and public at large 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and Reasons/Justification. Although the project was extended twice no changes were made to the development objectives, the outcome or results indicators. 1.4 Original Components The project had four major components and six sub-components: 1. Budget planning and execution (methodologies and information systems) a. Budget formulation methodology b. Budget execution methodology c. Financial management information system 2. Internal control and audit a. Normative and legal framework for internal control and audit b. Transformation of financial control (compliance) into internal audit c. Extending the internal audit system to line ministries 3. Financial management training capacity and training 4. Project management, monitoring and evaluation 1.5 Revised Components There were no changes to the scope of the project until the final project restructuring in December 2013 when the scope of the Financial Management Information System (FMIS) and related training activities was reduced. 1.6 Other Significant Changes The project underwent four restructurings (the first two carried out as extensions, with no formal restructuring). In March 2009 the closing date was extended from June 2010 to December 2011. In December 2011 the project was extended by a further year to December 2012, and finally in December 2012 the project was extended to December 31, 2013. The third project extension applied only to FMIS-related activities as all efforts were to be focused on the completion of 5 FMIS implementation and only the IDA credit was extended. As a result of the eventual reduction in scope of the FMIS, MOF requested a partial cancellation of the IDA credit, cancelling US$ 3,300,000 equivalent as of December 11, 2013. Other grants (trust funded [TF]) co-financing the project, TF055170 and TF054017, were closed on December 31, 2012. Most of the activities under these grants were successfully completed, delivering improved budget methodology, MTEF and internal audit improvements. Unused funds were returned to the donors (US$ 236,000 under TF055170 and US$ 354,000 under TF054017). The bidding process for the FMIS resulted in the selection of a solution costing substantially more than the estimated cost. In 2009, additional grant financing of around US$ 2 million was mobilized to fill the financing gap for the FMIS. Swedish International Cooperation (Sida) allocated additional grant funds to cover the shortfall through a successor TF (94801). These funds financed initial payments under FMIS contract. A new trust fund was established for this purpose because it was not possible to add funds to the TF054017 due to changes in TF rules. 1.7 Original Policy Areas Supported by the Program (as approved) Due to weak governance capacity and a difficult political environment, the proposed project focused on interventions that did not require sensitive policy actions. Instead the project focused on improving technical capacities of the public administration in public resource management. In the course of project implementation it was hoped that improvements in the budget planning process could stimulate an interest in developing capacity for strategic policy making at the center of government and in sector ministries. 1.8 Revised Policy Areas The policy areas were not revised during the life of the project. 1.9 Main Beneficiaries The project focused on the management of public financial resources. It did not directly address public services. Accordingly, the direct beneficiaries of the project were public officials responsible for budgeting and financial management, who were to benefit from improved information and streamlined processes, and decision-makers within the Government and Parliament who were to benefit from improvements in the quality and timeliness of financial information. Indirect beneficiaries included the non-governmental entities that take an interest in public finances and, more broadly, citizens who would benefit from greater transparency, better targeting of public programs and reduced corruption. 6 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The project development objective was linked to goals of poverty reduction, following the trend at that time. The outcome indicator of improved overall fiscal discipline was beyond the ability of the project to deliver. Nevertheless, during the design phase, the Quality Enhancement Review (QER) and Peer Reviewers warned of excessive ambition and the team took this advice into account in formulating other indicators. The project design had a solid analytical foundation in the form of up-to-date diagnostic work, including a CFAA and a CPAR carried out in 2002-03, as well as European Union (EU) and IMF diagnostic reports. The project also benefited from preparatory work for a public sector reform adjustment operation that was later dropped. At the Country Director’s request the team carried out a QER and a political economy analysis which sought to identify implementation risks and develop appropriate mitigation measures. Authorities’ project preparation was supported by technical assistance financed from a recipient executed Policy and Human Resources Development (PHRD) grant and a parallel grant provided by the Dutch directly to the MOF. These grants were used to fund diagnostics, equipment of the MOF training class and Project Implementation Unit (PIU) capacity building. An extension of the project effectiveness deadline, agreed prior to project negotiations, allowed the authorities to finalize preparation and fully use the PHRD grant. The design focused on addressing existing capacity constraints and took into account the MOF’s lack of experience in managing large complex Information Technology (IT) projects. Following the recommendations of a study of IT project management arrangements, the authorities created a PIU that was integrated within the MOF as a Project Office (PO) and supported with technical expertise. One design weakness was the formulation of the PDO. While the essence of the PDO was to support the improvement of public financial management in Moldova, which was realistic and attainable, the PDO was formulated in much broader terms. This was partly due to the then prevailing approach in the Bank to link back to "higher level” objectives – in this case reducing poverty and increasing economic growth – which a project of this type could not hope to influence. The project would have benefitted from more advance work on the definition of FMIS requirements and architecture during projection preparation. Much of the work of business process re-engineering design, preparation of functional requirement specifications, interoperability strategies, development of information architecture blueprints and application architecture blueprints was undertaken during implementation. Lack of clarify and agreement on FMIS requirements and specifications led to clarification requests, extensions of bid submission deadlines and change requests during contract implementation and increased cost for suppliers during contract implementation. 7 2.2 Implementation Implementation of the Moldova Public Financial Management Project (PFMP) began well notwithstanding some delays: indeed until 2009 the project was seen as a model to be emulated by others. There was strong government ownership, effective consultation and commitment on the part of the project team. Implementation of the TA components was slower than expected. Multiple changes of the counterparts affected the pace of implementation and there was some resistance to change, for example to the introduction of internal audit. Nevertheless, many of the changes required for the improvement of public finance management were eventually delivered, notably the MTEF, modernization of budget classification, reform of internal audit and the establishment of PFM training programs. The Government was initially well prepared for the FMIS procurement both technically and from the perspective of procurement. Arrangements included the establishment of a Project Office (PO) in charge of procurement and financial management (FM) issues, recruitment and training of a procurement specialist and appointment of an international Contract Management Specialist to support design and implementation. Implementation challenges were related to the complexity of the procurement process and the limited capacity and experience of MOF staff to manage contract implementation. The Bank staff strongly recommended further strengthening of the technical advisory team for the FMIS, including advisory support for the development of the software and accelerated recruitment of the Contract Management Consultant. The FMIS procurement process was launched in March 2007 as a two-stage bidding process for "Supply and Installation of Information Systems". The final bid submission deadline for the second stage bids was June 2008 and the Bid Evaluation Report was cleared in October 2008. However, the contract was only signed in March 2009 and became effective in July 2009. In 2009-10 the project took a negative turn, following a political crisis and a change of Government. During 2010-11 the Government became dissatisfied with the performance of the main supplier and its sub-contractors leading to a breakdown of trust. The budget execution module of the FMIS was the most problematic area. The suppliers initially developed a poor quality blueprint which undermined the Ministry’s confidence. The new Minister expressed initial skepticism regarding the purpose and benefits of the project and was dissatisfied with the Bank’s support. Thereafter, successive attempts to put the FMIS component of the project back on track failed. There was a substantial failure on the part of the supplier to deliver an acceptable technical solution for the budget execution module, despite multiple contract extensions. The project was extended for a total of three years given the delay in implementing the FMIS. In the fall of 2013 the Government, in consultation with the World Bank, concluded that the full FMIS could not be delivered within the scope of project. After evaluating different options, including termination of the contract, the MOF and HP management, the main contractor of the FMIS, reached an agreement on reductions in the contract scope and price for the delivery of a partially acceptable FMIS solution before the closing date of December 31, 2013. Partial delivery covered the Budget Preparation System (BPS), BES Master Data Management (MDM), 8 Collection of Non-Financial Data (CNFD), Cash Liquidity Forecasting (CLF), and some of the Business Warehouse (BW) modules. The Budget Execution System (BES) (except MDM), Accounting (ACC), related BW reporting tools, and Web Services were removed from the contract scope. The ICT infrastructure (main and backup data centers, workstations, and network equipment) was also accepted. A number of factors adversely affected implementation: i. Although Government maintained strong commitment to the project, poor supplier performance and discontinuities in leadership resulted in strained relationships between the Ministry of Finance, the main supplier, and subsequently the Bank team. Key staff changed on all sides, resulting in loss of institutional memory and the need to invest time and effort in rebuilding relationships. These were not fully successful. ii. Poor performance on the part of the main contractor, HP, was an important factor contributing to the failure to fully implement the FMIS. The quality of the HP team was variable over the lifetime of the project and at times their presence on the ground was also weak. HP managers changed frequently during the project lifetime, with some long gaps between appointments. The MOF viewed HP as bureaucratic, with the project manager not empowered to take decisions. With the benefit of hindsight it would have been better to terminate the contract with HP when it was clear that the MOF had lost trust in its supplier. iii. The complex nature of the consortium that HP put together to implement the project contributed to the contractor’s poor performance. Communication between HP and sub- contractors was poor. Sub-contractors were sometimes unaware of agreements that had been reached between the MOF and HP management while HP repudiated agreements reached directly between the MOF and sub-contractors. The detailed FMIS system design (Blueprint) was of poor quality and showed lack of understanding of the core business processes. Loss of critical project management staff, including those with expertise in contract management and IT, undermined project management on the Government side. This proved to be a critical weakness, contributing to problems in managing contract implementation and the relationship with the main supplier. Attempts to rebuild a strong team with the necessary contract skills were only partially successful and for long periods of time the PIU lacked a procurement specialist and IT project management expert. At the same time, the presence of an international consultant did not result in a significant improvement of the contract management. iv. Project extensions were granted for relatively short periods of time. While these extensions were intended to encourage the authorities and the contractor to follow up quickly, the extensions proved insufficient to reach agreement on contract amendments, resolve design issues and develop adequate software solutions. 9 Adequacy of Government’s Commitment Strong government commitment was identified from the outset as being a key to success. This commitment was strongly evident in the early years of implementation. The first Minister of Finance who initiated the project later became the first Deputy Prime Minister and then Prime Minister, which allowed the project to benefit from the highest level political support till the radical political change in 2009. The new Government, which took office in 2009, viewed the project with a more critical eye and did not immediately assume ownership. Those within the Ministry who were comfortable with the existing Treasury system exerted some influence on decision making and were able to point to the poor quality blueprint for the core budget execution module. Despite their reservations, the MOF decided to support the project and advocated the later extensions, recognizing the critical role of the FMIS in delivering the expected benefits of the project as a whole. Soundness of Background Analysis The design and implementation of the PFMP was built on a strong foundation of analytical work: the Bank’s Public Economic Management Review, the Country Financial Accountability Assessment, and the Country Procurement Assessment Review; an EU-funded Treasury Diagnostic Review; IMF reports; and a summary of the stakeholder analysis for the Moldova. The experience that the World Bank has gathered in comprehensive improvement of public financial management was also used to guide the design and implementation of the program. The UK Department for International Development (DFID), SIDA and the Dutch Government also supported analytical work on reforming and restructuring public financial management. The analytical work was used to inform project design, in particular the strengthened implementation arrangements. This analytical work focused on the institutional and technical dimensions of reform. There was no explicit political economy analysis. A better understanding of stakeholders’ interests, identifying winners and losers and potential sources of resistance to change might have led to a more realistic appreciation of implementation risks and helped identify more effective change management strategies. Assessment of the Operation’s Design The design of the operation was assessed as satisfactory by the Quality Assessment of the Lending Portfolio (QALP-2). The project focused on addressing existing capacity constraints. The Minister and the Director of the State Treasury Department assumed leading roles in project governance and management. Recognizing that public financial management engaged all line ministries, agencies and local authorities, these entities were represented in the Steering Committee and were engaged in operations by a project Task Force comprising the key departments of the Ministry and the Academy of Public Administration (APA). Grant co- financing was mobilized so that the IDA credit would be used for investment in the FMIS component, as requested by the MOF. The MOF, and specifically the Project Office, were responsible for the management of the credit, grants and counterpart funds and for financial reporting. The project funds were kept in Treasury’s Special Accounts, with different accounts for each funding source, opened in a competitively selected commercial bank. 10 Despite the complexity of the comprehensive PFM reform program and the number of entities involved, the reform areas addressed by the project were well identified and focused through the four project components. The Government of Moldova repeatedly indicated its appreciation of the Bank’s assistance, using the operation to improve program design and to enhance discipline in reform implementation. The fact that the Bank remained engaged in the reform effort despite the deterioration in project performance helped dialogue and made the Bank a privileged partner. Other development partners were successfully brought within the program. Assistance for the development of a Medium-Term Expenditure Framework was provided by DFID. SIDA assisted the MOF on debt management and training and helped build capacity of the Court of Accounts. The Government of the Netherlands supported training on managing large information systems for key IT staff of the MOF. The presence of many development partners posed a challenge and led the project to put in place special coordination arrangements to avoid overlap and address key issues on a timely and priority basis. The Task Force, chaired by the Head of Treasury, and comprising heads of departments of the MOF directly responsible for implementing project components/sub- components, was formed to coordinate efforts related to both technical and financial support provided to the government. With the benefit of hindsight the project might have benefited from several modifications to the design. Project implementation might have gone more smoothly had agreement been reached on FMIS requirements and the redesign of business processes during project preparation. The project would have been able to realize the benefits of PFM reforms if these had been supported by interim upgrades to the existing system, in parallel with the development of the new system. Furthermore, investments in communications and change management might have reduced the level of internal resistance to the project. Relevance of Risks Identified The risks identified during project design were relevant but not all the important risks were identified. Loss of political commitment resulting from the changes in government was identified as one of the risks and was highly relevant. To minimize the likelihood that the government would revert to the old system of public finance management, the project attempted to build broad-based support for improvement of public financial management emphasizing benefits for stakeholders and its significance for achieving EGPRSP objectives. The project and Bank teams briefed new Ministers soon after they were appointed in order to inform them about the project and advocate for continuity in implementation. Despite this proactive approach political commitment wavered, particularly towards the end of the project. The importance of the views of internal stakeholders below the ministerial level was not sufficiently appreciated or understood. Not all stakeholders were supportive of the project in practice and their resistance impacted on project implementation. Weak central policy coordination capacity was identified as a risk that might reduce the effectiveness of resource allocation within the MTEF process. The project included 11 representatives of the Council of Ministers (COM) and the Presidency in the coordination framework to strengthen coordination. Policy coordination capacity did not turn out to be a significant risk factor. The risk of political pressures to approve over-optimistic revenue forecasts that may undermine fiscal discipline was correctly identified and was managed to some extent. The MOF was able to review revenue forecasts and compare them with the projections of International Financial Institutions to improve realism of macroeconomic framework. The Project Appraisal Document (PAD) argued that poor management of the procurement process could potentially delay the timely installation of equipment and system testing. This was addressed by ensuring that the procurement specialists of the Project Office were trained in procurement management and project planning so that relevant infrastructure and manpower would be put in place prior to the implementation of the corresponding project component. Despite these mitigation measures substantial delays did occur in the procurement process. The borrower opted for an ambitious and complex solution for the FMIS contract and the main supplier struggled to manage the performance of its sub-contractors. This complex contract structure contributed to project failure. The project correctly identified risks to the introduction of internal audit arising from the culture of strong central control and weak internal management controls. This was successfully managed through study tours to learn more about IA practices; management training and awareness building for managers; and a successful transition was made to IA. The risk that the APA would fail to re-orient its mandate to provide in-service training using external trainers was not realized. A mitigating measure was provided in the form of establishment of a Department for Civil Service Training in the APA with adequate autonomy from the departments providing higher education programs. Finally, the risk related to the Project Executive Officer (PEO) failing to establish and maintain proper coordination of project implementation was addressed by relieving the PEO from some of the time-consuming tasks through shifting some responsibilities to other Deputy Ministers and ensuring the sustainability of the PEO position during project implementation regardless of any structural changes in the Government of Moldova. A significant risk that the initial risk assessment did not identify was related to lack of experience in the Ministry of managing a complex IT project with an international supplier. Ministry officials had no previous experience of this scale of contract and were reluctant to make use of experienced professionals with international experience, preferring to rely on less experienced local advisers. The project design relied on a strong PIU that was established at the start of the project. Technical advisors and IT specialists were hired but most of the key PIU staff and technical specialists left the project team for other positions and outside posts. Capacity issues emerged starting from 2010 and the Bank teams suggested hiring international advisors to compensate for the loss in capacity. During 2010-2012, the MOF had considerable difficulty dealing with project and contract management but it was only in 2012 that MOF agreed to hire an international advisor to support these activities. 12 2.3 Monitoring and Evaluation M&E Design The design of the M&E arrangements was satisfactory. A set of qualitative and quantitative intermediate outcome indicators were defined to monitor progress towards the achievement of the Program Development Objective (PDO). There were clear output measures in terms of training delivered and legislation developed and approved. The project design provided that key performance indicators would be tracked on an annual basis, through project-financed surveys and analyses. M&E Implementation Performance indicators were presented by the Project Office to the Steering Committee and the Task Force as part of regular progress reports. These were presented to the Steering Committee, the Task Force, and the development partners on a semi-annual basis and were used to track the progress in project implementation against the baselines. Later in the life of the project there was little reporting on results, since most of the project activities had been completed and the project was focused almost exclusively on FMIS implementation. The main shortcoming in evaluation was that the Government was unable to prepare an evaluation report at the project completion stage, notwithstanding reminders from the World Bank, citing the dissolution of the Project Office and lack of financing as the reasons for non- compliance with this requirement. M&E Utilization Some use was made of the M&E framework. The project team and MOF used the monitoring system to track progress in implementation of the PFM reforms. Specific indicators focused on progress in fiscal discipline of the national public budget, implementation of the budget execution methodology, establishment of FMIS, and independent audit mechanisms and reforming training system for civil servants. Monitoring information helped identify adjustments in technical design, such as alignment of budget preparation and execution methodology, the warrants system, the Chart of Accounts and Budget Classification implementation with the FMIS preparation and implementation schedule. Furthermore, the development of the PFM legal framework benefited from the project’s extension so that legal drafting better reflected what is happening on the ground. This also allowed for discussion and debate as well as analysis of the practices/lessons learned from other countries that have had a similar recent experience. The project staff also monitored progress in implementing the training component. Based on monitoring, activities related to IA training and IA function decentralization were prioritized and reevaluated. Training activities needs were also redirected to ensure the sustainability of specialist training. 13 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Substantial The project’s development objective was to achieve effective and transparent management of public finances, as an enabler of poverty-reducing programs and economic growth. This objective was and continues to be highly relevant to Moldova’s development strategy, its PFM reforms and current Bank priorities. In the aftermath of the financial crisis, and resulting downward pressure on state revenues, support for public financial management systems remains a high priority for both Bank and national authorities. The Country Partnership Strategy (CPS) for FY2014-17 continues to identify public finance management as an important enabling theme and the country program is supporting other projects to strengthen PFM in tax administration, auditing and public investment management. 3.2 Achievement of Program Development Objectives The project closed on December 31, 2013. The project accomplished those results that were not directly dependent on the successful implementation of the FMIS. Component 1: Budget Planning and Execution System Starting with the draft budget for 2014 the new budget classification developed under the PFM Project will become effective. In addition a Law on Public Finances and Budgetary-fiscal Responsibility is at the final reading stage and is expected to be adopted in 2014. The project supported the introduction of the single treasury account from March 2007. This contributed to increased transparency and efficiency of budget management by consolidating cash management. The PEFA 2011 report concluded that the data demonstrated very strong linkages (minimal variance) between MTEFs annual Budget appropriations demonstrating credibility in the process and that the MTEF and annual budget process are properly integrated and part of one unified cycle. The 2011 PEFA report showed the budget classification score improving from C to B. Alignment of the newly developed budget classification and chart of accounts with international standards has laid a solid foundation for improved transparency of budget and financial reporting. Status of component 1c (FMIS) is described in detail in the paragraph below. Under component 1c, the development of a fully functional FMIS, to support the PFM reforms initiated by the Government, was the largest project activity. There were significant delays in the execution of FMIS contract originally signed with the Supplier on March 27, 2009. The Supplier failed to deliver a fully functioning FMIS. Instead, MOF and the Supplier signed a contract amendment on December 4, 2013 to reduce the contract scope and price, for the delivery of a partially acceptable FMIS solution before the closing date of December 31, 2013. The approved FMIS application software represents 44 percent of the functionality foreseen under the original contract. The ICT infrastructure (main and backup data centers, workstations, and network equipment) has also been accepted. The reduced total contract price is 67.2 percent of the total initial price, including the payments already made for the ICT infrastructure and FMIS detailed system design. 14 As it is, the data center equipment and network infrastructure is currently being used by the MOF and Tax Department to support existing information systems. However, the budget preparation and budget reporting portions of FMIS cannot be used unless the MOF integrates them with existing systems. A MOF decision is pending on this integration. MOF is currently looking into options for completing the system through financing by its budget or from other sources Component 2: Internal Audit The project was successful in introducing an internal audit function following the European Union’s decentralized Public Internal Financial Control (PIFC) model. Legislation was passed. A methodology developed by the Ministry of Finance provides overall direction of internal audit. Internal auditors have been appointed and trained. All central government ministries now have established IA units and around 130 IA repots are produced annually. IA is being extended to district level with about 40 percent coverage. Staffing and recruitment remains a challenge. Component 3: Training capacity and training The training capacity and training component has been partially completed. The Academy of Public Administration developed 12 training modules, plus another 6 developed jointly with the Ministry of Finance. In total 217 courses were run, financed from the project, providing training to more than 5000 people, with an additional 6,000 financed from the state budget. The project also financed training materials and equipment including computers. However, some of the training activities under the project that are closely linked to the launch of FMIS and new methodologies to be introduced with the FMIS. Due to the partial acceptance of FMIS modules, the training component was scaled down to cover budget preparation and basic ICT support and reporting needs. In addition to the APA training, between 2011 and 2013 the Ministry of Finance developed and delivered two distinct training modules on budget classification and program-based budgeting. 40 trainers were trained for each module from the MOF and CPA and training was conducted both for the relevant staff from the CPA (120 persons), as well as for the decision makers from the Government and Parliament, as well as officials from the local public authorities (94 people). 3.3 Efficiency This was an institutional development project and standard computations of economic and financial rates of return were not assessed as part of project preparation. The project did contribute to improvements in budget management and control of public expenditures, reduction in arrears and borrowing and expenditure efficiency savings. The introduction of the single Treasury Account supported by the project made an important contribution to the elimination of arrears by 2009 (from 3 percent of GDP in 2005). However, these benefits are difficult to attribute solely to the project and their economic and financial benefits difficult to quantify. Furthermore, the FMIS component, which represented US$8.5 million (68 percent) of the total project cost of $12.4 million, did not result in the expected improvements in control because the budget execution module effectively non-functioning. This substantially reduced the economic efficiency of the project. 15 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory The Internal Control and Audit component was completed satisfactorily with the objectives fully met. The Financial Management Training Capacity and Training is rated moderately satisfactory since a sustainable training capacity was put in place and institutionalized but training activities were not completed since training to support implementation of the FMIS was never undertaken. The Budget Planning and Execution component is rated unsatisfactory owing to the project’s failure to deliver a working financial management information system, notwithstanding three project closing date extensions between 2011 and 2013 in order to provide time for completion of this component. The partial acceptance of the FMIS system is unsatisfactory because the modules that have been accepted are effectively non-functioning. Furthermore, the redesigned core PFM business processes that were successfully completed, including those for budget preparation and the new chart of accounts, cannot be put fully into effect unless and until the MOF decides to integrate the new FMIS modules with the existing treasury system. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The project did not specifically target poverty impacts or social development, but the modernization of the PFMS involved assistance to manage a revenue collection environment that could allow for the effective and transparent management of public finances as an enabler of poverty reducing programs and economic growth. No gender dimensions were identified at the design stage or tracked by the project. The ultimate result was expected to be a better management of expenditures. Indirectly, this was expected to lead to improvements in social benefits available to the population; therefore, the project, would instill more confidence in government, lead to greater social inclusion, and increase equity and transparency. (b) Institutional Change/Strengthening The project was successful in developing long terms institutional capacity in three important areas. Modern concepts of internal audit, consistent with EU PIFC framework have been established and functioning units set up across much of Government. Second, the APA has established long term capacity to train civil servants in public finance management. Thirdly the project helped to build substantial capacity in IT systems development and operational management in Fintehinform. The project was also successful in bringing about changes in processes. The medium term budget process delivered closer alignment of the budget with policy priorities and largely eliminated extra budgetary funding. Despite the failure to implement IFMIS the project brought about improvements is processes and controls that contributed to better cash management, and reduction of idle balances in Government’s accounts.. (c) Other Unintended Outcomes and Impacts (positive of negative, if any) Not applicable 16 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable 4 Assessment of Risk to Development Outcomes Rating: High The Risk to Development Outcomes is considered high. Most of the benefits of the project were contingent on the successful implementation of the FMIS system. Although there has been partial acceptance of the system it is effectively non-functioning because key components of the integrated system are missing, most notably the budget execution and accounting modules. Realization of the project development objective will now depend on the Government finding alternative financing to complete development of the FMIS. Assessment of Bank and Borrower Performance 4.1 Bank Performance (a) Bank Performance in Ensuring Quality of Entry Rating: Moderately Satisfactory Bank performance was rated as satisfactory at entry by a QALP assessment. Sound preparatory and analytical work, strong policy dialogue and good working relationships with government counterparts, high level of consultation with participants, and Bank expertise led to the design of an operation which took into account the need to build and strengthen public financial management through institutional capacity building and modernization of budget formulation and execution (including treasury operations), internal control and audit, and financial management training capacity. Furthermore, the Bank established close coordination with the development partners actively supporting PFM agenda, namely DFID and Sida, to help the government build the necessary capacity to implement the program. However, the decision to implement the project within five years was unrealistic. The task team had originally advised a seven year implementation period, which was consistent with experience in similar projects but this was rejected at the design stage. Notwithstanding the positive QALP assessment there were deficiencies in project preparation. The functional design of the FMIS was only finalized in 2007 well over a year after project effectiveness. Even then the lack of clarity over FMIS specifications and the business process reengineering requirements contributed to disagreements between the authorities and the contractor. Furthermore project preparation failed to identify potential for resistance to change from some staff within the Ministry. These factors contributed to the failure of the FMIS component. (b) Quality of Supervision Rating: Moderately Satisfactory 17 Bank supervision performance is rated as moderately satisfactory. Although there were several changes in task team leadership, the task team made effective handover arrangements and remained on top of emerging issues. The team has made good use of quarterly reports from the Project Office and supported the Office’s efforts to improve the quality of reporting. The team produced comprehensive and candid aide-memoires. Qualified, experienced consultants were deployed to support the core task team. The task team also drew on diagnostic and analytical work undertaken by the Bank, such as the PEFA Assessment and PFM Performance Report of July 2008, to inform dialogue with the authorities on the reform priorities and technical issues in project implementation. The task team worked closely with the authorities to engage in dialogue with stakeholders to help overcome resistance to reform. Under the internal audit component, the Inspectorate initially resisted the elimination of its functions. The task team helped the authorities overcome this resistance by introducing Inspectorate officials to "Best Practice” in the European Union and demonstrating how the modernization of internal audit is a prerequisite for EU accession. This proved a powerful incentive for reform. The task team helped the Project Office and MOF manage four transitions between Ministers of Finance during the lifetime of the project. Each time a new Minister of Finance was appointed, the task team leader (TTL) mounted a mission as soon as feasible in order to explain the objectives of the project, its components, the next steps and challenges in implementation. The Bank's country office in Chisinau provided valuable back-up support and follow-up throughout. Supervision intensified as project performance deteriorated. Implementation support visits focused on the FMIS component, which accounted for the bulk of the IDA credit and was critical for the achievement of the PDO. In addition to the regular monitoring of progress in the field, the Bank team conducted monthly and later weekly monitoring meetings though video conferencing. These meetings brought together the MOF, the supplier and subcontractors and focused on problem solving and work planning. Sector (ECSP4) and the CMU (ECCU2) management paid particular attention to the project when agreements had to be reached on extensions and when solutions had to be found to accelerate implementation (October 2011 – December 2012). This included: several trips to Chisinau by ECSP4 Sector Manager to discuss the situation directly with the Minister, including an implementation support visit directly led by the Sector Manager on the Minister’s request in February 2012. The CMU was closely involved and the Country Manager and Country Director held multiple meetings with the Minister. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The rating of moderately satisfactory is based on the following considerations. The task team was assiduous in monitoring and reporting on project progress. Key risks were flagged and candidly discussed with the client. The Bank team and management made a considerable effort to help MOF resolve disagreements with its FMIS contractors. The CMU was closely involved and supportive of the project at critical stages even as performance deteriorated. 18 Notwithstanding, the robust supervision that the team put in place the team failed to ensure that the Ministry retained adequate capacity to effectively manage a complex IT project with an international supplier. This contributed to the breakdown in the relationship with the supplier and made it hard for the Government to enforce penalty clauses in the contract. This problem was identified as a risk during implementation but was never adequately addressed by the team or by management. With the benefit of hindsight, the Bank could have been more decisive in cutting its losses and closing the project when faced with irreconcilable disagreements between the MOF, the lead contractors and sub-contractors and their persistent failure to meet agreed delivery schedules. The task team and Bank management discussed alternative project restructuring options in November – December 2011. At Bank management’s request, the task team prepared a paper for discussion with the Minister which included options for scaling back project activities and early closure. Regrettably, MOF management was not willing to discuss these options and asked for further project extensions. Again with the benefit of hindsight, project extensions were too short in duration to signal the Bank’s full commitment to the successful conclusion of the project. One year extensions provided insufficient time for the authorities to negotiate contract amendments, contractors to commit the resources needed to accelerate project implementation and the project to demonstrate progress in system development. 4.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory From the design to implementation stage, the government was actively seeking advice from the Bank where needed to implement its strategic reform agenda. The high-level commitment shown by the management of the MOF during the early years of the project led to the satisfactory outcomes for components 1a, 1b and 2 of the project. The Government that initiated the project sustained this leadership and consensus over several years through several changes of Ministers. (b) Implementing Agency Performance Rating: Moderately Unsatisfactory Following the change in Government in 2009, client commitment wavered at a critical point in the development of the FMIS system. Contract performance issues and an alleged incident of bribery contributed to a loss of trust between the Minister and the supplier. The Minister of Finance also expressed the opinion that the project had been imposed by the World Bank and questioned the Bank’s position vis–à–vis HP – the main contractor for the software. This resulted in a loss of confidence between the Minister and the TTL. The project never fully recovered from this loss of confidence. MOF senior management lost trust in both the main supplier of the FMIS and the Bank team leader. The Bank TTL was 19 changed. At the same time, the departure of experts with experience in managing development of a complex IT project hindered efforts to bring project implementation back on track. Once it was clear that the FMIS component was at risk of failure, the Ministry took steps to strengthen management. The Deputy Minister of Finance was appointed as Project Manager. Additional project staff were hired. The Project Office and the Deputy Minister began to provide weekly monitoring reports. However, gaps persisted in technical project management and procurement skills. By the time an experienced IT project management adviser was appointed in 2012 it proved too late. Communication between the MOF and the contractor had deteriorated. MOF senior management no longer trusted the contractor’s ability or commitment to deliver an operational FMIS system. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory Based on the evaluation of the Government performance and the implementing agency performance, the overall Borrower performance is rated as Moderately Unsatisfactory. 5 Lessons Learned This section addresses why project substantially failed to deliver its objectives, and identifies lessons learned that could potentially be applied to future PFM projects, especially those that contain substantial FMIS components. Despite the focus on the failure of the FMIS system, it should be noted that other project components that were not linked to FMIS implementation were successful. Medium term budgeting was rolled out successfully and has contributed to improved budget discipline and declining deficit. Improvements in internal controls, such as registering all government contracts, have contributed to a reduction in payment arrears. The project has laid the foundation for modern internal audit system which is now operating successfully across government. Key PFM training programs have been developed and are being delivered through a government institution financed from the State budget. The FY14-17 Country Partnership Strategy presents preliminary lessons from the implementation of ICT projects in Moldova. Many of these lessons are directly applicable to the implementation of FMIS projects. Based on review of the project documentation and discussions with stakeholders, including Government, Bank staff and suppliers, the main lessons learned from the Public Financial Management Technical Assistance Project are as follows: It is essential that Ministries of Finance implementing complex ICT projects retain advisers with expertise in managing large and complex ICT systems implementation projects. It is possible that the problems in communications and mismatched expectations could have been avoided had the initially strong team been retained or replaced with equally effective experts. Contract management capacity is particularly important when disputes arise. Better documentation of supplier performance issues, for example, could have put the Government in a stronger negotiating position when it came to negotiating the terms of the final contract amendment. ICT projects should retain ICT experts to provide strategic advice on integration of ICTs into the sector specific domain and examine the domain for business architecture and 20 business process re-engineering needs. The resultant requirements need to be planned and executed by a client PIU that possesses in-house expertise to prepare technical bidding documents and ensure appropriate vendor and contract management Projects can be more effective if PFM reforms are not linked to the implementation of the FMIS system. For example, the new budget classification system could have been implemented if the existing treasury system had been updated as an interim step. This was proposed by the task team during implementation but rejected by the MOF on the grounds that they were not prepared to test the new budget classification system through a parallel system in the middle of a complex reform and did not have the means and time needed to run and test these interim systems in parallel. Development of existing systems or interim stand-alone solutions could be used to improve the quality of financial management and potentially increase the chances of a successful transition to a full IFMIS. A realistic time frame for the design and implementation of a complex FMIS project is seven to eight years. Implementation time frames can be reduced by completing FMIS and business process reengineering design work during project preparation. However, design adjustments and resulting delays are not uncommon: Bank-wide, some 70 percent of technical assistance/financial management operations have had to be extended at least once, sometimes two or three times. A clear understanding of the nature of IT projects at their start, their implementation challenges and unusual back-ended disbursement profile can help reduce the frustrations and disagreements among key stakeholders. Project extensions should be long enough to allow the stakeholders to bring about changes in project performance. Both the Ministry of Finance and the main supplier commented that the extension periods were too short for any major restructuring to take place. Several months of each extension period was spent in negotiating the contract amendment, leaving little time for development work. The design of World Bank standard bidding documents for IT projects should be reviewed to assess their suitability. Suppliers complained that the provisions on system testing and acceptance in particular gave rise to problems during implementation. International competitive bidding (ICB) documents are complex and not designed to support complex information system implementation. Qualification requirements and lengthy procedures (which are difficult to understand for many client countries without external advisory support) lead to delays in procurement phase (the two-stage ICB took about 2 years in this project). The standard contract does not provide for any penalties to be applied for delays due to the Purchaser. Resistance to change should be anticipated and the project should retain consensus building and communications arrangements throughout implementation. The project was built on a consensus of major stakeholders, followed a careful process of consultation, and strong leadership form the Minister of Finance. This consensus building process was not followed through to project implementation. Consequently, the project was particularly vulnerable to political changes and resistance to change on the part of Ministry officials. A better understanding of the winners and losers and more active steps to manage potential resistance to change could have helped in identifying and managing risks before and during implementation. 21 6 Comments on Issues Raised by Borrower/Implementing Agencies/Partners When asked to comment on the Bank’s performance both the borrower and the supplier focused on the dispute that lay at the heart of the project’s failure. In the supplier’s view the Bank system is not well adapted to IT projects and is more geared towards the delivery of goods and works. In particular the supplier felt that the right of inspection, as interpreted by the MoF, contributed to delays, because the Ministry carried out extensive testing of software prior to it being fully ready for normal acceptance testing. In the view of the Government the standard contract should require a higher level of on-site presence by the main supplier. The Government also criticized the contract provisions on payment, which they saw as favoring the contractor and forcing the Government to pay out the majority of the contract value in spite of its failure to deliver of a working FMIS system. However, the payment conditions were defined by the MOF and later on revised by both parties (MoF and supplier) through jointly accepted amendments, approved by the Bank. Despite project closing MoF management has stated that they wish complete the development of the budget execution system and other components so that they can have an integrated FMIS solution. MOF has discussed options for financing this from ongoing World Bank-financed projects. The Minister proposed two options: (i) using e-Moldova project funds to support the development of the new Treasury (budget execution) system with; or (ii) initiating a new project to complete the development of integrated FMIS. The team suggested the improvement of existing treasury system as an interim solution to support new budget classification / chart of accounts in 2014, since a new project, or reallocation of funds from other projects may not be possible in the near future. 22 Annex 1: Project Cost and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Budget Planning and Execution 11.2 8.50 76% System Internal Control and Audit 1.2 1.2 100% Training Capacity and Training 2.0 1.9 95% Project Management, Monitoring 0.9 0.8 88% and Evaluation Total Baseline Cost 14.7 12.4.50 84% Physical Contingencies 0.7 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 15.3 12.4 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00 Total Financing Required 15.3 12.4 81% (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (USD millions) (USD millions) Borrower 1.19 NA NA International Development Association 8.55 4.680.00 55% (IDA) NETHERLANDS: Min. of Foreign 3.56 3.88 109% Affairs / Min. of Dev. Coop. SWEDEN: Swedish Intl. Dev. 2.13 3.84 180% Cooperation Agency (Sida) 23 Annex 2: Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Anarkan Akerova Counsel LEGEM Sr. Public Sector Management Svetlana Proskurovska ECSPE TTL Specialist Roland N. Clarke Sr. Country Economist ECSPE Cem Dener Sr. Public Sector Mgmt. Spec. ECSPE IT specialist Richard Gargrave Sr. Procurement Spec. ECSPS Vitaly Kazakov Financial Management Specialist ECSPS Craig R. Neal Sr. Public Sector Spec. ECSPE Elena Nikulina Senior Economist ECSPE PFM specialist Philip W. Thomas Sr. Public Sector Mgmt. Spec. ECSPE Anna Wielogorska Sr. Procurement Specialist ECSPS Supervision/ICR Irina Babich Financial Management Specialist ECSPS Andrei Busuioc Financial Management Specialist ECCAT Sr. Public Sector Management Svetlana Prosurovska ECSPE TTL till June 2009 Specialist Cem Dener Sr. Public Sector Mgmt. Spec. ECSPE TTL June – Oleksii Balabushko Senior Public Sector Specialist ECSPE December 2012 Andriy Kharchenko Finance Analyst LOADM PFM specialist, TTL Elena Nikulina Senior Public Sector Specialist ECSPE June 2009 – June 2012 Anna L Wielogorska Senior Procurement Specialist ECSPS Knut Leipold Senior Procurement Specialist ECSPS Lilia Razlog ETC ECSPE Ruxandra Costache Counsel LEGEM 24 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) FY03 0 11.78 FY04 59 211.32 FY05 75 249.33 FY06 33 102.83 FY07 32 104.03 FY08 34 124.57 FY09 28 86.73 FY10 21 116.78 FY11 36 121.40 FY12 33 134.26 FY13 25 145.23 FY14 23 138.84 Total: 398 1547.11 25 Annex 3: Beneficiary Survey Results Not Applicable. 26 Annex 4: Stakeholder Workshop Report and Results There were no stakeholder workshops. 27 Annex 5: List of Supporting Documents Program Document, dated May 15, 2005 (Report No. 31777-MD) Mid Term Review of Public Financial Management Project Aide-Memoires, Implementation Status Results and Reports Moldova Court of Accounts Report on FMIS Summary of the Stakeholder Analysis CPS CR Annex 2: Preliminary Lessons from implementation of Projects with ICT Components 28 Annex 6: Country Map MOLDOVA: PUBLIC FINANCIAL MANAGEMENT 29