ICRR 13961 Report Number : ICRR13961 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/17/2012 Country : Afghanistan Project ID : P099980 Appraisal Actual Project Name : Public Financial US$M ): Project Costs (US$M): 33.4 33.4 Management Reform Project L/C Number : CH298 Loan/ US$M): Loan /Credit (US$M): 33.4 33.4 Sector Board : Financial Cofinancing (US$M): US$M ): Management Cofinanciers : Board Approval Date : 05/29/2007 Closing Date : 12/31/2010 12/31/2011 Sector (s): Central government administration (96%); Sub-national government administration (4%) Theme (s): Public expenditure; financial management and procurement (33% - P); Gender (17% - S); Other public sector governance (17% - S); Administrative and civil service reform (17% - S); Other trade and integration (16% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Mariluz Cortes-Gorman Robert Mark Lacey Navin Girishankar IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Technical Annex (page 20) to the Memorandum and Recommendation to the President, the project's two development objectives are to : (i) develop an efficient and effective public financial management system; and (ii) develop the human resource capacity of the Ministry of Finance and the Control and Audit Office to ensure better operation of public financial management . The statements of project objectives in the Technical Annex and the Financing Agreement (page 4) are the same. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: Component 1. Strengthening Institutional Development (Appraisal Estimate US$11.4 million; Actual US$9.8 million). This component supported the Ministry of Finance and the Control and Audit Office to: (i) strengthen procurement related institutional arrangements and promote social accountability, including institutional support for the newly created Procurement Policy Unit; (ii) expand the automated treasury systems and rolling them out to selected line ministries and provincial treasury offices (Mustoufiats); (iii) develop internal audit practices and establish an anti -fraud unit; and (iv) develop external audit policy and procedures . Component 2. Strengthening Human Resources Capacity Development (Appraisal Estimate: US$5.0 million; Actual US$3.0 million). This component supported human resource capacity building activities of: (a) the Ministry of Finance and line ministries in the areas of : (i) procurement, (ii) financial management, and (iii) internal audit; and (b) the Control and Audit Office in the area of external audit. Component 3. Strengthening Reform Management (Appraisal Estimate: US$ 1.0 million; Actual US$0.6 million). This component assisted the newly created Reform Implementation Management Unit (RIMU) in ensuring coordination in the implementation of the Ministry of Finance functions and monitoring implementation of said functions across departments and Mustoufiats . Component 4. Operational Support for Public Financial Management (Appraisal Estimate: US$16.0 million; Actual US$20.0 million). This component provided direct operational assistance to line ministries through the provision of goods and consultants ’ services, to maintain high fiduciary standards in procurement and treasury operations, as well as in the external audit of these operations . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The estimated project cost at appraisal was SDR 22.4 million (US$33.4 million equivalent). There was no Borrower contribution. The total amount disbursed was SDR 22.09 million (UD$33.4 million equivalent). The closing date was extended by one year from December 31, 2010 to December 31, 2011, because the late mobilization of two key contracts pushed back their implementation closing date beyond the project’s original closing date. 3. Relevance of Objectives & Design: a. Relevance of Objectives: high . Building an effective state --one that can The relevance of the project objectives is rated as high. provide security and services to the people -- is an overarching objective that remains at the heart of the reconstruction effort in Afghanistan, but Government capacity is still weak despite improvements, and the pace of implementation of reconstruction programs has been short of popular expectations . In particular, strengthening the country’s public financial management to accelerate aid utilization, provide faster and better services to Afghan people, and ensure transparency and accountability of public expenditure is a top priority for the government . The project’s objectives are relevant, and important to Afghanistan’s economic development, and are timely and appropriate to the current needs of the country's public financial management sector . The project objectives are also relevant to the Bank ’s current engagement strategy with Afghanistan, which is spelled out in the Interim Strategy Note for the period FY 12-FY14. The ISN is closely aligned with the Afghanistan National Development Strategy. Bank support to Afghanistan over this period is based on supporting the delivery of some of the country's most important national priorities . It is also grounded in helping the government manage the critical transition from security and development dominated by the international community to one led by the government of Afghanistan by the end of 2014. Bank support emphasizes national programs to improve the lives of Afghans across the country, including in the areas of health, education, rural development, and public finance management. b. Relevance of Design: The relevance of project design is rated as modest . While the selection of components were generally appropriate, the activities and outputs financed under the project were insufficient to achieve such broad objectives within the time frame. Further, the results framework did not present a convincing causal chain between activities, outputs and outcomes with respect to both project objectives. Particularly weak were the causal links between outputs and outcomes under the second project objective (i.e., developing human resources capacity of the Ministry of Finance and the Control and Audit Office to ensure better operation of public financial management ). 4. Achievement of Objectives (Efficacy): First Objective : To develop an efficient and effective public financial management system . As indicated in section 3, the objective of developing an efficient and effective PFM system is an overarching objective that cannot be achieved in one single operation in a country that lacks a tradition of public accountability and transparency . Nevertheless, the improvements achieved in . The causal chain of PFM in Afghanistan justify an efficacy rating of substantial for this objective. actions, outputs and outcomes operated as expected to improve Afghanistan’s public finance management systems. However, the auditing capacity of the Supreme Auditing Institution (SAI) remains limited due to shortcomings in the legal framework and the capabilities of the staff involved (see Section 7). Actions and outputs in support of this objective are discussed below . (a) Procurement Institutional Development and Social Accountability : Institutional development and capacity assessments for stand-alone procurement of six line ministries have been completed . Public access to procurement information has been enhanced with the Procurement Policy Unit website and the Procurement Management Information System has been developed and is operational. (b) Treasury Systems Development and Roll-out: The Afghanistan Financial Management Information System roll-out of core modules has been completed for all line ministries and all Mustoufiats. The Qatia (annual national budget execution statement) and the World Bank loan and grant operations’ financial statements are audited and submitted to the National Assembly every year . All 34 Mustoufiats have completed implementation of restructuring and new processes of the Mustoufiat Reform program. (c ) Internal Audit Work Practices: Internal audit work practices have been developed along with capacity in the Internal Audit Department of Ministry of Finance . Four line ministries were audited for SY1389 and another four line ministries were audited for SY1390. A Fraud Investigation Unit was established in the Internal Audit Department in January 2008. (d) Supports for External Audit. The draft audit law is currently placed before the National Assembly for enactment. The Qatia audit has been reported to the legislature on time and has been strengthened through the application of better auditing methodologies. (e) Targeted Restructuring of Internal MoF Business Arrangements. All Ministry of Finance departments have been restructured and re-graded and revision of job descriptions for all 6100 staff, (in line with new Civil Servants Law and pay and grading system ) was approved at the beginning of SY1388 and implemented since then. (f) Staff Re-grading under the Government of Afghanistan Pay/Grading Policy. About 6750 Ministry of Finance employees have been transferred to the Pay and Grading System since SY 1388. Outcome Indicators for the First Objective . Coherence of budget, treasury and revenue estimates with the Afghanistan National Development Strategy (ANDS) priorities and the Medium Term Financing Framework (MTFF). At appraisal, there was no coherence between ANDS and MTFF. This target has been met. By project closing date, the annual budget document approved by parliament presents the MTFF by National Priority Program which is the Afghanistan National Development Strategy . Timeliness of budget, treasury and revenue preparation and execution . At appraisal, the timeliness of these activities was between irregular and moderate . The target outcome was that annual cash plans had to be realized within 80 percent of forecast. This target has been met. Currently, the Treasury Department prepares an annual cash plan, at a highly aggregated level, with sources and uses of funds. Both the uses and sources of funds of this plan came within 80 percent of actual results for SY 1389 ( FY 2011). Based on the above assessment, IEG rates achievement of the first objective as substantial . Second Objective : To develop the human resources capacity of the Ministry of Finance and the Control and Audit Office to ensure better operation of public financial management . The achievement of this objective is rated as substantial . The activities and outputs under this objective operated as expected to develop the human resources capacity of both agencies on public finance management. The fact that annual financial statements are still not audited in line with international standards underscores the continued importance of linking skills development and training to institutional incentives that affect the audit function . Furthermore, the weak links between activities/outputs and outcome indicators explain why only one of the outcome targets was fully met . Actions and outputs in support of this objective are discussed below . (a) Human Resources Capacity Development in Procurement. About 3550 staff of line ministries and Mustoufiats were trained in basic, intermediate, and advanced special courses in procurement during SY1387-1390, with an average of 885 staff trained per year. (b) Human Resources Capacity Development in Financial Management . In preparation of the rollout of Afghanistan Financial Management Information System (AFMIS), basic training in financial management was provided to line ministries staff, Mustoufiat ’s staff and Treasury interns. In SY1389, for instance, more than 700 staff and in SY1390, 900 staff in line ministries and provincial Mustoufiats were trained in areas relating to the AFMIS roll-out, problem solving, and financial management more generally. The successful rollout of AFMIS is a measure of the impact of these training efforts . (c) Human Resources Capacity Development in Internal and External Audit . Fourteen auditors were given on-the-job training by US Treasury Technical Advisors on techniques for accomplishing an assessment, procedures for auditing revenue processes and conducting risk assessment in line ministries. Training courses were provided to 81 internal auditors on: a) Customs, Revenue and Civil Service Law; b) Procurement Law and Regulations; c) Financial Mathematics; and d) Toolkits usage. Training on fraud investigation techniques and methodologies was conducted by a UNDP International consultant. In-classroom and on-the-job training for external auditors in grant audits was also provided . In addition to a quarterly basic training program was introduced in the CAO and scaled up with 30 trained in various disciplines in 2008, 70 in 2009, 120 in 2010, and 75 in 2011. An advanced course in IT and grant audits was offered for 46 staff over the life of the project, along with study tours to the European Commission and Turkey for about 10 CAO staff. As a result, the training in external audit contributed to application of better auditing methodologies in review of the annual budget statement (Qatia) and to timely reporting to the legislature. The CAO audits for donors grants were also strengthened, and for the for the first time since program initiation, met the reporting timetable in 2010 and 2011. (d) Support for the Human Resources Management Department (HRMD). The RIMU made good progress in the support and development of the HRMD and implementation of the administrative reforms; assistance in monitoring and evaluation of the Ministry of Finance strategic plan, technical assistance and project coordination and administration . A results-framework of the human resources operations was produced indicating the targets for the three years SY1389-1391. About 80 staff of HRMD were given different duty-specific training. (e) Preparation of New Human Resources Management Initiatives. The number of international technical assistant consultants has decreased by 30 percent at the Ministry of Finance. Outcome Indicators for the Second Objective . Level and perception of good governance in PFM operations. At project appraisal, the low share of donor funds going through Core Budget was an indication of the perception of poor governance in PFM operations. The target was a 15 percent increase in the share of donor funds going through Core Budget from SY1385 (base year). It is unclear whether this target has been met. By project closing the amount of total donor funding going through budget had increased significantly, but the level of increase in the share cannot be measured because there is real difficulty in establishing reliable figures for the total off -budget assistance. Number of ministries/agencies performing assigned treasury functions . By appraisal, no ministries were performing assigned treasury functions. The target was for 6 Line Ministries to have on-line access for payment requests and recording allotments directly in AFMIS . This target has been substantially met. By closing date, all ministries had on -line access for submission of expense vouchers in AFMIS. However, line ministries are not performing all the treasury functions foreseen in the target as the central authorities continue to exercise oversight on certain stages of transaction processing, and payments are subject to review of documents by the Ministry of Finance . Allotments, commitments, and payments are posted in a centralized manner in Treasury and in provincial Mustofiats. Number of line ministries conducting stand -alone procurement. At appraisal only one ministry conducted stand-alone procurement. The target was for seven ministries to do so. This target has been only partially met. By project closing, institutional development and capacity assessment was done for stand-alone procurement of six line ministries of which 3 were certified. Seven more agencies were under review. Organizational changes and capacity building to absorb responsibility for procurement has progressed across government. All ministries have in place or have proposed adequate organizational structures. Reliance on international technical assistance diminishing . The target was a decrease of 50 percent in reliance on international technical assistance in public financial management from levels in SY1385 (base year). This target has been partially met. By project closing there had been a reduction of 30 percent in the number of international advisors in the Ministry of Finance . Number of Mustoufiats which have implemented restructuring and new processes of Mustoufiat Reform program. At appraisal no Mustoufiat had implemented restructuring and new processes . The target outcome was for 6 Mustoufiats to do so. This target has been surpassed since by closing date all Mustofiats had carried out restructuring and introduced new processes including use of AFMIS for payments and accounting. IEG assesses achievement of the second objective to be substantial . However, only one target was met in part because the outcome indicators were not appropriate to the second objective (see Section 3b on relevance of design). 5. Efficiency: There is no cost effectiveness analysis to properly assess the efficiency of the project, and there is no information in the ICR to compare this project to similar projects in other countries . Based on the lack of evidence, and on the fact that some of the key contracts had a higher than initially expected cost due to the cost of providing protection and transport to the foreign consultants, efficiency is rated as modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The PFMR project led to improvements in basic financial management capacity throughout the government. Despite considerable training, the auditing capacity of the Control and Audit Office is constrained by shortcomings in the legal framework, and the base effect (i.e., starting from very low levels of initial capacity ). Also, reliance on international technical assistance for public financial management activities remains higher than anticipated . The overall outcome rating is based on the relevance of objectives (assessed as high), relevance of design (modest), efficacy in meeting the first and second project objectives (substantial for both), and the efficiency of the project (modest). Based on the IEG/OPCS Harmonized Evaluation Criteria, these split ratings result in an outcome rating of moderately satisfactory. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is rated as high based on the following factors, although the risks related to internal and external audit and to civil servant incentives could be mitigated by reforms under PFMRII and the Capacity Building for Results projects : The independence of the Supreme Audit Institution (SAI) is not established until the new Audit Act,currently in Parliament, is enacted and provides the principles of independence contained in the International Standards of Supreme Audit Institutions. Also, SAI capabilities will very likely not reach a standard such that the Bank and other donors could rely on CAO to carry out audits of technical assistance projects; or competently audit an Afghan annual financial statement prepared according to International Public Accounting Standards, without oversight and support of an audit agent. Furthermore, There seems to be continued misunderstanding and lack of acceptance of the role of internal audit within some ministries . In spite of a sound new grading and pay system being adopted, remuneration in the public sector remains uncompetitive for persons with the skills required to make public financial management function effectively. The deteriorating security situation may slow down implementation and discourage contractors from bidding, and agencies may not be able to source high level TA on a timely basis and at reasonable cost. Further, escalating insurgency-related violence has increased the level of fragility in the political environment, which is also limiting the ability of the government to deliver services and programs. Corruption continues to be widespread and pervasive. Afghanistan’s ranking in Transparency International’s Corruption Perceptions Index has slipped from 176 out of 180 countries surveyed in 2008 to 179 out of 180 countries surveyed in 2009. While the Procurement Law and the Appeal and Review mechanism provide a modern legal system for procurement, effective implementation of the law may encounter difficulties in the current weak institutional structure and capacity of the government . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: The PFMR project was the fourth Bank operation to support public financial management performance and reforms in the Afghanistan public sector. Together with the Civil Service Reform project, the PFMR envisioned to strengthen core business practices within government through appropriate regulatory, institutional and technical reforms . The appraisal team built on the experience of past and ongoing projects relevant to public finance management reforms in Afghanistan and on the assessment of Afghanistan’s public financial management and procurement performance conducted under the auspices of the Public Expenditure and Financial Accountability (PEFA) program in 2005. The PEFA confirmed that the execution cycle of budget was working well thanks in part to the centralized controls under Ministry of Finance for accounting and payments and ARDS for procurement oversight. This lead to the continued emphasis on support for these functions in the design. Some of the lessons learned from previous projects in Afghanistan incorporated in the design of the PFMR were the following : (i) for an emergency operation to be effective in a severely damaged post-conflict environment, intensive Bank supervision is essential; (ii) contracts for direct operations support need to be separated from capacity building activities to ensure that both are performed simultaneously; (iii) the expectations for capacity building and institutional strengthening need to be closely aligned with the availability and skill levels of the government counterparts; and (iv) the government needs to continue to engage the services of advisors for as long as it takes to build sufficient, demonstrable capacity in core public administration areas, even as the roles of the advisors become increasingly transitional. The design of this fourth operation responded well to Afghanistan's evolving conditions and challenges through an enhanced focus on capacity building in treasury, procurement and audit, and planned more use of local resources to deliver training . The structure of the PFMR project around human development and institutional development allowed progress across all targeted entities in these areas to be monitored. The project’s M&E was well designed to assess the project's achievements, although some of the outcome indicators measure intermediate outcomes. Project preparation was carried out with an adequate number of specialists who provided the technical skill mix necessary to address sector concerns and a good project design . The ICR reports that the Bank provided adequate resources to ensure quality preparation and appraisal work. Also according to the ICR, the Bank had a consistently good working relationship with the Borrower during preparation and appraisal, which facilitated development of the project design based on the Government public finance management priorities and strategies . at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: The Bank's supervision of the project has been continuous, comprehensive, and highly responsive to the needs of the Borrower. The task team prepared Aide-Memoires regularly and alerted the government and the implementing agencies to problems with project execution and facilitated remedies in a timely manner, in conformity with Bank procedures . The Implementation Status Reports (ISRs) realistically rated the performance of the project both in terms of achievement of development objectives and project implementation . The task team also monitored fiduciary compliances. The task team carried out a Mid-Term Review in April 2009, and assessed progress to date on all project components, and recommended actions to be taken to ensure the successful completion of the project. The task team was successful in convincing the government to restore key provisions of the Procurement Law that had been amended with the loss of transparency; to reverse the suspension of Article 61 of the Public Expenditure and Financial Management Law, which limited the purview of the Ministry of Finance internal audits; and to include provisions for the establishment of an independent and modern external audit function in the External Audit Bill which is now in Parliament.. One important aspect of the Bank's performance and contribution was the continuity of the Task Team, including the TTL and other key sector specialists, from project inception through completion. This continuity engendered consistency, depth and follow -up in the dialogue with the government and provided expertise to help the government analyze issues and implement actions as they emerged during supervision. Further, having the TTL on the ground in Kabul was strongly positive, given weak implementation capacity and thus the need for Bank assistance in drawing up the terms of reference for various consultancies and assisting the government in the contracting process. There was considerable input from technical staff . The Bank’s efforts kept the government focused on key reforms even at a time when staff were preoccupied about their new pay scale and there were many competing crises and appeals within Government to relax the application of the emerging PFM framework. The Bank team also ensured continuation of the efforts to achieve the objectives of developing an efficient and effective public financial management system; and the capacity of human resources performing PFM functions through the PFMRII approved on June 2011. This follow up TA operation finances consultants, training and equipment to assist with policy formulation, systems development, operations support and human capacity development for public procurement, budgeting, treasury and audit functions across government. Particular emphasis is placed on these functions in line ministries . Quality of Supervision Rating : Highly Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government had substantial involvement in defining the project's scope and identifying the agencies and ministries where consultants would be located to best advantage . There were also strong champions at senior levels of government. Government officials worked closely with the Bank's project team on a continual basis, and cooperated fully with the task team . The government consistently demonstrated and maintained full commitment to the project and its objectives throughout the implementation. Examples of active implementation interventions on the part of the government are: (i) directing other donors support to complement the project activities; (ii) investing domestic resources in a modern data center for the operation of the AFMIS in Treasury; (iii) preparing a PFM Roadmap for presentation to the donor community with significant actions to support the project development objectives; (iv) agreeing to memorandums of understanding with donors committing the Government to address the legal framework on internal audit and improve the legal framework for external audit; (v) leading the management and coordination of the project through the Project Steering Committee chaired by the Deputy Minister of Finance which approved all aide memoires and project budgets; and (vi) preparing periodic monitoring reports and communicating issues freely and openly . Government Performance Rating Satisfactory b. Implementing Agency Performance: The Ministry of Finance managed the project effectively in support of the Government’s public finance management reform strategy focusing on developing core competencies and institutional systems. The Ministry was fully committed to the aims of the project . RIMU) performed satisfactorily in keeping focus The Reform Implementation Management Unit (RIMU) on improvements in basic operational matters according to a MoF plan, and addressed human resources management issues. RIMU: (i) prepared the overall plan of actions with the contributions of the various departments and donors; (ii) coordinated actions to assure mutual support and cross functional efficiency; (iii) monitored progress; and (iv) evaluated the intended results. RIMU's Director convened periodic meetings of the working level authorities of various departments to identify and resolve implementation issues that were outside the scope of their individual authorities. The Treasury Department ’s performance was highly satisfactory in terms of executing the planned activities and raising the public finance management performance . The Internal Audit Department ’s performance was satisfactory as it executed the activities planned and raised the skills of the civil servants in particular . However, the department’s efforts to extend coverage to line ministries was stifled by interference from the Control and Audit Office . The Procurement Policy Unit was responsible for both formulating and then overseeing the national procurement legal and institutional framework . It needed the support of an international firm while the framework was being developed and training material was prepared . It is currently operating on its own and the training is on-going under the Afghan Civil Service Institute. Control and Audit Office . The direct operations support by CAO was successful and the performance on project audits in terms of timeliness and quality is excellent . Training was also delivered and planning and execution of audits, in addition to project audits, is improving . One shortcoming was the failure to enact a new legal framework for external government audit during project implementation, but an External Audit bill, satisfactory to the Bank, is now in Parliament .. Ministry of Economy . The role of Afghanistan Reconstruction and Development Services (ARDS) in the Ministry of Economy was to ensure that all procurement over an agreed threshold complied with the rules and regulations. ARDS managed to do this even when there was no technical assistance contract in place during the transition between PFMR and PFMRII . Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project M&E was well designed with a set of specific monitoring indicators of outputs and outcomes and conducive to appropriate evaluation given project objectives and available data . However, some of the outcome indicators measure, at best, intermediate outcomes . Others were not directly linked to project objectives such as those relating to building human resources capacities (see Section 3b and Section 4). Specific benchmarks were agreed between the government and technical assistance providers early on to measure specific progress in institutional strengthening and capacity building objectives. b. M&E Implementation: RIMU’s Monitoring and Evaluation Unit (MEU) led the monitoring and evaluation process and established a network of M&E focal point persons in each major operating line unit of the Ministry of Finance. Implementation data was collected on a regular basis, and Annual Performance Reports for the ministry were prepared using a results-based M&E performance framework. Good data was made available except for donor information of off budget operations. c. M&E Utilization: Appropriate data collected from Annual Performance Reports was evaluated and used to inform decision-making on certain activities. For example, the data contributed to the design of the follow-on project, PFMRP II. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: No safeguard policies were triggered for this Category "C" project. b. Fiduciary Compliance: Fiduciary compliance under the project was generally good . There were no significant deviations or waivers from the Bank’s fiduciary policies and procedures during the implementation of the project . There has been no misprocurement under the project and very little generally in the Bank's Afghanistan portfolio during the period of Bank re-engagement (since 2002). By and large procurement under the project was carried out in line with plan . Similarly, the were no ineligible expenditures outstanding at project closing and only minor items during the life of the project that had to be recovered. Audits in FY 2011 and 2010 were timely presented although the audit for FY 2009 was late due to difficulty in mobilizing the technical assistance for the auditor general . Audits for every year were accepted as satisfactory. Audits for the project do contain a qualification which is that the audits do not include a review of the component of the Auditor General's Office itself because it is not in a position to audit its own component - this was considered in the design of the financial management arrangements. c. Unintended Impacts (positive or negative): The project created the environment for developing and strengthening the accounting and auditing profession in Afghanistan. d. Other: The project had a substantial institutional development impact in the areas of procurement, treasury, audit, and reform management. For example: (i) a full legal framework was developed under PPU and structure in place to monitor its compliance . There is increased procurement capacity and skills in the line ministries. There is a full country coverage of AFMIS for on-line real time processing of provincial financial transactions . IAD’s capacity to conduct audits has been elevated to a high standard to include audits of computer systems. IAD has also raised the capacity of civil service staff and management in particular – this stands out in the Ministry of Finance, where the reliance on local consultants had been the rule for high level functions . Monitoring capacity in the RIMU has increased to follow PFM reforms both in the Ministry of Finance and across the government . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Relevance of objectives was high, yet Satisfactory relevance of design was modest . Efficacy was substantial for both objectives. Yet project efficiency was modest. Based on Harmonized Criteria, outcome is rated moderately satisfactory. Risk to Development High High Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are taken from the ICR with some adaptation of language : Continuity of the team working on a project, specially the TTL is very important for both the Bank and the Borrower as it will engender consistency and follow -up in the dialogue and provide expertise to help the government to analyze issues and implement actions as they emerge during supervision. It is important to recognize that an operation aimed at changing institutions is fundamentally about changing behavior and attitudes, social relationships and organizational culture, not strictly installing technical solutions . When there is no tradition of public accountability, there is a need for sustained additional efforts to raise the level of willing adoption of new auditing concepts and practices . The phasing out of foreign consultants in PFM may take longer than anticipated . The PFMR provided heavy support for using contractual consulting services to actually perform many PFM functions, even though a longer term goal was to develop a Ministry of Finance led system managed by the civil service. The Bank team resisted understandable impatience on the part of the government to take responsibility for financial operations, but was correct in working towards a more macro-level outcome—maintaining transparency and accountability acceptable to those providing the resources. Another important lesson drawn by IEG is that building up a country ’s public financial management system in countries that do not have a tradition of public accountability and transparency cannot be done in a short time or under one operation. It takes a number of technical assistance operations in which incremental improvements are made in the legal, institutional and human resources involved in public financial management. The PFMR was the fourth Bank operation to support PFM and reforms in the Afghanistan public sector, and was complemented by the Civil Service Reform Project, also aimed at strengthening core business practices within the Afghan government . The efforts to achieve the objectives of developing an efficient and effective public financial management system continue under the PFMRII and the Capacity Building for Results projects . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clearly written and consistent with Bank guidelines . It presents good evidence and analysis of the achievements of the project objectives . The discussion of the Bank and Borrower performance is perceptive and the ratings are consistent with the analysis . The lessons learned are insightful and based on evidence and analysis . However, the discussion of efficiency is cursory and that of fiduciary compliance is insufficient, particularly for a country that has no tradition of public accountability and transparency. a.Quality of ICR Rating : Satisfactory