DocuiDent.ef The WOrld Bant Repo.rt No: ICROOOOtl74 IMPLEMENtATION COMJ>LJ;nON ANDRESlJI:rS REPORT (lBRl).72530) ONA' LOAN .IN THE AMOUNT OF USSl2.l2.MlLLION . ~nm FEDERATIVE REPUBLIC OF BRAZIL F9RA '.' " - ' - ,\ - . ' - , -,' \ .' StJS'J'!JNABLE AND EQUITABLE GROWTH TECHNICAL ASSISTANCE , PROJECT IN Sl1PPt')RT OF THE FIRST PHASE OF THE SUSTAlNA8LE.AND EQUITABLE GROWTH TECHNICAL.ASSISTANCE .- PROGRAM June 3, 2011 - , Finance and Private Sector Development ·BrJzil.Coontry Management Unit . ~ Atnerica and the Caribbean Region • "'~~cY~EQUwAtbtTs ~• • ~m~ti'~¢APnI~,.20111 -. I': t' ,curtencyUtU(";;'IkazitiariW ' BRLl,~~tJS$'O.634' " USSI.OO.J.'S7: BRL ',' ~_~I~XJAlt • y-'" , ",', . - ,,,- ".A8BUVlATIONSANDACRONYMS ANTAQ' National waterborntt~ R~ Apncy, AN'IT- ' NationaiLandTI'8IlSpOrt Regulatory Ageney . \ ' ,.APL Adaptalne }SrpgramLo8n"" J " CIiDE Economic Defense ~ii ' CAS Country'ASmstange StrateiY ' .CDM' Clean Development M~stn CVM Securities and EXehangeCommissi()D UNIT Na$ional'.~ forTJ1II)Spbrt ~ .DPL\ ' Devel~Por ", ',Lending' ' "'1'_ " ICY. , FM FinancialManagelllent, , " , ' " FFMTAL , 'Fiql~Finarici81 M~erttTecluiicalAssiStaneeLoaa ,'GDP ~' Gross·'~',Product,' ~ " '\" ' ,HR'I cHuman~", .," ,Ieee brter..in~ commissiOn for Climate Chailge leR ImPlemeDtation, COlnpleti~ anciRe$11lts'RepQrt lOis' 1n~"OutplltIndic8tors IRI , Jnvestme1lt Rateoflnterest ISR ImpJ~tation'Statusand R~Report KIRs, Key Implenientation Results MOle , Ministry ofDeyelopment, Jn4ustnr an"~,-.,, ~,~~,_ ' Susan G. Goldmark ,!,,,,,,c,,,,,,,,,,,,~,"W'"" Pr()ject Team Leader: iRo~elio C. Marchetti Aymeric-Albin Meyer ICR Team Leader: Rogelio C. Marchetti I<:::,~.,~riIIll:1ryJ\lIt~or: Eric Palladini 11 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of this project was: to assist the Borrower in preparing and implementing the next phase of its national economic growth program. This project was part of a technical assistance program, the overall objective of which was to assist the Government in implementing its economic growth program, as described in the Minister's of Finance development policy letter dated January 20, 2004 and in the Program Document for the First Programmatic Loan for Sustainable and Equitable Growth (Report n. 27507-BR). The technical assistance program was intended to contribute to the objectives of a strategically selected sub-set of activities of the government's growth program, and thus to sustainable and equitable economic growth, and to poverty reduction. This was to be achieved by (1) reducing logistic costs, (2) improving the business environment, (3) enhancing the efficiency and depth of the financial system, and (4) increasing innovation capacity. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) . Original Target . Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Critical mass of results achieved. Value quantitative or 0 'd' 14 o fI6 In lcators.120f14 d' t 12 of 14 indicators In lca ors Qualitative) Date achieved 12/31/2005 12/31/2008 12/31/2008 12/20/20 I 0 The original success criteria called for satisfactory completion of 14 out of the Comments original 16 KIRs (87.5%). At restructuring 5 were eliminated and 3 added. Then, (incl. % ,out of the remaining 14, 12 (86%) are successfully (see Annex 10 for detail achievement) re~ult~). (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Revised Achieved at Indicator Baseline Value approval Completion or Target Values documents) Target Years Customs strategic plan and plan for re-engineering of Customs clearance Indicator 1 : proceduresliPproved and disclosed (2). Value KIR#1 (quantitative None Diagnostic of Brazil: or Qualitative) customs done Date achieved 12/3112008 12/20/2010 III iSatisfactory Comments Diagnostic of Brazil customs done by World Organization of Customs. (incl. % Elaboration of a Customs .achievement) Mode~i~~i()ll~lall.~a~d ~p.,?J1~ell~ve dillso:ostic' ... . . Customs strategic plan and plan for re-engineering of Customs clearance Indicator 2 : procedures approved lind ~isclosed (2)~ Value Diagnostic of Brazil: (quantitative None customs done ''?!. Q~llli~ti,,~~ ... . _ . Date achieved 12/31/2008 12/20/2010 KIR#1 Comments Satisfactory (incl. % Diagnostic of Brazil customs done by World Organization of Customs. achievement) ;Elaboration of a Customs .'M()dernization Plan ~1l~~uP'?J1 the above.~illso:ostic . .. Port reform plan, dock companies restructuring program, and structure of Indicator 3': ANTAQregulatory accounting system approved and .disclose~(3). KIR#2 The Agencia Nacional de Transportes Aquaviluios (ANTAQ) developed 2 (ofthe Value 3) relevant studies: (quantitative None .1) Evaluation of :or Qualitative) Port Management Performance; 2) Elaboration of guidelines for the Promotion and Defense of Competition in the ,Ports Subsector. Date achieved 12/3112005 12/20/2010 Moderately Unsatisfactory. Comments ANTAQ in the end achieved part of what was expected owing to the fact that it (incl. % had no Agency Director for some time, which delayed reforms. achievement) ...... Indicator 4 .: Highway strategic' plan and management reform program approved and disclosed .<2).. Value KIR#3 (quantitative None The study was or Qualitative) cO~p'Ic;,~~~~ .... Dat~'''ach-jev~~~f'''' "1"2/3"iI2005'- 12/20/2010 Moderately Satisfactory. Comments Study done with funding from another Bank loan (PROGEM). (incl. % Date ·achievement) 20-Dec-2010 IV Action plan for MOT PPP capacity strengthening approved and disclosed, Indicator 5: ,structures of MoT M&E system established, and ANTI concession eva]l1ation models apPruved and disclosed (3). KIR#4 ANTI developed Value various important (quantitative None ,activities in 'or Qualitative) institutional . _....~~!l~~!l.iIlg: Date achieved 12/311200S ·]2/20/2010 :Satisfactory. Comments (incl. % The ANTI component activities strengthened achievement) ANTI and operationalized norms fo~ oversight of fiduciary and regulatory responsibilities of services conceded. Indicator 6 : Recommendations for improvement of anti-trust system processes .__.~Pp~()y~.~ __ ._~~~~~~~y.~~~~~fa~!~ .... ~()~~~t~lyS.atisfac!()ry 3.97 -",_.- . 12 . . !>_~~2_~!~~lQ .. ~.~P~~~~t.~lyS.!l~~f.~C::~~!Y...:.M~~~':l!~~Y.s.!l!~~f~ory . . ....." ... _.. ,"._ .... 4.67 -, 13 ..Q~!!>~!~!> 1~_._M~_~~)~ly_ s.flti~X~(;tol}'._M~~~~~~ly_S~t~sfa(;t()ry . 4.90 H. Restructuring (if any) . . __ .... ISR Ratings at Amount Board }lestructuri!lg Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PD~ Change DO IP inUSD ..................... -. millions . .......... ". .._.. .... ... ,- ~ ~ 12/15/2008 MS MS 2.88 I. Disbursement Profile - Original ---- Formally Revised - - Aaual 10 lit C 0 Iii !i vt 5 !S vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal This Implementation Completion Report (lCR) describes the results of the Sustainable and Equitable Growth Technical Assistance Project (SEGTAP) which supported a programmatic lending operation that was effective through most of President Lula's administration. The main objective of the Development Policy Lending (DPL) series was to support sustainable and equitable growth in Brazil. a. Macro-economic situation at appraisal The Sustainable and Equitable Growth Technical Assistance Project (SEGTAP) was prepared when the Brazilian government had already launched its "Plano Real", and was addressing hyper-inflation and increased internal and external debt. The "Plano Real", implemented during the Cardoso administration (1995-2003), included strict controls on spending, currency reform, the tightening of monetary policy, and openness to foreign investment. By the end of the Cardoso administration, "Plano Real" had stabilized the economy, and the government was able to turn its attention to a new agenda that included poverty reduction and various sectoral issues related to economic growth. I The growth agenda included institutional changes, such as judicial, transport, customs, and infrastructure reforms. In 2003, the election of Luiz Ignacio Lula da Silva brought a new administration with strong support from social movements. However the new administration faced low investor confidence, currency depreciation and inflation. The administration introduced a number of reforms, increased interest rates and cut spending. As a result, inflation and inflationary expectations were brought down. The reform agenda focused on (i) changes to the public-sector social security system and the country's system of social assistance transfers, (ii) efforts to improve economic efficiency and (iii) increasing investment and savings. The focus of the reform agenda were very much aligned with the Bank's Country Partnership Strategy. The support of the Bank to the reform agenda at the time of transition in the administration was important to improve investor's confidence. This continuity of fiscal and institutional reform contributed to an average growth rate of 4 percent, approximately, for the decade. The sustained economic growth, the additional social programs, and general economic stability contributed to an impressive macroeconomic growth record and significant reductions in poverty levels. By the end of the Lula administration, Brazil ranked seventh among the world's largest economies, government debt was rated investment grade, and the country had become an International Monitory Fung (IMF) creditor. IWorld Bank. Brazil; Forging a Strategic Partnership/or Results OED Evaluation. Washington, DC. 2004 b. The Development Policy Lending loans I and II At the time of project design (2003), the Bank's practice was to provide targeted technical assistance to specific sectors with growth potential through programmatic Development Policy Lending (DPL) accompanied by technical assistance loans. To that end, the Bank planned a series of three DPLs supported by an APL series of TA loans. Two DPL loans (First and Second Programmatic Loan for Sustainable and Equitable Growth loans IBRD-72180 and IBRD-73860) were approved and implemented (see implementation completion ratings below). The third planned DPL was not pursued, since the country's fiscal position had improved considerably. For the same reason, the second DPL was cancelled after partial disbursement and only one TA loan was approved to accompany the DPL series. These two DPL projects targeted four topics: (i) logistics and transport, (ii) the business environment, (iii) the financial sector, and (iv) innovation. The objectives of SEGTAP were determined by the objectives of the DPL series, which gave the reason of being for the loan. The POOs of the DPLs were listed in" the respective document with identical wording to those of SEGTAP i.e. "support the sustainable and equitable economic growth potential". BlIZilrilst ~Lollifbr: isustaUable ud "----------------------"".-.~~~.~~-.---~ "" 1Cfr! ~Rb~,>,"" }~RD~ 12~8o__ CR Type ~ ______.f:.EN ICR ____,,_. __~ _~_: GOVERNMENT or __________ B~~ ________._.__, IUSD516,2M """"'_< _ _""" __ N""' _ _ _ _' ; ~~----_r----~.----~------------------~: Secaad Pmcrmmatic Sustaimble ud IE itabJe GroNth LOlli" IBRD-73850 CenICR IBRAZILIAN IFEDERAL GOVERNMENT--M!' IUSD 150.CIvI 2 ~~.~~~~.~!Jr.t.J:":~~~.~~~.~:.~~~. . . .. ~!:~..___.___..""~... _... _."_. __ ._._.__ ...... ..... de~1!~.~_~~~~.~... __.. _ _. . . . . _ . ___ .. Risk~P~Io~!~_~~. ____.__. __._ . ..___ .~1!.... __ ._._.." .__ .__ .... __ ... _ __. ___"._. __ J ~mc Pe~~______~______ .~ .--.-------~.--.----.--------1 BCDM'er PeJf'armm:e atisiic!my l ~~~~~.IiQw.~~I:~===~:=~==~:= 0u1l:amas ' . Risk to Develo ntOutao_ The Sustainable and Equitable Growth Technical Assistance Project (SEGTAP) was approved on July 8, 2004 and became effective on October 31, 2005 due to the lengthy Senate approval process. SEGTAP's approval followed the first OPL loan as part of an effort to facilitate progress towards and compliance with indicators and programs targeted by the OPL loans. The PAD further contemplated a second-phase Adaptable Program Loan (APL) to be triggered upon "satisfactory project implementation and satisfactory contribution of the project to the policy development and dialogue on the implementation of Brazil's economic growth program." This ICR is set in the context of the overall program of loans, including the OPLs. The two OPL-funded projects were rated moderately satisfactory because significant legal and institutional changes were implemented in the original four areas many of which had measurable development inputs. However, because there were no follow-up activities from a third project, some of the initially planned reforms were not completed. 2 Both OPLs were rated moderately satisfactory when considering outcomes. c. Rationale for Bank Assistance The Bank's Country Assistance Strategy (CAS) of Oecember 2003 saw a role for the Bank in supporting the government's growth agenda through a series of OPLs that would provide the Government with the funding needed to move forward. As part of the engagement with the Government with the OPL a set of companion programmed loans that would provide the necessary technical assistance in key areas of the reform. The CAS identified high logistics costs and inadequate transport infrastructure as having major negative impacts on Brazil's competitiveness. In addition, the CAS identified a number of constraints affecting the country's business environment (administrative costs, regulations, anti-trust enforcement, and corporate insolvency) and financial system. Finally, the CAS asserted that an improved innovation environment (such as links between universities and the private sector and an innovation law). would enhance Brazil's growth agenda. The Bank's program, as described in the CAS, sought to address these and other challenges, beginning with the OPL loan series and related technical assistance .loans. 3 2 World Bank; Implementation, Completion, and Results Report for a Series of Programmatic Loans for Sustainable and Equitable Growth - Loans I and II, Report # ICR0000972, p. 7. . 3 World Bank. Country Assistance Strategy 2003-2007 for the Federative Republic of Brazil in Support of A More Equitable, Sustainable, and Competitive Brazil, 2003. 3 The DPL series and the companion SEGTAP were then aligned with the vision of the CAS. The two DPL loans (#71280-BR and #73860-BR) contributed to this progress in four areas: logistics and transport, the business environment, the financial system, and innovation. In keeping with this line, SEGTAP's Project Appraisal Document (PAD) recognized Brazil's recent "deep macroeconomic reforms," but pointed out that these first-generation reforms were not enough, in the absence of other microeconomic and institutional reforms. Brazil's unemployment rate was high and its growth lagged behind other Latin American economies. Despite liberalization, trade remained low compared to other large· economies. Privatization in transport and electricity had not generated sufficient competition or expansion of service to support strong economic growth. 1.2 Original Project Development Objectives (PDO) and Key Indicators The project's objective which was aligned with the DPLS was defined "to assist the Borrower in preparing and implementing its economic growth program." The program called for an Adaptable Program Lending (APL) series of Technical Assistant loans to assist in implementing the program. The main objective of the related programmatic DPL lending was to support sustainable and equitable growth in Brazil. The program would raise Brazil's sustainable economic growth potential, thus increasing employment and reducing poverty. The T A loans were designed to be flexible (PAD pages 7 - 8). New activities consistent with the growth agenda could substitute other activities that had been completed or were no longer needed. The inclusion of new activities was to be subject to, among other considerations, an official request from the Borrower. The activities consisted, among others, of the provision of technical assistance and training services, office equipment, and software to the project beneficiaries to carry out activities needed to achieve the project's objectives. The technical assistance program was to help key Government entities in developing the capabilities needed for preparing and implementing its economic growth program of the Borrower, Additionally assist in moving forward micro-economic policy and institutional reforms, and be ready for the implementation of the next phase of the growth program. Specifically, the TA program focused on the needs of four Ministries to move forward five secondary objectives key to sustainable and equitable growth: • Improve productivity and facilitate trade by reducing logistics costs, increasing the effectiveness of customs, reducing port costs and delays, reducing road transport costs, and promoting efficient multimodal transport options; • Improve the business environment, by strengthening the regulatory framework for infrastructure, anti-trust enforcement, corporate insolvency resolution, and by reducing administrative costs to. businesses; • Enhance the efficiency and depth of the financial system, by increasing competition in the banking sector, fostering mobilization of long:-term capital through strengthened markets for risk and stronger institutional investors in the insurance sector, and improving access to financial services; • Increase Brazil's technological innovation capacity to transform knowledge into productivity gains, by increasing the· effectiveness of public research and development (R&D), improving the incentive regime for private R&D, and structuring/managing the Clean Development Mechanism (CDM); and • Support reforms and institutional measures to achieve other objectives of the Government's growth program, such as deepening trade integration, reducing tax distortions, and strengthening labor markets. Within the first four secondary objectives, the PAD identified the following results: (i) Reduce logistics costs: • Strategic plan defined for Customs' modernization, clearance procedures reengineered (with time release measurement in place), automation of Customs' systems improved, and port and cargo security enhancement plan approved; • Port reform and dock companies restructuring plans prepared and approved; and • National Waterborne Transport Regulatory Agency (ANTAQ) regulatory accounting and performance monitoring systems established; • Highway strategic plan and management reform program prepared and approved; and • Public-private partnership capacity and program monitoring and evaluation system established in the Ministry of Transport; and National Land Transport Regulatory Agency (ANTT) concession evaluation models operational. (ii) Improve the business environment: • Anti-trust system capacity strengthened through improved processes, systems, and training; • Bankruptcy framework strengthened through appropriate regulations and training of judges and court personnel; and • Performance assessment of judicial system in resolving economic disputes in public and private sectors completed. (iii) Enhance efficiency and depth of the financial system: • Diagnostic studies and action plans prepared for reducing the cost of credit and reducing financial spreads through studies on fmancial system competition; • Regulations and procedures in place to ensure application of anti-trust law in the banking sector; • Studies undertaken on financial access for the poor which includes an evaluation of recent measures adopted; and • Action plan developed for strengthening insurance regulations and supervision. (iv) Increase technological innovation capacity: • Ministry of Science and Technology's sector funds monitoring and evaluation system established and reform plan approved; • Capacity for appraising and certifying clean development mechanism projects in place; • Pilot internet technology/social inclusion project completed and evaluated; and • Ministry of Science and Technology's strategy to foster nanotechnology industry growth approved . . These intermediate results could be adjusted in the course of project implementation. 5 1.3 Revised PDO and Key Indicators, and reasons/justification The Project objectives were originally designed taking into consideration: (i) the high demand from several agencies of the Borrower involved in the execution of the Program; (ii) the limited fiscal space to accommodate a larger loan for technical assistance; (iii) an agreement reached between the Bank and the Borrower in relation to the importance of a long term relationship aimed at addressing key issues affecting Brazil's growth. Therefore, the Project aimed at benefiting four ministries (Finance, Justice, Science and Technology and Transport). At mid-term, the supervision team reviewed the progress of the components and recommended a formal restructuring, which was approved in December of 2008. The World Bank·and the government also agreed to a two-year extension of the closing date, but a fiscal constraint in 2009 (a two-year loan extension would have required a special budgetary approval) led to a one-year extension. Subsequently, after the first year, the loan was extended an additional year, meeting the Borrower's request. At the time of the restructuring the initial intention of additional Adaptable Program Lending (APL) was abandoned. The restructuring sought to "strengthen implementation arrangements and facilitate timely disbursement" of funds. It also: (i) added a new component on fiscal and public resource management; (ii) strengthened the financial component with new activities to be developed by the stock exchange regulator, the Securities and Exchange Commission (CVM); (iii) reassigned project implementation duties to the Treasury Secretariat, in the Finance Ministry; and (iv) eliminated: a component on innovation and technology 4 • These modifications did not change the environmental classification. The three new indicators listed below were added to the existing indicatorss: • The development by the Ministry of Planning of a taxonomy of the existing models of public institutions inCluding, inter alia, their funding, financial and administrative internal controls, accountability, and labor regimes; • Preparation of an Operationalization for Economic Co-operation and Development (OECD) peer review on human resource issues in Brazil; and • Operationalization by the Treasury Secretariat of specific processes (risk assessment, planning, strategies, auctions, budgetary and financial programming) for public debt management completed. 1.4 Main Beneficiaries Direct beneficiaries were the four ministries, secretariats, agencies, and authorities involved in the project. In the Finance Ministry, the beneficiaries included the Customs Administration (SRF), the Economic Policy Secretariat (SPE), the Insurance Supervisory Agency (SDSEP), the Economic- Monitoring Secretariat (SEAE), and the stock exchange regulatory agency, the Securities and Exchange Commission (CVM). In 4 Disbursement of the component for the Ministry of Science and Technology were stalled because of the Ministry's insistence on sole source procurement. As this was not consistent with Bank's rules, the Ministry opted to drop out of the program. S Due to the elimination of the Ministry of Science and Technology, the associated five indicators were not to be tracked, but were not eliminated from the ISR system. 6 the Justice Ministry, the beneficiaries included the Economic Law Secretariat (SDE), the Judicial Refonn Secretariat (SRJ), and the Economic Defense Council (CADE). In the Transport Ministry, the beneficiaries included the National Transport Policies Secretariat (SPNT), the Transport Infrastructure Department (DNIT), the Land Transport Regulatory Agency (ANTI), and the Water Transport Regulatory Agency (ANTAQ). In the Science and the Technology Ministry (MCT), the beneficiaries were the Executive Secretariat (SE), the Strategic Scientific Development Policies Secretariat (SEPED), and the Strategic Development Policy Secretariat (SETEC). After restructuring, the Science and Technology Ministry was removed, while the Ministry of Planning (MPOG) and the Treasury Secretariat (STN) were added. 1.5 Original Components The project provided technical assistance, training services, office equipment and software to the four ministries, to carry out activities needed to achieve the project's objective: Below there is' a summary of the components. Ministry of Finance Components (US$4.3 million) a. Customs Administration (US$2.98 million) The component consists of the first set of activities needed to implement the recently approved customs modernization strategy: (i) Strategic plan for Customs' modernization (ii) Reengineering of Customs procedures and integration of systems (iii) 'Port security improvements b. Secretary of Economic Policy (US$0.42 million) The component consisted of studies for the efficient development of financial markets and the reduction of spreads arising from weak creditor rights. c. Secretariat for Economic Monitoring - SEAE (US$O.3 million) The component sought to strengthen SEAE allowing a more efficient involvement of the Secretariat in the analysis of the economic impacts of anti-trust cases. d. Insurance Supervisory Authority - SUSEP (US$O.6 million) The component was set to provide the platfonn for the institutional development of SUSEP and the continued improvement in the regulation, supervision and oversight of the insurance sector. Ministry of Justice Component (US$ 1.61 million) a. Secretariat of Economy Law (US$O.77 million) The component was designed to provide support to SDE prepare for the implementation and enforcement of the new bankruptcy law. b. Secretariat of Judicial Refonn (US$O.24 million) The component was set to help design improvements to the judicial system to resolve economic disputes in the public and private sectors (e.g. to collect bad debts), and to 7 enforce property·rights and contracts. The aim was a more reliable judicial environment in which business would take place more efficiently. c. Anti - trust Council - CADE (US$O.60 million) The component was devised to help strengthen the recently-created Economic Defense Council (CADE) efforts to improve the Brazilian anti-trust system. Ministry of Transport Component (US$3.14 million) a. Secretariat for National Transport Policies (US$O.36 million) The component was structured to support the development of the Transport Ministry's integrated, long term planning and project financing capacity. b. Administrative Secretariat (US$0.36 million) The component was structured to support Government efforts aimed at improving the operational performance reducing the average cargo transit time through ports for both imports and exports. c. National Department for Transport Infrastructure - DNIT (US$0.40 million) The component was developed to strengthen the DNIT's capacity to manage federal road network. d. National Land Transport Regulatory Agency - ANTI (US$I.07 million) The component was arranged to help strengthen the recently-created agency and to improve the regulatory framework in the road, railways and interstate bus transportation sectors. e. National Water Transport Regulatory Agency - ANTAQ (US$O.95 million) The component was designed to strengthen the recently-created agency, as well as improve regulatory framework for ports and water transportation. Ministry ofScience and Technology Component (US$2.49 million) a. Executive Secretariat (US$O.95 million) The component was devised to help the Ministry: (a) strengthen its capacity to improve the efficiency of public expenditures in the sector, and (b) strengthen Brazil's innovation framework. b. Secretariat of Strategic Scientific Development Policies (US$O. 71 ) The component was arranged to support the strengthening of the Ministry's capacity to structure and manage the clean development mechanism in Brazil. c. Secretariat of Social Inclusion (US$O.65 million) The component was structured to support Ministry's efforts in fostering development in poor communities, providing access to computers and Internet to groups of small producers for which such access would allow them to generate significant productivity gains. d. Secretariat for Strategic Development Policy (US$O.18 million) 8 The component was devised to support the Ministry's efforts to foster the development of nanotechnology industry in Brazil. 1.6 Revised Components At restructuring, the supervision team worked extensively with the Borrower to define an updated set of activities that, while remaining within the original intent and scope of the Project, would improve disbursement and better meet the current needs of the Government. New Intermediate Results Indicators (IRIs) and activities were added. The new activities fit well into the broad PD~ and involved important previous Bank clients with relevant work programs - and with whom the Bank desired to maintain an active dialogue. The new activities were: • Strengthening the regulation of the Securities and Exchange Commission (CVM, Brazilian SEC) to develop risk based supervision and strengthening management systems. • Strengthening debt management and public resources management in the Ministry of Finance, through further development of the public debt information system. This new component would contribute directly to the strengthening of the Borrower's public resource management and indirectly to improving the business environment. • Strengthening human resource management and other administrative and budget reforms in the Ministry of Planning, Budget, and Management (MPOG), including support for budget reform and an assessment of the Government's human resources management system. • Strengthening customs administration reform within the Ministry of Finance. The following table shows 'by component (ministries), activities and the amount as they relate to each of the loan secondary objectives after the restructuring was implemented. Table 2 Secondary objectives by Ministry Enlta1M Strengtben Reduce Improve Business . . .. Emci~~e~tlnd Public Ministr'! Logistics Costs . . . . Ibviromnept Dept~ Ple ~~~uree Fina m M.ltilgement U$Stnillioo USSmmiOli US$mmion tmmfUion Finance 2.98 0.5 1.82 2.00 7.30 Justice 1.20 0.15 1.35 Transport 2.24 0.40 2.64 Planning 0.5 0.71 0.20 & Budget Total 5.22 1.70 2.17 2.50 12.00 (*) The US$0.45 million that was unallocated before the restructuring was used to absorb the amount spent by Ministry of Science and Technology and the rest was reallocated among components. (*) IfUS$O, 12 million of front end fee is included the amount is US$12.12 million, 9 1.7 Other significant changes The second extension (December 2009) was intended to solve the budgetary time constraints the Government experienced in the first extension, when a longer time period could not be accommodated. The extension also provided the time to complete procurement and project work. An important change to improve project implementation and increase the pace of disbursements was to move the Project Coordination Unit (PCU) from the Ministry of Finance Economic Policy Secretariat (SPE) to its Treasury Secretariat (STN). Because participating agencies had no prior experience in Bank project implementation, the proactive role of the Project Coordination Unit (PCU) was critical. The restructuring provided the opportunity to address this problem. The sm was a good choice because it had recently completed a large debt management component in the Fiscal and Financial Management Technical Assistance Project (Loan 4604-BR) and it was a frequent interlocutor with the Bank. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry - The Project was designed to support the activities developed under the DPLs program. As can be seen in Annex 2, the links among the two DPLS and SEGTAP were well aligned and structured. Project design was based on previous Bank experience and given the complexity and broad scope of activities, a dual project coordinator responsibility together with project coordinators in each participating Ministry was recommend. The design team recognized the complexity of the supervision and tried to systematize the monitoring process by tracking a composite PD~, reflecting accomplishments of the four ministries. A. Assessment of Project Design Implementation was the responsibility of two Project Coordination Units (PCUs), one in the Economic Policy Secretariat of the Finance Ministry and the other in the Executive Secretariat of the Ministry of Science and Technology. There were also three project coordinators (PCs) in the National Department for Transport Infrastructure (DNIT), the National Agency for Land Transport (ANTI) and the National Agency for Waterborne Transport (ANTAQ). The Finance Ministry delegated procurement responsibilities to the United Nations Development Program (UNDP). The PAD advocated a decentralized implementation arrangement as "key to ensure satisfactory implementation performance." However, this support for dissimilar activities across a large number of agencies in a decentralized management structure resulted in implementation problems for the project. Few agencies had recent and relevant experience working with Bank-financed projects. This, combined with the relatively small amounts of funds and relatively high transaction costs, led to low disbursement rates. The PCU in the Economic Policy Secretariat was not able to manage the overall project, since it lacked the status and leverage to push the ministries and agencies to implement their programs efficiently. Centralizing greater authority in the PCU or the ministries might have been viable alternatives. However, to the credit of the Bank project team 10 this issue was recognized and actions were taken. During the final two years, after the project's extension, the PCU was moved to the Treasury Secretariat in the Ministry of Finance. B. Ad~quacy of Government's Commitment The project was designed to support a government facing specific challenges. However, by the time of effectiveness in 2005, the country had already shown signs of economic growth that were unprecedented and could not have been predicted. By 2010, Brazil's economy had become the seventh largest in the world; its government debt had improved to investment grade quality; and it was an IMF creditor. The budget support represented by the DPLs and the project was no longer as critical. In this new scenario, the implementing agencies used USS2.0 million of their own resources when they were not able to meet Bank procurement requirements. This demonstrates the high degree of relevance of key activities supported by the project. This also demonstrates adequate Government commitment. C. Assessment of Risks The PAD rated the project as "high-risk, high-reward." The two main risks were insufficient budgetary allocations from the government and inadequate supervision from the Bank. The PAD recognized that the project was complex, given the large number of beneficiaries and activities and the inclusion of diverse sectors. The document stated that successful implementation would provide a "base for a continuous, extensive and integrated dialogue on economic growth" with key stakeholders within the Government. To that end, the project would require a "strong commitment," "effective project supervision" and "adequate resources" from the government. In response to this requirement, the project organized a ''joint coordination mechanism" to evaluate implementation progress, develop action plans to address major issues, and reallocate resources from weak to stronger government ministries and agencies. Procurement arrangements were addressed in the Project Appraisal Document but, despite the "high risk" rating, there was no proposed action plan. In 2003, it seemed reasonable to assign project management to the existing Project Coordination Units (PCUs) and project coordinators (PCs) in the participating ministries. However, capacity in the ministries and agencies was weak, given a lack of experience with Bank procurement requirements and practices. Project preparation did not fully recognize the degree of risk posed by working with multiple government ministries with little experience with procurement and project management and monitoring. As such, training for the PCUs and PCs was not contemplated. 2.2 Implementation The project was declared effective on October 31, 2005. There were early impressive successes in the areas of customs, transport and road networks that led to reductions in logistic costs. The Units responsible for restructuring the administrative procedures and developing regulatory framework for railroads and buses as well as for the valuation models for the concession supervision system quickly hired the key consultants and began to implement activities. The same can be said for those counterparts responsible 11 for improving business environment by strengthening infrastructure regulation reinforcing the legal and institutional framework for Private Public Partnerships (PPP). Implementation Status Reports (ISRs) in 2006 indicated the loan had a high likelihood of achieving development objectives given that (i) the Government growth agenda was on track, (ii) a number of project beneficiaries had undertaken most of the background work necessary for implementation of a first phase of project activities, and (iii) many of them were even utilizing significant amounts of their own resources to initiate activities. Since the four-year project was only declared effective in October 2005, a midterm review was undertaken in November 2008, discussions were timely regarding the need to restructure and extend the project closing date, as some project activities were delayed and the Government indicated interest in additional activities. The 2008 restructuring was significant, justified and fit in well with SEGTAP's PDO. The demand-driven reallocation to align the activities of the loan with the new priorities of the Government was consistent with the flexible structure that was envisioned for the loan. In addition, the supervision team worked with the Borrower to provide additional support to those ministries and agencies that had shown implementation capacity and could boost disbursements over the two year extension. To facilitate SEGTAP implementation, an experienced consultant was retained in Brasilia. The consultant was an important player in keeping the project moving forward and advising the new Bank task managers of political and operational matters. The problems of slow disbursement and poor performance of some of the participating agencies might not have had ready solutions at the time, given the evolving political economy and project design. Procurement was an issue requiring sustained attention. Procurement was a major contributing factor in project delays due to coordination challenges. For example, the Ministry of Science and Technology withdrew from the project over a single disagreement regarding a sole source contract. The "Procuradoria Geraf' ruled against the Department of Transport Infrastructure for failure to follow the national Law #8666 on procurement. Procurement problems also included difficulties working with UNDP, which complicated implementation of the new components following restructuring. In several instances there was disagreement as to which procurement rules were to apply (UNDP or the Bank), for example with the purchase of computer equipment. In the four-month grace period following loan closure, a case arose regarding information technology (IT) purchases and consultant contracts valued at around US$500,000, which had not followed proper Bank processes. There were costs to the use of the "flexible design". Monitoring was difficult, with so many key implementation results and intermediate output indicators. In addition, some were not adequately defined, making it difficult for the ISRs to provide adequate progress on all the indicators. The ISRs focused heavily on implementation issues such as procurement and disbursement. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Project design was based on previous Bank knowledge and followed the experience available at the time for decentralization of implementation arrangements. The PAD 12 recognized that past experience in Brazil and elsewhere with centralization of implementation arrangements had caused tensions between the government entity responsible for implementation and the other entities with lower accountability. The design team recognized the complexity of the supervision of a decentralized arrangement, and tried to systematize the monitoring process by tracking a composite PD~. The ISRs were structured to track a single composite PD~ over the life of the loan: "critical mass of 14 out of 16 Key Intermediate Results achieved." Results Indicators were described as "(an unspecified) number of related activities funded under the project satisfactorily completed." Results Indicators were described as "corrective measures during activities' implementation for quality assurance and timeliness of products to be delivered." Further description included "Reallocation of funds from non-performing to performing activities / entities." The activities aligned with the DPL series however, were not detailed in the Results Framework. The original project target of achieving 14 KIRs was not adjusted during restructuring. Five KIRs associated with the Ministry of Science and Technology activities were no longer relevant when the component was officially dropped at restructuring. Three new indicators were added (regarding planning, human resources, and public debt management), resulting in 14 KIRs. The original criterion for success called for the successful completion of 14 out of the original 16 KIRs. As a result it was unclear how many KIRs would satisfy the equivalent of the original project compliance target of 14 out of 16. Monitoring and Evaluation (M&E) and, subsequently, the ISRs process, were complicated due to the issues mentioned above. The complexity of the project design, with four participating ministries, made tracking costs difficult. The cost accounting system tracked costs by ministry and not by KIR or activity. In some cases UNDP and non-UNDP transactions needed to be aggregated and reported. The project had. two distinct phases, including (i) the design and problematic first implementation stage, which also saw some impressive gains in customs, transport and logistics costs, followed by (ii) the phase of restructuring and smoother implementation. Depending on the phase being evaluated, the outcomes and performance of key actors (the Borrower, the Bank and the implementing agencies) are very different. These issues will be further reviewed in the respective Bank and Borrower performance. Nonetheless, the relative success of the two DPLs supported by this TA Loan show the significant value added provided to the four ministries. 2.4 Safeguard and Fiduciary Compliance Safeguards. There were no safeguard issues. The project was categorized as a "C." There were no direct environmental implications. Fiduciary Compliance There was one covenant which required annual progress reviews and adjustments where needed. The covenant was rated "in compliance" every year and the Progress Reports were useful. 13 Financial Management (FM) was rated moderately unsatisfactory in implementation status reports (lSRs) from June 2006 to December 2007. A June 2007 annual audit detected several errors and incorrect actions executed by the project. Based upon this report, the task manager sent a letter to the project coordinator suggesting urgent measures to correct flaws. The letter pointed out that the FM rating was unsatisfactory because: (i) an action plan was not observed; (ii) a supervision mission was postponed because of a lack of PCU structure at Treasury; (iii) staff were not formally assigned to the project; (iv) weak internal controls; (v) a need for an updated Operations Manual to reflect the new PCU; and (vi) the delay in replying to the Bank's review letter pertaining to the 2006 audit report. After the Bank's audit review letter was sent, the project coordinator took corrective measures to solve the issues. A financial management supervision mission in 2007 reviewed the situation and found that most issues were resolved and the remaining ones were in the process of being resolved. From there on, FM arrangements were rated moderately satisfactory in all ISRs until the closure of the loan. The involvement and experience of the Government Audit Department played an important role in keeping the project aligned. The Audit Department was involved in many other activities related to the Bank and was quite knowledgeable about both Bank and Government procedures. The approach utilized during the audits was very helpful in addressing the discrepancies between Bank's and Borrower's procedures. 2.5 Post-completion Operation/Next Phase There are no transition arrangements for a follow-up project. Although the PAD had envisioned a possible APL series, significant changes in circumstances and the impressive rapid Brazilian economic growth made this less attractive and appropriate. In fact, the issue was not included in the restructuring and there is no project under implementation or in the pipeline to carry on SEGTAP interventions. Nevertheless, SEGTAP's work is very important for the government agencies, and they are using their own resources to advance the agenda and continue many of the activities that were initiated by the project. These agencies include, inter alia (i) the Ministry of Planning on strategic planning and completion of studies relevant to its organization, (ii) the Securities and Exchange Commission (CVM) on deepening of capital markets, (iii) the Treasury Secretariat on the final phase of managing government debt and (iv) the Justice Ministry's Conselho Administrativo de Defesa Econ6mica (CADE) on refinement of the Insolvency Law. Brazil's recent growing pains pose significant challenges and provide important opportunities for more lending and technical assistance. Useful follow-up might begin with a dialogue with the new administration regarding useful public sector management reforms and how the Bank can help. One option could be a large judicial sector loan to strengthen some reforms. In addition, there is much more to be done in areas first developed by SEGTAP, such as debt management in the Finance Ministry and in the Planning Ministry. Other significant opportunities include strengthening governance and public sector management, supervision of gov~rnment pensions, innovation, and sub-national fiscal management, to name a few. 14 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating: Moderately Unsatisfactory SEGTAP was developed to support and extend reforms begun under the first DPL project. Its design faithfully reflected the DPL prior conditions and was aligned with the Brazil's growth agenda, as described in the Bank's Country Assistance Strategy (CAS) of December 2003. The CAS called for a focus on creating a country that was "more equitable, sustainable, and competitive." The CAS saw a role for the Bank in supporting the government's growth agenda through programmed loans that would provide the necessary technical assistance. The initial DPL series and SEGTAP were designed together to reach those goals by supporting the government's "efforts toward higher productivity and investment." The participation of four ministries. the KIRs and 101 activities were driven by the DPL series. The definition of the PD~ as facilitating the implementation of the DPL growth agenda resulted in the participation of four ministries and a high number of indicators to match identified prior actions in the DPLs. The overall objectives remained relevant throughout the implementation period, and the DPLs and SEGTAP helped consolidate progress made by providing technical advice, training, and equipment to participating ministries and agencies. Underlining the continued relevance of the SEGTAP objectives and the importance of the Bank's support, many of the project's activities, that had not been finalized utilizing Bank's funding, were concluded by the ministries utilizing their own resources. The. Bank's teams supervising the loan provided significant guidance and the local consultant in Brasilia provided essential continuity and assistance to beneficiary entities and made up, to some extent, for PCU weakness. Also, transport and logistics activities were achieved fully within the original project schedule. However, given the 51 percent disbursement rate, procurement problems overall implementation was not strong. 3.2 Achievement of Project Development Objectives PD~ Rating: Moderately Satisfactory The Project Development Objective was largely met. Many of the objectives (KJRs) and activities were partially or fully met through efforts funded either by SEGTAP or the Borrnwer's own resources, or both. These activities included a number of studies and reports, training and reorganization of agencies or processes, that improved existing capabilities. With improved capabilities these agencies were able to carry out an important agenda that relates to the development objectives of the DPLs and SEGAP. The list below describes the progress made by the different Ministries as they relate to the different secondary objectives. 15 Reduce Logistics Costs - Satisfactory: • Customs strategic plan and plan for re-engineering Customs' procedures approved and cleared. Port reform plan, dock companies restructuring program and the structure of the ANTAQ regulatory system approved and disclosed. • Highway strategic plan and management reform approved and disclosed. • Action plan for the MoT PPP capacity strengthening approved and disclosed, structures of MoT M&E system established, and ANTI concession evaluation models approved and disclosed. Selected examples: • Customs Administration (SRF) completed reengineering of procedures, systems, risk management, bonded warehouses, and broker procedures. The government simplified export procedures and reformed the clearing systems and the management and information systems of customs. The expected outcome of reducing release times was achieved, which now can be done in 3 days (previously 5 days). An importer can clear customs in three days and an exporter can clear customs in about haIfa day (previously it took two days). According to Doing Business 2010, the time needed to complete an export decreased from 18 days in 2007, 14 days in 2009 to 13 days in 2010. Similarly, the number of days to complete an import decreased from 24 days to 19 days and to 17 days during the same sequence of years. • As part of the reform agenda, the government created the Secretariat for· Ports (SEP) and defmed policies and guidelines for port development. The Decree 6,620 issued in 2008 defines policies and guidelines for the development of the port sector are important steps setting up the stage for ports reform and infrastructure rehabilitation. Three studies were conducted to support the reform. Ports have improved their operations and port handling times have fallen from 13.8 days to four days for imports and from 8.4 days to three days for exports. • The project completed an environmental diagnosis for complete port reform, with action plans and recommendations. Port and cargo security improvements and compliance with international norms (ISPS) have also been completed. The reform of ANTAQ was initiated but it is yet to be completed, turnover in management did not help in the consolidation of the ref~rm. • The government reorganized the Federal Transport Administration and transferred the remaining trunk roads from the federal network to state management. Transport times on roads have fallen and road conditions have improved. The federal road network has passed the 30 percent target for output- based maintenance contracts. • The Transport Ministry developed its capacity in long-term planning and project financing. Developed system to manage road work: Sistema de Gestiio dos Servifos de Fiscalizafiio - SIGFIS. • The Monitoring and Evaluation (M&E) system of transport projects and programs was completed. It was designed to reduce transport costs on federal roads. This included information technology and consolidation of policies and standards for road maintenance and rehab contracts. 16 • The government restructured railways concessions and made progress in the regulation of the railways sector, although it has not yet made operational the Inter-Ministerial Committee for the Integration of Transport Policies. The work on PPP was initiated, but it is yet to be finished with Units' other resources. • The National Land Transport Regulatory Agency (ANTT) upgraded land valuation models and provided hardware and software. It developed procedures and systems for efficient concessions supervision and trained agency staff in valuation models and strategies. It also elaborated financial-economic equilibrium clause rules in concession contracts valuation models. • The ANTI also completed the regulatory and normative framework for land transport including multimodal transport. It also developed uniform rules for concessionaires and permit holders including information and data base upgrading. Improve Business Environment - Moderately Satisfactory: • Recommendations for improvement of anti-trust system processes approved and disclosed. • Action plan for strengthening of corporate insolvency law framework approved and disclosed. • Recommendation for improved performance of judicial system in resolving economic disputes in public and private sectors approved and disclosed. Selected examples: • The Anti-trust Council reformed the information system, reviewed procedures and training needs, and acquired hardWare and software expansion. • The Economic Law Secretariat (SDE) implemented important training for judges and provided hardware and software. A technical assistance program trained judges in bankruptcy court proceedings. The two courses trained 80 judges, who transferred their knowledge to their respective courts and colleagues. • The Judicial Reform Secretariat (SRJ) improved the courts system and conducted cost-benefit studies. Approval of the guidelines to simplify and integrate the registry and legalization of firms and creation of the National Network for Simplifying Registry and Legalization of firms congruent with Law 11.598/2007 • Two studies on ways to improve judicial actions in economic disputes were done and in addition a performance survey was prepared. The survey is to include recommendations for improvement. • The Economic Monitoring Secretariat (SEAE) modernized its procedures and strengthened capacity to review recommendations and resolve issues related to on-the-job training. It also financed a study to upgrade procedures to open and close firms. 6 Despite these successful refonns, Brazil is still among the countries with the most procedures and days required to start a business. The Doing Business 2011 report, which used data from Sao Paulo, can be found at www.doingbusiness.org. 17 Enhance Efficiency and Depth o/the Financial System - Moderately Satisfactory Action plans approved and disclosed for: • Promotion of compe~tive structures in banking sector • Facilitation of access to credit • Development of capital markets and institutional strengthening of SUSEP Selected examples: • The Economic Policy Secretariat (SPE) completed studies for the efficient development of financial markets and for improved enforcement of creditor rights. One study was a judicial analysis of pUblic-private partnerships. The other study focused on the micro-economy of social transfer programs. • SPE developed regulation of payroll deduction for loans was introduqed to facilitate access to credit to low income groups. • The Agency developed studies for improved financial sector competition, access to credit, and capital markets development. • Securities and Exchange Commission - CVM (added at restructuring) developed a method to assess the costs and benefits of regulating capital market, and created training materials for basic education program aimed at increasing the level of financial education of Brazilian investors. Initiated the move to Risk Based Supervision. , • The Insurance Supervisory Authority (SUSEP) was strengthened with a development plan in regulation, supervision, human resources, and infrastructure. It developed circulars, manual on procedures, training. It also was supplied with hardware and software. By the beginning of the second extension, SUSEP had already implemented the majority of the planned activities. Due to changes in priorities, the strategic planning and neural networks studies were canceled. Increase Innovation Capacity (removed at restructuring, 2008) Strengthen Public Resource Management (added at restructuring, 2008) - Moderately Satisfactory Included the following: • Development ofa taxonomy of the existing models of public institutions • Preparation of an OECD peer review on human resources issues in Brazil • Operational incorporation by the National Treasury Secretary of specific processes for debt management Selected examples: • The Planning Ministry developed the guidelines and TORs for the taxonomy studies using SEGTAP funds. The Organization for Economic Co-operation and Development (OECD) was awarded the contract. SEGTAP funds could not be 18 used to pay the OECD for the actual studies. The government paid for the studies with its own resources. • The Treasury Secretariat (SlN) completed the design for the Integrated Public Debt Management System (SID), which built capacity to forecast analyze growth and trajectory of government debt. And built capacity through training for budget execution and public expenditure management. • The Ministry of Planning. Budget, and Administration (MPOG) completed assessments in the government's human resources management system, indirect administration and budget reform efforts. Despite these accomplishments, the project faced some challenges, which slowed progress and resulted in a 51 percent disbursement rate. • Procurement disagreements with the UNDP highlighted some of the problems that resulted in some of the activities being funded by the client's own resources. The Science and Technology Ministry withdrew from the project in 2008 due to a disagreement over a single-source procurement contract for a web· site. However, these activities were part of the project's planning and implementation. Some of the implementation problems were beyond the project's control. • Beneficiary performance was uneven and many of the fifty-plus activities described in the PAD were only partially undertaken. • The PCU never claimed the Bank-approved US$I.2 million of retroactive expenqitures. . 3.3 Efficiency Rating: Moderately Satisfactory The costs in achieving project objectives were reasonable in comparison with the benefits. In the end, there was sufficient value for the US$6.22 million disbursed to justify a moderately satisfactory rating. The specific activities funded were significant, although, in some cases, the amount disbursed may have been insufficient to meet the KIR. The activities that were undertaken were important and contributed to the objectives, and their cost was reasonable. During the second extension year (final year of the loan), the project was on track to reach 86% of loan proceeds. However, due to the restructuring process of UNDP, contracts were delayed and disbursement reached 51.13%. However, the clients were significantly committed with the process to the point of using their own funds to complete activities that were delayed by UNDP procurement process. As such, and despite the low disbursement, much of what was funded by the client's own resources reflected SEGTAP's design and efforts. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory The rating of Moderately Satisfactory is justified because the project was able to meet a substantial portion of its objectives despite a low disbursement rate, an ambitious agenda and problematic monitoring. SEGTAP made an important contribution to the Brazilian government's growth agenda by contributing to a pro-reform environment that will 19 continue to be felt well after project closing. The beneficiary agencies did accomplish important reforms, and they are continuing many of the project's reform activities with their own resources on what was built with SEGTAP funding. This is especially true in the areas of judicial reform, transport, and debt management. The project's impact will surely be felt as· the government continues its reform program despite the transitory disagreements over procurement. 3.5 Overarehing Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, aod Social Development - Moderately Satisfactory SEGTAP was a straightforward technical assistance operation whose impact on poverty, gender and social development aspects was almost entirely indirect. A number of its activities supported the strengthening of SPE in its work to improve credit access for the poor, micro-credit and credit for small and medium enterprises (SMEs) during the project period. (b) Institutional ChangelStrengthening - Satisfactory There were several important laws and decrees, prepared by agencies that benefited from SEGTAP that contributed to the PD~. The following examples can be mentioned: insolvency law, bankruptcy procedures, regulations for opening and closing firms, credit access, and anti-trust, judicial system performance in economic cases, banking sector competitiveness, and capital market deepening. There were important and sustainable gains in terms of institutional strengthening of the Customs Administration, the Insurance Supervisory Authority (SUSEP), and the Transport Ministry (ANTI). These changes were the result of a combination of project strengthening agencies capabilities and government funding and deserve a satisfactory rating. (c) Other Unintended Outcomes and Impacts (positive) During the supervision missions that visited the Securities and Exchange Commission (CVM), the Bank project team came to learn about the pilot program on financial literacy being developed for high school students. The pilot is part of the Brazilian Financial Education Strategy (Estrategia NacionaZ·de Educafiio Financeira), a national priority of· the government. 7 The strategy recognizes that financial education is a potential tool for social inclusion, improving the lives of citizens and promoting stability, competition and efficiency of the financial system. The team identified implementation issues and the lack of impact evaluation of the program. An impact evaluation mission was subsequently organized and the discussions led to a request for NLT A to design and conduct an impact evaluation of the pilot project. As a direct result of this project, the Bank became a valued partner of the program that is reaching 24,000 students in 880 schools distributed in six states and the Federal District. The impact evaluation program has successfully secured over a million dollars from international and local donors and has attracted interest from the OECD and other 7 See http://www.vidaedinheiro.gov.br 20 countries in the region. While not directly related to the original loan, these programs emerged from the discussion on financial access, which took place under this loan. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N.A. 4. Assessment of Risk to Development Outcome Rating: Moderate The question of "risk" refers to whether outcomes' sustainability was at risk at the time of the ICR.. In the case of SEGTAP the question is complicated because the PD~ was designed to support the facilitation of a series of DPLs and it was defined by ambiguously worded and unclear outcomes. The SEGTAP successfully supported two DPLs, whose reforms have been sustainable and have been rated moderately satisfactory. Even though it is.hard to determine just how much SEGTAP contributed to institutional strengthening with its studies, training and processes review, almost all were very relevant for the units that undertook them. This is highlighted by the fact that many units utilizing own resources continued and finalized the studies. Sustainability risk "ambiguity" is made worse because the KIRs and the overall PDQ' were linked to the DPL series, which was truncated at 25 percent disbursement of the second DPL. Nevertheless, Brazil today enjoys a much more favorable economic and judicial governance framework than at the time of loan preparation and has weathered the worldwide recession well in large measure because of interventions taken during the period of loan implementation. Furthermore, the laws, decrees, and institutional strengthening activities completed by the beneficiary agencies using project and non- project resources are largely sustainable. For example, the insurance supervisor SUSEP strengthened its capabilities (developing circulars, manuals on procedures, training and acquiring technological infrastructure - both hardware and software); the preparation work for CVM's studies (assessing the cost and benefits of regulating capital markets) and the studies done by the Economic Policy Secretariat (improved enforcement of creditor's rights). Also many of the activities that could not be funded with SEGTAP resources due to procurement issues, have been carried out with beneficiaries own resources. On balance the risks to the development outcomes are perceived to be moderate. s. Assessment of Bank and Borrower Performance S.l Bank Performance (a> Bank Performance in Ensuring Quality at Entry Rating: Moderately: Unsatisfactory The project objectives were strategically relevant. The objectives were aligned with Government priorities and Bank strategies and the DPL loans it supported. The loan was designed based on prior research that pointed to a broad decentralized cross-cutting approach combined with a flexible design that would allow the loan to accommodate changing priorities. However, the design was overly complex with many activities and stakeholders, which led to some implementation challenges and also the achievement of the PD~ could not be easily measured. Eight factors were used to assess quality at entry 21 and come to a holistic moderately unsatisfactory rating. This reflects the lack of participation of the participating agencies in the four ministries at the design stage, the very broad project development objective, the choice of key performance indicators, and the overall complexity of the design of the technical assistance loan. This rating recognizes that the technical assistance loan matched the programmatic focus of the two DPLs, and the choice of instrument allowed the project to respond to changing needs of the counterpart and complement the work of other Bank projects. Quality at Entry: More thorough initial background analysis could have detected low level of preparedness of the units that were going to implement loan activities and prepare an action plan to tackle weaknesses. Strategic: Relevance and Approach: Given the size of the DPLs and the strategic importance of the government's growth agenda, this TA loan was timely and relevant. Political Ownership: Despite problems with disbursement, the Borrower showed continued interest in the project's reform agenda. The continuation of the activities using government funds highlights the importance of the work. Policy and Institutional Aspects: At the time of preparation the operation was well aligned with the government's growth agenda of the DPLs. The project's reform agenda design remained unchanged throughout the life of the loan. However, the institutional capacity was not well assessed. Technical, Financial and Economic Aspects: Indicators were congruent with DPL and CAS objectives and remained so throughout the life of the operation. However, some of the participating entities were ill-prepared to carry out a Bank loan. Bank Inputs and Processes: There was sufficient knowledge of the components within the Bank's team, which included experts in most of the areas covered (transport, finance, insurance, environment, legal, etc.) There were synergies in the supervision with the teams that managed the DPLs loans. The interactive and open dialogue provided an opportunity to engage with the counterpart in other areas (such as financial literacy). Risk Assessment: There were gaps in the risk assessment specifically given the proliferation of beneficiary entities with little or no experience in Bank lending. Preparedness of some of the units involved could have been better assessed so action plans to tackle the issue could have been prepared. Implementation Arrangements: Bank efforts to keep the loan on track are commendable. One of the PCU was moved to the Treasury for the final years of implementation. Realizing that the additional project coordinators did not have sufficient knowledge of Bank procedures, the team retained an experienced external consultant through the life of the loan to provide guidance, continuity and consistency. Procurement problems slowed progress and the lack of a properly specified results matrix did not provide the detail necessary for well dermed lOIs. Monitoring and Evaluation Arrangements: The unwieldy number and specification of KlR and IRI activities and especially the specification issues of thePDO, KlR and IRI results made supervision difficult and subsequently led to a focus on procurement and disbursement. Poverty, Gender and Social Development Aspects: See comments on 3.5 a. 22 Environmental Aspects: N.A. (b) Quality of Supervision Rating: Moderately Satisfactory Five factors were used to assess quality of supervision: adequacy of supervision, fiduciary safeguards, focus on development, candor and quality of reporting and adequate transition arrangements. . Adequacy of Supervision of Inputs and Processes: Given the project's complex design, the Bank had to provide close and intense supervision through the project's life. Given the 'flexible' design the Bank was proactive and restructured at mid-term to meet the changing priorities of the Government. After the restructuring implementation activity peaked. During that period the Bank had daily communication with the PCU and implementing units and provided detailed feedback on a significant. amount of "no objection" requests. For activities where specific skills were needed, the team found an appropriate expert to review the issues before issuing the "no objection." However, the supervision team could have been more skeptical about the government implementation schedules particularly when procurement problems continued surfacing. Supervision of Fiduciary and Safeguard Aspects: The audit process was comprehensive and while there were no major issues, there were several qualified audit reports. Adequate response was always provided. There were also few processing issues in the beginning. For example, Financial Management systems were slow to be put in place and their coverage was inadequate and staffing the original and new PCUs was delayed . . However, fiduciary management issues that surfaced in the early years of the loan were properly addressed by the counterpart and corrective measures were taken. The financial management supervision that reviewed the situatjon after corrective measures were taken confirmed the reversal of the situation. Focus on Development Impact: Many key outcomes were identified and achieved. Additionally, the midterm review and resulting restructuring increased the development impact by providing resources to responsive ministries and agencies. However, poor specification of the PDO and KIR did not contribute to efforts in this area. Due to poor specification, too much emphasis had to be placed on procurement plans and disbursement and too little on results to ensure development impact. Also at the time of restructuring, the team should have refocused the monitoring and evaluation framework. Candor and Quality of Performance Reporting: ISRs lacked important detail reporting on KIRs and Component IRIs in the Results Framework and questionable loan design and risk analyses. However, procurement and component progress reporting was moderately satisfactory. Role of Ensuring Adequate Transition Arrangements: The APL series was not included in the restructuring, and no formal transition arrangements were left in the pipeline. However, three ministries using their own resources continued follow-up work in these areas using TORs, action plans and technical work provided under the loan. This was a strong indication of an appropriate transition, given the major unexpected changes in the macroeconomic situation. 23 (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The project had two stages of implementation. Some of the activities were completed satisfactorily within the original project timeline (including transport, logistics and regulatory gains in railroads and buses and concession supervision). Significant effort and resources were spent on preparation, review, launch, implementation, supervision, and portfolio review. The project was complex, with four components (run by four ministries), 16 KIRs, 5 IRIs, and a composite PDO. The Bank project team proactively restructured the project to eliminate the slow disbursement components and supervision missions were conducted generally every three to four months. The turnover of task team leaders affected the overall project supervision. However, efforts were made to perform frequent supervision missions, track key indicators carefully,and hire a field- based experienced local consultant, who joined the team. in early 2007 and remained until loan completion. Counterbalancing this effort was the late restructuring, and the initial inability to force counterparts to improve slow disbursements and slow implementation. Taking into account the positive and negative aspects of the Bank's performance, the rating has been set at Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory The Federal Government was directly responsible for the design, and there is little evidence that implementing agencies participated in a consultation process during design. The Federal Government was the actor responsible for the reform framework that led to rapid economic growth during the life of the project. In this regard a top-down approach with the Borrower's imposition of some parts of the growth agenda - supported· by the loan - on some of the Ministries and implementing agencies was considered appropriate in times of transition to a new administration. In addition to these considerations, nine factors were considered in assessing government performance. Government ownership and commitment to development objectives: Borrower's commitment was reasonable, despite the departure of the Ministry of Science and Technology due to a procurement dispute. In the other cases, the government was engaged and committed. Several units used their own funding (in excess of US$2.0 million) to continue the work related to SEGTAP after the loan was closed. This includes: US$280,OOO for consulting on strategic planning and development of a taxonomy study of the public sector, US$400,000 for debt management system, US$75,000 for implementation of Risk Based Supervision approach in CVM, US$150,000 for research on effectiveness of Judicial System (more detail can be found in Annex 10). Enabling environment, supportive policies; The Government maintained its commitment to support the reforms developed in the original program. In line with the objectives of 24 both DPLs and SEGTAP, the Government introduced significant pieces of legislation to support the program, inter alia a legal change to unifY tax collection at federal, state and municipal level for micro and small companies, new bankruptcy law and tax codes (passed by the lower house of Congress), introduction in Congress of a new law extending application of antitrust to banks, and completion of the geographical restructuring of the railways concession. Adequacy of beneficiary/stakeholder consultations and involvement: Despite the top- down approach there was an intense dialogue with the executing units especially during and after the restructuring. Representatives of the ministries consulted the Bank for advice on project management. The most frequent topic was procurement requirements of the Bank. After the restructuring, the new PCU took a leading role in seeking advice from the Bank on procurement issues related to UNDP systems. Readiness for implementation: The initial PCU was not completely staffed; financial management and internal control needed strengthening and as late as March 2007 a Bank financial management specialist rated SEGTAP financial management as Moderately Unsatisfactory. The four ministries were also not adequately trained and staffed, which contributed to the slow start, US$1.2 million of retroactive expenditures that were approved before Effectiveness could have been claimed but due to lapses they were not. Timely resolution of implementation issues: The restructuring was justified, but irregular performance across beneficiary entities and serious procurement disagreements with UNDP remained unresolved. However, it is not clear that the client could have done anything more to resolve the latter issues. Fiduciary Oversight: Despite some technical delays in submitting information to the Bank, in general most agencies acted diligently in handling the process. One area that should be commended is the work of the Government external audit agency that oversaw the actions of the different units. This unit is well prepared and versed in dealings with the Bank. In several instances the audit agency raised questions about the eligibility of certain expenses and inadequate planning. Adequacy of Monitoring and Evaluation Arrangements: Most reports, such as end-of- year reports and other ad-hoc responses to Bank questions, were provided on a timely basis, procurement tables were updated, and there was reasonable monitoring of progress on procurement plans. However, the agencies were not focused in measuring impact. Relationships and coordination with partners and stakeholders: The initial PCU's performance and leverage with the ministries were insufficient to allow progress to pro~eed at the projected pace Only when the Ministry of Finance Economic Policy Secretariat was replaced by the Ministry's Treasury Office did the situation improve. The relationship with UNDP was a continuing stakeholder problem. Adequacy of transition arrangements after loan closing: The APL series was not included in the restructuring, and the no formal transition arrangements were left in the pipeline. However, several agencies using their own resources continued follow up work iIi specific areas utilizing work provided under the loan. 25 (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory As mentioned before (in the Monitoring and Evaluation section), the project had two distinct phases, (i) the design and problematic first implementation stage followed by (ii) the phase of restructuring and smoother implementation. This coincides with two levels of commitment and performance of the implementing agencies. During the first phase, these agencies were given a new assignment and the funds provided offered no additionality. This meant the participating agencies had to adjust their work plan to dedicate resources and staff to the activities included in the project. After the restructuring, due to the preparation work done and the involvement of the supervisor team, participating agencies performance improved. There was better coordination and buy-in from agencies However, UNDP's systems and procedures which in many cases· diverged from Bank's procedure, materially affected the rate of implementation and disbursement under the project. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The top-down approach from the new government which was undertaking a reform program was justified. Despite shortcomings in the implementing agencies performance, the project achieved several important outcomes already discussed, justifying an overall moderately satisfactory rating. The satisfactory rating granted to Borrowers Performance by the two DPLs also supports the conclusion. Many of the ministries offices found important value on the activities and utilized their own resources to complete these activities when procurement issues prevented them from using SEGTAP funding. 6. Lessons Learned First and most important lesson keep the project structure simple. An overly complex design with too many components and activities led to a project that was difficult to implement, supervise and evaluate. Also, the project specifically tried to address the possibility of change in Government priorities with flexible architecture. This flexibility comes with a price at the time of-evaluation when the original objectives had to be evaluated, although it was appreciated by the Government and matched changing conditions. Strong involvement of participating agencies. The big difference between the first phase and the restructured phase of implementation was a far greater understanding of the ways the technical assistance loan could reinforce reform efforts supported by the DPLs and the government's reform agenda. This led to a greater buy-in and an improved rate of disbursements. By giving the implementing ministries and agencies more of a voice in choosing specific activities, there was a greater response and appreciation for the reforms and for project support. Third party procurement support can be useful or complicate project implementation. The UNDP has been involved in a number of Bank operations in the 26 past. It is still being used to reduce the risk of judicial complaints and to compensate for low capacity of some government counterparts. If the SEGTAP experience is any indication, a third party whose ability and willingness to diligently and efficiently assist the Bank and the Brazilian Government to implement loans could be a questionable strategy. In the case of SEGTAP the centralization of UNDP's operational structure did not help, with the management of many activities taken care of outside the country. In addition, UNDP's insistence on applying its policies regarding procurement rules created serious delays. To a certain degree, the UNDP's approach negatively impacted the overall disbursement of the loan, forcing some of the agencies to use their own resources to complete project activities. If a stronger project implementing unit had been responsible for implementation in the early phase of the project, it may not have been necessary to rely on UNDP for this support and more efficient procurement systems using Bank practices might have increased project activity significantly. Small T ALs can be useful to support DPLs. Notwithstanding the design related problems with this TAL, it is generally useful to have such an operation in place to support programmatic DPLs and to maintain Bank involvement in reforms and a place at the table when issues are being sorted out. If this is not possible, then a fee-for-service arrangement or other possibilities should be explored. A changing political economic environment can challenge the relevance of a project's initial design. Great care must be taken in designing projects that will span more than one government administration and in adapting loan design to changing environments. Despite the flexible design there are few effective mitigation measures. A smaller number of SUb-components and implementing agencies might have been a suitable response to the risks inherent in working on government reform in a changing political economy. Also the willingness to consider cancelling the loan at an early stage should have been kept present in the realm of possible courses of action. PD~, Results and Results Framework, below is a summary of issues related to the PDQs and results. • There can be too many indicators as well as too few. • PDQs should be well specified and measurable; a PD~ that facilitates an agenda is neither. • PDQ indicators should be detailed and not indicated as a number or percentage of total indicators. They should each be monitored and evaluated individually. • PDQ indicators should be properly specified and protocols used to clarify their meaning. • PDQ indicators' dimensioning should be commensurate and aligned with loan objectives. Great care should be taken to align loan activities with the desired outcome. • The Results Matrix should be thoroughly and comprehensively specified and then downloaded to the ISR. • PD~ indicators need to be selected with care and to reflect important loan objectives. They should also be reviewed and appropriately amended during any restructuring. 27 Risk Analysis, this should be elaborated first around political economy, the Results Matrix and implementation arrangements. Expenditures should reflect Results Matrix priorities. Mitigating measures Ileed to be specific and useful, not generic. Dedicated Staff in Country, any· loan but especially complicated ones will benefit from having experienced staff or consultants on the ground, whose job descriptions and experience are adequate for supervising and supporting implementation is a good practice worthy of replication whenever warranted by circumstances. 7. Comments on Issues Raised by Borrowerllmplementhig Agencies/Partners (a) Borrower/implementing agencies .(b) Co financiers There were no other co fmanciers (c) Other partners .and stakeholders N.A. 28 Annex 1. Project Costs and Financing (a) Project Cost by Component (in usn Million equivalent) Original . Restructured World Bank World Bank Actual Percentage of Components Financing Financing (USn Appraisal (USn (USD millions) millions) millions) . MINISTRY OF FINANCE 4.30 7.37 3.38 45.86 MINISTRY OF JUSTICE 1.61 1.35 0.36 26.67 MINISTRY OF TRANSPORT 3.14 2.64 2.20 83.34 MINISTRY OF SCIENCE AND 2.50 0.10 0.04 TECHNOLOGY MINISTRY OF PLANNING (Component added at restructuring) 0.00 0.70 0.18 25.71 Unallocated 0.45 0 0.00 0.00 Total Program Cost 12.00 12.06 6.22 51.1 Front-end Fee 0.12 0.06 0.60 100.00 Total Financing 12.12 12.12 6.22 51.13 (b) Financing Appraisal ActualILatest Type of Co Source of Funds ."st'Ima t e . I'..St'Ima t e . PercentageIof '" . '" financing (USD millions)(USD millions) ApprSlsa . Borrower 2.28 2.28 100.0 International Bank for 12.12 6.22 51.13 Reconstruction and Development 29 Annex 2. Component of DPL Loans 1 and 2, and SEGTAP Activities DPLl ................................. > DPLl ~ "-C;""!;'i-] .•... I Results Framework ..... I~ lt~~,eeLogisticsC.. ' ...•. , '. .•. .•. •. >... ••.. .. . •••••••••••••••• .' .' • I • .· .... Improve customs Improve customs effectiveness: Customs Administration: Customs modernization, reengineering procedures and effectiveness: custom selectivity levels reduced from 40% systems, risk management, bonded warehouses and broker procedures. reform strategy approved to 30% and average net release tie reduced by 20% Reduce port costs and delays: Port and cargo security improvements and compliance with ISPS norms. approval of Port agenda Diagnose environment for complete port reform~ do action plan and recommendations to address issues. Reduce transport costs on Reduce Transport Costs on Federal Design system for M&E of transport projects and programs. IT system and data federal Roads networks: Road Networks: bases~ Law reorganizing Federal Output based maintenance and Consolidate policies and standards for road maintenance and rehab activities. Transport Administration rehabilitation contracts on 30% of approved and federal road network. Streamline admin procedures for management of road maintenance and rehab implemented, transfer of contracts. 9% of federal non-truck roads to state management. Foster Multimodal Foster Multimodal Transport: See item above Transport: Geographic geographical restructuring of restructuring of railway railway concessions completed concessions underway. H. Improve Business Environment Strengthen Infrastructure Strengthen National Land Transport Regulatory Agency (ANTI) Regulation: Creation of Upgrade land valuation models; supply hardware and software. land and water transport 30 regulatory agencies. Develop procedures and system for efficient concessions supervision; Train agency staff in valuation models and strategies. Complete regulatory and normative framework for land transport including multimodal transport. Define uniform rules for concessionaires and permit holders including information and data base upgrading. Elaborate financial-economic equilibrium clause rules in concession contracts valuation models. Strengthen National Water Transport Regulatory Agency (ANTAQ) Strengthen organizational structure, management and supervision processes. Complete normative framework for port administrations and integrate port information flows. Complete regulatory framework including of ports for sub nationals; delegate inland water transport services Improve regulatory information and accounting system. Develop regulatory information and accounting procedures and management system for supervision of concessions and permissions including competition, accounting rules and standards and performance indicators; hardware/software system. Secretariat for Economic Monitoring to modernize procedures and strengthen capacity to review recommendations, resolve issues related to on-the-job training. Enhance the Strengthen Infrastructure Strengthen Public-Private Partnership unit capabilities of Project identification, competitiveness Regulation: PPP Law and Law on Financing, and provide staff Training Environment: Public- Career Development Plan for Private Partnership (PPP) Regulators approved by Congress Law submitted to Congress and Amendment to antitrust law approved by the inter- ministerial committee 31 Simplify Entry and Simplify Entry and Business Study to upgrade procedures to open and close firms Business Operations: Operations: Constitutional amendment approved to tax regime to facilitate refund of unify tax collection at indirect taxes on acquisition or federal and state importation of capital goods by municipal levels for exporting firms. micro and small companies, simplify procedures for companies' registration adopted in some cities, export norms simplified by the Ministry of Development, Industry and External Commerce (MOIC). Strengthen Corporate Facilitate Entry and Exit Ministry of Justice, Secretariat of Economic Lavv: Insolvency Framevvork: (Bankruptcy and Antitrust Regime): Improve judicial instruction, anti-competitive behavior ~n.d proc~dures, nevv Bankruptcy lavv and Bankruptcy lavv enacted and hardvvare/softvvare, improve bankruptcy framevvork, trammg for Judges/court staff Tax Code Amendments training programs for courts started and procedures in bankruptcy cases; (ii) Secretariat of Judicial Ref~rm for passed by lovver house and Amendment to Tax Code improving courts system; cost benefit studies; (iii) Antitru~t Coun~tl to structure enacted information system, supply hardvvare and softvvare expanSIon, reVIevv procedures and training needs. Improve Judicial Contract See above Enforcement: constitutional Amendment No. 45 approved and Lavv No. 11,187/2005 approved by Congress. . ....•... . .... . ............. Increase Financial Competition: Draft Complementary Lavv 32 extending application of Antitrust to banking submitted to congress. Sound fundamental Sound fundamental legislation and legislation and systemic systemic risk control: evaluation of risk control: residual risk in the payments system Constitutional completed and blue print prepared amendment (Article 192) for second phase payments reform approved new large'value (retail payment). payments system installed and operating successfully. Mobilize long term Mobilize long term resources in :stren~~tlum Insurance Supervisory Authority (SUSEP) for development plan: resources in insurance insurance sector. Submit reform of regulatory, supervisory, HR, infrastructure; circulars, procedures manuals, training sector: regulation reinsurance market to Congress. to implement; supply hardware and software. strengthened on asset allocation, eligibility, registration, and custody and audit requirements enacted. Improve efficient access Improve efficient access to financial Studies and TORs for improving financial sector competition, access to credit and to financial services to services to poor and SMEs: development of capital markets. poor and SMEs: Regulation and enactment of Provisional law and payroll deduction loans resolutions passed to expand financial access to banks. 33 and Development (R&D) Innovation law approved and legal Evaluate sector funds' design, management, perfonnance, recommendations. effectiveness. Innovation framework for direct subsidy to Evaluate and prepare recommendations to improve technological extension law sent to Congress. private sector R&D established. program. Implement and disseminate current actions related to Bill of Law on Innovation including drafting regulations and norms. Supply hardware and software. Secretariat for Strategic Development Policy: Evaluate industry and prepare recommendations to foster R&D. Do pilot project evaluation. Secretariat for Social Inclusion: Define criteria to select groups of small producers, design evaluation methodology and participatory mechanisms to defme and support communities' hardware/software needs. Design and implement pilot projects in 15 communities. Supply hardware and software to communities and provide training. Do pilot project evaluation. Foster Private Innovation: Foster Private Innovation: refonn of See above Regulation of Fundo Fundos Setoriais and tax breaks and Verde-Amarelo and other tax incentives for private R&D mechanisms to support developed. private R&D introduced. Create Innovation in Secretariat for Strategic Scientific Development Policies Environmental Markets: Design processes, methodologies, criteria to evaluate, approve and certify COM Kyoto Protocol ratified, project proposals and M&E ICCC operational and Clean Development Implement processes above; supply hardware and software Mechanism (COM) Design COM outreach program project approval Strengthen Climate Change coordination Unit; office equipment mechanism published. 34 Added Secretariat of the Treaswy (STN) Further develop the public debt information system Management Build capacity to forecast and analyze growth . Build capacity through training for budget execution and public expenditure management. Added Ministry of Planning, Budget and Administration (MPOG) Assess government's human resources management systems Assess indirect administration Assess budget reform efforts Expanded Improve Added Comissao de Va/ores Mobilitirios (CVM) Effectiveness and Depth Develop risk based supervision of the Financial Sector Develop institutional performance evaluations Strengthen management systems Expanded Reduce InCrec.rse'lfljlJU1lrng: Customs Administration Secretaria da Receita Logistics Costs Reduce time and costs to clear customs (Phase IV of the HARPIA systemL (Improve Business Environment) . Eliminated Increase Withdr'aw,a1ofMinistry of Science and Technology Innovation Capacity to Transform Knowledge into Productivity Growth Source for DPL I and DPL 2 columns: Implementation Completion and Results Report for a Series of Loans for Sustainable and Equitable Growth-Loans I and II. World Bank Report No. ICR0000972. February 27, 2009. 35 Annex 3. Economic and Financial Analysis Most technical assistance loans linked to large DPLs are not expected to provide detailed economic and financial analysis, since they are catalysts linked intimately to the outcomes of the , DPLs. This is the case for this project, which provided technical assistance to the following DPLs: ~i!R!!!l-~~~.;---~~'--'-__r-_'_~'~_'_~_"~'~~'~'''''''''"''''_''''; 1 : BlaZil Filst :," : , _ . . . . : . •• ip' Prog:tammatic Loan »r ..,....... r lDgraR! Name Sustamable am , : I~_==~~~~~~~t~~r ,~,RAZIL isbursed AnDUnt USD SI6,2M: ! Co6nimI:ien a:nd Other E:ri!mal PariJaw t~i"""",,~',:~,:,~,,' : ,' M', """""'"", j _ _" " " " " " " , Second Programmatic :., __ r;675-~-:;~.c~=fi~L~, __ ~"__ "___,,,_,,~~P2._,,,,~_...,,J£~I~~_,....,.,,,.,,.. !=.5!!!~! .. ,,,,,, BRAZILIAN FEDERAL GOVERNM.~'I::;:,M:p':,~ lusD 6J1.5M bisbursed AnDUnt USD ISO.OM """"'" ,L, """,""""'" ~JJ~J'~~&:x-..rSultainahJe8Jlll,Et.1IibIbJeG:vwth- !V808%7 IOu U : o n e s d e l < l : t e l y S , . l : t i s f a c k l r y '" ~~~~i=--- !S. ._~ S.......Je...a It 'UiubJe G1rowd\Loaa- "PO!Jm5 ' " """'"""""", • i " " " " " " " " '" """" '" """""""".,,,,, , ," " , , , I Ouu:ones __ fh-"''''~'" "'''''''".''''''''N<,,",'m~''''N''''''''.~ """"'''"",,,-m.'._.,,·._,<,,,",,.,w>,,,,,",,>.,,,,. deRtelySatisfacby ",,','»,"<'''''m.,,',-,''''''''''''''''''''''>'','''''''''' >.w,"-',<"","kO'''''","-,,,,.''"'''''''''''''''''''''''i''''· ~" In addition, this technical assistance loan strengthened four ministries, created action plans co- financed by the government, and contributed to transportation system improvements, judicial system reform and reliability, and customs efficiency. It is difficult to use standard economic and financial analysis to calculate the value added of this loan. 36 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Aymeric-Albin Meyer Sr. Transport Specialist LCSFT Task Team Leader Paulo Guilherme Correa Sr. Economist LCSFR Team member Tulio Henrique Lima Correa Financial Sector Specialist LCOAA Financial Specialist Werner L. Kornexl PSD Specialist LCSRF Carbon Emission Specialist Anjali Kumar Lead Financial Economist LCSFF Economist - Legal Sergio Margulis Lead Environmental Economist AFTESSO Environment Craig W. Thorburn Sr. Financial Sector Specialist OPO Insurance Jose Augusto Carvalho Lead Counsel LEGLA Legal Mariam Oayoub Junior Professional Associate LCSFF Research Solange P. Van Veldhuizen Program Assistant LCSFT TTL assistant Cassia Coutinho Barreto Consultant Project analyst Maritza Castro Consultant Project analyst Cristina Oliveira Roriz Consultant Project analyst Jacques L Cellier Consultant Transport and Customs Roberto Mosse Consultant Financial Management Smriti Goyal Consultant Project analyst SupervisionllCR Senior Financial Sector Rogelio Marchetti LCSPF Task Team Leader Specialist Cassia Coutinho Barreto Consultant LCSTR Project Management Jose Augusto Carvalho Consultant LCSPT Legal Paulo Guilherme Correa Lead Economist ECSFI Team Member Financial Management Tulio Henrique Lima Correa LCSFM Financial Sector Specialist Mariam Oayoub Junior Professional Associate LCSPF Research Pablo Fajnzylber Sector Leader LCSPR Economist Carbon Emission Specialist Werner L. Kornexl Sr. Technical Spec. ENVCF and innovation Anjali Kumar Lead Economist IEGPS Justice Eric R. Lancelot Sr. Transport. Engr. LCSTR Transport Patricia Rodrigues de Melo Language Program Assistant LCSPF Assistant Aymeric-Albin Meyer Senior Operations Officer OPCIL Transport 37 Monica Torrelio :Program Assistant LCSPF Assistant Craig W. Thorburn Sr Financial Sector Spec. GCMNBInsurance Luciano Wuerzius Procurement Specialist LCSPT Procurement Denise von Gersdorff Consultant LCSPF Project Management b) Staff Time and Cost . . ..~~~!f'!~~~I!~~!>.s!.(I:I~n~ ~tlcJg~t()l1ly) . . Stage of Project Cycle USD Thousands (including· No. of staff weeks travel and consultant costs) Lending FY04 25 224.58 FY05 3 11.'82 FY06 0.00 FY07 0.00 FY08 0.00 Total: 28 236.40 Supervision/l CR FY04 2.79 FY05 17 106.48 FY06 28 124.01 FY07 17 91.38 FY08 107.54 FY09 0.00 Total: 82 432.20 38 Lending PE-P083533-LEN-BB (until 10104/2005) Fiscal Year Weeks Total 2004 24.89 224,578.02 2005 2.73 11,823.76 2006 0.00 0.00 2007 0.00 0.00 2008 0.00 0.00 2009 0.00 0.00 2010 0.00 0.00 2011 0.00 0.00 Supervision PE-P083533-SPN-BB (2005 -2011) Fiscal Year Weeks Total 2004 0.58 2,793.35 2005 2.00 178.14 2006 27.62 124,008.25 2007 16.35 91,376.24 2008 16.06 107,537.59 2009 10.25 106,812.85 2010 13.12 103,962.21 2011 3.82 60,388.78 39 Annex 5. Beneficiary Survey Results NIA 40 Annex 6. Stakeholder Workshop Report and Results N.A. 41 Annex 7. Summary of Borrower's ICR aDd/or Comments on DraftICR The ICR report represents an overview of the entire project's progress and it shows its evolution reliably. We concur with the contents and note that our suggestions have been incorporated. The Borrower represented by Under Secretary of Corporate Affairs responded to the letter sent by the Bank responded on June 3 2011 in the attached, whose translation states the following: Ministry of Finance National Treasury Secretary Under Secretary of Corporate Affairs Reference: Administrative Affaires Other - Other PACE Project In reference to the letter dated May 24, 2011 sent by the Bank and the Final Report on the Implementation of the Loan 7253-BR; upon reviewing the analysis performed I concur that the report represents an accurate overview of the development and implementation of the PACE project. In addition, I express agreement with the edits made to the preliminary text. Best regards. Undersecretary of Corporate Affairs. 42 ~iinisl~rio da F:mmdu Seer ~'lilTia do Tcsouro Nacional Subseerelari3 de Assunllls ('(lrp~.ralivo~ COllrdcna~ao· Gcral de Dcscnvolvimcnto Instuuc:ional E~planada lIos I...finistilios • ~Iinisl~rio da Fazendll' Ed. Ancxo· Bloco "P" •.\ia ",," • Terre," Sala 27 • Esplanada dus Mini!l\cril\S 70048·900 • 8rasilia - Df I{,l) 3412-3500 (61) 3412·394$ 161) 3412-.'96J ;'In\"fal.endn.gov.hr Oliciu rl" ItJOf201IiCODIN/SLJCOP/STN/MF-DF Hrasilia· OF. 3 de jllnh~l de :!O) I. A Sun Scnhoriu 0 Scnhor Rogelio Marchetti Especialisla Seni(lr para 0 Setor FimmC'eiro Hll n<., I Mundial • Ed. Corporure Financial Center· SeN QD. 02 81. A 7" andur - Sctor ('ol1lcrcinl Nnrte 7()7 ) :!-900 - Rrasilia - OF A..,sUllIo: Assuntot; Admini~trath'os - Outroll • Outroll • Projrto P.<\CI'; Prczado Senhllr. I. Em refer~ncia a<'''U1u de 24 de mnio de 2011 encaminhada por VOS/ill Senhnria e que tratn du Relllt6rio Final de Implel11l,lnlafi:io do Acordo dl: Elllpl-estirno 725J-BR. informo LlUC. <1-. unalise ~Ietuuda conclui-se que 0 C:lladu relat{'rto represema um8 \';sao glob