Document of The World Bank Report No: ICR00003984 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-78880) ON A LOAN IN THE AMOUNT OF EUR 26 MILLION (US$ 36.3 MILLION EQUIVALENT) TO THE REPUBLIC OF CROATIA FOR A JUSTICE SECTOR SUPPORT PROJECT December 9, 2016 Governance Global Practice Croatia Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective November 1, 2016) Currency Unit = Kuna HRK 1.00 = US$ 0.15 US$ 1.00 = 6.85 FISCAL YEAR January 1st – December 31st ABBREVIATIONS AND ACRONYMS ADR Alternative Dispute Resolution AmCham The American Chamber of Commerce CEPEJ The European Commission for the Efficiency of Justice CPS Country Partnership Strategy CTS Case Tracking System DA Designated Account DO Development Objectives EC European Commission ECA Europe and Central Asia EFO Externally Financed Output EU European Union FM Financial Management FMR Financial Management Report GDP Gross Domestic Product ICMS Integrated Case Management System ICR Implementation Completion Report ICT Information and Communications Technology IPA Instrument for Pre-Accession Assistance ISR Implementation Status Report IT Information Technology JSPEIR Justice Sector Public Expenditure and Institutional Review JSSP Croatia Justice Sector Support Project M&E Monitoring and Evaluation MFA Ministry of Foreign Affairs MOF Ministry of Finance MOJ Ministry of Justice MTEF Medium-Term Expenditure Framework MTR Mid-Term Review PAD Project Appraisal Document PDO Project Development Objectives PIU Project Implementation Unit PPA Project Preparation Advance PPF Project Preparation Facility QER Quality Enhancement Review SAO State Attorneys’ Office SIL Specific Investment Loan TA Technical Assistance TTL Task Team Leader USKOK Office for the Suppression of Corruption and Organized Crime Senior Global Practice Director: Deborah L. Wetzel Practice Manager: Adrian Fozzard Project Team Leader: Georgia Harley ICR Team Leader: Ismail Radwan Republic of Croatia Justice Sector Support Project Table of Contents Data sheet ........................................................................................................................................ v Executive Summary ........................................................................................................................ 1 1 Project Context, Development Objectives and Design ........................................................... 1 1.1 Context at Appraisal ........................................................................................................ 1 1.2 Original Project Development Objectives (PDO) and Key Indicators ............................ 1 1.3 Revised PDO ................................................................................................................... 2 1.4 Main Beneficiaries........................................................................................................... 2 1.5 Original Components....................................................................................................... 2 1.6 Revised Components ....................................................................................................... 2 1.7 Other significant changes ................................................................................................ 3 2 Key Factors Affecting Implementation and Outcomes .......................................................... 3 2.1 Project Preparation, Design and Quality at Entry............................................................ 3 2.2 Implementation ................................................................................................................ 4 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization ................. 5 2.4 Safeguard and Fiduciary Compliance.............................................................................. 5 2.5 Post-completion Operation/Next Phase ........................................................................... 5 3 Assessment of Outcomes ........................................................................................................ 6 3.1 Relevance of Objectives, Design and Implementation .................................................... 6 3.2 Achievement of Project Development Objectives: Substantial ....................................... 7 3.3 Efficiency ...................................................................................................................... 12 3.4 Justification of Overall Outcome Rating ....................................................................... 12 3.5 Overarching Themes, Other Outcomes and Impact ...................................................... 13 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops .............. 14 4 Assessment of Risk to Development Outcome ..................................................................... 16 5 Assessment of Bank and Borrower Performance ................................................................. 16 5.1 Bank Performance ......................................................................................................... 16 5.2 Borrower Performance .................................................................................................. 17 5.3 Lessons Learned ............................................................................................................ 18 5.4 Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............... 19 Annex 1. Project Costs and Financing ...................................................................................... 21 Annex 2. Outputs by Component .............................................................................................. 26 Annex 3. Economic and Financial Analysis ............................................................................. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes ......................... 30 Annex 5. Beneficiary Survey Results ....................................................................................... 32 Annex 6. Stakeholder Workshop Report and Results ............................................................... 35 Main Stakeholder Workshop .................................................................................................... 35 Annex 7. Summary of Borrower's ICR ..................................................................................... 39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................................... 47 Annex 9. Project timeline: Key events in the implementation.................................................. 42 Annex 10. List of Supporting Documents................................................................................. 43 MAP .............................................................................................................................................. 44 List of Tables Table 1: Achievement of PDO Indicator 1 (Substantial) ................................................................ 7 Table 2: Reduced processing times 2010-2016 ............................................................................... 8 Table 3: User satisfaction 2010 and 2016 ....................................................................................... 9 Table 4: ICMS usage in Croatian courts 2010-2016 ..................................................................... 10 Table 5: CTS usage in the Croatian SAO 2010-2016 ................................................................... 11 Table 6: Assessment of the Overall Project Outcome Rating ....................................................... 12 Table 7: Estimated Economic Benefits due to Efficiency Gains ................................................... 30 Table 8: Overall satisfaction with the court, premises and employees in surveyed groups .......... 33 Table 9: Satisfaction with courts, premises and judicial clerks in observed cities ........................ 34 Table 10: List of participants invited to the JSSP Workshop ........................................................ 37 List of Figures Figure 1: Net Economic Benefits, 2017 - 2030 (EUR mil.) .......................................................... 31 Figure 2: Trust in State institutions - 2010-2016 comparison ....................................................... 32 Figure 3: Perceptions of changes in judicial system - 2010-2016 ................................................. 33 Data sheet A. Basic Information Justice Sector Support Country: Croatia Project Name: Project Project ID: P104749 L/C/TF Number(s): IBRD-78880 ICR Date: 10/04/2016 ICR Type: Core ICR Lending Instrument: SIL Borrower: Republic of Croatia Original Total USD 36.30M Disbursed Amount: USD 21.41M Commitment: Revised Amount: USD 23.73M Environmental Category: B Implementing Agencies: Ministry of Justice (MOJ) Co-financiers and Other External Partners: Ministry of Foreign Affairs of the Kingdom of the Netherlands (partner) Ministry of Foreign Affairs of the Kingdom of Norway (partner) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 04/16/2007 Effectiveness: 07/06/2010 07/12/2010 12/22/2013 Appraisal: 01/29/2010 Restructuring(s): 05/30/2015 Approval: 04/06/2010 Mid-term Review: N/A N/A Closing: 06/30/2015 06/30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project No Quality at Entry: None at any time (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision: DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Law and justice 100 100 Theme Code (as % of total Bank financing) Judicial and other dispute resolution mechanisms 75 75 Public expenditure, financial management and 25 25 procurement E. Bank Staff Positions At ICR At Approval Vice President: Cyril E Muller Philippe H. Le Houerou Country Director: Arup Banerji Peter C. Harrold Practice Manager: Adrian Fozzard William Leslie Dorotinsky Project Team Leader: Georgia Harley Amitabha Mukherjee ICR Team Leader: Ismail Radwan ICR Primary Authors: Ismail Radwan Georgia Harley Elena Gabriela Ardelean F. Results Framework Analysis Project Development Objectives The objective of the project is to improve the efficiency of Croatia's justice system. Revised Project Development Objectives N/A (a) PDO Indicator(s) Original Formally Actual Value Target Values Baseline Revised Achieved at Indicator (from Achievement Value Target Completion or approval Values Target Years documents) Clearance rate for courts 96.39 Increased by Increased by 112.16 Fully achieved. (Percentage, Custom 10% over 10% over Supplement)* baseline. baseline. (+16%) Clearance rate for SAO 98.90 Increased by Increased by 96.2 Not achieved. and USKOK 10% over 10% over (Percentage, Custom baseline. baseline. (-3%) Supplement)* Disposition time for 280.18 Reduced by Reduced by 238.13 Fully achieved. courts (Days, Custom 15% over 15% over Supplement) baseline. baseline. (+15%) Aggregate case backlogs 437,892 Reduced by Reduced by 349,051 Fully achieved. for Courts (Number, 10% over 10% over Custom Supplement)* baseline. baseline. (-20%) Aggregate case backlogs 11,273 Reduced by Reduced by 10,415 Partially for SAO and USKOK 10% over 10% over achieved (Number, Custom baseline. baseline. (-7.61%) Supplement)* Reduced case 838 Reduced by Reduced by 756 Fully achieved. processing times in 10% over 10% over Courts (Days, Custom) baseline. baseline. (-10%) Improved user ratings 3.5 Third user Third user 3.4 Not achieved. for efficiency for survey: 10% survey: 10% project-financed courts improvement improvement (-3%) and SAO offices over baseline over baseline (Percentage, Custom) value value (b) Intermediate Outcome Indicator(s) Original Actual Value Formally Target Values Achieved at Baseline Revised Indicator (from Completion Achievements Value Target approval or Target Values documents) Years Court Presidents and N Y Y Y Fully Court Secretaries in at achieved. least thirty priority municipal and country courts develop resource management plans (Yes/No, Custom) Numbers of users of 4,100 Increase by Increase by 6,254 Fully ICMS 30% over 30% over achieved. baseline. baseline. (+52.5%) Number of transaction 18,500 Increase by Increase by 74,560 Fully made in the ICMS daily 20% p.a. over 20% p.a. achieved. baseline. over (+26.2%) baseline. Number of cases 467,000 Increase by Increase by 6,062,259 Fully registered in ICMS 10% over 10% over achieved. baseline. baseline. (+1198%) Consolidated and N/A Consolidated Consolidated Y Revised target modernized facilities for a and and achieved. functioning (a) SAO HQ modernized modernized and USKOK in Zagreb facilities for (a) facilities for and (b) SAO in Pula SAO HQ and (a) (Text, USKOK in SAO HQ Custom)** Zagreb and (b) and SAO in Pula USKOK in Zagreb and (b) SAO in Pula Upgraded and Updated N Y Y Y Fully Operational Information achieved. Systems Improve SAO efficiency (Text, Custom) Numbers of users – CTS 737 Increase by Increase by 1,820 Fully Operational Information 10% over 10% over achieved. Systems baseline. baseline. (+16% p.a.) Number of transaction on N/A Increase by Increase by 74,234 Fully cases per month – CTS 10% over 10% over achieved. Operational Information baseline. baseline. Systems Number of cases – CTS 6,393 Increase by Increase by 354,449 Fully Operational Information 20% over 20% over achieved. Systems baseline. baseline. (+95% p.a.) A medium term No MTEF MTEF Dropped N/A Dropped. expenditure framework exists for developed by for the justice sector is Justice MOJ developed by MOJ on the sector, not basis of justice sector prioritized expenditure and investment institutional review (Text, program Custom) or a clear set of actionable indicators. Strengthened MOJ N Upgraded pilot Dropped N/A Dropped. After information system and information delay due to their management system budget evidenced by: (i) an functional constrains operational ICT asset across all procurement management system; (ii) correctional was initiated functioning Data facilities, but it had to be Warehouse System (Text, county courts cancelled due Custom) and all to technical misdemeanor issues. courts ii) (specific integrated ICT technical asset equipment was management not available). system containing all major ICT assets under administrative control, digital communication system, modern document management system operating in MOJ Timely project N Report Report Y Revised target implementation including (including (including achieved. adherence to procurement survey results survey schedules (Yes/No, as appropriate). results as Custom)*** Clean Audit appropriate). reports. Clean Audit reports. Conduct of periodic user N Y Y Y Fully surveys and achieved. publication/dissemination Survey of the results of such conducted for surveys (Text, Custom) evaluating Court users’ satisfaction. * Indicators revised following the 2013 restructuring which conducted to the split of PO 1 Indicator and PO Indicator 2. **Indicator revised following the 2013 restructuring which decreased the number of courts to be modernized from 5 to 2, namely SAO HQ and USKOK in Zagreb and (b) SAO in Pula *** Indicators revised following the 2015 restructuring. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 01/25/2011 Satisfactory Satisfactory 0.67 2 10/11/2011 Satisfactory Satisfactory 2.02 3 05/19/2012 Moderately Satisfactory Moderately Satisfactory 6.28 4 04/25/2013 Moderately Satisfactory Moderately Satisfactory 8.98 5 12/15/2013 Moderately Satisfactory Moderately Satisfactory 11.06 6 06/29/2014 Moderately Satisfactory Moderately Satisfactory 13.64 7 01/07/2015 Moderately Satisfactory Moderately Satisfactory 17.24 8 05/22/2015 Moderately Satisfactory Moderately Satisfactory 17.24 9 11/10/2015 Moderately Satisfactory Moderately Satisfactory 20.81 10 06/04/2016 Moderately Satisfactory Moderately Satisfactory 21.41 11 06/29/2016 Moderately Satisfactory Moderately Satisfactory 21.41 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Partial cancellation of loan 12/22/2013 N/A MS MS 11.06 proceeds and two civil works sites. Request for a one year project 11/05/2015 N/A MS MS 20.81 extension. I. Disbursement Profile Executive Summary The Croatia Justice Sector Support Project (JSSP) was approved on April 6, 2010 with the objective of improving the efficiency of the justice system. The project supported the EU pre- accession agenda in the period up to Croatia’s accession as the 28th EU Member State on July 1, 2013. The ICR concludes that the project outcome was successful. Efficiency of the justice system improved over the project period. Four out of seven PDO targets were achieved and one partially achieved. All the intermediate results targets were met and some exceeded by large margins. JSSP had four components: (i) improving the efficiency of the court system and (ii) the State Attorney’s Offices (SAO’s); (iii) strengthening the management of the ministry of justice; and (iv) support for project management and implementation. The initial project cost was EUR 26 million. The project was restructured twice. The first restructuring, in December 2013, following a change in Government, led to the cancellation of proposed civil works and EUR 9 million of the loan amount. The second restructuring, in May 2015, extended the project closing date by a year to June 30, 2016 enabling the project to support new reforms to personal bankruptcy. The project contributed to important reforms in the justice sector: notably the upgrading and scaling up of the Integrated Case Management System (ICMS) in the courts and the Case Tracking System (CTS) in SAOs. These two systems are now used throughout Croatia and have transformed the way that courts and SAOs operate, improving efficiency, transparency and integrity of case management. In the courts, ICT has helped to increase clearance rates and reduce the duration and backlog of cases. In the SAOs, efficiency was increased but the project targets were not met because workloads increased. Court staff, the business community and the legal profession report a positive experience of the post-reform system and are committed to upgrading these systems in coming years. The civil works in Zagreb and Pula were completed to a high quality, albeit with some delays. The new facilities have helped to house and protect the relevant ICT investments, ensure compliance with EU standards, and offer state of the art archiving facilities. Stakeholders report that the facilities improve the efficiency of work flows and the quality and integrity of their work. The project also financed designs for courts & SAO offices in four additional locations, the construction of which is financed from EU IPA funds. Beneficiaries value infrastructure investments highly. The MOJ plans to use the Pula designs as the benchmark for future civil works across Croatia. Three lessons for the design and implementation of justice reform projects are highlighted here. First, JSSP demonstrated that Bank projects can produce development results in the criminal justice sector. JSSP was the first Bank operation to finance criminal justice sector reforms as a significant part of the project. Concerns about reputational risks did not materialize. SAO proved a reliable and proactive counterpart. Future projects can now be more assertive and move beyond support to efficiency to include reforms that promote transparency, integrity and quality of justice. Second, the importance of change management cannot be overstated. Borrowers tend to be reluctant to allocated loan funds to change management activities even though under-investment in this areas can limit the effectiveness of the ‘hard’ investments and jeopardize policy goals. More could have been done, particularly at the strategic level, to embed a culture of change and support the transition within the Croatian judiciary as the ICMS and CTS were being developed and rolled out. The Bank should be more proactive in emphasizing the value of such investments with clients. Third, projects need to unpack the meaning ‘judicial efficiency’ to ensure that the results framework captures the desired outcomes, they should avoid using indicators that do not compensate for changes in workload outside of the project’s control. 1 After project close in June 2016, the Borrower allocated resources from the State budget and EU funds to continue upgrading the ICT environment and implementing the reform agenda. This signals political commitment and suggests that the risk to development outcomes is moderate. 2 1 Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context: At project appraisal in 2010, the process for Croatia’s accession the European Union (EU) was well underway. Negotiations with the EU started in October 2005 and concluded in June 2011. On July 1, 2013 Croatia joined the EU as its 28th Member State. When project preparation began, 30 out of 35 negotiating chapters of the Acquis Communautaire had been opened, 17 had provisionally closed, and Chapter 23 (Judiciary and Fundamental Rights) and Chapter 24 (Justice, Freedom and Security) were considered among the toughest chapters still to finalize. The 2009 Progress Report of the European Commission (EC) 1 acknowledged Croatia’s progress under Chapters 23 and 24 but underlined the need to accelerate reforms to improve the efficiency of the justice system. The 2008 Croatian Government Program recognized the importance of the justice sector, identifying the completion of judicial reforms as one of its five priorities. 2. Bank Rationale: The Croatian authorities approached the World Bank for support to close the implementation gap in priority areas required by the accession process. The Bank’s Country Partnership Strategy (CPS) for FY09-122 focused on assisting Croatia with its accession process; convergence with EU Member states’ income levels in a fiscally, socially and environmentally sustainable manner; and providing a decent quality of life for its citizens. The Bank had been engaged for more than a decade in Croatia’s justice sector, including through two operations (Court and Bankruptcy Administration and Real Property Registration and Cadastre Project). The Bank had significant experience in justice sector reform throughout the region including EU accession countries. The Justice Sector Support Project (JSSP) was conceived as the instrument to support judicial reforms. 1.2 Original Project Development Objectives (PDO) and Key Indicators 3. The project’s development objective was to improve the efficiency of Croatia's justice system. The project would enhance efficiency by financing: ICT investments (building on a previous Bank financed-project); civil works, reconstruction and rehabilitation of selected courts and SAOs for which there was no alternative financing. It would consolidate the court and SAO networks; and provide capacity-building on issues which leverage the Bank’s expertise (such as user surveys) and address justice sector resource management constraints (such as management, budgeting and performance measurement). 4. The project development indicators were: a) Reduced case backlogs and improved case disposal ratios in project-financed courts and prosecution offices. b) Reduced processing times, in pilot courts financed by the project, for key stages of the judicial process subject to long delays, namely: service of notice to parties; time between first and final hearings; and testimony of expert witnesses. c) Improved user ratings for efficiency for project-financed courts and prosecution offices. 1 http://ec.europa.eu/enlargement/pdf/press_corner/key-documents/reports_nov_2009/croatia_progress_report_en.pdf 2 World Bank. Country Partnership Strategy for the Republic of Croatia. August 13, 2008. Report No. 44-879-HR. 1 1.3 Revised PDO 5. The PDO remained unchanged for the duration of the project. 1.4 Main Beneficiaries 6. The direct beneficiaries of this project were the main actors in the Croatian justice system, namely, the MOJ, the courts and the SAOs, and through it, the Office for the Suppression of Corruption and Organized Crime (USKOK). The MOJ was the implementing agency. The PAD also anticipated that benefits would flow to a broader set of beneficiaries who use the court system, including, but not limited to the bar associations, association of notaries, civil society, chambers of commerce, academic institutions, and the media. Secondary beneficiaries of a more efficient justice system would include businesses and the general public. 1.5 Original Components 7. The original JSSP project had four components: 8. Component 1: Improving the Efficiency of the Court System (Cost EUR 14.68 M). The original activities included: a) Rehabilitation and renovation of three courthouses in Split, Karlovac and Pula, housing five county and municipal courts and the provision of access for the disabled in the courts covered by the project; b) Court resource management capacity-building (training court presidents and secretaries to prepare and use resource management plans); c) Modernization of operational information technology (IT) systems for decision-making and performance monitoring in courts; d) Refinement of case management standards, guidelines, processes and practices; and e) Improving the efficiency of enforcement of judicial decisions. 9. Component 2: Improving the Efficiency of the SAOs (Cost EUR 7.73 M). The original activities included: a) Rehabilitation of the SAO Zagreb headquarters and USKOK, and rehabilitation of an existing building for the Pula SAO; b) Institutional capacity strengthening for the SAO; and c) Case management and IT systems including automating SAO case management and case tracking. 10. Component 3: Strengthening Management Functions of the MOJ (Cost EUR 1.58 M). The original activities included: a) Strengthening justice sector resource management and performance management capacity, including the development of a medium-term expenditure framework for the justice sector (MTEF) by the MOJ; and b) Strengthening MOJ information systems and their use in management decision-making, including initial steps to modernize the inmate information system and strengthen its connectivity with the justice information system; interconnecting the information systems of key justice sector entities; implementing an integrated MOJ document management system; and strengthening information and communications technology (ICT) technical and management skills of selected MOJ personnel. 11. Component 4: Support for Project Management and Implementation (Cost EUR 1.20 M). The original activities included: a) Project monitoring and evaluation (M&E), change management and outreach; b) Operating costs (including logistical, administrative and other expert technical support); and c) User surveys and the Preparation Advance. 1.6 Revised Components 12. Two level 2 restructurings took place in December 2013 and May 2015. The 2013 restructuring did not change the components, but some activities were cancelled or modified and new activities were included requiring an updated implementation schedule and procurement plan. 13. Component 1: Improving the Efficiency of the Court System. The funds allocated to this component were cut by 73 percent from EUR 14.68 million to EUR 4.00 million. Two civil works were cancelled in Split and Karlovac. 2 14. Component 2: Improving the Efficiency of the SAOs. The cost of this component was increased by 16 percent from EUR 7.73 million to EUR 9.00 million. The restructuring introduced new activities: (a) upgrading the CTS; (b) enhancing CTS operational efficiency and automatization; and (c) preparing, enabling and facilitating national roll out of the upgraded CTS. 15. Component 3: Strengthening Management Functions of the MOJ. The cost of this component was increased by 102 percent from EUR 1.58 million to EUR 3.2 million. Support to an inmate information system was cancelled. Activities to support court enforcement were modified. Three new activities were introduced: (a) an analysis of the court and prosecution network with recommendation for further rationalization; (b) an infrastructure stock-take; and (c) the development of justice sector performance indicators. The 2015 restructuring added activities related to reforms of personal bankruptcy. 16. Component 4: Support for Project Management and Implementation. Funding was cut by 33 percent from EUR 1.20 million to EUR 0.80 million. The activities foreseen were not changed but the staffing of the Project Implementation Unit (PIU) was reduced. 1.7 Other significant changes 17. Results Framework. The 2013 restructuring simplified the outcome indicators and revised the intermediate results indicators to align with the revised project activities. The main changes were: (a) PDO Indicator 1 was simplified and merged with PDO Indicator 2; (b) intermediate results indicators were added to measure the uptake and usage of project-financed ICT systems to improve efficiency; (c) intermediate results indicators have been added to measure the availability and usage of performance information generated by improved information systems; and (d) an intermediate results indicator has been added to assess progress in the government’s efforts to rationalize the court and prosecution network. These changes are presented in Annex 2. 18. Financing. The 2013 restructuring reduced the estimated total amount needed for effective implementation of the Project from EUR 26.0 million to EUR 18.0 million, of which EUR 17.0 million was proposed to be financed from the loan proceeds. The borrower requested cancellation of EUR 9.0 million of the original loan. 19. Duration. The 2015 restructuring extended the project by one year to June 2016. The restructuring enabled the project to support the rollout of reforms to the personal bankruptcy framework, which was a priority reform for the Government at that time. It also extended the time available to roll out upgrades to the ICMS, the CTS and ICT equipment that were critical to the rationalization of the court network which came into effect in July 2015. There were no other changes introduced in this restructuring. 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 20. The concept review meeting took place in April 2007. It recommended a Quality Enhancement Review at Entry (QER) but the QER did not take place. Project preparation was initially led by the Bank’s Legal group, but was re-mapped to the public sector management group. 21. Project preparation was complemented with financing from the Dutch Government who were keen to support the development of Croatia’s justice sector in the pre -accession context. They pledged EUR 1 million to an externally funded output (EFO). The EFO was split into three components. EUR 0.2 million was allocated to support preparation activities while EUR 0.3 million was ear-marked for a justice sector public expenditure and institutional review (JSPEIR). 3 The remainder was allocated to technical assistance including exploratory work on public- private-partnerships (PPPs) for the proposed development of the Zagreb Judicial Square: an ambitious project to consolidate justice sector buildings in a single site in the capital. 22. Preparation was enhanced through a Project Preparation Advance (PPA) of US$ 653,000. Under the PPA, the project team was established and the technical designs were prepared before appraisal, including civil works and a number of ICT procurements. However, project complexity and lack of leadership from the Borrower side caused delays during the preparation phase. Preparation was also affected by political turbulence. The Bank team had to familiarize new officials with the project objective and design following a change in Government in July, 2009, immediately before project negotiations. 2.2 Implementation 23. Project implementation had to contend with a turbulent environment: three electoral cycles, four different Ministers of Justice, the global financial crisis and the EU accession process. There were significant changes personnel too: six different PIU Directors, and five different heads of the ICT sector of the MOJ. Frequent changes in counterparts slowed project implementation as incoming decision-makers developed their understanding and ownership of the project objectives. Priorities shifted between administrations, new policy initiatives were identified but were often not pursued by their successors. 24. Following Bank approval on April 6, 2010, the project was quickly approved by Parliament and became effective in July 2010. The project was ready for implementation at effectiveness. Tenders for works in Pula and Zagreb were advertised shortly after effectiveness in October 2010 and contracted in March 2011. The quick start resulted in a project disbursement profile that was initially higher than the original estimates. 25. Despite the good start to implementation, the pace of implementation had slowed in 2012. With the accession chapters closed, justice stakeholders suffered from reform fatigue. Elections in late-2011 brought in a new Government and the incoming authorities began to question the rationale for the project. In late 2012, a high-level meeting was held to consider dropping the ICMS altogether, due to concerns that it was expensive and not user-friendly. At this time, the system was not yet operating in all courts; it suffered from glitches; the interface was not intuitive; and staff complained that they had to duplicate electronic and paper processes. The project team and technical-level MOJ staff managed to convince the authorities to persevere with the ICMS. Further investments were made in upgrades, patches, and training to overcome these teething challenges. 26. The new Government faced significant fiscal pressures following the financial crisis. In December 2013, the Government requested a project restructuring to cancel the construction of two court complexes in Split and Karlovac 3 and reduce project costs by EUR 9 million. Restructuring also cut the PIU budget without a significant reduction in workload. Attention shifted to ICT implementation and the change management activities supporting the reforms. A number of new project activities were introduced: an analysis of the court and prosecution network with recommendation for further rationalization; an infrastructure stock-take; data warehousing; and the development of justice sector performance indicators. None of those activities were implemented. By the time the restructuring was complete, the client’s priorities had changed and interest had waned. 3 Instead of using loan financing, the authorities secured a Norwegian Grant in order to complete the Karlovac complex using World Bank designs. 4 27. In 2015, the Borrower requested a second project restructuring to extend the project for one year until June 2016. The second restructuring enabled the project to support the Government’s personal bankruptcy reform agenda. It also extended the rollout of ICMS upgrades and ICT equipment to support the new court network. These activities were completed successfully. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 28. The PAD provides a clear M&E framework and an action plan. The MOJ project team gathered information regularly and reported on the indicators to the Bank and the Government. However, the initial M&E framework design was ambitious and data collection proved a challenge. Some of the indicators were no longer relevant because the activities that underpinned them had been dropped. Along with the pilot courts, activities related to strengthening the management functions of the MOJ were dropped along with the two indicators corresponding to these activities. Further, the indicators related to the clearance rate and case backlogs were split, reflecting separately the courts and SAO/USKOK. Consequently, the results framework was changed and the key performance indicators modified during the 2013 restructuring. 29. The stakeholder workshops confirmed that information in the revised M&E framework, especially that derived from the ICMS and CTS, were used as a management tool to guide project activities, resource allocations and supervision by MOJ and the PIU. Information for the PDO indicators and IOIs was updated throughout the Project. 2.4 Safeguard and Fiduciary Compliance 30. Financial management (FM) arrangements were reviewed during project preparation in June 2009, updated in October 2009 and February 2010, and found to be satisfactory. The FM assessment confirmed that: an Internal Audit Department had been established; a FM consultant was already working, financed by the PPA; and the project's Designated Account was maintained in a commercial bank acceptable to the Bank. While the corruption risk was assessed as high, adequate mitigation measures were put in place and closely monitored to ensure that the residual project risk was acceptable. The PIU maintained satisfactory FM arrangements throughout implementation. FMRs were always delivered on time and deemed satisfactory by the Bank. 31. The 2005 Country Procurement Assessment Review assessed the general environment in Croatia for public procurement as “average.” A new Public Procurement Law was enacted in 2007 and became effective January 1, 2008. With subsequent amendments effective January 2009, the law was aligned with the EU Procurement Directives. However, the project continued to use Bank procurement procedures which were appreciated by the borrower. Project procurement was managed by the PIU and rated as satisfactory throughout implementation. 32. The environmental safeguards (OP, BP 4.01) were triggered for civil works. Environmental Management Plans (EMPs) were undertaken as required. The safeguards for cultural property (OP, BP 4.11) were triggered because works were envisaged in historical urban areas and subject to potential chance finds. There were no significant deviations or waivers from the Bank safeguards and fiduciary policy procedures during implementation. However, some incompliances were noted in implementation of the EMPs defined measures, monitoring and reporting including the omission to construct a grease and oil separator for the parking space at the Pula and Zagreb SAO buildings. 2.5 Post-completion Operation/Next Phase 33. In June 2016, three weeks prior to project closure, the MOJ requested a further extension of the project for six months in order to prepare a follow-on project or additional financing to the JSSP. The reform agenda built on the progress to date under the JSSP, and aimed to further 5 increase the quality and efficiency of the justice system, with a particular focus on commercial courts to improve the business climate. Further digitization would include the integration of e- filing and other two-way online interactions with users, along with a full document management system, speech-to-text automation, and centralized printing. Other areas of reform included the equalization of caseloads and workloads, the introduction of quality assurance in case processing, the promotion of alternative dispute resolution (ADR) mechanisms, and the strengthening training for judges and staff. The reform agenda was technically sound. However, the request arrived at the Bank on the day the Government collapsed and Bank management declined to issue the extension in the uncertain environment. Instead, the Bank agreed to continue the policy dialogue with the Croatian Government in the justice sector with a view to further collaboration using any of the Bank’s available instruments. 34. At project closure in 2016, the ICMS was operational in all courts across Croatia and was compliant with legislation. Judges and their staff used the ICMS as an essential tool to process and manage their cases. ICMS reports were also routinely used to inform managerial decision- making, especially among managers at the MOJ, court presidents and heads of court departments. Similarly, at project closure, the CTS was operational in all SAO offices, compliant with relevant legislation and generated performance reports. The project had introduced WAN and LAN infrastructure and fitted out courts and SAOs with more than 8,000 computers and 6,000 printers and scanners to run the new systems. All of these systems were fully operational at the time of the ICR. 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives: High 35. The PDO, which aimed to improve the efficiency of the justice system, was aligned with Croatia’s national program for EU accession. At the time of preparation, there was consensus among the EC, Croatian Government and the World Bank that improving the performance of the judiciary was a critical area of reform. The Bank agreed to focus on supporting improvements in efficiency, as this was a high priority, built on the Bank’s earlier projects and reflected the Bank’s comparative advantage. The PDO was aligned with the FY09-12 Country Partnership Strategy (CPS) 4 which supported the EU accession agenda as a cross-cutting theme for the Bank’s program. Relevance of Design: Modest 36. Scope and Components: The project addressed the challenges highlighted in the EC progress report 5 including the backlog of cases, lengthy time in courts, ineffective case party notification system, weak case management system and insufficient capacity at the MOJ. It also built on an earlier Bank-financed project (Court and Bankruptcy project, 2001-2007), an IDF grant and extensive sectoral dialogue supported by a serious of programmatic adjustment loans. At the outset, the project focused most resources on civil works, the rationale being that implementation of reforms required upgraded facilities. 37. At preparation, the EC had a strong focus on anti-corruption efforts and the project provided much-needed facilities, equipment and training for the country’s anti-corruption agency, 4 World Bank. Country Partnership Strategy for the Republic of Croatia. August 13 th 2008. Report No. 44-879-HR 5 http://ec.europa.eu/enlargement/pdf/key_documents/2009/hr_rapport_2009_en.pdf 6 USKOK. However, the project design combined the support to the SAOs and USKOK and did not track any impacts in terms of tackling corruption. 38. Investments in ICT were critical to the achievement of the intended outcomes. So too were resources ear-marked for system reform elements, training and change management. These activities could have been better designed and change management and training could have started earlier in the project. This would have helped overcome the resistance of the judges and other stakeholders. The project could also have been more explicit regarding the length of time required to manage the change. Nonetheless, the Bank’s and the authorities’ flexibility combined to make the needed mid-course adjustments to overcome challenges during implementation 39. Covenants. JSSP had an extensive and onerous list of eleven loan covenants that added little value to the project design. Covenants included requirements for the beneficiary institutions publish quarterly implementation reports on their websites, establish a MTEF for the justice sector and finance various studies. The covenant on the MTEF was never complied with, in part because the Bank-executed justice sector expenditure review that was supposed to provide the inputs for expenditure analysis was only delivered in 2014, towards the end of the project, but also because this exercise was owned by the Bank rather than the client. 40. Choice of instrument: The choice of Specific Investment Loan (SIL) was appropriate for the purpose, given the number of civil works, the ICT procurement, the training and capacity building activities and the need to reinforce implementation capacity. A policy-based operation was considered but rejected due to uncertainty in the timing of a broad-based justice-sector reform. Besides, a policy-based operation would not have helped the authorities implement the civil works and ICT investments that were at the heart of the project. The PIU and the use of Bank procurement rules helped the authorities address fiduciary risks. A larger loan was also considered to support the development plans of the Zagreb Judicial Square and Zagreb Commercial Court among other investments. Given uncertainty regarding the pace of Croatia’s EU accession and the turmoil from the global financial crisis, opting for a small, focused project was a sound choice. The PPA helped ensure that the project was implementation-ready by facilitating the preparation of designs and procurement. 3.2 Achievement of Project Development Objectives: Substantial 41. The project had a single PDO, “to improve the efficiency of Croatia’s justice system”. The PDO was clearly defined and project components and activities were linked to the achievement of that objective. Annex 2 on the Assessment of Achievements by Objectives reconstructs the Results Framework in the PAD including the changes made at the first restructuring. This section evaluates the achievement of the PDO drawing on information from a variety of sources including: Bank aide-memoires and ISRs, stakeholder data and surveys, the borrower completion report, Doing Business Surveys, and the stakeholder workshop.  PDO Indicator 1: Reduced case backlogs Table 1: Achievement of PDO Indicator 1 PDO Indicator Baseline Actual Target Actual Status Case backlogs for 437,892 349,051 15% 20.3% Fully achieved. Courts reduction reduction Case backlogs for 11,273 10,415 15% 7.6% Partially achieved. SAO and reduction reduction USKOK Clearance rate for 96.39 112.16 15% 16% increase Fully achieved courts increase 7 Clearance rate for 98.90 96.2 15% 2.73% Not achieved. SAO and increase reduction USKOK 42. The first PDO indicator tracked increased efficiency in the court system through the reduction in backlogged cases and an improved clearance rate. The project addressed these challenges in the courts and SAO’s office with different results. Efforts in the court system produced results that exceeded the original project targets. In the SAO’s office and USKOK the clearance rate deteriorated and while the backlog of cases was reduced it did not meet the original target. Underlying improvements in the efficiency of the SAO were unable to compensate for unforeseen increases in the SAO’s workload and reductions in staffing. From 2014 to 2015, the SAO had to process 1,400 additional cases. All of these cases required additional processing because of changes in criminal procedure6. Following a Supreme Court ruling in May 20147, a further 889 cases were submitted to the competent SAO’s between 2014 and 2015 . These cases, previously handled by investigative judges, were particularly complex and time consuming, requiring an average of three years to process. At the same time, organizational reforms significantly reduced capacity: the number of municipal SAO offices (which deal with 95 percent of criminal cases) was cut by one third from 33 to 22 and between 2010 and 2015 the number of state attorneys in criminal departments was reduced from 412 to 397 in 2015. CEPEJ data confirm reduction in the number of prosecutors in Croatia since 2010. In 2014, Croatia had 13.4 prosecutors for every 100,000 inhabitants which is 18 percent lower than the EU11 average. Furthermore, SAO argues that the implementation of the CTS generated considerable additional burdens to SAO staff during project implementation and only recently began to facilitate case processing.  PDO Indicator 2: Reduced processing times 43. The second PDO indicator tracked reduced processing times in pilot courts financed by the project. At appraisal, this indicator was broken down by the three key stages subject to long delays: service of notice to parties; time between first and final hearings; and testimony of expert witnesses. However, the activities underpinning these indicators were not implemented. At the first restructuring in 2013, this indicator was simplified and tracked only the case processing time which is presented in the table below. The project achieved a 9.8 percent reduction in processing times but did not meet the 10 percent target within the project deadline. Table 2: Reduced processing times 2010-2016 2010 Target 2016 Actual Processing time 838 days 755 days 756 days (10 percent reduction) 44. On average almost 40 percent of cases take more than two years to complete with about 10 percent taking more than four years. A 2016 survey found that only 22 percent of respondents thought that the length of proceedings was improving, as opposed to 29 percent in 2010. The survey also revealed that the general public identified speeding up the court procedures as the 6 Act on amendments to the Criminal Procedure Act on 15 December 2013. 7 Su-IV, K-24/14-37 8 number one priority to improve the judicial system, ranking even higher than reducing corruption or reducing political influence. 45. The World Bank’s Doing Business survey paints a more positive picture of case processing. Contract enforcement in Croatia ranks highly: 9th in 2015 and 10th in 2016. Doing Business estimates that the time taken to enforce a contract is 572 days in 2016. The high overall rank is driven by relatively low costs for court fees, which the judiciary argues is not a positive feature and encourages excessive litigation. The ranking is also driven by high marks in the Doing Business methodology for court structure and proceedings that allow for random assignment of cases (through the ICMS), separate small claims procedures and the availability of ADR mechanisms.  PDO Indicator 3: Improved user ratings for efficiency 46. The third PDO indicator was intended to track user ratings of the efficiency of project- financed courts and the SAO offices. However, rather than tracking court users, the surveys undertaken by the project examined views of the general public. As 85 percent of the respondents of the survey had no interaction with the judicial system, the survey results are skewed by perceptions rather than experiences of the court system. Ratings remained static throughout the project. Survey ratings for the functioning of the judicial system in Croatia experienced a slight decline with those having a very negative view of the judiciary increasing from 17 percent in 2010 to 22 percent in 2016.8 Table 3: User satisfaction 2010 and 2016 2010 Target 2016 Actual User satisfaction 3.5 (out of 5) 3.85 3.4 (10 percent increase) (2.8 percent reduction) 47. Given the project’s design focus, it would have been better to track this indicator through direct surveys of court users rather than the general public. Meetings with the private sector in Croatia indicate that the business community are more supportive of the digitization in the courts although they also report that more should be done to incentivize judges to close cases quicker. Judges are currently assessed according to the “framework criteria” which sets various KPIs for the judges. Unfortunately judges can meet all their relevant targets while the system as a whole still suffers from continued delays and backlogs. 48. The private sector have suggested replacing the framework criteria with a case weighted system. They have also proposed that the Supreme Court should adopt the ICMS and drive behavioral change from the top. Greater incentives for ADR mechanisms – currently used in less than 0.2 percent of cases – would help to reduce the burden of cases. So too would the removal of administrative land registry and company registry transactions from the courts. The Croatian authorities are working on these reforms. It will take time for developments in ICT systems to impact court performance. Court presidents are starting to use the data created by the ICMS and CTS as a management tool to better distribute workloads and assess the performance of judges. As these management techniques are gradually adopted case processing is expected to accelerate and user satisfaction increase. Intermediate results: Improving the efficiency of the court system (Highly Satisfactory) 8 Evaluation on quality of selected justice services in the Republic of Croatia in 2016. IPSOS report, June 29, 2016. 9 49. The PAD presented three intermediate results indicators: the consolidation of the court network; improved court resource management; and the upgraded ICMS. At the first restructuring in December 2013, these indicators were revised to focus on: preparation of resource development plans in 30 municipal and county courts; backlog volumes and case disposal ratios in all regular and commercial courts; and quarterly publication of ICMS-based justice system performance data on average case duration per major types of cases. 50. The take-up and usage of the ICMS system and the data produced has been very impressive: all of the sub-indicator targets have been surpassed. The number of users of the ICMS increased by over 50 percent during the project implementation period. By the project closure, 100 percent of cases were registered in the system, more than a ten-fold increase over the baseline. The number of daily transactions also greatly surpassed the original target. Table 4: ICMS usage in Croatian courts 2010-2016 Indicator 2010 Target 2016 Actual Outcome Number of users 4,100 5,330 6,264 Target (30 % increase) (53 % increase) exceeded Transactions 18,500 55,241 74,560 Target registered per day (20% p.a. increase) (26% p.a. increase) exceeded (#) Cases registered (#) 467,000 513,700 6,062,259 Target (10% increase) (1,198% increase) exceeded 51. The new ICMS system has been extremely successful. In order to achieve these results the MOJ had to overcome tremendous resistance to change. Initially, the pilot system was not user-friendly. Judges and court officials complained that it took a long time to enter basic data and workloads were duplicated because they were asked to maintain paper-based files. However, the implementation team managed to improve the product in consultation with the vendor and resisted calls for the system to be scrapped. Court officers became familiar with the system through training, hands-on support and a help desk. The stakeholder workshop and discussions at the court complex in Pula confirm the efficacy of and user satisfaction with the ICMS system. The system is here to stay. Discussions now are focused around how to upgrade the system and introduce it in the Supreme Court. While the Supreme Court handles few cases and so an ICMS is not strictly necessary, its adoption of the ICMS would send a positive signal throughout the court hierarchy. Possible upgrades have been suggested to integrate a full document management system into the ICMS and move to an entirely paperless system based on e-filing and e-signatures. Users have also proposed linking the ICMS and CTS to the land registry, business register, tax and police information systems and other government systems such as pensions and health insurance to facilitate case processing and reduce documentation requirements. Intermediate Results: Improving the efficiency of the SAOs (Highly Satisfactory) 52. Support to the SAO was monitored using two intermediate results indicators which did not change throughout the life of the project: the consolidation of the SAO network and the upgraded IT system. SAO facilities were upgraded and constructed in Zagreb and Pula respectively. Despite some delays in the construction schedule and some teething issues around safeguards and use permits, the two facilities were completed during implementation. The SAO Zagreb office now provides modern facilities for the largest number of state attorneys in the country, including those working on criminal and civil matters, and for USKOK. The Pula complex includes offices for the court, land registry and SAO. Users moved into the newly completed building at the beginning of October 2016 and have expressed satisfaction with the construction quality and the impact of facilities on the efficiency and quality of their daily work. 10 Table 5: CTS usage in the Croatian SAO 2010-2016 Indicator 2010 Target 2016 Actual Outcome Number of users 737 1,306 1,820 Target (10% p.a. (16% p.a. increase) exceeded increase) Transactions registered N/A 74,234 Target per month (#) exceeded Cases registered (#) 6,393 19,089 354,449 Target (20% p.a. (195% p.a. increase) exceeded increase) 53. The CTS system in the SAO has been a tremendous success. Use of the system has far surpassed initial expectations. The SAOs delivered a structured schedule of training and workshops on the use of the CTS in Zagreb and in regional centers. Feedback on these sessions has been positive and the SAO plans to continue training. Training outcomes should also have been an intermediate results indicator for the project. Intermediate Results: Strengthening Management Functions of the MOJ (Moderately Satisfactory) 54. The project aimed to strengthen resource management and information systems in the MOJ. This would help the MOJ improve the management of justice sector resources. The Croatian authorities agreed to develop a medium-term expenditure framework (MTEF) for the justice sector that would include a prioritized program of investments and a set of actionable justice sector performance indicators. This agreement was captured in a Loan Covenant. 55. The MTEF was originally supposed to be delivered within two years of project effectiveness, taking into account the policy recommendations of a Bank-executed JSPEIR, to be delivered in the first year. However, the JSPEIR was only completed in October 2014 by which time decisions had been taken on court rationalization and interest in the proposed MTEF had waned. Towards the end of the project’s life, Bank teams attempted to interest the Croatian authorities in a revised set of activities that could have complied with the covenant but expenditure planning was not seen as relevant in the absence of an agreement on medium-term expenditure limits with the Ministry of Finance and so these proposals never materialized. 56. The original project design and 2013 restructuring both envisaged support for management and systems development in the MOJ. This included an inmate information system for prisons, an infrastructure stock-take to inform the prioritization of future capital investments, an ICT asset management system, a data warehouse, the development of justice sector indicators and more. Client interest in these activities waxed and waned over the period, depending on the personnel directing the PIU at the time. In the end, none of these activities were implemented. 57. On a positive note, the project helped the MOJ to absorb significant funding from other donors. The project funded the design of more than seven courthouses, which are being upgraded using IPA funding. It also funded an ICT enterprise architecture assessment and ICT strategy which provided the blueprint for the next phase of ICT reforms across the sector. After civil works at Karlovac were cancelled, the Norwegian Ministry of Foreign Affairs agreed to fund the works, along with a package of technical assistance to boost performance in that court and preparation of the designs and investment program for the Zagreb Judicial Square. 58. Project management was tracked using two indicators that remained constant throughout the project implementation: conducting user surveys and timely project implementation. User surveys were implemented as planned and on time. They produced valuable data that was used by 11 the MOJ and is presented in various sections of this report. The surveys would have been more helpful if they had been conducted more frequently because this would have allowed them to be used to inform ongoing decision-making. On project implementation more broadly, the PIU managed to stick closely to procurement schedules, despite significant reductions in staffing. Although the implementation often experienced delays, most of these were beyond the control of the project, such as changing municipal regulations which delayed the finalization of civil works. 3.3 Efficiency 59. An economic and financial analysis of the project is presented in Annex 3. This analysis uses the same indicators of efficiency that were presented in the original PAD (such as average cost per case, average time per case and the economic benefits from the quicker recovery of claims). The PAD estimates the net economic benefit of the JSSP over five years at EUR 23.0 million (IRR 189 percent). While the economic analysis in the original PAD makes various assumptions of the benefits the analysis presented in this report is based on actual data from the ICMS system and other sources. 60. The economic analysis of the project considers the direct benefits and costs associated with changes in economic welfare arising from the project. The financial analysis considers the costs and benefits of the project from a government perspective using a cash flow approach. 61. The economic analysis addresses only the economic value of unlocking the amounts tied up in Croatia’s judicial system earlier than would have been the case without the project, as a result of the reduced duration of court cases. The improvements resulting from the project are more far reaching but these are not easily quantifiable, including greater transparency and predictability of the judicial system, reduced corruption, increased trust in government, and a more stable investment climate and greater business confidence. 62. Similarly for the financial analysis, our calculation covers only selected interventions (i.e., those quantifiable) and does not include the positive impact of the project on improved prestige of judiciary, improvement of morale among judges, the efficiency of the buildings in terms of case processing, maintenance and energy efficiency, and the impact of the buildings on the surrounding districts, or advantages stemming from access to vulnerable groups such as the disabled. 63. Under these very conservative assumptions, the net financial benefit of the JSSP discounted over a fourteen year period is estimated at Euro 31.36 million generating an internal rate of return of 43 percent which justifies the project interventions. 64. The economic benefit of the JSSP over fourteen years is estimated at EUR 6.22 million (IRR 18 percent) which also justifies the project interventions. The differences between the IRR in the PAD and the final analysis can largely be attributed to PAD’s overoptimistic assu mptions regarding the economic benefit derived from the rollout of the ICMS. 3.4 Justification of Overall Outcome Rating Table 6: Assessment of the Overall Project Outcome Rating Relevance of Efficiency Outcome Objectives Design High Modest Substantial Substantial 65. The project had a single objective which was to improve the efficiency of the Croatian justice system. The project’s impact on the achievement of this outcome was substantial. The 12 project succeeded in constructing and refurbishing two large facilities, rolling-out the ICMS nationwide and helped the Government leverage additional resources from other donors to scale- up some of the project achievements. The JSSP project also greatly contributed to improving the transparency and efficiency of the court and SAOs. It provided much needed office space for the anti-corruption office USKOK. The PDO indicators on reducing the backlog of cases and improving the disposition rate were achieved in the courts but other developments in the SAOs, including changing legislation, increased caseloads and reduced staff numbers made the PDO targets difficult to achieve despite significant improvements in efficiency. The PDO indicator on a reduction in processing times was almost met and is likely to be reached fairly soon as improvements continue to be made in the Croatian justice system. 66. The one PDO indicator that was not met was the improvement in user ratings. However, this is partly explained by the survey design which targeted the general public rather than actual users of the court system. This indicator was not the most appropriate to track improvements in user satisfaction. When the survey data are disaggregated it is clear that lawyers and businesses have improved their perceptions of the justice system in Croatia. The number of court managers with a positive view of the system has increased from 25 percent in 2010 to 33 percent in 2016. It is likely that improvements in back-office functions will continue to be made following the implementation of the project, as court users become more familiar with the new IT systems and begin using it to better allocate resources. 67. All thirteen of the intermediate results indicators were achieved (see Annex 2). The intermediate results indicators related to the uptake of the ICMS and CTS systems have been greatly surpassed in every category. The buildings in Zagreb and Pula have been completed on time and the PIU has maintained the procurement schedule, despite budget cutbacks. The before- and-after surveys have been undertaken and have informed the ongoing reform agenda. 68. Building on these project outcomes, in November 2016, the MOJ finalized procurements for further ICT upgrades using EU funds. The MOJ is also working on a number of other activities started under the project including quality management, embedding the court rationalization through the creation of hubs, intensified training, increasing the use of ADR and a greater emphasis on change management. All bode well for the sustainability of the project achievements. There are also ongoing discussions over the proposed Zagreb Judicial Square. 3.5 Overarching Themes, Other Outcomes and Impact (a) Poverty Impacts, Gender Aspects, and Social Development 69. The principal poverty and social development impacts of justice sector reforms are indirect: improvements in the transparency, efficiency and predictability of the justice system improve the investment climate and enable increased economic activity. Approximately EUR 8 billion – approximately 25 percent of Croatia’s GDP – is still claimed in pending backlogged court cases. Clearing the backlog and speeding up case processing would free up some of this capital and encourage private investment. Unfortunately, it has not been possible to demonstrate these impacts: JSSP coincided with a prolonged recession in Croatia and so any improvement in the investment climate have been overshadowed by a decline in economic activity. 70. The transparency and impartiality of the justice system is a major pillar in the fight against corruption. The ICMS supported these objectives in several ways. By introducing random allocation algorithm to case assignment, the ICMS significantly reduced opportunities for judges and lawyers to influence the assignment of judges to cases. The automation of case processes reduced opportunities for malfeasance among court staff. ICMS increased integrity and accountability in the courthouse because it shows who made what changes to the case file. The 2016 survey indicated less than 5 percent of respondents had experienced corruption within the 13 past year, which is on par with averages in the EU11 and Southern and Eastern Europe. Nonetheless, more could be done in this regard especially through audio and visual recordings of the court proceedings. The Bank-financed Court and Bankruptcy Project had purchased recording equipment for 89 court venues but this was never installed. Given the Bank’s role in supporting recordings of court hearings in other countries, the Bank team could have followed up on this opportunity even though it was not within the remit of the original project. 71. Most of the project beneficiaries are women. As of 2014, women comprised 70 percent of all judges and 66 percent of all state attorneys in Croatia. These ratios are higher than the averages in the EU11, all EU countries and ECA. Furthermore, court and SAO staff are overwhelmingly women – though the project lacks precise figures this is estimated at over 90 percent. By supporting these beneficiaries to improve their skills, productivity and working conditions, it may be said that the project had a positive gender impact. Unfortunately the survey data collected by the project is not disaggregated by gender, so it is not possible to analyze gender impacts in more detail. (b) Institutional Change/Strengthening 72. The project contributed to significant institutional changes, notably through development of the ICMS in courts and the CTS in SAOs. The ICMS in its initial form had been piloted in nine courts through the earlier Bank-financed Court and Bankruptcy Project. The JSSP improved upon ICMS design, upgraded it and rolled it out across the country. This entailed a nation-wide change in the way that cases are managed and tracked through the justice system. Extensive training for court officials on the new IT systems built the capacity of the system. The new system allows for faster proceedings as each step is tracked and accompanied by new time limits which are respected in the vast majority of new cases. The random assignment of judges improves the transparency and impartiality of the justice system. A portal (E-Case) now provides parties with online case information. 73. In future, as court presidents, heads of departments and policymakers use the ICMS data to inform management decisions, the ICMS will increase institutional capacity as workloads are better distributed and the performance of individuals within the system can be measured and appraised. These improvements are starting to be seen during the project implementation period and will gain prominence in the years to come. This is a key issue in Croatia as the amount of spending per capita on the justice system remains significantly higher than the EU average as does the number of judges per citizen. More difficult to quantify but even more important is the change in mindset, moving towards a model of openness and transparency where information is available publicly and the judiciary move towards a service-based culture promoted through the project. (c) Other Unintended Outcomes and Impacts (positive or negative) 74. An interesting unintended consequence of the project activities was highlighted during the visit to the newly completed court complex in Pula. Court, SAO and land registry official highlighted the improvement in the character and safety of the neighborhood within which the complex now resides. Prior to the construction of the new building the site had been derelict site that attracted delinquents and criminal activity. The safety and aesthetic of the neighborhood has improved due to better lighting and 24-hour security for the government building. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Beneficiary Survey 75. A beneficiary survey was carried out by IPSOS consulting firm on behalf of the PIU and was delivered on June 29, 2016 just prior to the project closing. The survey provided a detailed 14 picture of the before and after status of justice services in Croatia. The findings of the survey are summarized in Annex 5. Stakeholder Workshops 76. A stakeholder workshop was convened at the Bank office in Zagreb on October 11th 2016. A summary of the proceedings is presented in Annex 6. It was followed by a roundtable discussion with the private sector convened at the AmCham offices in Zagreb on October 12th 2016. 77. Overall, participants to the stakeholder workshop were supportive of the project and highlighted the transparency, efficiency and cost-savings and improved justice services that it delivered. They also noted the strong performance of the PIU and the continuity provided by the PIU’s project coordinator, who worked with the previous Bank project and with JSSP through to project closure. 78. In particular, stakeholders placed value on the construction of buildings as a tangible outcome of the project. Many participants felt that the Bank should have used its leverage to persuade the Government to continue construction in other sites, especially Split due to its economic importance. They appreciated that the Government used Bank finance to complement Norwegian funds in Karlovac, and EU funds in four additional locations (Sisak, Osijek, Benkovac, Ivanić Grad). Participants also appreciated the Bank’s relative flexibility in changing course and adding and removing activities. Follow-up site visits revealed a high degree of satisfaction with the new facilities and an appreciation for the Bank’s procurement and oversight of the construction. 79. The stakeholder workshop highlighted the tremendous support for the ICT systems and a sense of urgency to complete what remains an unfinished agenda. The plans that were put together at appraisal are now out of date and participants suggested a number of additional developments that could help move to an entirely paper-less system linked to the other relevant government databases. Participants criticized the change management process which was described as weak especially at the outset of the project. The bar association would have liked to have been consulted to a much greater extent during the project preparation and implementation. They were represented on the project steering committee which met infrequently during implementation and it was felt that this was inadequate given that they would be important stakeholders and users of the new system. A more effective change management process could have helped to avoid the tremendous resistance to the ICT system that at one point threatened to derail the process. 80. Consultations with the private sector highlighted a desire from the business community to hold the judiciary more accountable for results. Participants argued that judges’ performance should be measured more accurately and transparently and judges should be encouraged to improve their performance through positive incentives without compromising their independence. They would like to see the Supreme Court leading the change management process to ensure that the statistics and capabilities of the new systems are used to their fullest extent. The private sector highlighted the need to professionalize the judicial cadre with more demanding recruitment and promotion processes and mandatory in-service training for judges so that they are aware of new legislation. Participants also advocated a number of procedural and institutional reforms, notably: review of the procedures relating to expert witnesses to ensure that they have relevant qualifications and are independent; upgrading of land and company registries and transfer of these functions outside of the courts; expansion of ADR mechanisms and specialized training for a small number of judges on intellectual property rights. 15 4 Assessment of Risk to Development Outcome Rating: Moderate 81. While the project made significant progress in improving efficiency, there is more work to be done to reduce the backlog of cases and improve processing times. Nonetheless, there is consensus among once-recalcitrant stakeholders that reforms to increase efficiency must continue and deepen. 82. The risk to the project-financed investments especially the ICT systems is moderate. These investments equip the judiciary with the tools and skills to manage their workloads and improve efficiency in the future. The ICMS and the CTS are embedded into daily operations and now form the backbone of the judicial system. Training and capacity building opportunities have strengthened skills of judges, prosecutors and their staff and they can now work efficiently with these systems. The data generated by these systems is used to inform decision-making and guide reforms. MOJ plans to continue upgrading the ICMS, which bodes well for its sustainability. The SAO has similar plans and will allocate additional state budget for the continued training of attorneys and staff. 83. The civil works in Zagreb and Pula have improved facilities permanently. Pula provides a model that the SAO intends to replicate to future infrastructure upgrades. The MOJ has allocated adequate budget for ongoing operations and maintenance costs to ensure that these facilities will serve the courts and SAOs for many years to come. 5 Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 84. Project preparation was lengthy: almost three years. Very little analytical work was undertaken and – although recommended – no QER took place. The team processed a PPA in late 2008 which financed a number of preparatory activities prior to effectiveness, including a detailed procurement plan and several procurement packages. These were approved soon after effectiveness allowing the project to make its first disbursements in the first quarter following effectiveness. As a result the project was ready for implementation and disbursements exceeded initial estimates in the first two years after effectiveness. (b) Quality of Supervision Rating: Moderately Satisfactory 85. The team respected the standards for reporting and monitoring the project. Fiduciary and safeguards supervision was satisfactory throughout. The fiduciary team supported the PIU through the various FM and procurement challenges and risks were managed well. A Zagreb- based environmental specialist was available to address safeguards issues that arose during the supervision of the construction in Zagreb and Pula. The implementation schedule was met, except in the case of construction delays, but these were largely due to changes in municipal regulations and were outside of the control of the Bank or the Borrower. A Zagreb-based Senior Economist monitored the project throughout its duration and maintained stakeholder dialogue. 86. The team did not undertake a MTR. This was a missed opportunity. The MTR could have provided an opportunity to consult with stakeholders through structured dialogue to take stock, adjust the project design to meet changing context and emerging government priorities. A MTR 16 could have informed the 2013 restructuring and renewed political commitment for the project. The Bank team also missed a number of opportunities to influence important reforms, including the rationalization of the court network, anti-corruption reforms, and focus attention on the core, change management activities. 87. Delays in the JSPEIR reflected poorly on the Bank’s performance. The PAD portrays the JSPEIR as a critical analytical tool to drive sector reform through the development of the MTEF, which was required as a loan covenant. However, it took six years from approval of the EFO in 2008 to publication of the JSPEIR in October 2014. By that time, analysis was too late to inform the rationalization of the court network as intended in the PAD. With changes in MOJ personnel, there was no interest or ownership in the report. The Bank did not convene any dissemination or dialogue around the analysis, which further limited the report’s readership. By the time of the ICR, the EFO contributor, the Dutch MFA, had not received a copy. 88. Towards the end of the project, the team responded to requests that reflected emerging priorities, including the IT strategy and information architecture to leverage EU funds and intensified training and capacity to embed case management systems. Support to the personal bankruptcy reforms responded to an urgent client need and brought together teams from the Bank’s Governance and Finance and Markets practices to provide advice on policy and legislation, support project-financed training and accreditation, upgrades of systems and business processes and a public awareness campaign. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 89. Supervision was in many ways satisfactory. However, the Bank could have provided more support to the client if it had completed the JSPEIR earlier and undertaken a MTR to keep the focus on the need to continue reforms, mitigate risks and keep the project on track. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 90. The Government met the majority of the loan covenants and complied with co-financing commitments. However, following the global financial crisis the Croatian government implemented an austerity program that sought to impose hard budget ceilings on the spending ministries. Each ministry was presented with a total expenditure ceiling. The judiciary’s budget ceiling included amounts financed under the project. The Ministry of Justice curtailed the project spending in order not to exceed the total expenditure ceiling. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 91. MOJ ownership and commitment to the project waxed and waned over the implementation period due to changes in key personnel. Over the six years, the project endured through three electoral cycles, four Ministers of Justice, more than six PIU Directors (de facto and de jure) and five Heads of Sector for ICT. Some continuity in the PIU and technical-level personnel mitigated the turnover at the political level. The PIU remained stable but was often disconnected with leadership in the MOJ and across the sector. The steering committee met infrequently. During the ICR consultations, several stakeholders argued that the client could have drawn greater benefits from the project if they had demonstrated greater continuity and follow- through. 17 92. Nonetheless, the ICT reforms endured and were the project’s key success . The implementing agencies demonstrated a clear commitment to using ICT to drive efficiency gains in the judiciary. Within the MOJ, a core team of technical-level staff was dedicated to ensuring the success of the ICMS. They demonstrated skill and tenacity in seeing through a complex and challenging automation process. 93. Similarly, there were clear champions for change in the SAO, which provided the driving force to roll out the CTS across Croatia. Despite initial concerns during preparation regarding stakeholder fragmentation and engagement in criminal justice reform, the implementation of the component supporting the SAO worked well: activities were implemented as envisaged and on time, and political and technical commitment was maintained throughout. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 94. Borrower performance was strongest in support of the ICT components. These are complex and challenging processes that have frequently failed in other Bank-financed public administration projects. With support from the Bank, and a dedicated team of champions predominantly at the technical level, the judiciary was able to develop a system that meets their needs. Over time, they managed to embed these systems into daily operations, transforming the way that courts and SAOs operate. In areas beyond ICT, borrower performance was variable, depending on the interest of agency leadership at the time. 5.3 Lessons Learned 95. The project demonstrated that Bank projects can produce development results in the criminal justice sector. JSSP was the first Bank operation to finance engagement in the criminal justice sector as a significant part of the project, with prosecution service leading an entire component. The loan was approved and became effective prior to the issuance in 2012 of the Legal Note on Bank Engagement in the Criminal Justice Sector.9 There were concerns during project preparation about the reputational risks of this work, in particular potential exposure to politically motivated prosecutions. These risks did not materialize. The SAO proved a reliable and proactive counterpart. During the project period, SAO’s performance improved and the USKOK developed a strong track record in tackling corruption. Future projects could now be more assertive about supporting to this important sub-sector of the justice system, and could move beyond support to efficiency to include support to reforms that promote transparency, integrity and quality of justice. 96. The importance of change management cannot be overstated. Borrowers tend to be reluctant to allocated loan funds to change management activities and other ‘soft’ investments . Yet, under-investment in these areas can limit the effectiveness of the ‘hard’ investments and jeopardize policy goals. The Bank should be more proactive in emphasizing the value of such investments with clients. Under JSSP, substantial resources were dedicated to change management broadly described, through consultancy services, meetings, workshops, and training events that built consensus, fed in user feedback, and reduced resistance, but this was mainly at the technical level and was not explicit in the design. More could have been done, particularly at the strategic level, to embed a culture of change and support the transition within the Croatian judiciary as the ICMS and CTS were being developed and rolled out. At project closure, the client recognizes the importance of change management and intends to invest further in these areas in future. 9 (http://siteresources.worldbank.org/INTLAWJUSTINST/Resources/CriminalJusticeLegalNote.pdf). 18 97. Teams working on justice projects should work with clients to unpack the meaning of commonly used terms such as ‘judicial efficiency’ to ensure that the results framework captures the desired outcomes. The PDO indicators for this project (case duration in days, numbers of backlogged cases, clearance rate) were not unit measures and therefore not measures of efficiency strictly speaking. For example, backlog reduction is a worthy goal in Croatia, but measuring the absolute number of backlogged cases says little of the resources and effort invested in producing that goal. An increased clearance rate (the ratio of incoming to outgoing cases) is a common indicator in courts, but this indicator is affected by many factors outside of the court’s control, this is particularly true in dynamic developing country contexts with changing legislative frameworks like Croatia. A better indicator of efficiency might have been ‘the average number of resolved cases per judge per year’, where an increase in resolved cases per judge demonstrates the improved productivity of that unit. During JSSP preparation, project indicators were drawn from national plans and monitoring data that was already being collected by the institutions. In future, teams should unpack the indicators to ensure that they measure the intended outcome. 98. A programmatic approach combining various sources of financing can maximize impacts. JSSP complemented other external partners in several ways. The ICMS was originally developed and piloted under an earlier Bank project; it was further upgraded using IPA funds and was then scaled-up and implemented nation-wide under the JSSP. JSSP financed the ICT enterprise architecture and ICT strategy for the future justice sector which are now being implemented using IPA funds. JSSP also financed designs for court and SAO offices in four additional locations, which are being completed using EU funds. The Karlovac courthouse is being completed with funds from the Norwegian Government. In this way JSSP improved Croatia’s absorption and multiplied project impacts. 99. MTRs should be mandatory. A MTR in 2013 would have provided an opportunity to restructure the project, renew political commitment and focus on critical change management activities. The MTEF loan covenant should have been removed by this time, given that there was no Government interest and no clear rationale for this activity. The results framework should have incorporated new activities that supported the rollout of the personal bankruptcy reform agenda. This was identified as the motive for the restructuring. Appropriate restructuring following an MTR could have set the project on track for a Satisfactory or Highly Satisfactory rating. 5.4 Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 100. The Borrower’s Implementation Completion Report (ICR) was received on November 13th 2016. In accordance with the Minister of Finance’s request, the team has reproduced the ICR verbatim at Annex 7. There are no issues in the Borrower’s ICR that warrant a response here. 101. The ICR team notes the tremendous support and commitment to the ICR process that has been shown by the MOJ and Ministry of Finance counterparts. The information that they have provided and their leadership in the stakeholder workshop has greatly assisted the team in the completion of this report. (b) Co-financiers 102. There were no co-financiers of the project. However, the Dutch government provided an EFO in the amount of EUR 1 million. Their views were solicited as part of the ICR mission and are presented in Annex 8. (c) Other partners and stakeholders 19 103. A stakeholder workshop including the MOJ, courts, private sector, chambers of commerce, IT companies and academics was convened on October 11, 2016. Follow-up meetings were convened on October 12, 2016 with the private sector and October 14, 2016 with the SAO. A summary of the proceedings of the workshop and related consultations are presented at Annex 6. 20 Annex 1. Project Costs and Financing a) Project Cost by Component (in EUR Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (EUR (EUR millions) Appraisal millions) Improving the Efficiency of the 14.68 4.00 27.25 Court System Improving the efficiency of the 7.73 9.00 116.43 SAOs Strengthening management functions of the Ministry of 1.58 3.2 202.53 Justice and increasing efficiency Support for project management 1.2 0.8 66.67 and implementation Total Baseline Cost 26.00 17.00 65.38 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 26.00 17.00 65.38 Front-end fee PPF 0.00 0.00 00.00 Front-end fee IBRD 0.00 0.00 00.00 Total Financing Required 00.00 0.00 b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (EUR (EUR Appraisal millions) millions) Borrower 0.00 1.91 1.00 52.36 International Bank for Reconstruction 0.00 26.00 17.00 65.38 and Development 21 Annex 2. Outputs by Component Results Indicator Baseline Actual End of Project Target Achievement Project development objective indicators 1. Clearance rate for courts 96.39 112.16 Increased by 10% over Fully achieved. (Percentage, Custom Supplement)* baseline. 2. Clearance rate for SAO and 98.90 96.2 Increased by 10% over Not fully achieved. USKOK (Percentage, Custom baseline. Supplement)* 3. Disposition time for courts (Days, 280.18 238.13 Reduced by 15% over Fully achieved. Custom Supplement) baseline. 4. Aggregate case backlogs for Courts 437.892 349.051 Reduced by 10% over Fully achieved. Target exceeded (- (Number, Custom Supplement)* baseline. 20% not -10%). 5. Aggregate case backlogs for SAO 11.273 10.415 Reduced by 10% over Not achieved but substantially met (- and USKOK (Number, Custom baseline. 7.61% not -10%). Supplement)* 6. Reduced case processing times in 838.00 756 Reduced by 10% over Fully achieved. Processing time has pilot courts financed by the project (Days, baseline. been reduced significantly reaching Custom) 582 days in 3rd quarter of 2014. 7. Improved user ratings for efficiency 3.5 3.4 Third user survey: 10% Not fully achieved. It appears that for project-financed courts and SAO improvement over the aggregate user ratings have offices (Percentage, Custom) baseline value stagnated, even fallen slightly, and that the perceptions of Business have improved and the perceptions of citizens has deteriorated. 26 Intermediate results indicator – Component 1 Improving the Efficiency of the Court System Court Presidents and Court Secretaries in at N Y Y Fully achieved. least thirty priority municipal and country courts develop resource management plans (Yes/No, Custom) Numbers of users of ICMS 4.100 6.254 Increase by 30% over Fully achieved. Target exceeded + baseline. 52.78%. Number of transaction made in the ICMS 18.500 74.560 Increase by 20% over Fully achieved. Target exceeded + daily baseline. 26.2%. Number of cases registered in ICMS 467.000 6.062.259 Increase by 10% over Fully achieved. Target exceeded + baseline. 1.298%. Intermediate results indicator – Component 2 Improving the Efficiency of the SAOs Consolidated and modernized facilities for N/A Y Consolidated Revised target achieved. a functioning (a) SAO HQ and USKOK in and Zagreb and (b) SAO in Pula (Text, modernized Custom)** facilities for (a) SAO HQ and USKOK in Zagreb and (b) SAO in Pula Upgraded and Updated Operational N Y Y Fully achieved. Information Systems Improve SAO efficiency (Text, Custom) Numbers of users - Operational Information 737 1.820 Increase by 10% over Fully achieved. Target exceeded + Systems baseline. 16%. Number of transaction on cases per month - N/A 74.234 Increase by 10% over Fully achieved. Operational Information Systems baseline. Number of cases - Operational Information 6.393 354.449 Increase by 20% over Fully achieved. Systems baseline. 27 Intermediate results indicator – Component 3 Strengthening Management Functions of the Ministry of Justice A medium term expenditure framework for No MTEF exists Dropped MTEF developed by Dropped. After numerous delays due the justice sector is developed by MOJ on for Justice MOJ to the budget constraints and slow the basis of justice sector expenditure and sector, not decision making process in MOJ, institutional review (Text, Custom) prioritized MOJ decided to develop MTEF with investment their own resources. program or a clear set of actionable indicators. Strengthened MOJ information system and N Dropped. Upgraded pilot Dropped. After delay due to budged their management evidenced by: (i) an information system constrains procurement was initiated operational ICT asset management system; functional across all but it had to be cancelled due to (ii) functioning Data Warehouse System correctional facilities, technical issues. (specific technical (Text, Custom) county courts and all equipment was not available). misdemeanor courts ii) integrated ICT asset management system containing all major ICT assets under administrative control, digital communication system, modern document management system operating in MOJ Intermediate results indicator – Component 4 Support for Project Management and Implementation Timely project implementation including N Y Report (including Revised indicator achieved. adherence to procurement schedules survey results as (Yes/No, Custom)*** appropriate). Clean Audit reports. 28 Conduct of periodic user surveys and N Y Y Fully achieved. Survey conducted publication/dissemination of the results of for evaluating Court users’ such surveys (Text, Custom) satisfaction. * Indicators revised following the 2013 restructuring which conducted to the split of PO 1 Indicator and PO Indicator 2. **Indicator revised following the 2013 restructuring which decreased the number of courts to be modernized from 5 to 2, namely SAO HQ and USKOK in Zagreb and (b) SAO in Pula *** Indicators revised following the 2015 restructuring. 29 Annex 3. Economic and Financial Analysis Context 1. Croatia became the 28th member of the EU on July 1st, 2013. The EU accession process was accompanied by significant justice reforms, including the implementation of new laws and amendment of others, improved case management and enforcement and computerization of land registers. The rationalization of the courts network and of state prosecution offices is a continuing process. Economic and Financial Analysis 2. The economic analysis of the project considers the direct benefits and costs associated with changes in economic welfare arising from the project. The financial analysis considers the project only from the perspective of the Ministry of Justice and ignores the broader social and economic results of the project. The analysis presented below uses data for 2010-2016 obtained from the Ministry of Justice ICMS, the World Bank Doing Business survey and the Bank’s Croatia Justice Sector Public Expenditure Review. 3. In order to simplify the analysis, the cost side considers only the project expenditure and does not also factor in the Bank staff time and supervision costs. The later are not relevant from a government perspective and are in any case insignificant to the outcome. 4. The financial analysis considers the costs and benefits of the project using a cash flow approach. It considers the savings generated by the project from an increased number of cases cleared by the system and estimates a reduction in total costs generated by this increased efficiency. We use Doing Business data to estimate the average costs of a case and check this against the Ministry budgets and the total caseload as measured by the system. 5. The economic analysis of the project considers the benefits for society. As mentioned above there is an estimated Euro 8 billion locked up in backlogged cases. If the duration of these cases is reduced it should unlock the disputed amounts for investments earlier. We assume that such investments once cleared of court procedures could be invested and earn a 2 percent rate of return. The economic analysis calculates these benefits drawing on project data obtained from the PIU. The economic analysis does not consider any other time savings for the general public or increased investments that might come as a result of a more efficient judicial system. 6. There are significant additional economic benefits which are difficult to quantify. These include greater transparency and accountability of the courts and SAO system, reduced corruption and increased trust in the judicial system and government. Moreover, indirect benefits include a more predictable and uniform application of the rule of law, a more stable investment climate, and strengthened business confidence. Together, these factors could help spur investment and promote economic growth but are not considered in this analysis in order to provide for a very conservative estimate of the benefits. Our calculation does not include the positive impact of the project on improved prestige of judiciary, improvement of self-esteem of judges, beauty of the buildings that make the districts better living spaces, or advantages stemming from access to vulnerable groups such as the disabled. Assumptions 7. A more detailed discussion of this analysis and assumptions used for the period after the project completion are provided below: 30  The EU inflation rate is 1.5%, and the inflation rate in Croatia is -0.9 per annum. 10 The exchange rate is 7.5 Kuna to Euro.  Number of disposed cases for courts is 294,500 by project closure, and for SAO is 709,211. It is estimated to grow at 1% p.a. after 2016.  The average cost per case is estimated at EUR 22,000 from Doing Business 2016.  In addition, the rollout of the new framework and trainings generated significant performance improvements, both for the courts and the SAO. However, SAO and USKOK experienced a critical moment not related to the project implementation but to control staffing determined by MOJ. Position control was an essential element for budget control, however not flexible enough to address operational inefficiencies. In our assumptions we estimate a flat budget for the MOJ that does not address possible increases or decreases in staffing over time. In reality this is again a conservative estimate given the high number of judges and court personnel per capita. Conclusion 8. Under these assumptions, the net financial benefit of the JSSP discounted over a fourteen year period is estimated at Euro 31.36 million generating an internal rate of return of 43 percent which justifies the project interventions. 9. The economic benefit of the JSSP over fourteen years is estimated at EUR 6.22 million (IRR 18 percent) which justifies the project interventions. Figure 1: Net Economic Benefits, 2017 - 2030 (EUR mil.) Net Economic Benefits 14.0 (mil EUR) 12.0 10.0 8.0 6.0 4.0 2.0 0.0 Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10 Year 11 Year 12 Year 13 Year 14 10 Economic Outlook, data published on November 9, 2016 31 Table 7: Estimated Financial and Economic Benefits due to Efficiency Gains Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 Economic costs Financing -3.00 -3.00 -3.00 -3.00 -3.00 -3.00 Economic benefits Number cases cleared without project courts 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 250,325 SAO 689,849 689,849 686,082 682,316 678,549 674,782 671,016 671,016 671,016 671,016 671,016 671,016 671,016 671,016 671,016 671,016 671,016 Number cases cleared with project courts 250,325 257,688 266,523 275,358 284,193 293,028 294,500 297,445 300,419 303,424 306,458 309,522 312,618 315,744 318,901 322,090 325,311 SAO 689,849 693,076 696,303 699,530 702,757 705,984 709,211 709,211 716,303 723,466 730,700 738,007 745,387 752,841 760,370 767,973 775,653 Benefits from cost savings Average spend without project € 210 € 210 € 211 € 212 € 213 € 214 € 215 € 215 € 215 € 215 € 215 € 215 € 215 € 215 € 215 € 215 € 215 Average spend with project € 210 € 208 € 206 € 203 € 201 € 198 € 197 € 197 € 195 € 193 € 191 € 189 € 187 € 185 € 183 € 182 € 180 Cost saving per case €0 €2 €6 €9 € 13 € 16 € 18 € 18 € 20 € 22 € 24 € 26 € 28 € 30 € 31 € 33 € 35 Total cost savings €0 €1 €2 €3 €4 €5 €5 €5 €6 €7 €7 €8 €9 €9 € 10 € 11 € 11 Benefits from unlocking backlogged cases Number of backlogged cases 437,892 423,658 409,424 395,189 380,955 366,721 349,051 314145.9 282731.3 254,458 229,012 206,111 185,500 166,950 150,255 135,230 121,707 Amount unlocked €0 € 326 € 326 € 326 € 326 € 326 € 405 € 800 € 720 € 648 € 583 € 525 € 472 € 425 € 383 € 344 € 310 Benefits (assuming 2% return) €0 €7 €7 €7 €7 €7 €8 € 16 € 14 € 13 € 12 € 10 €9 €9 €8 €7 €6 Benefits from time savings 838 Duration of case without project (days) 838 838 838 838 838 838 838 838 838 838 838 838 838 838 838 838 Duration of case with project (days) 838 816 799 780 770 768 756 744 733 721 710 699 688 677 667 656 646 Time saved in days 0 22 39 58 68 70 82 94 105 117 128 139 150 161 171 182 192 Total benefits €0 €0 €1 €1 €1 €1 €2 €4 €4 €4 €4 €4 €4 €4 €4 €3 €3 Net benefit Financial benefit 0.0 -2.4 -1.5 -0.5 0.6 1.6 2.2 5.4 6.1 6.7 7.4 8.0 8.7 9.3 10.0 10.7 11.4 Economic benefit 0.0 -2.6 -2.3 -2.0 -1.8 -1.7 -1.2 4.1 4.2 4.2 4.1 4.0 3.9 3.7 3.6 3.4 3.3 Financial NPV (@10%) € 31.36 Economic NPV (@10%) € 6.22 IRR 43% EIRR 18% 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes a) Task Team members Responsibility/ Names Title Unit Specialty Lending Amitabha Mukherjee Lead Public Sector Specialist GGO15 Task Team Leader Lamija Marijanovic Financial Management Specialist GGO21 FM Sanja Mazdarevic-Sujster Senior Economist GMF03 Craig Neal Consultant GGO15 Justice Charles R. Simpson Consultant GGO15 Justice Natasa Vetma Senior Env. Safeguards Specialist GEN03 Env. Safeguards Antonia Viyachka Procurement Specialist ECCHR Procurement Supervision Amitabha Mukherjee Lead Public Sector Specialist GGO15 Task Team Leader Charles R. Simpson Consultant GGO15 Justice Dubravka Jerman Craig Neal Consultant GGO15 Justice Daniel Mullins Consultant GGO15 Justice Sherry Yang Barry Raymond Walsh Senior Counsel Justice Andrew Vorkink Viorela Voinea Financial Management Analyst GGO21 FM Rajeev Kumar Swami Lead FM Specialist GGO21 FM Alberto Leyton Senior Public Sector Specialist GGO15 Task Team Leader Olga Schwartz Consultant GGO15 Eric Peterson Sayyora Krulikowski Operations Officer Anita Komlodi Matija Laco Operations Officer ECCU5 Valeria Nikolaeva Procurement Specialist GGOGI Procurement Ireneusz M. Smolewski Senior Procurement Specialist GGO03 Procurement Georgia Harley Public Sector Specialist GGO15 Task Team Leader Milena Joksimovic Consultant GGO15 Danijel Marasovic Architect GGO15 Infrastructure Ivana Ivcic Env. Safeguards Consultant GEN03 Env. Safeguards Antonia Viyachka Procurement Specialist ECCHR Procurement Sanja Mazdarevic-Sujster Senior Economist GMF03 Ljiljana Tarade Operational Officer ECCHR Operations Lamija Marijanovic Financial Management Specialist GGO21 FM Ruzica Jugovic Program Assistant ECCHR ICR Ismail Radwan Lead Public Sector Specialist GGO15 Elena Gabriela Ardelean Consultant GGO15 Georgia Harley Public Sector Specialist GGO15 30 b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY07 5.18 49.19 FY08 17.54 200.41 FY09 21.54 275.50 FY10 21.47 198.88 FY11 0.30 36.41 Total: 66.03 760.39 Supervision FY10 0.00 9.48 FY11 22.48 181.04 FY12 9.37 582.10 FY13 7.63 171.51 FY14 11.90 89.26 FY15 13.89 98.36 FY16 12.79 78.17 Total: 78.06 1,209.92 ICR FY17 3.75 35.49 Total: 3.75 35.49 31 Annex 5. Beneficiary Survey Results 1. A beneficiary survey was finalized in June 2016, by IPSOS consulting on behalf of the PIU. Following the survey a report was produced that presents the results and the comparison of the results of the initial study in 2010. Three different surveys have been conducted and the main conclusions are presented here below: a. survey of trust in the justice system among the general public; b. survey of satisfaction among three groups of users of courts which are covered by the Project; c. employee satisfaction survey in the judicial bodies covered by the Project. 2. Conclusion #1: Public opinion survey among the citizens of Croatia shows that, even after six years of reforms, more than half of citizens do not trust the justice system. 39 percent of citizens of general population trust in SAO’s work, while 38 percent trust the Courts, putting them in the lower half of the list of trusting institutions (Figure 1).11 Figure 2: Trust in State institutions - 2010-2016 comparison Source: Evaluation on Quality of Selected Justice Services in the Republic of Croatia in 2016, Ipsos 11 Evaluation on Quality of Selected Justice Services in the Republic of Croatia in 2016, Ipsos 32 Figure 3: Perceptions of changes in judicial system - 2010-2016 Source: Evaluation on Quality of Selected Justice Services in the Republic of Croatia in 2016, Ipsos 3. Conclusion #2: The greatest perceived shortcomings of the justice system are duration of procedures, frequent changes in laws and their vagueness. Almost identical percentage of respondents from the general population believes that the court proceedings last too long (74 percent in 2010 and 75 percent in 2016) and they find that it is, with the political influence on courts, the biggest problem of the judicial system. Although there are minor differences among the indicators in observed years, the attitude of general population towards either of them did not change significantly. 4. Conclusion #3: Perception of corruption among respondents from the general population has not changed compared to 2010: about 50 percent of respondents believe that corruption is the biggest problem in the Croatian judiciary. 5. Conclusion #4: Compared to 2010, when about 40 percent of cases took more than two years, in 2016 in three observed samples, 38 percent of disputes still take longer than two years. 6. Conclusion #5: The level of general satisfaction with the court compared to six years ago is almost without change, and respondents, on average, evaluate the courts with neutral mark (3.4 in 2016 vs. 3.5 in 2010). Table 8: Overall satisfaction with the court, premises and employees in surveyed groups Lawyers The average of Natural Managers and all three groups persons notaries 201 201 201 201 201 2010 2010 2016 6 0 6 0 6 Overall satisfaction with the court 3,5 3,4 3,3 3,4 3,5 3,2 3,7 3,7 Overall satisfaction with the work 3,7 3,8 3,3 3,7 4,1 3,8 3,7 4,0 of judges Overall satisfaction with the work 3,9 4,1 3,5 4,0 4,3 4,2 4,0 4,2 of judicial clerks Arrangement and comfort of 3,2 2,5 3,2 2,7 3,4 2,4 3,1 2,4 premises 33 Adequately equipped rooms 3,2 2,5 3,1 2,7 3,4 2,4 3,2 2,4 Marks and ease of reference in the 3,8 3,1 3,7 3,5 3,9 2,9 3,9 2,9 building 7. Conclusion # 6: Respondents were generally satisfied with the work of judges in their case. The greatest satisfaction (4.0) is expressed by professional court users, while legal entities (3.7) and natural persons (3.8) were somewhat more critical in their evaluations. If we compare the ratings with those of 2010, differences in scores were significantly better / worse depending on the studied group. Table 9: Satisfaction with courts, premises and judicial clerks in observed cities MC and CC MC MC SPLIT MC ZADAR PULA KARLOVAC 2010 2016 2010 2016 2010 2016 2010 2016 Overall satisfaction with the 3,8 3,8 3,2 3,3 4,0 3,1 3,2 3,8 court Overall satisfaction with the 4,0 3,9 3,7 3,8 3,9 3,7 3,1 3,9 work of judges Overall satisfaction with the 4,2 4,0 3,9 4,1 4,1 4,3 3,5 4,1 work of judicial clerks Arrangement and comfort of 3,4 4,0 2,7 1,8 3,7 2,2 4,1 3,5 the rooms Adequately equipped rooms 3,3 4,0 3,2 1,8 3,8 2,5 3,9 3,4 Marks and ease of reference 4,3 4,1 3,2 2,5 4,2 2,6 4,3 3,9 in the building 34 Annex 6. Stakeholder Workshop Report and Results Main Stakeholder Workshop 1. The World Bank in collaboration with the MOJ organized a stakeholder workshop on October 11, 2016. The main objective of the workshop was to discuss the achievements of the Justice Sector Support Project. The ICR team solicited feedback from stakeholders on how the collaboration of the MOJ and the Bank on the Project could have been improved and in what areas the Ministry could make further progress to improve its efficiency. The lists of participants is included in Table 9. 2. The workshop program started with opening remarks by the MOJ and the Bank followed by presentations on the main components and results of the Project and concluded with an open discussion. This report reflects the main highlights of the presentations and discussion with the stakeholders. 3. The MOJ emphasized that the project aimed to improve the efficiency of Croatia’s justice system in many areas, but the key outputs of the project that had influenced or would affect improvements in the justice system efficiency the most were: design, development and implementation of the CTS at SAOs, upgrades on ICMS in the courts, rehabilitation of Pula complex and SAO/USKOK offices in Zagreb, and training for court and SAO personnel. 4. The MOJ highlighted the context, initial objectives and final results of the Project sharing with the audience the ministry’s overall satisfaction with the outcomes of the project, emphasizing the civil works in Pula and Zagreb, as well as the IT component. MOJ highlighted the important role of the Bank during implementation in providing continuous support and showing flexibility in changing course. The ministry expressed its regret that they had to reduce the number of courts to be modernized from 5 to 2, due to budget constraints. Securing construction funds from either the state budget or the European Commission has proved to be extremely difficult and the Bank project played a useful role in supplementing other funding sources. 5. The PIU, delivered a high-level presentation on challenges encountered during the five- year project cycle highlighting the tremendous changes in project leadership that accompanied government and electoral changes. It was noted that changes in minister were uniformly accompanied by a large turnover in the managerial cadre of the ministry which often stalled progress until new personnel were fully informed and committed to the project objectives and activities. 6. The Bank team presented the changes in the project, such as the two restructurings (first one in December 2013, second one in June 2015), as well as the project components and activities, and the progress against development objective indicators. The overall project outcome rating was moderately satisfactory, all intermediate results indicators were achieved. Key highlights of discussion 7. The discussion was structured and introduced five questions, as follows:  What worked well?  What could have been done differently?  Do you think the money was well spent?  Has the project impacted the work of courts? And if so how?  What was left undone and what would you focus on in the future? 8. Overall, the workshop participants were supportive of the project and highlighted the good performance of the PIU and continuity of leadership represented by Mr Nenad Vukodinovic 35 who had started working with the Bank on the earlier project and provided continuity during the entire five-year project cycle. 9. The workshop participants noted a strong preference for civil works as tangible outcomes of the project. The updates and developments regarding the IT system were highly appreciated and participants showed extreme satisfaction in using ICT systems. However, they highlighted the urgency to complete what remains for further development in order to upgrade the ICT system to allow for: an entirely paper-less system, digital signatures, mobile access, court recordings and links to relevant government information sources e.g. the land registry, police, pensions and health insurance databases. 10. The participants also noted that the project would have benefitted from better outcomes if the change management dimension was better developed. The participants discussed they encountered tremendous resistance to the change that almost conducted to process abandon. In addition, the lawyers’ association suggested that they would have preferred to have been involved and consulted to a much greater extend during the project preparation and implementation as the key users of the various systems. All participants agreed that the real results of the project are only just beginning to be seen and that far greater benefits will come in the future as court presidents, the MOJ, court users and other stakeholders increasingly use the information produced by the ICT systems for management purposes. Discussions with the Private Sector 11. On October 12th 2016, the American Chamber of Commerce (AMCHAM) in Zagreb convened a roundtable discussion on the project’s impact and remaining outstanding issues in the justice sector. The meeting indicated that the private sector in Croatia are more supportive of the digitization in the courts although they also report that more should be done to incentivize judges to close cases quicker. Judges are currently assessed according to the “framework criteria” which sets various KPIs for the judges to meet. Unfortunately judges can meet all their relevant quotas while the entire system still suffers from continued delays and backlogs. 12. The private sector have suggested replacing the framework criteria with a case weighted system and also putting pressure on the supreme court to adopt the ICMS and CTS and drive behavioral change from the top. Greater incentives for ADR (currently used in less than 0.2 percent of cases would help to reduce the burden of cases as would taking land registry and company registry out of the judicial process. These are all issues that the Croatian authorities are working on and it will take time for the systemic improvements to be seen more clearly as the supreme court adopt the new ICT system and more and more court presidents start using the data created by the ICMS and CTS as a management tool to better distribute workloads and assess the performance of judges. Other consultations with stakeholders – SAO Zagreb 13. As part of the stakeholder consultations, the World Bank team visited the SAO office in Zagreb on October 14, 2016. The SAO Zagreb did not attend the workshop on October 11th, 2016 due to other commitments but agreed to be interviewed separately. The team met with Mr. Dragan Novosel, Former Deputy State Attorney, and Ms. Silvija Kumic, Registrar of Criminal Department. The discussion followed the workshop’s structure and addressed the same five questions mentioned above. 14. Overall, the SAO was very satisfied with the project and its results. The SAO’s structure is different from the Courts’ and has therefore specific needs which were very important to identify and integrate them in the design of the JSSP project. SAO representatives highlighted the excellent relationships and communication throughout the project, though multiple actors 36 involved. The SAO benefitted from direct communication with the Bank that ensured project success. 15. The project supported the CTS roll out and all personnel were provided with comprehensive training. From a managerial perspective, the CTS has proved to be a very powerful tool which is now used on a daily basis offering tremendous improvements in the workflow. However, additional training is needed and in this perspective, the SAO Zagreb, is considering including a request for funding for continued CTS training in its budget submission. The SAO representatives also expressed their enthusiasm for future collaboration with the Bank. Table 10: List of participants invited to the JSSP Workshop Name Title Institution Ms. Maja Grubisin Assistant Minister Ministry of Justice Ms. Renata Duka Assistant Minister Ministry of Justice Ms. Sandra Artukovic- Former Deputy Minister of Ministry of Justice Kunst Justice Mr. Nenad Vukadinovic Project Management Advisor Ministry of Justice Ms. Ana Krnic Former ICMS Administrator Ministry of Justice Ms. Martina Vrdoljak Head of Statistics Unit Ministry of Justice Mr. Silvije Seremet Head of IT Department Ministry of Justice Ms. Suzana Hudin ICMS Administrator Ministry of Justice Ms. Gordana Andjelic Head of Department Ministry of Justice Ms. Maja Barberic Acting Secretary General Ministry of Justice Ms. Anka Sertic Head of Finance Department Ministry of Justice Ms. Branka Head of Sector for Projects and Ministry of Justice Augustinovic Investments Ms. Marijana Balenovic Head of Service for Projects Ministry of Justice Arbutin Ms. Marija Jurman Head of Department Ministry of Justice Ms. Zrinka Oreb PR Department Ministry of Justice Mr. Igor Vidra Head of Sector Ministry of Justice Ms. Tatjana Pinhak ICT Advisor Ministry of Justice Mr. Dejan Vukelic Chief IT Advisor Ministry of Justice Mr. Kristijan Turkalj International EU expert International Management Group Ms. Andrea Posavec Director Judicial Academy Franic Ms. Silvija Belajec Head of Sector Ministry of Finance Mr. Tomislav Maras President of Municipal Court in Municipal Court in Karlovac Karlovac Mr. Ivica Botica President of Municipal Court in Municipal Court in Split Split Mr. Dinko Cvitan State Attorney General State Attorney Office Mr. Mladen Bajic Deputy State Attorney General State Attorney Office Ms. Melina Ralis Senior Advisor State Attorney Office Ms. Silvija Kumic Registrar of Criminal Departmen State Attorney Office Mr. Dragan Novosel Former Deputy State Attorney State Attorney Office Mr. Robert Travas President of the Croatian Bar Bar Association Association 37 Ms. Lucija Popov President of the Croatian Notaries Chamber Notaries Chamber Mr. Branko Hrvatin President of the Supreme Court Supreme Court Ms. Davorka Curko Judge, President of the Civil Municipal Civil Court Nasic Department Ms. Ana Lovrinov Judge, President of the Municipal Civil Court Enforcement Department Ms. Nina Poznic Coordinator of Statistics and e- Municipal Civil Court Spis (ICMS) Ms. Zlatica Grahovac Head of the Special Registrar Municipal Civil Court Stojanovic Ms. Nada Plevko Judge Zagreb Commercial Court Ms. Nevenka Baran Judge High Commercial Court of the Republic of Croatia Mr. Srdjan Simac Association of Mediators Ms. Manica Hauptman European Semester Officer European Commission Mr. Mark van der Deputy Head of Mission Embassy of the Netherlands Linden Mr. Davor Majetic Director General Employers’ Association Mr. Darjan Dragicevic American Chamber of Commerce Mr. Dražen Penzar IBM Mr. Tomislav Mesic IBM 38 Annex 7. Summary of Borrower's ICR GENERAL PROJECT INFORMATION Justice Sector Support Project, which was prepared during the period late 2008 – late 2010 and described in the Project Appraisal Document dated January 27, 2010, has been designed as a development project with a goal to achieve further improvements in efficiency, service to citizens and consolidation of the Croatian justice system by strengthening capacity and improving various aspects of the justice system; as well as to assist Croatia's efforts on harmonizing its justice system with the requirements of accession to the EU. The Loan Agreement for financing of the Project was signed in April 2010 and came into effect in July 2010. Project Preparation The Project preparation was financed through a Project Preparation Advance (PPA, USD 653.000), which greatly contributed to the readiness of major project activities for implementation at the time when Loan Agreement was signed. This included setting up the Project Team within the Ministry of Justice, preparation and review of bidding documents for major civil works, as well as for several larger IT procurements. This resulted in a rather quick start with Project implementation where after the Project effectiveness in July 2010, the first civil works – CW-4 SAO and Court Building in Pula; and CW-3 SAO and USKOK Headquarters in Zagreb - were advertised in October and November 2010, respectively. During the PPA the core of the Project Team – the PIU – was also actively involved in designing the Project and in facilitating administrative and legal procedures during the project negotiations, signing of the Loan Agreement and its ratification. Project Components The Project was to reach its objectives through numerous activities grouped into 4 major Project Components: A. Improving the Efficiency of the Court System. This component was to finance: (i) support for consolidation of court network (rehabilitation and renovation of three courthouse locations in Split, Karlovac and Pula, housing five county and municipal courts; (ii) improving courts’ resource management capacity (training for court presidents and secretaries; (iii) modernization of operational information technology (IT) systems for decision-making and performance monitoring in courts; (iv) refining case management standards, guidelines, processes and practices; and (v) realizing efficiency gains in the enforcement of judicial decisions. The locations for civil works (Pula, Split and Karlovac) satisfied the key criteria, in conformity with the project objective, of significant impact, demonstrable efficiency gains, and readiness for implementation. B. Improving the Efficiency of the State Attorney’s Office. Sub-components comprised of the following: (i) consolidation of the State Attorney’s Office (SAO) network (SAO Zagreb headquarters complex including USKOK) and rehabilitation of an existing building for the Pula SAO; (ii) institutional capacity strengthening for the SAO; and (iii) case management and IT systems including automating SAO case management and case tracking. 39 C. Strengthening MOJ Management Functions. Sub-components comprised of: (i) technical support to strengthen justice sector budgeting, resource management and performance framework including activities to enable the MOJ to develop a sector-wide expenditure and performance framework; and (ii) strengthening MOJ information systems and their use in management decision-making, including initial steps to modernize the inmate information system and strengthen its connectivity with the justice information system. D. Project Governance and Implementation. Sub-components comprised of: (i) project monitoring, evaluation and change management; (ii) operating costs (including logistical, administrative and other expert technical support; (iii) user surveys; and the Project Preparation Facility. All activities were intended to complement each other, as well as other ongoing or planned MOJ's projects; and to provide constant assistance to MOJ's efforts in harmonizing the system with the EU requirements. Institutional and Implementation Arrangements According to the Loan Agreement the MOJ was the agency responsible for project implementation with the Minister of Justice as Loan Coordinator. Further, various MOJ directorates were to lead the implementation of project activities falling within their respective domain. The Attorney General’s Office was to lead the procedural and organizational innovations for the SAO, and the Office of the President of the Supreme Court was consulted on various activities relating to the court system. An MOJ Project Team, with the core Project Implementation Unit (PIU), was led by the designated MOJ official (State Secretary, later a Director, and finally Assistant Minister), and supported by advisers and consultants, and was to manage project implementation and be responsible for project procurement, financial management, accounting, disbursement, change management, monitoring and evaluation, and reporting. Project governance was vested in a Steering Committee comprising representatives of the Supreme Court, the State Attorney’s Office, the participating Municipal Courts and SAO offices (Split, Karlovac, Pula, Zagreb), the Ministry of Finance, the MOJ, the Association of Croatian Judges and the Association of Croatian Prosecutors. The Steering Committee was chaired by the Minister of Justice and supported by technical working groups on IT, civil works, enforcement issues and case-management issues. The staff and management of the MOJ Project Team were performing secretariat functions for the Steering Committee and working groups. Majority of these arrangements were in place already during the PPA implementation, and the structure was fully completed by the date of the Loan Agreement effectiveness. Although there were many changes in MOJ personnel during the Project implementation, as well as changes in circumstances (e.g. changes in internal organization of the MOJ), this structure proved to be sufficiently robust and sustainable and in fact it represented a stable environment for Project implementation throughout the Project life cycle. Here a role of the Steering Committee and Steering Committee sessions should be mentioned. During the first phase of Project implementation 3 Steering Committee sessions were organized. These were attended by all stakeholders – i.e. the Minister of Justice, the President of the Supreme Court, the State Attorney General, all Project Team members, as well as representatives 40 of the Bar Association, Association of Public Notaries, Ministry of Finance, faculties of law, numerous invited judges and prosecutors, and representatives of other assistance providers (foreign and local, governmental and non-governmental), plus the media. In short, all three SC sessions turned to be a very high-visibility events. These usually lasted two days, and the Project Team would prepare a range of presentations on various relevant topics and related Project activities. These were usually followed by a lively discussion by the participants and a set of conclusions was drafted from the session. During the later phase of the Project implementation (period 2012 – 2016) only one low-key SC session was held. Monitoring and Evaluation of Outcomes/Results During the PPA implementation and the negotiations on the Loan Agreement considerable thought and effort was invested in designing a meaningful, yet manageable Results Framework, including the project results indicators and the Intermediate Outcome Indicators (of project progress). The MOJ Project Team was to administer information gathering and reporting processes in the Results Framework (in conjunction with routine project reporting) to the Bank and the GOC via the Steering Committee and the Ministry of Finance (MOF) IFI’s Directorate. The results indicators were intended to inform the authorities’ decision-making processes regarding the reform process and resource allocations. Project results and intermediate indicators, including results from user surveys, were to be made public via the project web site and informational events organized by the MOJ Project Team. In parallel with the Results Framework monitoring and evaluation (M&E) process, the project- financed enhancements and generalization of the information systems for the courts and the prosecutors’ offices (the Integrated Case Management System (ICMS) and Case Tracking System (CTS), respectively), were expected to greatly enhance the information base for case processing (e.g., speed, mix, etc.). The authorities, notably the Supreme Court and the MOJ, were to use these continuous streams of detailed operational information to strengthen the (evidence-based) management and reforms of court/prosecutorial operations, as well as corresponding resource allocations. However, during the first half of the Project implementation it became apparent that some of the Indicators were designed somewhat over ambitiously – partly because the IT systems that were to generate necessary data did not reach the expected levels of their sophistication in time. This created considerable problems for the Project Team, where a lot of time and energy was periodically spent on gathering such data from sources that were not always reliable. Also, since the Project was involved in almost all segments of the justice system, and at the same time in many of these areas it represented only a small fraction of overall activities in their reform, it was sometimes difficult to precisely attribute specific results to project activities per se. This led to the revision of the Results Framework within the Project restructuring process that took place in late 2013. Implementation Civil Works - CW-3, SAO Building in Zagreb and CW-4, Justice Building in Pula - Completed As mentioned, the MOJ started implementing the Project immediately after the Loan Agreement effectiveness (July 2010). First procurements for the planned civil works were advertised in October and November 2010, respectively, and contracts for (i) refurbishment of the ex-military building in the center of Pula for county and municipal SAO offices and land registry department 41 of the Municipal Court in Pula (CW-4); and (ii) for extension of the SAO headquarters in Zagreb (CW-3) were signed in March and April 2011, respectively. At the same time services of supervising engineers, as required by the law (CS-15 and CS-16) were successfully procured and contracted for both sites as well. However, the first months at construction sites revealed that architectural designs for these works were far from perfect (this especially for CW-4 Pula), and that contractors had difficulties in proceeding with works based on them. Since the problem could not have been solved through the on-the-site solutions and decisions, the MOJ, in agreement with the World Bank, contracted the same architects on sole-source basis to review and revise the designs. These contracts were negotiated and prepared for signing by late 2011. But in November 2011 parliamentary elections in Croatia were scheduled and eventually led to the change of government, so these contracts were actually signed only in late February 2012. This fact resulted in significant delays in early stages of the works that were never compensated for. Hence both civil works were completed later than anticipated – CW-3 Zagreb in late 2013 (instead of within 18 months from contract signature), and CW-4 Pula in early 2016 (instead within 11 months from contract signature). In spite of that both buildings were completed, furnished and equipped, and occupied prior to the Project end. Notwithstanding these difficulties and delays, the MOJ assesses these activities as successful and as a significant improvement to the state of overall justice sector infrastructure. The MOJ is also expressing its gratitude and appreciation for all the assistance and understanding that the World Bank has provided in addressing these issues, which has enabled these activities to be completed to the satisfaction of all stakeholders involved. - CW-1, Municipal Court Building in Split and CW-2, Judicial Building in Karlovac - Cancelled Learning on its experience with designs for CW-3 and CW-4, the MOJ has at the same time contracted an independent architect to review the designs for CW-1 (Municipal Court Building in Split) and CW-2 (Judicial Building in Karlovac). The review has pointed to the need of revising these architectural designs as well, and such was contracted on SSS basis with architects who originally made them. Based on these changes to bidding documents and designs IFB for CW-1 Split was published in June 2012, and bids were received and evaluated by the end of 2012. IFB for Karlovac was ready to be published in late 2012. However, at that moment the MOJ was forced to cancel these works. Namely, as Croatia was at the time heavily hit by economic and financial crisis, the new government introduced strict stringency measures. Within such circumstances the MOJ budget was considerably cut, and the MOJ had no space in its limited budget to accommodate the costs of these activities. On top of that it was assessed that the building intended for adaptation for the needs of the Municipal Court in Split (originally built for a supermarket) no longer satisfied the space needs of that court, since within the consolidation of the court network the MC Split physically took over judges and staff of three minor local courts that were merged to it and closed down. In other words, the building was not large enough to accommodate the enlarged MC Split. Fortunately, not long after the decision to cancel the CW-1 and CW-2, the MOJ started talks with the Embassy of the Kingdom of Norway on possible assistance to Croatia’s justice system through a bilateral project. These eventually led to a project that, among other, included refurbishing of the court building in Karlovac. All the (revised) design documentation prepared within the JSSP Project was used for tendering the works on the Karlovac court building, and 42 these works were successfully contracted and are ongoing at the time of this Report. Originally the JSSP Procurement Plan envisaged another minor civil work contract – the one on minor adaptation of the existing court building in Pula, after parts of the building would be vacated by the users moving to the building refurbished under the CW-4 (County SAO offices, Municipal SAO offices and Land Registry Department of the Municipal Court in Pula). However, since the new building in Pula received the Use Permit just shortly prior to the JSSP Project closing, there was no time to contract and complete these works (although the IFB was published and bids evaluated). The two buildings refurbished through the Project were also fully equipped and furnished (all furniture, various necessary equipment including archive equipment, AV equipment, security equipment, etc.) through separate contracts financed from the JSSP Loan. Information Technologies In the design of the JSSP Project considerable thought was given to building upon the vast effort that was invested in automatization of Croatia’s justice sector in previous years. Notably, at the time when the JSSP Project was designed, several major IT applications were already developed for the justice sector and these were in various phases of their operation. These included the Integrated Case Management System (ICMS) developed for courts and operating in majority of Croatian courts (however not all of them), the E-Statistics system operating in all municipal and county courts, the Case Tracking System (CTS) for State Attorney’s Offices that was in pilot phase in 4 SAO offices, the JCMS system for Municipal Courts that was in planning stage at the time, and several IT systems developed for the MOJ and used by the MOJ. Based on the development and operation of these systems the MOJ had acquired considerable experience on various issues and problems connected with operation and administration of such complex IT systems, as well as knowledge on what the next steps in their development should be. Another challenging feature of such IT development lied in the fact that majority of these systems were, at one time or another, developed through various projects and with various partners (through the EU pre-accession funds, bilateral projects, MOJ projects), which meant that they were not always fully coordinated and sometimes lacked clear approach to their sustainability. For this reason the IT component within the JSSP Project was designed to assist all stakeholders (primarily the MOJ) in (i) developing the strategy and operation plans for future development and better integration, coordination and management of these systems, including a more intensive training of users; and (ii) continuous improvement of these systems and basic IT infrastructure necessary for their full utilization. However, the Project had mixed success in both components. On one hand ICMS and CTS systems were considerably improved during the Project, through several contracts. Also the Project has enabled replacement of almost all outdated basic IT equipment (PCs, printers, etc.) in judicial bodies that was in some instances so old that the newly developed systems did not work on them, and LAN/WAN networks were built in 21 judicial bodies. These activities can also unequivocally be qualified as very successful and as significant contribution of the JSSP Project to overall improvements in Croatian justice system over the past six years. On the other hand some other important activities planned for strengthening the capacity of the justice system to operate in the new ICT environment did not happen, for various reasons. These activities would include a new, systematic approach to training of users of IT systems in justice sector; better connection of justice systems with the systems/data bases of other governmental 43 institutions (Tax Administration, Pension Insurance, Interior Ministry, etc.); design and procurement of data warehousing system; possibly development of first steps towards electronic filing; strengthening the capacity for absorption of EU funds in IT segment; IT asset management, and similar. Naturally, for each of these efforts an in-depth analysis of the current situation and needs was envisaged. As mentioned, reasons for this were various. The desire to be as prudent as possible within the given financial situation often led the MOJ to prefer the (“free”) EU pre -accession funds and structural funds for financing such activities rather than the proceeds of the Loan (which is to be repaid). As a result several of the above mentioned activities were “removed” from the JSSP Project and applied for EU financing, where time for contracting and implementation tend to be considerably longer. Other reasons for this would include frequent changes in MOJ IT management. Finally, in spite of heavy accent on change management in the JSSP Project design, this element was not fully achieved. For example, majority of IT training envisaged by the JSSP project was intended for employees other than judges (such as typists, registrars, and similar) – since they are by far the largest group using these systems, and are the most frequent users. However, this group is traditionally receiving the least amount of training. Judicial Academy has the mandate to offer trainings for judicial officials (judges and prosecutors), while training of other public servants remains in the domain of the Ministry of Administration. Hence, there was an organizational issue connected with how should such training be organized and conducted. As a conclusion it could be said that the JSSP Project had some major positive effects on further automatization of the Croatia’s justice sector, however, in hindsight, maybe even more could have been achieved. Technical Assistance Activities Activities involving analysis of the current status in various segments of justice sector administration, IT management, financial operations, facilities management, or even court case management and business process, followed by recommendations for their improvement, was heavily accented in the early JSSP Project design. It also must be said that several of these planned activities were envisaged as very novel and advanced approaches to relevant issues if compared to the existing state of how these issues were traditionally viewed in Croatia while some others were aimed at topics that were novel in itself. There were some examples of technical assistance activities planned in the first versions of the JSSP Procurement Plan that turned out to be over ambitious due to the reasons mentioned above. These included: “technical support to strengthen justice sector budgeting, resource management and performance framework including activities to enable the MOJ to develop a sector-wide expenditure and performance framework”, or simply “performance budgeting”, as a concept that was never tried in Croatia’s justice sector before; public-private partnership as a vehicle to fund new investments in judicial infrastructure (specifically for development of the “Justice Square” in Zagreb and possibly other major justice centers); analysis of the system of court fees in Croatian courts – as a means to collect some desired funds (how much?), control the inflow of cases (to what level?), strengthen procedural discipline, and at the same time remove some bottlenecks which the court fee collection system at the time was reportedly creating; performance framework that would be based on empirical data collected through the new IT systems in place; development of a sector-wide, mid-term IT strategy and implementation plan that would clearly determine the achieved level, set forth the desired future state, list all the needs and prepare appropriate projects for various sources of funding. 44 For one, after the lengthy and demanding process of negotiation for EU membership Croatia suffered a clear case of the “reform fatigue”. In combination with the financial crisis and change of government this resulted in noticeable decrease of energy and willingness for new reforms across the public sector in Croatia. In spite of this the MOJ did achieve several breakthroughs in these “soft elements” through the JSSP Project. To name only the most important ones: Financial analysis of options for financing the “Justice Square” in Zagreb was completed. An expert from Poland was engaged in 2011 to assist the MOJ to better prepare for absorption of future EU funds. An “Architectural, Developmental and Investment Strategy for Justice Sector Infrastructure” was drafted. The JSPEIR, or “Justice S ector Public Expenditure and Investment Review” was made by the World Bank with inputs from the MOJ. Several training programs for court and SAO employees were conducted. Statistical and management reporting on justice sector performance was considerably discussed and improved during the period. The Project assisted with introduction of personal bankruptcy into Croatia’s legal system through financing training of various groups participating in the proceedings, and necessary adjustments to the ICMS system. The “Justice Sector IT Strategy and Enterprise Architecture” was prepared based on an extensive analysis of the existing status and future needs. Finally, two User Surveys were made (one at the beginning, one at the end of the Project) where citizens, businesses, professional users and insiders were questioned about their views and opinions on various aspects of operation of the justice system. Although not all of these activities, “strategies” and “reports” happened in the best possible times, or maybe led to dramatic and quick results, they have certainly changed the way how these issues will be approached in the future. Also, these efforts have exposed numerous personnel from the MOJ, courts and SAO offices to these topics and to new concepts, and have altogether certainly widened the institutional knowledge and understanding of the underlying issues. Conclusion As the Borrower, we find that the Project has achieved all of its main objectives and some among them even to the fullest possible extent. The Project was started with very high expectations; it was a cross-sector project touching upon a vast number of segments and sub-segments of the justice system, many of which were not in the focus beforehand. To the extent that any criticism may be voiced, we are aware that it can be addressed to no one but ourselves. However, we also believe that majority of decisions that have possibly affected Project implementation and results – such as cancellation of certain activities and portion of the Loan - were at the same time prudent and financially wise. We are also aware that in some areas and in some instances we still have to work on achieving the capacity required for implementation of such and similar projects in public sector. Change management also remains as something that should be more seriously approached in future endeavors with such implications as this one. Notwithstanding this, we are of an opinion that Croatia’s justice sector – especially the court and prosecution system and the MOJ – is in a much better situation at the end of this Project than it was at the beginning. And in many areas – be they specialized “niches” or overall system performance - these improvements may be directly attributed to the Justice Sector Support Project. At the end we would like to express our sincere gratitude to the World Bank, and to the WB team and team leaders working with us on this Project for all their assistance, professionalism, patience, understanding and flexibility that made implementation of this Project possible through 45 such a long time and all the changes, which eventually enabled these positive results without a single negative tone. 46 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 1. As a Partner in the JSSP Project, the Dutch Government financed an Externally Funded Output (EFO) in the amount of EUR 1 million. As explained above EUR 0.3 million of this budget was allocated through AgentschapNL, to finance the Justice Sector Public Expenditure and Institutional Review 12 which was published in October, 2014. In this context, the ICR mission visited the Embassy of the Netherlands in Zagreb on October 10th, 2016. The World Bank team met with Mr. Marc van der Linden, Deputy Head of Mission and Mr. Robert Csongei, Assistant. Mr. Robert Csongei also attended the JSSP workshop on October 11th, 2016. 2. The Embassy had limited information on JSSP. Exchanges between the borrower, the Bank and the Dutch Government were very limited, consisting primarily of e-mail exchanges and a press release dated October 7th, 2008. 3. The Dutch Government’s commitment to support the Project was mainly due to the fact that the World Bank’s proposal met the Dutch’s priority to help support the pre-accession, reorganization of the judicial sector in Croatia. In 2008, the Embassy offered to provide financial support and suggested to involve Dutch experts with experience in the Eastern bloc. 4. Dutch assistance offered to new EU countries, such as Croatia, is financed through MATRA Funds. These funds are managed by the Dutch Ministry of Economy and dedicated to address EU policy priorities and strengthen the rule of law. Following Croatia’s accession, the Netherlands Government drastically cut its financing to the justice sector and now maintains a very limited mostly observational role. 5. The private business sector is one of the investment areas that seems to be of greater interest to the Netherlands Government. The Embassy highlighted that though progress has been made in reforming the judicial sector, there is still room for improvement in order to ensure an investment friendly environment for foreign companies. 6. The Norwegian government provided financing for designs for the proposed Zagreb Judicial Square but declined to make a statement on this ICR. The Government of Croatia remains in discussion with the Norwegians for possible financing for the implementation of the Judicial Square. 12 See Justice Sector Public Expenditure and Institutional Review, Resourcing the Justice Sector for Efficiency and Performance, World Bank, October, 2014 47 Annex 9. Project timeline: Key events in the implementation 42 Annex 10. List of Supporting Documents  Project Appraisal Document, Report No. 51133-HR dated March 3, 2010  Loan Agreement, Loan Number 7888 HR dated April 13, 2010  Project Information Document No. AB5296, dated March 30, 2010  Restructuring Paper, Report no. RES19188, dated December 18, 2013  Partial Cancelation of Loan Proceeds dated December 24, 2013  Implementation Status and Results Reports No. 1 to 11 dated between January 25, 2011 and June 29, 2016  Croatia Country Assistance Strategy, November 24, 2004  Croatia CPS Progress Report, May 2007  World Bank. Country Partnership Strategy for the Republic of Croatia. August 13, 2008. Report No. 44-879-HR  Aide Memoirs of Bank missions from 2010 to 2016  IPSOS Report on Evaluation on Quality Of Selected Justice Services in the Republic Of Croatia in 2016, dated June 29, 2016  Investment Project Financing Economic Analysis Guidance Note, OPSPQ, October 2014  ICR Guidelines, May, 2011  IEG Guidelines for reviewing World Bank ICRs, November 2013  Final Report on Preliminary Technical Feasibility Studies Pertaining to the Proposed Zagreb Judicial Square (ZJS), March 28, 2012  Analysis of Financing Options for the Development of Zagreb Justice Square – presentation of the draft report, July 4, 2012  Draft Report on Analysis of Financing Options for the Development of “Zagreb Justice Square”, with Recommendations, June 20, 2012  Bankruptcy proceedings: Analysis of responses for the Evaluation questionnaires on training for judges of municipal courts, Zagreb, April 14 and 15, 2016 43 MAP 44