Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) Report Number : ICRR0020649 1. Project Data Project ID Project Name P120321 AZ Capital Markets Modernization Country Practice Area(Lead) Azerbaijan Finance & Markets L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-80240 31-Dec-2015 12,000,000.00 Bank Approval Date Closing Date (Actual) 17-Mar-2011 01-Jul-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 12,000,000.00 0.00 Revised Commitment 11,964,451.00 0.00 Actual 11,964,451.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Nestor Ntungwanayo George T. K. Pitman Christopher David Nelson IEGFP (Unit 3) 2. Project Objectives and Components a. Objectives As stated in the Financial Agreement (FA, p. 6), the project development objective is "to increase the use of equity and corporate debt as financing and/or investment instruments through the adoption of an effective capital markets regulatory framework and infrastructure". b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? --- d. Components The project has four components as delineated below: 1. Streamlining and Automating Market Infrastructure ( Cost at appraisal in the amount of US$6.05 million, with actual cost of US$7.72 million.) Towards promoting compliance with best international practices, this component intended to: (i) increase automation of the market through establishing one depository center and consolidating clearance and settlement; (ii) open up the infrastructure’s membership and access; (iii) minimize off-market trades and improve price discovery; (iv) facilitate the operational introduction of investment funds; and (v) build better risk protection through establishing an updated trade guarantee mechanism. 2. Updating and Strengthening the Legal and Regulatory Framework (Cost at appraisal in the amount of US$1.89 million, with actual cost of US$1.42 million.) Toward strengthening the country's legal and regulatory framework, this component was to: (i) support the modernization of the capital market legislation through a new Capital Market Act and appropriate implementing regulations; (ii) enhance market confidence and safety; (iii) eliminate tax obstacles to capital formation for companies and double taxation, in collaboration with public entities overseeing taxation; (iv) allow pooled investments by insurance companies; (v) foster competitiveness by extending market participation; and (vi) ensure effective coordination with other regulatory authorities in the financial sector. 3. Stimulating Supply (Cost at appraisal in the amount of US$1.86 million, with actual cost of US$1.70 million.) Toward greater use of capital markets as a financing alternative, this component intended to: (i) create a more appropriate environment to encourage equity listings and the corporate bond market; (ii) reach out to corporations to promote benefits of financial market as a funding alternative; (iii) promote development of reliable benchmarks in the government bond market (in cooperation with IMF); (iv) support domestic capital market financing of the large infrastructure products; and (v) assess potential for introduction of new financial products. 4. Capacity Building (Cost at appraisal in the amount of US$3.84 million, with actual cost of US$2.70 million.) This component aimed to: (i) strengthen the State Committee for Securities' (SCS) knowledge management, and its policy formation processes and transparency; and (ii) strengthen the institutional capacity of the SCS’ surveillance, reporting and enforcement systems. The component's main activities were to: (i) improve public awareness of the capital market by carrying out a series of effective communication programs; (ii) increase potential investors’ understanding of capital market; and (iii) promote the Azeri market to international investors. This component intended also to support an increase in the professional knowledge base by: (i) developing a capital markets training center; (ii) gauging the current capacity of the capital markets participants; (iii) establishing curricula for qualification for licenses; and (iv) developing a sustainability plan for Training Center. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) Cost: This was a technical assistance (TA) loan whose actual cost amounted to US$17.36 million, provided by IBRD, a Swiss financing and a borrower contribution, in respective shares of 78.1 percent, 6.7 percent, and 12.7 percent of the total cost. Financing: The project was funded by n IBRD TAL in the amount of US$13.56million, a borrower contribution of US$2.2 million, and a Swiss State Secretariat for Economic affairs (SECO) financing in the amount of US$1.8 million. Borrower Contribution: The planned borrower contribution of US$1.80 million, of which 98% was disbursed. Dates: The project was approved on March 17, 2011, restructured once on May 06, 2015 to extend the closing date, in order to allow full project implementation. The project was closed on July 01, 2016. 3. Relevance of Objectives & Design a. Relevance of Objectives At project appraisal in 2011, the country's economy had largely recovered and stabilized after the 2008 global financial crisis. The Government's focus was on promoting economic diversification and growth in the private sector, and deepening the capital markets was seen as a priority for private companies to secure cheaper long term capital, and get involved in the non-oil economy. The project aimed specifically to strengthen the capital markets through market expansion toward non-bank long-term financing. While the project objective was part of the Government priorities at appraisal, the situation at project closure had changed, as the financial sector weakened during implementation, due to weakening economic activity, distress in the banking sector, currency devaluation and tighter liquidity On the Bank's side, modernization of the financial markets in Azerbaijan was not at the forefront of its priorities. The Bank's Country Partnership Strategy (CPS) at appraisal had a pillar aimed at improving the environment for private sector-led growth, with activities to support the expansion of business activity and increasing economic opportunity, but financial market development was not a focus area. The FY12-15 CPS also had a pillar supporting economic growth by tackling the bottlenecks to competitiveness and faster productivity growth, and activities to enhance access to finance, but with no specific emphasis on financial market development. In conclusion, given that the project objective was not consistent with the Bank's strategies, and that it was no longer part of the Government's priorities during the period preceding the project closure, relevance of objectives is rated as modest. Rating Modest b. Relevance of Design The project objective was clear and specific, and aimed to increase the use of equity and corporate debt as financing and/or investment instruments. There was a logical link between the identified activities, the Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) expected outputs and outcomes, and the targeted objective. Project's components fed logically into building up the required infrastructure and the legal and regulatory framework needed to underpin an improved financial market sector. For instance, effectiveness of the use of equity and corporate debt was to be demonstrated through increased issuance of bonds, and an increase in the cumulative number of companies with listed equity. The project design emphasized institutional strengthening and capacity building, and the reform of the regulatory regime and streamlining market infrastructure, including a security market law consistent with international standards, and the directives of the EU in particular. The project result framework and monitoring in Annex 1 of the PAD provided an overview of how identified components intended to generate the intermediate and final results needed to achieve the project objective. The relevance of design is rated as high. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective "To increase the use of equity and corporate debt as financing and/or investment instruments through the adoption of an effective capital markets regulatory framework and infrastructure". Rationale Outputs: (i) Toward streamlining and automating financial market Infrastructure, the National Depository Centre (NDC) was consolidated and made the sole Central Securities Depository (CSD) for government and corporate securities, and provider of clearing and settlement functions to members. The NDC was connected to the country’s Real Time Gross Settlement (RTGS) systems, ensuring Delivery Versus Payment (DVP) principles of both government and corporate securities. The NDC's methodology was harmonized with internationally recognized ISIN standards, and it became a member of the Association of National Numbering Agencies. 14 NDC rules and post-trading system rules were adopted in accordance with the new “On Securities Markets” Law. (ii) A new “On Securities Markets” Law was adopted in 2015, and was assessed by experts as a major effort to improve the legal and operational framework of the securities market. The Law provisions observe the Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) three core objectives of securities regulation, including (a) the protection of investors, (b) ensuring that markets are fair, efficient and transparent, and (c) the reduction of systemic risk. A number of 30 implementing regulations were adopted subsequent to the “On Securities Market” Law. (iii) The modernization of the tax legislation in respect of the securities market operations allowed the Parliament to adopt in January 2016 an amendment to the taxation code withdrawing profits tax on dividends and coupon payments for the next 3 fiscal years. (iv) After the adoption of the “On Securities Market Law”, the listing rules and regulations were updated to comply with the provisions of the new Law. In line with the new Listing regulations, key developments included the separation of bond and stock market, and the creation of market segments, with minimum capital requirements tailored to market segments and compulsory free float rates. After the approval of new Listing regulations, the transition from a 2-tiers listing structure to a 3-tiers listing structure was realized. This development made the ground for the companies to be rearranged to these market segments according to their financial and other indicators. Other main contributions of the new Listing regulations included the minimum capital requirements of the companies. and the creation of free-float rates required to be listed in each tier. (v) A Listing Advisory Program (LAP) was launched in April 2014 and implemented by the Baku Stock Exchange (BSE), and aimed to build on success in making a “highly effective securities market” in order to attract financial resources to the real sector, move capital between sectors and increase effectiveness of the economic system. (vi) The BSE totally revised its operational rules and procedures, and new BSE rules and procedures were harmonized with new technological trading and post-trading platform (the Capital Markets Information System) launched early 2016 as well. Nine new operational trading rules and regulations were implemented in early 2016 by BSE. (vii) The State Committee for Securities (SCS) adopted an action plan on Capital Market Related Nationwide Education Program and created a Financial Laboratory fully equipped for seminar workshops and handed over to the University by BSE. The SCS was later on dissolved and replaced by the Financial Market Supervision Authority (FMSA), an agency that currently has a regulatory and supervisory responsibility over the financial and banking sector. (viii) A Capital market Training Center (CMTC) was launched in March 2014, and a strategic and marketing plan was elaborated, targeting: (a) capacity building for the SCS, BSE and NDC personnel, and (b) capital markets based training for wider professional audience including attestation exam preparations for market participants. Outcomes: (i) The target for increased issuance of corporate bonds was fully achieved. The increase rose steadily to reach a high of 24% in 2015, but decreased down to 14% as of May 2016, against a target of 3.8%, and a baseline in 2010 was 1.9%. In retrospect, this target might have been set at a less ambitiuos level, given that the result of 14% was achieved, even under challenging economic conditions. (ii) The target of increased number of companies with listed equity was partially achieved, as only five were listed at project closure, against the original target range of 8-10 companies with listed equity. Price volatility, lack of market confidence and tightening liquidity contributed to a stagnation in the equity markets. However, there was a 100% ratio of market participants electronically linked to market infrastructure (trading, clearing and settlements). (iii) The target of improved price transparency defined as the ratio of competitive transactions to total equity was exceeded, the ratio of competitive transactions reaching 100 percent, against a target of 90 percent. Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) “Competitive transactions” are non-block transactions as block transactions are not considered competitive and transparent. However, the BSE has taken measures that are designed to make block (or cross trades) more competitive by allowing a 60 minute price discovery period to allow counter-bid/offers and prepay requirements have been eliminated for purchase orders. Rating High PHREVDELTBL PHREVISEDTBL 5. Efficiency This was a technical assistance project, and there was no economic or financial analysis conducted either at appraisal or at project closure. The ICR indicated that the markets were too illiquid at closure to perform an economic analysis to assess the cost savings accruing to issuers of corporate debt or equities in comparison to the alternative of short term bank financing, and the review is in agreement with this assessement. Administrative and operational performance of the project implementation was overall satisfactory. The project helped facilitate the modernization of the debt and equity trading platform for the BSE and inform the creation of the Financial Markets Supervision Authority (FIMSA) and subsequent liquidation of the SCS. However delays were experienced in the early stages of the project due to unanticipated complexities surrounding the TOR, tendering and ultimate installation and testing of the CMIS system. The successful completion of the later tasks necessitated a six month extension of the project closing date. Aside from this particular delay, the amount of progress made was significant, including the new “On Securities Markets” Law, new NDC and post-trading system rules and tailored implementing regulations were adopted in support of the new securities law. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  0 Not Applicable 0 ICR Estimate  0 Not Applicable Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of project objectives was modest, but that of design was high. Efficacy was high, as most key expected results were achieved, including an automated financial market Infrastructure together with an improved legal and operational framework, an increase in the issuance of corporate bonds and the number of companies with listed equity, and an improved price transparency. Efficiency was substantial. While other Bank projects were instrumental in preparing a strategy that supported reform, supported by other donors independently, the above outcome was derived prominently from the activities supported by this project. a. Outcome Rating Satisfactory 7. Rationale for Risk to Development Outcome Rating In the short term, the key risk to project outcomes is the unstable banking and financial sector. The ICR noted that currency devaluation, lack of liquidity in the financial markets and the draining of deposit accounts in retail banks have affected people's confidence in the banking system (ICR p.25). The lack of regulatory transparency and uncertainty in the markets are major factors contributing to the fragility of the financial sector, and has led to the insolvency of large banks, thereby eroding confidence in the banking sector. While acquired financial market infrastructure and modernized legal and operational framework are not at risk of being reversed, financial market performance will be proportional to the health and strength of the country's financial sector, which was under stress when the project closed. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance a. Quality-at-Entry Overall, quality at entry was moderately satisfactory, as good project preparation was offset by weaknesses in the forward-looking analysis of developments in the macroeconomic environment and the financial sector. The PAD provided a comprehensive overview of the country, sector, and institutional context, and the project design relied on Bank's ground experience and existing projects, and on the synergies with activities supported by other donors. However, the thrust of the project and identified activities were not at the center of the Bank's country partnership strategies, and the Bank team could not identify potential macroeconomic Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) risks that could have an adverse effect on the financial sector and the overall success of the project. The project design, as well as implementation and M&E arrangements were appropriate and served well the project delivery. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank's team closely monitored the project with a minimum of two supervision missions per year for a total of ten missions which produced comprehensive aide-memoires, and detailed ISRs. The Bank team was responsive to client needs through active communication between the Bank's Task Team Leaders (TTLs), the PIU and implementing agencies. However, project implementation took place in a very challenging macroeconomic environment with a sharp reduction in economic activity, tightened liquidity and significant distress in the banking sector. The above developments created an acute need for financial market stability and management of the banking sector, and cast doubt on the sustainability of the project outcomes. There were delays early in the preparation stage, which affected the procurement process for the capital markets information system (CMIS), and the launch of the national public awareness program for capital markets, leading to a six month extension of the closing date, allowing the complete implementation of project components. Most key outcomes were achieved despite the unforeseen macroeconomic challenges that undermined the banking and financial sector. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance The Government, though its Ministry of Finance, demonstrated ownership and commitment during project preparation and implementation, by using a PIU that oversaw project coordination, procurement, financial management and M&E. Project effectiveness was delayed as a result of a lengthy approval process and the time needed to establish and staff the PIU, and to warrant the needed coordination with multiple Government's entities, including the Ministry of Education, the Ministry of Justice, and the Cabinet of Ministers. During project implementation, the Government worked in collaboration with the Bank's team and ensured coordination with other donors supporting the State Program on the development of the securities markets. Government Performance Rating Satisfactory Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) b. Implementing Agency Performance The primary implementing agency for the project was the country's State Committee for Securities (SCS), and the establishment of the PIU proved more problematic than initially anticipated. The early setbacks experienced during the project's launch (identified earlier under Section 8(b)) led to an extension of the closing date by an additional six months. Despite early implementation delays and unforeseen macroeconomic conditions, the PIU remained fully committed to the project and achieved most deliverables. The Borrower remained committed to completing the reforms and structural changes, including the creation of the integrated Financial Markets Supervision Authority (FIMSA) with a larger mandate, covering notably regulatory and supervisory responsibility over banks, non-bank credit institutions, capital market, and payment systems. Implementing Agency Performance Rating Satisfactory Overall Borrower Performance Rating Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The Annex 1 of the PAD provided a comprehensive table summarizing the project's results framework and monitoring, and comprised three PDO indicators and six intermediate results indicators. All indicators had baselines and end-targets, including annual targets. Indicators were overall relevant and realistic as they aimed to report progress on the key outputs and the pursued outcomes. Despite new developments in the economic setting, indicators were not changed, because expected outputs were already achieved, and impact on the outcomes indicators was minimal. b. M&E Implementation The PIU, initially located within the State Committee for Securities (SCS), was specifically responsible for project monitoring and evaluation (M&E), with close collaboration with the Government and the Bank. The Internal Audit Department of the SCS was specifically given the responsibility of M&E tracking and reporting, and to ensure that the implementation units were integrated into the management information results aggregation process. Reporting was provided on a regular and timely basis. The project was also closely monitored by the World Bank. Progress outputs and outcomes was reported through aide-memoires prepared after every mission, and recorded in the ISRs. The Mid-Term Review (MTR) provided an update of the results matrix, including the needed adjustments in order to achieved the expected results. c. M&E Utilization Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) M&E reports shared among project stakeholders through Aide-memoires and ISRs led to the decision of a six-month extension of the project, in order to make sure that all planned outputs and outcomes are achieved. M&E Quality Rating Substantial 11. Other Issues a. Safeguards The project was classified as Category C, and no Environmental Assessment was required. b. Fiduciary Compliance Financial management: Accounting records were consistently found to be accurate and with adequate controls to ensure accuracy of accounting records and reports, and regular audits were conducted by a reputable firm. The PIU provided timely reports on the project financial management and all annual audit opinions were unqualified. Procurement: The ICR did not report on the procurement performance of the project, but it was indicated that procurement specialists were present in the region to provide the needed support to keep the project implementation on track. Additional information from the TTL indicated that delay with procurement was primarily caused by the complexity of the procured IT/MIS systems, which resulted in significantly more time to analyze best practices and develop the respective technical specifications for a large value procurement. c. Unintended impacts (Positive or Negative) The ICR did not report any. d. Other The ICR did not report any. 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) Outcome Satisfactory Satisfactory --- Risk to Development Substantial Substantial --- Outcome Moderately Moderately Bank Performance --- Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The lessons learned put forward in the ICR are common and reflect the experience of the project design and implementation. Key lessons underscore: (i) the need to factor in the project design macroeconomic risks for economies relying on on a particular commodity or sector, (ii) the importance of having a PIU with an internal capacity to manage all aspects of the project, (iii) the determining factor of a committed Government for a successful project, and (iv) the difficult task of measuring the reforms supported by a TA project that take longer to be completed. This review is in agreement with the above lessons, and adds also that attribution of outcomes remains a major challenge when many other donors were supporting the same category of reforms. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR is comprehensive, concise, and to the point in presenting the context, the design and the implementation of the project. The ICR's focus was on the outcomes of the project, but the main report gives insufficient attention to the project outputs, without which the outcomes could not be achieved. The Borrower's ICR was sufficiently detailed on the generated outputs. The lessons learned are sound and reflect the story of the project design and implementation. Areas of the ICR that could be strengthened are (i) increased attention needed to be given to the outputs in the main text of the ICR, notably in updating the Annex 2, (ii) the outcomes attribution issue should have been discussed, given the large number of donors involved in the development of the financial markets reform agenda, ahead of and in parallel to the project implementation and, (iii) a systematic assessment of the procurement implementation under the fiduciary heading. Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AZ Capital Markets Modernization (P120321) a. Quality of ICR Rating Substantial Page 12 of 12