84050 World Bank Trust Fund for Statistical Capacity Building Evaluation Report Hallgrímur Snorrason Andrew J. Flatt Jette Jensen January 2010 1 Table of contents List of abbreviations .......................................................................................... 3 I. Executive summary .................................................................................... 5 II. Introduction ................................................................................................ 9 Background ................................................................................................ 9 The evaluation ............................................................................................ 10 III. Methodology and conduct of the evaluation ................................................ 11 IV. Main findings ............................................................................................. 14 Distribution of grants .................................................................................. 14 Relevance of the TFSCB grants .................................................................. 17 Experience of TFSCB projects – outcomes and impact ............................... 19 NSDS projects ............................................................................................ 19 SCB (non-NSDS) projects .......................................................................... 22 Regional projects ........................................................................................ 23 Global projects ........................................................................................... 23 Project management and implementation .................................................... 24 TFSCB management and governance ......................................................... 27 Overall usefulness of the TFSCB ................................................................ 30 V. Conclusions and recommendations ............................................................. 32 Conclusions ................................................................................................ 32 Recommendations ...................................................................................... 34 Annex 1 Survey among TFSCB stakeholders ................................................... 37 Annex 2 Reports on missions undertaken by the Evaluation Team ................... 51 Annex 3 List of persons met ............................................................................ 75 Annex 4 Project cycle ...................................................................................... 82 Annex 5 Financial status of the TFSCB ........................................................... 83 Annex 6 Terms of Reference for an Evaluation of the TFSCB ......................... 85 Acknowledgements The Evaluation Team would like to express its sincere thanks to all the persons with whom it met and discussed during its work, for their generous provision of assistance, friendly and professional cooperation, and enthusiasm for statistical capacity building in developing countries. Special thanks go to World Bank staff in Washington and in the countries visited who assisted the Team by helping to plan and conduct its missions and by providing documentation and information. The Team would also like to express its gratitude to all the persons who provided invaluable inputs into the evaluation by participating in the survey among TFSCB stakeholders. 2 List of abbreviations AfDB African Development Bank AFR Sub-Saharan Africa (World Bank region) AFRISTAT Observatoire économique et statistique d‟Afrique subsaharienne ANSD Agence Nationale de la Statistique et de la Démographie (Senegal) BCP Banco Central del Paraguay BPS Badan Pusat Statistik (Statistics Indonesia) CG Consultative Group DECDG Development Data Group DFID Department for International Development (United Kingdom) DGEEC Dirección General de Estadística, Encuestas y Censos (Paraguay) DGSEE Direction Générale des Statistiques et des Études Économiques (Gabon) ECA Europe and Central Asia (World Bank region) ECLAC Economic Commission for Latin America and the Caribbean EU European Union Eurostat Statistical Office of the European Communities FAO Food and Agriculture Organization of the United Nations GDDS General Data Dissemination System GFR Grant Funding Request GIS geographic information system GRM Grant Reporting and Monitoring System GSS Ghana Statistical Service HIES Household Income and Expenditure Survey IA implementing authority IADB Inter-American Development Bank IBRD International Bank for Reconstruction and Development IDA International Development Association IMC Internal Management Committee INE Instituto Nacional de Estadística (Uruguay) IRIS Integrated Records and Information System ISI International Statistical Institute JICA Japan International Cooperation Agency LAC Latin America and the Caribbean (World Bank region) LSMS Living Standards Measurement Study MAPS Marrakech Action Plan for Statistics MDG Millennium Development Goals MECOVI Mejoramiento de las Encuestas de las Condiciones de Vida MENA Middle East and North Africa (World Bank region) MONSTAT Strengthening the National Statistical System of Mongolia Project NBS National Bureau of Statistics (Nigeria) NIS National Institute of Statistics (Cambodia) NSDS national strategy for the development of statistics NSO national statistical office NSS national statistical system OECD Organisation for Economic Co-operation and Development PARIS21 Partnership in Statistics for Development in the 21st Century PCBS Palestinian Central Bureau of Statistics PRSP Poverty Reduction Strategy Paper 3 Renstra Rencana strategis (Strategic plan, Indonesia) SADC Southern African Development Community SCB statistical capacity building SDMX Statistical Data and Metadata Exchange SMP statistical master plan SRF Statistics for Results Facility STATCAP World Bank lending program for investment in statistical capacity TF Trust Fund TFAU Trust Fund Administration Unit TFSCB Trust Fund for Statistical Capacity Building ToR Terms of Reference TTL Task Team Leader UBOS Uganda Bureau of Statistics UN United Nations UNDP United Nations Development Programme UNECA United Nations Economic Commission for Africa UNECE United Nations Economic Commission for Europe UNFPA United Nations Population Fund UNICEF United Nations Children‟s Fund UNSC United Nations Statistical Commission WB World Bank WBI World Bank Institute 4 I. Executive summary I.1. The World Bank and the Consultative Group of the Trust Fund for Statistical Capacity Building (TFSCB) decided to organize an independent review of the operation and outcomes of the TFSCB in the course of 2009. The evaluation was conducted between June and October 2009. This report presents the findings and recommendations of the evaluation. I.2. The TFSCB was established in 1999 by the World Bank in order to build up and strengthen the capacity of statistical systems of developing countries. It has been closely coordinated with the work of the Partnership in Statistics for Development in the 21st Century (PARIS21). Since the adoption of the Marrakech Action Plan for Statistics (MAPS) in 2004, it provides the context for the World Bank‟s support to statistical systems in developing countries and related activities until the end of 2010. The TFSCB is funded by donors and makes grants for projects aimed at capacity building in official statistics. As of June 2009, the TFSCB had received US$40 million and committed US$31 million to specific projects. The current tranche, TFSCB III, is scheduled to close by the end of 2012. The TFSCB is governed by a Consultative Group, and the Development Data Group of the World Bank manages the approval and administration of grants according to well established procedures. I.3. The MAPS underscored the policy of laying the groundwork for statistical capacity building (SCB) in each developing country by designing a national strategy for the development of statistics (NSDS). Following the launch of MAPS, the TFSCB was used increasingly for this purpose. Two financing windows were introduced at this stage: an NSDS window and a more general SCB window for various development projects. From the beginning of its operation in November 2000 until mid-2009 the TFSCB gave grants to 149 projects. Of these, 101 went to national authorities in 73 countries, 29 were for funding regional projects, and the remaining 19 financed global projects. Of the grants to national authorities, just under half were for drawing up national strategies while the others were for various capacity building projects. I.4. Sub-Saharan Africa has been the largest recipient of the TFSCB grants. For the operational period of the TFSCB as a whole Africa has received 33% of the total disbursement of US$30.6 million, with Asian countries receiving 20%, Latin America and the Caribbean 16%, Europe and Central Asia 12%, and countries in the Middle East and North Africa region 5%. I.5. The TFSCB funding of NSDSs and other national capacity building projects has been highly relevant for the developing countries. They have been encouraged to establish national strategies or master plans for setting out the paths and priorities for their statistical capacity building. National authorities have also been encouraged to link their applications for the general SCB grants to specific actions or parts of their national strategies or master plans. This policy seems to have been fully accepted by the recipient countries. The emphasis placed on laying the groundwork by designing a national strategy or a statistical master plan (SMP) with a concomitant action plan to guide the development of statistical capacities seems also to have been fully embraced by the developing countries. Furthermore, the countries have been free to set their 5 own priorities and acquire grants on the basis of these priorities or the particular path which each country has paved out for itself. I.6. Evidence gathered in the course of the evaluation suggests that the NSDS projects have generally succeeded in achieving their planned outcomes. Thus, the countries have been largely successful in planning and designing an NSDS or SMP. The work has more often than not been carried out in cooperation with several partners within the statistical or administrative systems of the countries. The level of technical assistance or consultative work involved is generally thought to have ranged from satisfactory to excellent. Finally, the work has in most cases resulted in a proper strategy document outlining priorities and specifying a plan of action. The national statistical offices (NSOs) in the countries in question have assumed ownership of the strategies and plans, sometimes in cooperation with other government institutions, and have almost invariably expressed satisfaction with the NSDS. I.7. There are some, albeit very few, cases where the countries feel that the NSDS work has been less successful than anticipated. The extent to which the national statistical authorities have been successful in involving other partners within the statistical system varies considerably. In some countries this has been a proper cooperative exercise involving all the main statistical producers and even users throughout the project. In other cases, the participation has been more limited. I.8. Implementation of the strategies and action plans seems to give the largest cause for concern. Asked in the Team‟s survey about the stage of implementation of the NSDS, national authorities gave widely differing answers; in some cases implementation was ongoing while in others it was at a standstill or hampered by lack of funds. This reflects the fact that statistical needs have to compete for finance with other urgent needs under the very severe budget constraints of the developing countries. Asked about the major factors affecting implementation, national agencies most commonly cited insufficient funding. I.9. Regarding the impact of the non-NSDS grants, some of the early ones were connected to larger projects funded by other sources, such as poverty measurement projects and PRSP (Poverty Reduction Strategy Paper) programs. There is evidence that such supporting funding has provided good results. More recently, several projects focused on solving specific needs, for example in national accounts, price statistics and survey methodology, are reported to have been very successful. Linking non-NSDS projects to national strategies, master plans or action plans based on these seems to have worked well. It seems very likely that the grants given in addition to or alongside the NSDS grants may contribute substantially to the sustainability of the results achieved through the NSDS work. I.10. The evaluation revealed both limited and good outcomes of regional projects funded by TFSCB grants. The limited success has been experienced for instance in projects on sectoral statistics where low development levels of one or more of the countries, or political instability, prevented the countries in question from benefitting from the project. On the other hand, success has been reported for outcomes of regional projects of a more general capacity building nature, training in particular, which have enjoyed effective leadership and organization by the agencies which implemented the projects. 6 I.11. All the stakeholders whom the Evaluation Team either met or heard from through its questionnaires expressed satisfaction with the TFSCB scheme. Although small, the grants are considered very useful. The national strategies or master plans are very effective starting points for the statistical capacity building effort, both for mobilizing concerted action within the government administration to generate and make use of statistics and for facilitating cooperation with donors. The NSDS work is seen as being very important for raising the awareness of statistics and the need for a properly functioning national statistical system (NSS). At the same time it underscores the role of the government body specialized in statistics, the NSO, for coordinating the statistical capacity building process. I.12. One of the underlying assumptions of the TFSCB is that the financing of NSDSs and the linked projects will help the developing countries to acquire donor funding for a more comprehensive statistical capacity building effort, as well as facilitating the use of development loans for that purpose. Many stakeholders have contended that having an NSDS has proved decisive in raising new or additional donor funding. Furthermore, there is good evidence that donors are ready to accept the prioritization and plans of the NSDSs and to coordinate their assistance accordingly. I.13. Although the countries fully accept the importance of developing their official statistics, many of them balk at accepting loans for that purpose in view of their precarious economic and budgetary situations. This lessens the chances for the smooth or continuous progress of statistical capacity building based on the NSDS and for the sustainability of the actions put in place. Hence, there has been – and still is –a need for some rethinking and reorientation in this respect, in particular involving increased emphasis on assisting the implementation of the SCB plans of the countries. I.14. The World Bank has responded to this by designing a so-called Statistics for Results Facility (SRF) and a supporting SRF Catalytic Fund, in particular aimed at extending large grants to countries for accelerating statistical capacity building. The development of the SRF – still in its early stages – could be an important step forward. However, as the future relationship between the SRF and the TFSCB is still unclear and the aims of the two schemes differ considerably, replacing the TFSCB with the SRF does not seem a realistic option at the present time. Thus, while the SRF could be invaluable by financing large projects in a few countries at any one time, there will still be a need to provide small grants to many countries for statistical capacity building purposes. I.15. Notwithstanding the creation of the SRF the Evaluation Team finds that some reorientation of the TFSCB is called for. That should in particular recognize the present need to provide a better bridge between the development of national strategies and their implementation. I.16. Most of the TFSCB-funded projects seem to have been carried out in a timely manner within the foreseen budget and using inputs of high professional standard. The formulation and approval of projects work well following well-based established procedures, helped by recent World Bank grant requesting systems. Two factors beyond the control of the World Bank administrators delay and hamper smooth project implementation: the time required in some countries to formally approve an 7 externally funded project, and the difficulty faced by many implementing agencies, especially in Africa, in applying the complex World Bank procurement and disbursement rules. Monitoring of project progress is the responsibility of Task Team Leaders (TTLs) using a central computerized system which – at present – is not very suitable for management reporting. The TFSCB Administration Unit may give additional guidance to TTLs to promote uniform, timely and results-oriented progress reports. In the absence of a substantial completion report it is recommended that a short terminal report be introduced highlighting tangible results and lessons learned. I.17. The Team makes several recommendations on the future operation of the TFSCB:  First, it recommends that donor funding continue to be mobilized for the operations of the Trust Fund.  Second, it is recommended that the Consultative Group of TFSCB address the issue of financing the implementation of NSDSs and the action plans based on them with the aim of securing the gains achieved through the formulation of the national strategies.  Third, it is recommended that not less than 60% of the TFSCB‟s resources are allocated to the NSDS process including the formulation of NSDSs, their updating and particularly the financing of their implementation.  Fourth, it is recommended that the maximum amount of the SCB grants and their duration be reconsidered, not only to adjust for inflation but more particularly because more substantial grants are likely to be needed to start the implementation of NSDS action plans.  The Team makes several recommendations on project management and reporting, in particular on promoting progress reports and completion and evaluation reporting for the benefit of future projects, on facilitating the execution of TFSCB projects by UN agencies, and on the TFSCB becoming more proactive in disseminating information about its key features. 8 II. Introduction Background II.1 The Consultative Group (CG) of the Trust Fund for Statistical Capacity Building (TFSCB) decided in early 2009 that an independent evaluation of the TFSCB should be undertaken in the course of the year. The Steering Committee of the Partnership in Statistics for Development in the 21st Century – PARIS21 – decided similarly that the workings of PARIS21 should be subject to evaluation in 2009. The aim of undertaking the evaluations was to review the progress so far and to help to develop a coordinated strategy for the period 2010-2015. This report covers the evaluation of the TFSCB. II.2 The TFSCB was set up in November 2000 by the Development Data Group (DECDG) of the World Bank in order to build up and strengthen the capacity of statistical systems of developing countries. It has been closely coordinated with the work of PARIS21 and is part of a worldwide effort to reduce poverty by strengthening the evidence base for decision making at all levels. Since the adoption of the Marrakech Action Plan for Statistics (MAPS) in 2004, it provides the context for the World Bank‟s support to statistical systems in developing countries and related activities until the end of 2010. II.3 The TFSCB makes grants for projects aimed at capacity building in official statistics. The grants are rather small, a maximum of US$400,000, and can be said to be of four types. First, there are grants extended to statistical authorities of developing countries for financing the preparation of a national strategy for the development of statistics (NSDS). Second, there are grants to the same statistical authorities for other capacity building projects closely linked to the NSDS or its implementation. Third, the grants may also be extended to international organizations, regional commissions or other institutions for specific capacity building projects in a group of countries. Fourth, some grants have been of a global character, i.e., designed to fit a specific purpose applicable to many countries rather than directed towards particular countries or regions. As of June 2009 the TFSCB had awarded 149 grants to 73 countries and some 20 regional and international organizations. II.4 To date there have been three consecutive tranches of the TFSCB (referred to in this report as TFSCB I, II and III). As of June 2009, it had received US$40 million from donors and committed US$31 million to specific projects. The current tranche, TFSCB III, is scheduled to close by the end of 2012. The TFSCB is governed by a Consultative Group and the DECDG manages the approval and administration of grants according to well established procedures, with advice from an external Advisory Panel. II.5 By now, with the support of TFSCB, PARIS21 or other means, many countries have an NSDS in place and some have embarked on their implementation. A great deal of effort has gone into widening the understanding in developing countries and among the donor community of the important role played by statistics, and new support instruments are being launched. Against this background it is an opportune moment to take stock of the TFSCB‟s achievements so far and consider its future role. 9 The Evaluation II.6 This evaluation follows a number of other studies which have reviewed the TFSCB. In 2003 there was an independent evaluation of the TFSCB, undertaken in conjunction with an evaluation of PARIS21 earlier that year. This evaluation covered topics such as the relevance of the Trust Fund‟s projects to capacity building needs and the emerging procedures of the Fund, but given its timing could only reach very tentative assessments of the impact of the TFSCB‟s portfolio or the sustainability of individual projects. In 2005/06 an internal World Bank review of the TFSCB was conducted by its Independent Evaluation Group, covering program performance and Bank performance including governance issues. A third evaluation was conducted by the Department for International Development (DFID) in 2007/08 of its own support to the TFSCB, primarily to meet internal DFID reporting requirements; it was designed as a „light -touch‟ exercise and focused on strategic issues and mana gement and funding mechanisms. In 2008 the World Bank commissioned an independent evaluation of the Marrakech Action Plan for Statistics, which touched on TFSCB support to national strategies for the development of statistics. The present study is thus the first comprehensive external evaluation of the TFSCB since 2003, with the opportunity to review the impact of its interventions over a sustained period. II.7 The framework and the modalities for this evaluation of the TFSCB are set out in specific Terms of Reference (ToR) which are reproduced in Annex 6. The purpose of the evaluation is the following:  To assess the impact of the TFSCB in supporting the preparation of national strategies for the development of statistics and in helping to strengthen the capacity of national statistical systems;  To identify what changes might be needed in the design of the TFSCB or in the ways in which it operates to improve its effectiveness and impact in the future; and  In the light of this evaluation, the MAPS evaluation and the related evaluation of PARIS21, to recommend what changes might be needed to enable the TFSCB to support not only the preparation of strategies, but also their implementation. II.8 The scope of the evaluation covers a wide range of issues. The ToR list a number of topics as a guide for the evaluation, while other issues may also be considered. The main points are the following: Coverage: The ToR mention the number of projects by country, topic and other characteristics and the extent to which coverage has reflected TFSCB guidelines. Implementation: Attention should be given to the length of time for projects to be implemented and for activities to be completed as compared with the original timetables, problems encountered during implementation, and plans for budgets and activities compared with actual outcomes. Achievements and impact: The evaluation is expected to focus on the proposed projects and achieved outputs, the impact of a sample of projects, project ratings and evaluations. Follow-up and sustainability: The evaluation is asked to look specifically into the follow up of projects and sustainability of outputs. 10 Process: The processes associated with the TFSCB grants shall be considered, in particular with respect to accessibility of the Fund, problems encountered in getting grants approved and activities started, difficulties in dealing with the Bank‟s procedures concerning procurement, financial management and disbursement, the extent to which agencies may have been discouraged from accessing the TFSCB, and the main constraints they faced in that regard. II.9 Apart from these guidelines the Evaluation Team has also conducted its work with reference to the logical framework table accompanying the ToR, in particular the main assumptions set out there for the operations of the TFSCB. These are the following: 1. As regards purpose:  That the NSDSs lead to an improvement in capacity and output and national policy makers use better information to develop and implement pro-poor policies.  That national governments provide acceptable levels of resources to sustain improvements. 2. As regards outputs:  Projects are well designed and properly implemented and agreed outputs are delivered on time and within budget.  Countries are aware of and make use of the TFSCB.  NSDSs lead to funded implementation programs. 3. As regards activities:  Donors are willing to make regular contributions.  The Bank has sufficient capacity to provide appropriate technical assistance as required.  The Bank is able to provide adequate project supervision and monitoring. II.10 To undertake the evaluation of the TFSCB the World Bank hired a team of three experts, Mr Hallgrímur Snorrason, Team Leader and Statistical Manager, Mr Andrew J. Flatt, Statistician and Evaluation Specialist and Ms Jette Jensen, Project Management Specialist. The Evaluation Team convened and prepared its program in Washington D.C. at the end of June 2009. The Team planned a survey among stakeholders which was sent out in July, carried out its missions in August and September and, after incorporating comments from various stakeholders, finalized its evaluation report in January 2010. Apart from this evaluation report, the Team delivered an inception report as required by the ToR and detailed reports from each of its missions. III. Methodology and conduct of the evaluation III.1 The Terms of Reference for the evaluation (see Annex 6) sketched out the broad methods by which the Evaluation Team should proceed in its work. Within that framework, the Team decided on the detailed methodologies to be employed. These included desk studies of documentation on the TFSCB and its projects, the administration of questionnaires to different groups of stakeholders, field visits to a number of countries with TFSCB projects, and interviews with a range of individuals involved with the TFSCB in one capacity or another. 11 III.2 The Team had available for review English language documentation on the TFSCB website, including the brochure for the Fund, material on the Fund‟s guidelines and procedures, semi-annual progress reports, and reports of the meetings of the Fund‟s Consultative Group and Advisory Panel. Also available on the website is the text of the previous (2003) external evaluation of the TFSCB.1 The Team was given access to other assessments which are not in the public domain, including an internal review of the Trust Fund by the Bank‟s Independent Evaluation Group in 2005/06 and an evaluation of DFID support to the Trust Fund in early 2008. In addition, the independent evaluation of the Marrakech Action Plan for Statistics in 2008, which covers support by the TFSCB for national strategies for the development of statistics, was made available to the Team. III.3 The Team initially assembled in Washington from 25 June to 1 July 2009, a little later than originally planned. During this period the Team studied background material as well as the operational machinery and workings of the TFSCB, and met with its Manager and Administration Unit, members of its Internal Management Committee (IMC), some of the Task Team Leaders (TTLs) of TFSCB projects, and one member of the Advisory Panel. An inception report was delivered at the end of this initial visit, covering inter alia the collection of information via questionnaires, the program of field visits, and an outline of the final evaluation report. III.4 In order to gather information and views from the TFSCB‟s main stakeholders, the Team designed four questionnaires (see Annex 1) to be sent to the implementing authorities of Trust Fund projects (primarily national statistical offices), to Task Team Leaders of projects, to members of the Internal Management Committee, and to Trust Fund donors and other Consultative Group members. No sampling was involved – all agencies and individuals in the respective groups were to be contacted. Arrangements for the handling of the questionnaires, including their translation into French and Spanish, the establishment of a dedicated e-mail account, and the timelines for the despatch of questionnaires and the receipt of responses, were discussed with the Manager and Administration Unit of the Trust Fund. III.5 Some delays were experienced with the transmission of the questionnaires, and in some cases it was discovered that implementing agency contact persons and also TTLs on the TFSCB‟s records were no longer in charge of the projects. In addition the time of year was not conducive to eliciting replies. 2 Thus analysis of the questionnaire responses by the Team could only commence much later than originally planned. Follow-up by both Bank administrative staff and Team members assisted in boosting response rates to around 43 per cent overall.3 III.6 The ToR anticipated that up to 10 projects in two or three different continents would need to be studied during the course of the evaluation, and the Team 1 The World Bank Trust Fund for Statistical Capacity Building: An Evaluation. Oxford Policy Management, November 2003 2 The 2003 evaluation had made a similar observation, noting that it was difficult to contact World Bank staff in July and August 3 The response rates were 50 per cent for implementing agencies and for donors, 44 per cent for IMC members, and 43 per cent for TTLs; there were no replies from Consultative Group members who were not donors. 12 deliberated carefully on the countries and projects to be visited. It wished to include countries at various stages of statistical and general development, and to examine projects aimed at the formulation of national strategies for the development of statistics as well as some of a more „traditional‟ capacity-building nature. It also was interested to study countries where there had been a series of TFSCB projects. Additionally, the Team noted that the preponderance of field visits in earlier evaluations had been to Africa, with no missions at all to countries in Asia or Latin America. 4 Finally, it wished to take advantage of the fact that many heads and senior representatives of national statistical offices and other institutions (i.e., potential interviewees) would be attending the 57th session of the International Statistical Institute (ISI) in Durban, South Africa in mid-August 2009. Taking all these factors as well as logistical considerations into account, the Team decided to undertake three missions: (i) to Uruguay and Paraguay; (ii) to the ISI conference in Durban; and (iii) to Mongolia, Cambodia and Indonesia. III.7 Determining a mission schedule that suited the implementing authorities and other key agencies in the countries selected for visits, and that was feasible for the Team within the time and logistical constraints of the assignment, took somewhat longer than envisaged in the ToR. The mission to Uruguay and Paraguay took place in early August 2009, and that to Mongolia, Cambodia and Indonesia in the first half of September; two contact days were spent in each country. In all, seven projects which were ongoing or had recently been concluded were studied in detail. The Team also took the opportunity to examine, to the extent possible, the impact of three projects which had closed several years previously5, and to review the status of three other projects which had been approved but had yet to commence. With the assistance of the respective World Bank offices, discussions were held in all five countries with the project implementing agencies and other producers and users of official statistics, and as appropriate with Bank staff, donors and international agencies working in the country, and other relevant stakeholders – a total of some 37 meetings in all. Brief accounts of the country visits are given in Annex 2, and a list of persons met, structured by countries and institutions, appears in Annex 3. III.8 The International Statistical Institute‟s 57th session, held in Durban from 16 to 22 August 2009, presented the Team with a unique opportunity to meet with a large number of persons involved with the Trust Fund in one or another capacity. The Team had perused information about grants to individual countries and organizations with a view to identifying conference participants to be interviewed, but in the event was able to plan only a few meetings in advance and had to rely heavily on networking. Cooperation from the conference organizers and support from World Bank staff and other colleagues rendered this effort largely successful: meetings were held with representatives of PARIS21 and the PARIS21 evaluators, national statistical offices of African and other countries, and international and regional institutions concerned with statistical capacity building, as well as with the second member of the Advisory Panel and other individuals having experience of TFSCB activities. In addition to these 20 4 The 2003 evaluation of the TFSCB had covered Albania, Chad, and a Southern African Development Community (SADC) project; the evaluation of DFID support to the TFSCB had chosen Ghana, Kenya, Moldova and Sierra Leone as case studies; and the team evaluating the Marrakech Action Plan had visited Ethiopia, Malawi, Mali and Niger. 5 These projects had been financed under the first tranche of the Trust Fund, TFSCB I. 13 meetings with TFSCB stakeholders, the Team attended relevant conference sessions on statistical capacity building and national strategies for the development of statistics. Brief reports of the Team‟s meetings with country and organization representatives at the Durban conference appear in Annex 2. III.9 The ToR noted that the Marrakech Action Plan for Statistics provided the context for World Bank support to statistical systems in developing countries and related activities until the end of 2010; the current evaluation and that of PARIS21 were part of a review of progress to date, and would contribute to the development of a coordinated strategy for the period 2010 to 2015. In particular, it was envisaged that the TFSCB evaluation would be carried out in close coordination with the parallel evaluation of PARIS21 and that the two evaluation teams would to some extent collaborate on their tasks. That was not initially possible because of a delay in establishing the PARIS21 evaluation team. However, by late July 2009 the teams had started to exchange information by e-mail on their planned activities and had made their respective inception reports available to each other. The PARIS21 evaluators also decided to attend the ISI in Durban, and the teams met at the outset of the conference to discuss the scope of their cooperation and coordination. As agreed at Durban, the two teams have exchanged information and reports as they became available. III.10 The Team analysed all the national-level TFSCB grants by region, over time, and so far as it was able, by purpose of grant (for an NSDS or for more general statistical capacity building purposes). Grants of a regional and global character were also examined. III.11 The ToR had envisaged that the Evaluation Team would meet again in Washington towards the end of the assignment to present its draft final report to World Bank staff. However, it was not possible for this meeting to be held and the Team‟s draft final report was completed via correspondence and teleconferencing during late October 2009. Comments from World Bank staff including IMC members, and from the joint TFSCB/PARIS21 Evaluation Reference Group, were received in late December 2009, and the Team submitted its final report in January 2010. IV. Main findings Distribution of grants IV.1 The TFSCB was established in 1999 and commenced operations in the year 2000. The purpose of the fund was to support capacity building of statistical systems in developing countries by extending grants to such projects. As mentioned above, there have been four types of grants from the TFSCB: i) to national authorities for designing national strategies for the development of statistics (NSDS), ii) to national authorities for various statistical capacity building (SCB) purposes, iii) regional grants addressing SCB needs in groups of countries or sub-regions, and iv) grants of a global character. IV.2 From November 2000 until mid-2009 the TFSCB gave grants to 149 projects. Of these, 101 went to national authorities, 29 were for funding regional projects and 14 19 for global projects. It should be noted that some of the grants to national authorities have been advanced for both NSDS and other SCB purposes, and hence their classification into these two categories is not always very clear. In determining the main thrust of each project, the Evaluation Team has drawn principally on the project title although in some cases more information has been gathered from the individual project documents.6 Of the 101 grants to national authorities, the Team has classified a little under half (45 grants) as being for drawing up national strategies, with the remainder (56 grants) going to various statistical capacity building (non-NSDS) projects. IV.3 In the first years of operation most of the grants to individual developing countries went to various capacity building projects but a small number were granted specifically for designing national strategies. Thus, of 35 grants extended to national authorities during the operation of TFSCB I, November 2000-August 2004, 6 grants were explicitly given for drafting national strategies or statistical master plans (SMP) while 29 grants were disbursed to support various capacity building efforts (some of these included an SMP component). IV.4 The Marrakech Action Plan for Statistics (MAPS) underscored the policy of laying the groundwork for statistical development in each developing country by designing a national strategy. The TFSCB was thought to be a suitable instrument for supporting the work on national strategies, or national statistical master plans as they were called in some countries. Following the launch of MAPS, the TFSCB was used increasingly for this purpose. Two financing windows were introduced at this stage: an NSDS window and a more general SCB window for various development projects. The general operation of the SCB window continued as before, but with the important distinction that the grants were increasingly used to finance SCB projects which were directly related to or based on the NSDS or SMP of the country in question. In addition, this window was made more competitive by accepting applications twice a year. IV.5 During the operation of TFSCB II, October 2004-December 2007, there were 54 grants extended to national authorities. Of these, 34 were explicitly said to be for NSDS projects while the remaining 20 were for SCB projects of various kinds. Since the start of TFSCB III in January 2008, 12 country-level grants have been approved: 4 for NSDS purposes and 8 for addressing various SCB needs of national statistical authorities. IV.6 Of the US$30.6 million disbursed by the TFSCB from the outset until mid- 2009, regional projects received US$6.3 million and global projects US$4.2 million while the 101 grants to national authorities made up the remaining US$20 million. Based on the classification of grants discussed above, the total of grants to NSDS projects amounts to US$6.1 million or just under one-third of the grants to national authorities. Excluding TFSCB I, i.e. the period before the MAPS policy of prioritizing NSDS development was launched, the total of grants to NSDS projects amounts to US$5.5 million out of a total of US$12.3 million to national authorities. This equals 45% of the total to national authorities. The share of NSDS grants was highest during 6 Some project allotments for NSDS projects containing an SCB element were later divided into two grant agreements, one for NSDS and one for non-NSDS purposes, 15 TFSCB II: 52% of the total of grants to national authorities and 37% of the total of all TFSCB grants under that tranche. This can be compared with the policy expressed in the TFSCB Guidelines that the Fund allocates up to 60% of its resources for the purpose of preparing NSDSs in developing countries. One major reason for the difference in percentages is simply that only small amounts are needed to develop NSDS projects and consequently the NSDS grants tend to be small, often less than US$100,000. IV.7 Regional and global grants have been important elements of the TFSCB from the start of its operations. In TFSCB I, there were 13 grants of a regional character and 5 global grants out of a total of 53 grants. In the second period, the corresponding figures were 10 regional and 6 global out of 70 grants. The share of these grants seems to have been rising in the present operational period of the Fund; of 26 approved grants, 6 are regional and 8 are global. For the whole period of operation of the TFSCB, one third of the number of grants has been of regional or global character, or 48 out of a total of 149. It is of interest to note that during the operation of TFSCB I and II, the large majority of the regional grants went to Africa and for the period as a whole 15 out of 29 regional grants have been extended to Africa. Countries Approved TFSCB I-III receiving Number of projects Distribution Amount grants Regions and Number National Regional Total Million US$ % countries AFR (47) 30 40 15 55 10.0 33 MENA (13) 8 8 0 8 1.6 5 ASIA (30) 12 25 2 27 6.2 20 LAC (30) 10 13 7 20 4.8 16 ECA (23) 13 15 5 20 3.8 12 GLOBAL ... 0 19 19 4.2 14 TOTAL 73 101 48 149 30.6 100 IV.8 As regards the regional distribution of all the grants, Africa has been the largest recipient. During TFSCB I, 19 out of 53 grants went to Africa, in all amounting to US$3.9 million out of a total of US$11.9 million (33%). During TFSCB II, the share of Africa was greater still, or 30 out of 70 grants amounting to US$5.1 million out of a total of US$11.8 million (43%). The predominance of Africa has decreased during the operation of the present tranche, TFSCB III. The accompanying table shows the distribution of grants for the operational period of the TFSCB as a whole. The definition of regions follows that of the World Bank, except that „East Asia and Pacific‟ and „South Asia‟ have been combined into „Asia‟. Of the total disbursement of US$30.6 million, 33% has gone to Sub-Saharan Africa (AFR), 20% to Asia, 16% to Latin America and the Caribbean (LAC), 12% to European and Central Asian (ECA) countries and 5% to Middle East and North African (MENA) countries. The distribution is only partly explained by the number of countries in each region. In Sub-Saharan Africa, 30 countries (out of 47) have received 40 grants, 16 whereas the corresponding numbers are 12 Asian countries (out of 30) with 25 grants and 10 LAC countries (out of 30) with 13 grants. In Europe and Central Asia, 13 countries (out of 23) received 15 grants and 8 Middle Eastern and North African countries and territories (out of 13) received one grant each. 7 Different income levels in the regions also explain the distribution. Thus, out of a total of 64 IDA (International Development Association) countries in mid-2009, Sub-Saharan Africa has by far the highest number, 38, followed by Asia with 17. Here, it should be borne in mind that IDA status is one of the criteria used for determining TFSCB grants to countries. IV.9 The coverage of the 29 regional and 19 global projects has been wide ranging, but they may be roughly grouped as follows: research-oriented projects, training seminars and workshops, multi-country technical cooperation, and conference participation. Major research-oriented projects were carried out by the World Bank Institute (WBI) during TFSCB I and II, addressing „Poverty Analysis Data‟ and „Governance Data Capacity Building‟; the Organisation for Economic Co-operation and Development (OECD) also collaborated in a study on measurement of the non- observed economies in selected countries. Training events and learning networks have been funded by the TFSCB in topics such as national accounts (LAC), compilation of social and gender statistics (LAC), fundamentals of official statistics in economies in transition (ECA), assistance to statistical training institutes in the African region, and development of a labour market indicators library network (Global). Multi-country technical cooperation has been provided either via a regional organization, for example the Secretariat of the Pacific Community for a Pacific Survey program, or through projects executed by the World Bank itself; a regional approach was also applied to the formulation of NSDSs for a number of Caribbean, Andean and Central American countries. Finally, the TFSCB has supported the participation of developing country representatives in a number of conferences, notably the United Nations Statistical Commission (UNSC), the International Statistical Institute (ISI), and the Forum for African Statistical Development, but also meetings on more specific areas like the Statistical Data and Metadata Exchange (SDMX) and agricultural statistics. In delivering the regional and global projects, the TFSCB has so far worked with some 20 organizations, including seven United Nations agencies. Relevance of the TFSCB grants IV.10 In discussing the relevance of the TFSCB it is helpful to distinguish between the four types of grants mentioned in paragraph IV.1 above. Considering first the grants given to national authorities, these have been classified into those for the purpose of designing national strategies or master plans, or those for general capacity building purposes (in fact some of the grants have been for both of these purposes as mentioned above). In line with the stated policy of MAPS and the advocacy of PARIS21, developing countries have been explicitly encouraged to establish national strategies or master plans for setting out the paths and priorities for their statistical capacity building. Furthermore, national authorities have been encouraged to link 7 The number of countries by region refers to IDA/IBRD eligible countries according to the World Bank list as of July 1, 2009. The West Bank and Gaza is included in the number of MENA countries. 17 their applications for the general SCB grants to specific actions or parts of their national strategies or master plans. IV.11 This policy, which has been actively pursued by the TFSCB and PARIS21, seems to have been fully accepted by the recipient countries. First of all, the emphasis placed on laying the groundwork by designing an NSDS and having an NSDS or SMP action plan to guide the development of statistical capacities seems to have been fully embraced by the developing countries. Furthermore, in spite of the dependence of the more general SCB grants on the NSDSs the countries have been free to set their own priorities, both in their NSDSs and actions plans, and acquire grants on the basis of these priorities or the particular path which each country has paved out for itself. This is clearly borne out by the evidence gathered by the Evaluation Team. In particular, this has come to light in the survey among the recipient national authorities, both reflected by their direct answers to the survey questions and the great variety of needs which the countries have striven to address with the support of the TFSCB. IV.12 The question of relevance has loomed large in the direct discussions between the Evaluation Team and representatives of national statistical authorities. These discussions have shown clearly that the countries have defined their particular needs and pursued their own priority actions. Two factors might be borne in mind here: i) although the national authorities have received guidance and encouragement on their applications for TFSCB grants and technical assistance on the design of their NSDSs, their realities have been quite different and their solutions as well; ii) while there have been certain common features in the priority setting of the NSOs, such as national accounts, price statistics and poverty statistics, the individual priorities have varied considerably. IV.13 In view of this, the Evaluation Team is of the opinion that the TFSCB funding of NSDSs and other national capacity building projects has been highly relevant for the developing countries. Furthermore, the cooperation between the World Bank and PARIS21 experts vis-à-vis the individual countries and the joint work of the World Bank staff and domestic experts on the formulation of TFSCB applications should add an element of guarantee of the relevance of the projects to be supported. IV.14 As regards the regional grants from the TFSCB, these seem to have been initiated by a wide range of institutions including United Nations agencies and regional commissions and other bodies operating in specific regions or sub-regions, like AFRISTAT and the Southern African Development Community (SADC) in Africa. The question of relevance of the regional grants may be viewed both from the standpoint of the body requesting the grant and of the individual countries which, as a group, are the planned beneficiaries of the outcome of the project to be funded. It would seem more important that the regional grants are geared towards the needs of specific groups of countries, as in fact generally seems to have been the case, rather than being motivated by the priorities set by the institutions on the basis of their overall mandate. IV.15 The evidence gained by the Evaluation Team seems to suggest that in some cases the relevance to individual countries of regional projects has varied considerably. Thus, while a regional project may be considered to be fully relevant for a group of neighbouring countries with some common characteristics, the relevance of 18 the project may in fact vary from country to country depending on factors like the different level of statistical development, in general or in specific areas, in the individual countries. This may in particular apply to sectoral statistics where the country priorities and achievement levels may differ a lot. Regional projects typically have important advantages; they are likely to be cost effective, the project administration does not burden the recipient country and they often involve valuable exchange of experience. In discussions with both regional authorities and country representatives, training was frequently cited as a good subject for regional projects and was generally considered to be relevant by recipient countries. IV.16 It is not clear how to evaluate the relevance of the global projects. In the first years of its operation, the TFSCB extended a few large grants to fund or support programs which were global in nature, such as the grants extended to the World Bank Institute (see paragraph IV.9). The TFSCB has also extended grants to PARIS21, in 2001 and 2008. There is no doubt that some of the global grants have been given in support of programs of high relevance for statistical development in the developing countries in general, for example areas highlighted in the Marrakech Action Plan. Utilizing the TFSCB to fund projects which are run by related United Nations or similar international institutions may also make good sense provided that the projects involve statistical capacity building in accordance with the objectives of the Trust Fund. Such projects often provide a vehicle for utilizing specific expertise and experience in the different institutions. Experience of TFSCB projects – outcomes and impact IV.17 The Evaluation Team tried to throw light on the question of the experience gained of the statistical capacity building projects in its discussions with stakeholders, i.e., the national statistical authorities, the Internal Management Committee and Task Team Leaders, and through its surveys among these groups and representatives of donors and cooperating agencies. IV.18 Here, the discussion of the experience gained will be focused on two different facets: the outcomes of the work on the NSDS and on other SCB-funded projects in the different countries, and the impact of these projects on official statistics and the use of statistics for evidence-based decision making in the various countries. A distinction will also be drawn between NSDS projects and the projects funded through the more general SCB window. Finally, the outcomes and impact of regional and global projects will be considered separately. NSDS projects Outcomes IV.19 Altogether, the TFSCB has extended grants to design and work on national statistical strategies or master plans in at least 43 countries (the number may in fact be higher as some of the SCB grants have included an NSDS component). Of these, the Evaluation Team has had evidence from 23 countries, either through discussions with country representatives or the survey among national implementing authorities. The evidence suggests that the NSDS projects have generally succeeded in achieving the planned outcomes of these programs. What is meant by this is the following: 19  The countries have been largely successful in planning and designing an NSDS or SMP.  The work has more often than not been carried out in cooperation with or with the direct or indirect participation of several partners within the statistical or administrative systems of the countries.  The projects have usually involved identifying weaknesses or needs and setting priorities.  The level of technical assistance or consultative work involved is generally thought to have ranged from satisfactory to excellent.  The work has in most cases resulted in a proper strategy document outlining priorities and specifying a plan of action.  The NSOs in the countries in question seem to have assumed ownership of the strategies and plans, sometimes in cooperation with other government institutions.  The NSOs have almost invariably expressed satisfaction with the NSDS. IV.20 There are some deviations from this or different opinions on the quality and the strength of the outcomes. In this respect, the following considerations should be mentioned:  There are some, albeit very few, cases where the countries feel that the NSDS work has been less successful than anticipated. In the survey among national implementing authorities, one country stated that the results had not been achieved while in two cases the outcomes seemed to be unclear.  The extent to which the national statistical authorities have attempted to involve or have been successful in involving other partners within the statistical system varies considerably. In some countries this has been a proper cooperative exercise involving all the main statistical producers and even users throughout the project. Such involvement has in some countries extended to local as well as centrally placed statistical partners and involved training reaching far outside of the NSO, and has embraced external development partners as well. In other cases, the participation has been more limited, sometimes restricted to having some common workshops at the end of the exercise.  The criticism has been directed towards the NSDS work that it has in some countries been carried out more or less solely by international consultants, who have based the work on their own premises rather than the realities and actual needs of the countries in question. However, the Evaluation Team has not seen direct evidence of it nor has it been brought out in the discussions between the Team and the country representatives. Impact IV.21 The question of impact of the NSDS work is discussed here under four different headings: impact on the workings of the statistical system, impact on the regular statistical production of the country, impact on the use of statistics for policy making, monitoring and evaluation, and impact on donor funding and coordination in the different countries. 20 IV.22 The evidence of the evaluation exercise indicates clearly that the NSDS work has had considerable positive impact on the strength, focus and coordination of the national statistical system in the countries undertaking it. Many countries have reported that their statistical system has been strengthened through the NSDS work, that there is better understanding and cooperation among the different parts of the system, and that work on official statistics is better coordinated than before. But the sustainability of this may depend very much on the implementation of the strategies or the action plans which are based on the strategies. IV.23 Implementation of the strategies and action plans after the conclusion of an NSDS project seems to give the largest cause for concern. The TFSCB grant agreements for NSDS work frequently include government counterpart funding or active involvement entailing at least some contribution in kind. There is evidence that this government contribution is often realized while the NSDS project is ongoing. In some cases the NSDS effort has been accompanied by increased government funding of statistical activities. It is also clear from the material gathered by the Evaluation Team that the acquisition of the TFSCB grants has not displaced government spending on statistics. However, there is no decisive link between the NSDS work and the continuing statistical development effort set out in the action plans of the NSDSs and intended to be undertaken once the TFSCB project has closed. In other words, the decision to accept the TFSCB grant does not as such commit the governments to implement the actions prescribed by the NSDS and the establishment of an NSDS constitutes no guarantee for implementation. On the part of the TFSCB, the decision to extend a grant constitutes no guarantee for further funding although the creation of an NSDS is viewed as a prerequisite for organized statistical capacity building. IV.24 Asked in the Team‟s survey about the stage of implementation of the NSDS, national authorities gave widely differing answers; in some cases implementation was ongoing while in others it was at a standstill or hampered by lack of funds. This is quite understandable, as statistical needs have to compete for finance with other urgent needs under the very severe budget constraints of the developing countries. Asked about the major factors affecting implementation, national agencies most commonly cited insufficient funding. Other factors mentioned by several countries were lack of human resources possessing the necessary skills for statistical work, weak statistical infrastructure, insufficient motivation in partner institutions, and political instability. IV.25 The impact of the NSDSs on the production of statistics seems to vary a lot between the different countries. The main reason is presumably hesitant and insufficient implementation of the actions planned by the NSDS and lack of funding of statistical activities in general. In such situations it is clear that donor funding is often decisive for the operations of the NSO and other parts of the statistical system. IV.26 The NSDS projects have in several countries been said to have acted as an eye opener for the national governments to the importance of having regular and trustworthy statistics for decision making in economic and social policies. Hence, an NSDS may be a catalyst for increased respect for and use of statistics for public policies, monitoring and evaluation. The evidence seems to indicate that where it has been possible to properly implement NSDS or SMP action plans – by donor funding, special capacity building programs based on borrowed funds or even in some cases by 21 government funding – these activities have been accompanied by increased use of the statistics, in particular by government ministries, monetary authorities and other public policy makers. In the absence of proper implementation, however, the NSDS seems always in danger of being short-lived. IV.27 One of the more gratifying indications gathered by the Evaluation Team is the readiness of international donors to accept and work within the framework of the NSDS once it has been designed and adopted. This has been brought out in several of the discussions between the Team and NSOs as well as with representatives of international organizations. In some cases the NSO has been able to involve the donors in the plans for action thereby gaining acceptance of the NSDS action plan as a blueprint for the timing and conduct of periodic surveys and censuses. Hence, the NSDS action plan has become an important tool for coordination, not only of statistical activities but also of the participation of the different international donors in the important statistical undertakings as spelled out in the NSDS. SCB (non-NSDS) projects IV.28 Over its lifespan the TFSCB has extended a large number of grants to a great variety of SCB (non-NSDS) projects. During TFSCB I, 29 SCB projects were given grants. Judging by the titles of the projects, there were many grants of a general nature, such as for strengthening capacities of the NSO or the statistical system, for strengthening institutional capacities etc. There were also a number of grants linked to PRSP programs and poverty measurement. During TFSCB II, NSDS grants came to dominate, numbering 34 compared with 20 non-NSDS grants. It is from this period that the policy originates of linking non-NSDS grants to the NSDSs or action plans of the developing countries. In the present TFSCB III the Fund has so far approved 4 NSDS grants and 8 for non-NSDS purposes. IV.29 The Evaluation Team has looked into information about a number of the non- NSDS grants, including some of the early ones, which allow some conclusions to be drawn. Some of the early grants were connected to some larger projects funded by other sources. This may have been particularly the case with poverty measurement projects designed to support the then ongoing PRSP programs, which were receiving a lot of attention and funding in the early years of the TFSCB. There is evidence that such linking or supporting funding has provided good results. The Team has also been informed of projects focused on solving specific needs, for example in national accounts, price statistics and survey methodology, which have turned out to be very successful. IV.30 The encouragement and practice of linking non-NSDS projects to national strategies, master plans or action plans based on these seems to have worked well in the cases reported to the Evaluation Team. Such projects have had various aims including implementation of an NSDS or increasing statistical capacity building in general. There is evidence that some such funds have been used by the countries for training and in general for increasing the level of skills and quality of the staff. It seems very likely that the grants given in addition to or alongside the NSDS grants may contribute substantially to the sustainability of the results achieved through the NSDS work. It is of interest to note that in discussions about sustainability and the likelihood of success of the national strategies and similar capacity building projects, 22 the need for training is almost invariably mentioned by the national statistical authorities. Training was also cited by several countries as one of the main factors of importance for implementation and sustainability of the NSDS. Regional projects IV.31 The Evaluation Team has succeeded in obtaining information on the execution, outcomes and impact of several regional projects in Africa and in the Europe and Central Asia region but has little evidence of such projects in other parts of the world. The information that the Team has was mainly obtained through discussions with representatives of regional bodies present at the ISI conference in Durban. These discussions revealed both limited and good outcomes of regional projects funded by TFSCB grants. The limited success has been experienced for instance in projects on sectoral statistics, where low development levels of one or more of the countries prevented the countries in question from benefitting from the project. On the other hand, success has been reported for outcomes of regional projects of a more general capacity building nature, training in particular, which have enjoyed effective leadership and organization by the agencies which implemented the projects. IV.32 One of the arguments put forward for having international or regional organizations receiving TFSCB grants for regional projects is that it allows them to undertake activities which extend or complement their regular work and make good use of the expertise and experience of their staff for the benefit of a group of countries or a sub-region. This seems to work particularly well for training; in general statistics, in particular fields or with specific applications. Global projects IV.33 As discussed above, the “global” projects funded by the TFSCB have been of various kinds. Some of the early grants were relatively large or tied to large projects funded by others. During the last two or three years the global grants have mainly been extended to fund the participation of statisticians from developing countries in international conferences, either on specific subjects such as metadata, global warming, or SDMX, or the annual United Nations Statistical Commission meetings and the biennial ISI conferences. There are also recent examples of global grants to fund strategic planning for agricultural statistics and on specific issues such as the WBI Poverty Analysis and Data Initiative. IV.34 The global grants may well be worthwhile given that they are in accordance with and within the limits of the guidelines of the TFSCB. The impact of such global projects on building statistical capacities is of course not easy to assess. Assuming that there is a continuation of the projects, the funding of specific planning efforts, like the one for agricultural statistics mentioned above, seems likely to have the intended impact. The impact of funding participation in international conferences is uncertain but may well be substantial, both for the developing countries and the international organizations in question. For the developing countries, such participation may, on the one hand, bring home learning and experience and be a part of training. On the other hand, participation in conferences like the UNSC and similar meetings may be important for facilitating understanding and motivation to align 23 domestic practices with international ones and for networking among managers of statistical institutions, national as well as international. Such participation and the accompanying networking does also benefit the international organizations and strengthens those bodies themselves in terms of knowledge and sensitivity to statistical capacity building issues. IV.35 These views seem to be shared by several representatives of both countries and regional bodies with whom the Evaluation Team met in the course of its work, as well as by a number of IMC and Consultative Group members who participated in the Team’s survey. Project management and implementation IV.36 The Evaluation Team assessed project management and efficiency in project implementation by examining practices and experiences through the various stages of the project management cycle. The project cycle is shown in Annex 4 and the most important findings are highlighted below according to the stage of the cycle. 8 IV.37 The project idea, formulation and approval stages run efficiently based on well-established practices. The origin of project ideas is evenly divided among client agencies, PARIS21 and World Bank staff and does not seem to be supply-driven. The use of the World Bank Grant Funding Request (GFR) system, and guidance from the Trust Fund Administration Unit (TFAU) in using it, have helped speed up project formulation and approval. The Internal Management Committee reviews proposals against a range of set criteria. The IMC‟s decision process works well and decisions are well documented.9 The IMC members indicated that the quality of proposals has improved over recent years. IV.38 The project activation time from approval to start-up may take from 1.5 to 20 months; in the evaluation sample projects, it averaged 8 months. During this time, the TTL finalizes a grant agreement with the respective recipient agency, which is either a national agency or a regional or international organization. The duration is affected by two factors, namely determination of execution responsibilities and recipient agency/government procedures for approving the grant. Both are external factors beyond the control of the TFSCB management, but affect the timeliness of project activities. World Bank Trust Fund policies stipulate “recipient execution” of all grants, only allowing “Bank execution” on behalf of the recipient in exceptional cases, for example in countries with very fragile implementation capacity or in certain Board-approved resolutions that specifically permit Bank execution. 10 The determination of the execution arrangements often takes some time, and Bank execution requires approval at higher level. 11 In one case known to the Team, the 8 The TFAU has issued specific TFSCB guidelines and procedures, last updated in September 2009. These are mostly targeted at TTLs and cover the main aspects of project management. 9 The IMC can approve, conditionally approve or reject a proposal. Subsequently, the TFSCB window manager can approve a conditionally approved project once the condition is satisfied. 10 World Bank Operational Policy 14.40 - Trust Funds, Paragraph 4 (i) Recipient Executed Trust Funds, Footnote 7. The same policy mentions certain restrictions on Bank execution. 11 This decision typically includes an assessment of the client‟s administrative and financial management capabilities as an important element. In some cases, recipient country execution may be supported by a locally hired accounting firm or other administrative arrangements. 24 United Nations Development Programme (UNDP) was designated as the executing agency on behalf of the NSO, which worked well. IV.39 The recipient country authorities typically approve a grant agreement within three to six months. However, approval may take up to 20 months depending on the time in the financial year that the project proposal and grant agreement are introduced. Three of the sample projects reviewed by the Team had to await a new budget year, resulting in activation time of 18-20 months. In all cases, however, the recipients reported that the projects were still valid, and the TFAU extended the planned closing date accordingly. The recipient approval time did not appear to delay the start of projects executed by regional or international agencies. IV.40 Lengthy though the project activation process may appear, it reflects the formal ownership of recipient governments by incorporating externally funded projects into their national budgets and allocating appropriate counterpart funds. Hence, the process follows the principles of the Paris Declaration on Aid Effectiveness, 2005. IV.41 The ease and timeliness of project implementation vary greatly. Bank- executed projects were nearly always smoothly implemented. Recipient-executed projects were also smoothly implemented in countries or by organizations with well functioning administrations, and/or where the client agency could dedicate specialized staff to project administration matters. For all other recipient-executed projects, client agencies and TTLs unanimously cited the difficulty by client agencies in applying the complex World Bank procurement and disbursement rules to small grants as the greatest obstacle to smooth implementation. 12 Problems in applying World Bank rules were particularly noticeable among African NSOs but other countries also expressed such difficulties.13 IV.42 Many TFSCB client agencies, such as NSOs, do not command in-house professional procurement staff, and the project size does not merit hiring ad hoc financial management staff. This situation is sometimes addressed by using the Ministry of Finance as executing agency on behalf of the NSO, but this creates an additional bureaucratic layer and curbs NSO flexibility. Training of existing NSO staff is useful, but they are seldom the same staff who might later on administer large- scale World Bank loans. The hiring of firms following the tender procedures was found to be particularly cumbersome, and a World Bank procurement specialist recommended to the Team that recipient agencies be encouraged to use individual expert services instead. It was also suggested that where difficulties were anticipated with recipient execution, NSOs should allocate funding for administrative support. This practice, however, would reduce the resources available for statistical purposes. 12 Implementing agencies and TTLs (interviewees and respondents to surveys alike) specifically mentioned the complexity of World Bank rules, strict documentation requirements for disbursement leading to rejection, resubmission and delays, and the long time required for non-objection to procurement actions, possibly caused by incomplete documentation. 13 Five African NSOs (out of seven) and three TTLs for African projects (out of four) made this point. 25 IV.43 The principle of the World Bank policy of recipient execution is noble, but often places appreciable additional administrative burden on an already stretched NSO management, taking time away from more productive technical, statistical activities. Other factors influencing implementation are no different from those found in any development project, including absence of counterparts in NSOs to effectively transfer knowledge, lack of IT staff, lack of or late release of government funding or lack of commitment from other parts of government. IV.44 The quality of project inputs is highly rated. Consistently, NSO counterparts and survey respondents expressed satisfaction with the quality of expert services as well as the technical knowledge and support of the TTLs. The World Bank Country Office‟s staff also play a vital role throughout the project formulation, activation and implementation stages, as acknowledged by many respondents and during the Team‟s country visits. In general the Team found that World Bank Country Offices were well aware of the grants and appreciated their catalytic importance. IV.45 Project monitoring is primarily done by the TTL, while the TFAU window manager checks on the rate of disbursement as well as general project progress. The TTL must submit a progress report every six months in the Grant Reporting and Monitoring System (GRM) but the TFAU is concerned over low compliance. At the same time, the progress reports are designed to cater for the detailed needs of the TTL, his/her supervisors and other close collaborators, and are less adequate for management reporting. Project documentation is stored in the Integrated Records and Information System (IRIS) and GRM and is only accessible to those directly concerned. Currently, the GRM allows TTLs to rate progress towards project objectives as well as overall progress. Progress towards objectives was rated as “highly satisfactory” or “satisfactory” for 82 per cent of all rated projects in September 2008.14 While this appears to show good performance, the fact that it reflects self-assessment must be borne in mind. IV.46 Reporting could become more results oriented at the project level but this would require additional active guidance from the TFAU to TTLs on how to complete the GFR based on precise formulation of outputs, outcomes and associated indicators in the initial project proposals. Subsequently, the progress reporting would follow the same pattern and generate more harmonized reports presenting tangible results. Some IMC members called for more use of results monitoring frameworks. This may be an advantage for the largest SCB projects and especially for co-sponsored projects that have a relatively large budget, say above US$800,000. For smaller projects or for the NSDS window, it may not be worth the effort. In any event, the TFAU would need additional resources to guide such a process, and in general to monitor project implementation and follow up projects more effectively. IV.47 Overall, projects are completed within budget and within the regular two year time frame, at times extended by one year. The TFSCB Guidelines provide very detailed instructions on changes to the planned closing dates, but only one sentence on project completion procedures. Formal completion takes place by the TTL submitting a completion memorandum and the window manager declaring the project closed. Completion reports are mostly administrative in content and final reports are only 14 WB, DECDG: TFSCB Progress Report, May 1 to September 30, 2008, page 4. 26 accessible to a few staff directly concerned with the project. Given the small project size, evaluations are not mandatory, and in the course of its work the Team could not identify any project that had been evaluated either by the World Bank or independently. One TTL suggested having more systematic follow-up on global and regional projects. In view of the absence of evaluations and the administrative nature of the completion memorandum, it would be desirable that TTLs provide a short summary report of the project‟s main substantive and other achievements in the GRM which would be easily accessible to TFAU and TFSCB donors, if they so request.15 IV.48 United Nations (UN) agencies face special challenges in executing TFSCB co- funded activities. The TFSCB has supported a good number of activities which have been proposed and implemented by such agencies. Despite goodwill on both sides, UN agencies and the TFSCB Administration Unit have been finding it increasingly difficult to conclude grant agreements. The principal issue blocking the agreements is the administrative fee required by both organizations for disbursing funds received from external sources, which would result in double overhead being charged to projects. At least one project has been cancelled for this reason and another project remains inactive some nine months after its approval. 16 The grant agreements for other projects have been subject to lengthy negotiations. It is against the financial rules of the Bank and the UN agencies to waive their overhead fees and solutions can only be reached by obtaining sensitive, top-level exceptions. While by no means wishing to underestimate the difficulties involved, the Evaluation Team nevertheless notes that the total overhead amounts are modest compared with the time and effort spent in resolving these matters. The cases are few within the overall TFSCB project portfolio, and the delays run counter to the TFSCB‟s overall aim of delivering timely, focused, and demand driven technical assistance. 17 IV.49. Against this backdrop, the Team is pleased to learn that the TFSCB Administration Unit has recently obtained the acceptance of the three donors to TFSCB III to allow a reasonable degree of double overhead to be charged. Hence UN agencies will be able to charge a fee (within a certain limit) in executing TFSCB grants. The arrangement still requires the acceptance of the UN agencies concerned before it can become operational. TFSCB Management and Governance IV.50 The TFSCB is closely linked with the PARIS21 initiative and so are the various management and governance functions. The TFSCB and PARIS21 share the same overarching objective and are regarded, at least by the DECDG, as “two arms of 15 The Evaluation Team, for instance, relied mostly on recipient interviews and questionnaire responses to learn of project achievements. 16 Proposed in September 2008 and approved by the IMC in April 2009, the grant agreement for this project could not be finalized due to negotiations on the waiver of the UN overhead fee of 13%, and subsequently its reduction, which requires an exception, approved by the UN Controller -- in itself a lengthy process. At the time of writing (January 2010), the grant agreement has still not been finalized. 17 In the project described above, the UN overhead of 13 per cent would add less than US$44,000 to the project allotment, and much staff time was lost in tackling the administrative obstacles and in cancelling and re-arranging project activities. Ultimately, timely training in certain statistical techniques could not be delivered. 27 the same process”, where PARIS21 is responsible for advocacy and the TFSCB for operationalization of statistical capacity building. 18 IV.51 The TFAU located in the DECDG manages the TFSCB. In addition to the project or grant management described above, the Trust Fund management includes several functions such as servicing the IMC, facilitating the work of the Advisory Panel, reporting to the TFSCB Consultative Group (CG), following up on CG recommendations, Fund outreach, and coordination with other related initiatives within and outside the World Bank. These include PARIS21, the Marrakesh Action Plan, and internally the Development Grant Facility, which also provides grants for statistical capacity building and preparatory activities for STATCAP. The TFAU is also responsible for compliance with World Bank Trust Fund policies and regulations as they relate to the administration of TFSCB I, II, and III. IV.52 The Team‟s observations on the management and governance functions of the TFSCB are summarized below, keeping the close relationship with PARIS21 and the MAPS in mind. TFSCB outreach to potential clients and how clients approach the Fund IV.53 Judging by the information gathered by the Evaluation Team, the knowledge of governments and statistical authorities in developing countries about the existence and the role of the TFSCB varies considerably. During its missions the Team met with the representatives of many countries, most of whom had communicated with the TFAU or the Bank more generally. Some of these, in particular those with the longest history of dealing with the Bank, seemed to possess comprehensive knowledge of the TFSCB and the possibilities of acquiring grants for SCB purposes. Others were much less knowledgeable; they might have known about the NSDS grants but were less aware of those for non-NSDS purposes, or of the possibilities of obtaining SCB grants following the successful design of an NSDS. Some countries had been receiving financing from the Bank but had little knowledge of the specific channels through which these funds had been supplied. In general, the countries did not seem to be very well informed about how, when or for which purposes they might approach the TFSCB or about the detailed conditions for applying for and receiving grants. Instead, they seemed to rely very much on their contacts at the World Bank country office, not only for information but even for taking the lead in approaching the TFSCB and preparing an application. In this connection it should be pointed out that the Team found that in the countries visited the relevant Bank experts dealing with statistics were interacting well with the statistical authorities. IV.54 One sentiment expressed to the Team was that the TFSCB might be more proactive in promoting itself. While it was acknowledged that information on the Trust Fund and on the possibilities for acquiring funding for statistical capacity building was to be had from several different sources, such as the websites of the Bank and PARIS21 as well as from experts of these and other international and regional institutions, it was doubted that this is sufficient. Fears were also expressed to the Team that the use of English as the sole working language of the TFSCB might present a barrier to accessing the Fund. The Team raised this question in its 18 World Bank Independent Evaluation Group, “Global Program Review – TFSCB”, January 2006, DECDG response to findings, page 34. 28 discussions with the different non-English-speaking countries but in only one instance was this perceived as a real barrier. Internal Management Committee, project management IV.55 The IMC is responsible for reviewing, appraising and deciding on TFSCB funding for project proposals. The IMC currently has nine members. New candidates for the IMC are nominated by its Chair, based on their statistical expertise and knowledge of statistical conditions in various regions of the world, and agreed by the sitting IMC members. From the minutes of the IMC deliberations (in person as well as virtually) made available to the Team, it is clear that members have been serious in examining the proposals and in passing their best professional judgment. The IMC members expressed broad satisfaction with the review and approval process. As the TFSCB is an umbrella funding facility, the responsibility for each individual grant is devolved to the TTL as soon as it is approved. This is fully coherent with the management of such funding arrangements, but the TFAU has a pertinent claim to monitor the overall performance of the grants, emerging good practices and lessons learned. However, the general World Bank tools for this purpose, the Grant Reporting and Monitoring System (GRM), only started a few years back. For previous projects, only the TTL can locate the associated reports in the various systems, and if the TTL transfers, documentation can be lost or become very difficult to locate in the Integrated Records and Information System (IRIS).19 The Advisory Panel IV.56 The Advisory Panel consists of two experts who, once a year, review all IMC decisions on TFSCB proposals and record their findings on the overall direction of proposals, geographical composition of grants and so on in a report which is submitted to the Consultative Group (and studied carefully by DECDG and the IMC). In discussions with the Evaluation Team, the Panel members described their role as, in effect, one of quality control. Based mainly on the review of past project proposals, the Advisory Panel provides advice on the future direction of the TFSCB. The IMC members stated that the reports of the Panel were very useful. The Consultative Group IV.57 The governance of TFSCB is closely linked to that of PARIS21. The TFSCB is formally governed by a 12 member Consultative Group, including donors (Canada, France, Germany, Netherlands, Switzerland and United Kingdom), international bodies concerned with statistical capacity building (Eurostat, OECD, United Nations Statistics Division, and the International Monetary Fund), PARIS21 and the World Bank itself. The TFAU prepares a TFCSB semi-annual progress report for submission to the CG.20 The report lists approved projects by title and stage of implementation, the TFSCB financial situation, and the overall progress in terms of number of NSDS and non-NSDS projects. It also provides a listing of projects rejected by the IMC under the non-NSDS window. The progress reports are posted on the TFSCB web 19 For instance, access is available to the window manager but not to all IMC members. The difficulties faced by the Team in verifying the exact project status and obtaining project reports etc bear witness to this situation. 20 The reports are called “Annual report” but are in fact issued twice a year, generally in May and October. 29 site. The CG meets once a year in a joint meeting with the PARIS21 Consortium, which includes additional PARIS21 donors21. IV.58 The Evaluation Team noted that the current high-level arrangements for review of the TFSCB appear to be in a state of flux, and the Team finds itself in difficulty in commenting on them. The Proceedings of the PARIS21 and TFSCB Donors Meeting are listed on the PARIS21 website but not on the TFSCB website. This arrangement makes the TFSCB less visible than could be desired. The Team feels that, as a start, the DECDG should create a link from its own website to these reports and include a list of participants in the meetings. Coordination with other internal initiatives IV.59 The TFSCB is the World Bank‟s main grant instrument to support NSDS preparation and SCB activities. However, the World Bank Development Grant Facility also provides grants for SCB purposes, and a number of activities could be funded from other World Bank grants or loans of a wider scope and which include a statistical component. The World Bank has established an internal Statistical Capacity Building Committee to coordinate all such activities. Financial Management IV.60 The TFAU monitors the aggregate financial status of the TFSCB, and interacts with donors on contributions. The financial situation is reported on in the progress reports in terms of funds committed by donors and allocations towards approved projects, generally as of 30 April and 30 September. The posting on the website helps transparency. More details with exact figures on the revenues and expenditures would make the statements clearer. An overview of the TFSCB‟s financial status as of 31 October 2009 is provided in Annex 5. Overall usefulness of the TFSCB IV.61 In the discussions which the Evaluation Team had with stakeholders of the TFSCB, questions concerning the overall usefulness of the Fund for the developing countries were invariably addressed. In particular such discussions considered the direct use of the grants, their size, their role in directing the statistical capacity building effort of the countries, and as constituting leverage for further SCB funding by other donors. IV.62 All the stakeholders whom the Evaluation Team either met or heard from through its questionnaires expressed satisfaction with the TFSCB scheme and the grants. Although the grants are small they are still considered very useful. Here it is relevant that the projects on national strategies or master plans are not costly and in fact the large majority of the NSDS grants are US$100,000 or less. The non-NSDS grants need, however, to be larger if they are to have the desired impact. IV.63 One of the main advantages of the TFSCB scheme expressed by stakeholders is that the national strategies or master plans which the Fund finances are a very 21 PARIS21 donors include: Belgium, Canada, European Free Trade Association (EFTA), European Commission, Finland, France, Germany, Greece, Ireland, Japan, Netherlands, Norway, Spain, Sweden, Switzerland, and United Kingdom. (TFSCB donors underlined). 30 effective starting point for the statistical capacity building effort of a country, both for mobilizing concerted action within the government administration to generate and make use of statistics and for facilitating cooperation with donors. It seems clear that the whole process of preparing and applying for a TFSCB grant for designing an NSDS, planning the work, involving government and partner institutions, and carrying out the work is very important for raising the awareness of statistics and capacity building in that area. Also, it creates an awareness of the need for a properly functioning national statistical system and the position in it for the different actors involved, while at the same time it underscores the role of the government body specialized in statistics, the NSO, in coordinating the statistical capacity building process. IV.64 The outcomes of the NSDS work and the fact that a country subsequently has a national strategy and an action plan to guide the capacity building process is seen as being very important for mobilizing the assistance of donors and international organizations in the capacity building effort. One of the underlying assumptions of the TFSCB is that the financing of NSDSs and the linked projects will help the developing countries to acquire donor funding for a more comprehensive statistical capacity building effort as well as facilitate the use of development loans for that purpose. Asked about this, many stakeholders contended that having an NSDS has proved decisive for several countries in raising new or additional donor funding. Furthermore and as has already been discussed above, there is good evidence that donors are ready to accept the prioritization and plans of the NSDSs and coordinate their assistance accordingly. IV.65 However, although the countries fully accept the importance of developing their official statistics, many of them balk at accepting loans for that purpose for the simple reason of their precarious economic and budgetary situation. This lessens the chances for the smooth or continuous progress of statistical capacity building based on the NSDS and for the sustainability of the actions put in place. Hence, there has been – and still is – a need for some rethinking and reorientation in this respect, in particular involving increased emphasis on assisting the implementation of the SCB plans of the countries. IV.66 The Bank has responded to this by designing a so-called Statistics for Results Facility (SRF). The aim is to accelerate the strengthening of the statistical systems in the participating countries and help them to build the capacity to manage for development results, by linking national development plans and poverty reduction strategies more closely to statistical efforts, and accelerating the implementation of country-owned statistical improvement plans. Furthermore, it aims to trigger more effective donor and government collaboration for substantial scaling-up of investments in national statistical systems and results management. To support the SRF, the DECDG has set up an SRF Catalytic Fund which shall provide substantial grants to countries for activities that improve both the production and the use of statistics. This scheme is in its early stages of development. Five countries have been selected as pilot countries to participate in the scheme. IV.67 The development of the SRF could be an important step forward in accelerating statistical capacity building in developing countries. However, at the present time the future relationship between the SRF and the TFSCB is unclear; 31 certainly, replacing the TFSCB with the SRF does not seem to be a realistic option. The aims of the two schemes are different; the TFSCB aims at giving small grants to many countries for developing NSDSs and undertaking specific but limited capacity building tasks; the SRF aims at giving large grants for substantial investment in statistical capacity building, not least for managing development results. Thus, while the SRF could be invaluable by financing large projects in a few countries at any one time there will still be a need for providing small grants to many countries for SCB purposes. IV.68 Notwithstanding the development of the SRF, the Evaluation Team finds that some reorientation of the TFSCB is called for. That should in particular recognize the present need to provide a better bridge between the development of national strategies and the implementation of the strategies. Such a reorientation might involve three issues. First, the allocation of the TFSCB grants might be rebalanced, giving more weight to NSDS projects than the present targets prescribe. Second, the NSDS projects and grants might be redefined in such a way as to include a final component which would help finance the implementation of the first stages of NSDS action plans, thus preserving the momentum of statistical capacity building which all evidence shows is likely to be triggered by the NSDS work – and which indeed is a hallmark of an effective NSDS. Third, the size and duration of the TFSCB grants should be reconsidered. IV.69 As discussed above, projects for laying down national strategies, priorities and action plans are not expensive and their cost is well within the limits of the grants. The same does not apply to the subsequent or more general SCB grants. The maximum amount for the grants has remained unchanged at US$400,000 from the beginning of the TFSCB‟s operation. Inflation alone should be an argument for reconsidering this limit. Looking at the project amounts over the cycle of the TFSCB shows that the grants from the SCB window have frequently been around or just below the maximum, which may be an indication that the limits are becoming too narrow. Finally, if the TFSCB is to become more oriented towards the implementation of the plans for statistical capacity building, that in itself is a sufficient argument for reconsidering the size of the grants. V. Conclusions and recommendations Conclusions V.1 The evidence collected by the Evaluation Team indicates clearly that the TFSCB has been very important for the statistical capacity building of developing countries. At the same time, the role of the Fund is thought to be still valid and there continues to be a need for unrequited grants for statistical capacity building in the developing world. The advent of the SRF seems likely to become an important vehicle for strengthening statistical capacity building in the developing countries but it is additional to the TFSCB and does not diminish the need for the Fund‟s continued operation at the present time. V.2 The evaluation has shown that the projects financed by the TFSCB are highly relevant to the recipient countries and that they reflect the priorities of the countries themselves. On the whole the projects are judged to have achieved the planned 32 outcomes. The MAPS policy of laying the groundwork for statistical capacity building by developing national strategies seems to be embraced by the developing countries. The NSDS projects are considered to have been very useful, particularly as a starting point and guide for the capacity building process, for mobilizing action and cooperation within the government system for SCB purposes, and for securing new or additional and well-focused donor funding. V.3 The projects financed by the TFSCB are found to have had considerable impact on the generation and use of statistics. The information obtained by the Team shows that the non-NSDS projects have been very successful in solving methodological needs in specific areas as well as more general needs for training and for institutional and infrastructural capacity building. Similarly, the practice of linking non-NSDS projects to the national strategies seems to have worked well. V.4 The main weaknesses observed by the Evaluation Team concern the implementation of the strategies and their concomitant action plans. There is no decisive link between the NSDS work and the continuing effort at statistical capacity building. Hence, there is no guarantee that the development of an NSDS will be followed by implementation of the strategies and action plans. This shows up clearly in the information gathered by the Team; in some cases countries are not successful in maintaining the momentum created during their NSDS work by implementing the planned actions. This is mainly considered to be the result of a lack of funds although there may also be some other obstacles. Thus, the sustainability of NSDS actions funded by the TFSCB is threatened. The evidence of the evaluation work also shows that for many developing countries with weak economies and severe budget constraints it is not realistic to expect that the gap between the projects funded by the TFSCB and the unfunded action plans can be bridged by loans. V.5 The Evaluation Team noted that regional and global projects were a heterogeneous grouping, ranging from assistance at the sub-regional level for national strategy development, to financing developing country participation at global meetings. The Team also noted that, compared with national projects, it was generally more difficult to evaluate the outcomes and especially the impact of regional and global projects on statistical capacity building. Moreover, the TFSCB‟s guidelines and procedures and the criteria for selection of projects are arguably more focused on national rather than on regional or global projects. The Internal Management Committee should consider whether, in a situation of limited funding, it would have adequate tools to make informed choices between financing national and regional or global projects. V.6 As regards the execution of TFSCB-funded projects, most are carried out in a timely manner within the foreseen budget and having used inputs of high professional standard. The formulation and approval of projects work well following soundly based established procedures, helped by recent World Bank grant requesting systems. Two factors beyond the control of the World Bank administrators delay and hamper smooth project implementation: the time required in some countries to formally approve an externally funded project and the difficulty faced by many implementing agencies, especially in Africa, in applying the complex World Bank rules. Monitoring of project progress is decentralized, and does not lend itself, at least in its current form, to aggregate reporting of results. This may be improved by the TFAU 33 expanding its guidance to TTLs to cover not only approval procedures but also measures that will promote uniform and timely progress reports that include results. Improving project proposals in their definition of outputs, outcomes and indicators could be a starting point for better reporting. V.7. The application of the World Bank procurement and disbursement rules was the most frequently stated factor causing delay in project implementation. As these rules must be applied to all World Bank projects (large loans and modest grants alike), the difficulty in their application constitutes an external factor beyond the control of both the TFSCB administrators and the recipient governments. The difficulty was observed at two stages: (i) in the initial determination of grant execution responsibility (Bank or recipient executed), and (ii) in the application of the rules by recipient governments.22 Particular difficulty was noted when a client agency wanted to engage a firm using the Bank procurement rules. It should be noted that several survey respondents and interviewees during country visits commended World Bank staff for being very helpful in this process. Nevertheless, the cumbersome procedures and extensive documentation requirements – when applied to relatively small grants – have no doubt contributed to delays in implementation. The Team feels that this drawback should be one of the factors taken into consideration in future reviews of the Bank‟s procurement and disbursement rules. Recommendations V.8 The main recommendations of the Evaluation Team are as follows: 1. In the light of the conclusions on the operation and impact of the TFSCB and noting especially that there is still a great need to extend grants for statistical capacity building in developing countries, the Team recommends that donor funding continue to be mobilized for the operations of the Trust Fund. 2. The main weakness of the present scheme is the uncertainty about the implementation of the national strategies and the prioritized action plans based on them. Hence, it is recommended that the Consultative Group address the issue of financing the implementation of such action plans, with the aim of securing the gains achieved through the formulation of national strategies. This could be facilitated by redefining the present NSDS projects and grants and even by opening a special window for implementation grants. 3. At present the TFSCB attempts to allocate up to 60% of its resources to the preparation of NSDSs in developing countries. The Team recommends that in future a minimum of 60% of the TFSCB‟s resources sho uld be allocated to the NSDS process. Within this process a strong emphasis should be placed on contributing to the financing of the implementation of national strategies; resources should also continue to be allocated to the formulation of national strategies for countries still without them, to the updating of strategies where 22 The benefits of the client agency “lear ning-effect” were mentioned, but the Team noted that the difficulties mostly occurred in NSO administrations using existing staff to handle the grant. Those staff members were not likely to get involved in the administration of larger WB projects for which agencies typically would employ specialized financial management staff. Thus the “learning benefit” may be limited. 34 necessary, and to elaborating cost plans or action plans for strategy implementation where these do not exist or are inadequate. 4. At the same time, the maximum amount of the SCB grants and their duration should be reconsidered. This should be done with a view to both adjusting the grants for the inflation which has reduced their value over the last decade, and enabling the countries to start implementing the first stages of the action plans which their NSDSs involve. 5. Considering the difficulty experienced by many recipient agencies in applying the World Bank procurement and disbursement rules, the Team recommends that the Trust Fund Administration Unit amend the TFSCB Guidelines to encourage TTLs and client agencies to prepare projects in such a way that individual expert services rather than firms are used and to allow more use of TFSCB funds to hire local administrative staff. 6. The Trust Fund Administration Unit should expand its guidance to Task Team Leaders from the approval procedures to measures that will promote uniform and timely results-oriented progress reports. This could include revising and expanding the TFSCB Guidelines on progress reports and completion reports. The TFAU may also consider how tailor-made reports from the Grant Reporting and Monitoring System may further facilitate the work. 7. In the absence of substantive completion or evaluation reports, it is recommended that a short terminal report be introduced highlighting tangible results and major lessons learned. 8. The Evaluation Team is of the opinion that some form of evaluation could help to highlight lessons learned for the benefit of future projects. TFSCB management may consider applying a mixture of measures in this respect: i) The role of the Advisory Panel could be expanded to include ex-post evaluation of sample projects; ii) The CG may consider requesting thematic evaluations of a subset of projects; iii) The management of the TFSCB may consider requesting end-of-project evaluations of larger projects (around US$400,000 or more if combined with associated projects in the same country) or where the Fund is co-financing a large regional or global project. 9. The TFSCB should continue to co-fund statistical capacity building activities proposed by United Nations agencies as long as they fall within the mandate of the TFSCB. In view of the occasionally intractable problems affecting the ability of UN agencies to execute TFSCB-funded projects, and taking account of the recent acceptance by main donors of a modest degree of double overhead, the TFSCB management should continue to negotiate with the concerned UN agencies with a view to facilitating their execution of TFSCB projects while minimizing the total amount of overhead involved. The TFSCB should keep the Consultative Group informed of the outcome of these negotiations and seek its guidance on any ongoing difficulties. 10. In order to enhance the TFSCB‟s visibility and increase awareness among its potential clientele, its management should be more proactive in disseminating information about its key features. It is recommended that when the TFSCB brochure is next revised, it should be translated and placed on the Data for Development home pages in French, Spanish and Arabic. 35 11. The Evaluation Team noted that while the Consultative Group is the governing body of the TFSCB, it is not apparent from the Trust Fund‟s own web page how this governance is exercised owing to incomplete posting of CG reports there. Additionally, although more information may be considered better than less, it was not clear why progress reports are provided to the Consultative Group every six months when the Group usually meets once a year. The Team recommends that these matters be scrutinized with a view to improving the transparency of the TFSCB‟s governance arrangements as well as freeing resources currently devoted to the reporting process. 36 Annex 1 Survey among TFSCB stakeholders In order to gather information and views from the TFSCB‟s main stakeholders, the Team designed four questionnaires to be sent to the implementing authorities of Trust Fund projects (primarily national statistical offices), to Task Team Leaders of projects, to members of the Internal Management Committee, and to Trust Fund donors and other Consultative Group members. No sampling was involved – all agencies and individuals in the respective groups were to be contacted. Arrangements for the handling of the questionnaires, including their translation into French and Spanish, the establishment of a dedicated e-mail account, and the timelines for the despatch of questionnaires and the receipt of responses, were discussed with the Manager and Administration Unit of the TFSCB. Some delays were experienced with the transmission of the questionnaires, and in some cases it was discovered that implementing agency contact persons and also TTLs on the TFSCB‟s records were no longer in charge of the projects. In addition, the time of year was not conducive to eliciting replies. Thus, analysis of the questionnaire responses by the Team could only commence much later than originally planned. Follow-up by both Bank administrative staff and Team members assisted in boosting response rates to around 43 per cent overall. The response rates were 50 per cent for implementing agencies and for donors, 44 per cent for IMC members, and 43 per cent for TTLs; there were no replies from Consultative Group members who were not donors. This annex contains the questions and summaries of the responses in the survey among the implementing authorities and the Task Team Leaders as well as the questionnaires sent to the members of the Internal Management Committee and to TFSCB donors and other Consultative Group members. 37 Survey among implementing authorities: Questions and summary of responses With the assistance of the Administration Unit of the TFSCB, the Evaluation Team distributed a questionnaire to 58 implementing authorities (IA), mostly national statistical offices, in the recipient countries on the operations and the outcomes of the Fund. Responses were received from 29 country authorities as well as from one regional agency which was contacted separately. The questionnaire contained 12 questions which appear below together with summaries of the responses. IA Question 1. If your government/agency received assistance from the World Bank Trust Fund for Statistical Capacity Building (TFSCB) for the formulation of a National Strategy for the Development of Statistics (NSDS), what was your main motivation for wanting to prepare such a strategy? Of the 29 countries participating in the survey two countries had not received grants for developing national strategies but for other statistical capacity building purposes. The answers of the 27 countries to this question of motivation were mainly twofold; to improve coordination and strengthen the national statistical system was cited by 13 countries while 10 countries mentioned the need to establish a comprehensive strategy or a national plan for statistics. In fact, the difference between these two formulations may not be all that marked, which may indicate that the countries have embraced the stated policy of the MAPS to establish a national strategy for the statistical systems of the developing countries. Two countries mentioned improving statistics for the Millennium Development Goals (MDG) and poverty statistics as their main motivation, while another two countries stated that their main reason for preparing a strategy was to improve informed decision making. IA Question 2. If your agency received assistance from the TFSCB for statistical capacity building, what were the main needs you wanted to address? The 29 countries participating in the survey had all received or were about to receive TFSCB grants. Of these, 26 had received grants for establishing an NSDS for their respective statistical systems, one was about to embark on such a project and two countries had received grants for other capacity building purposes. Several countries mentioned many needs which they wanted to address; hence classifying the needs may not be very accurate. The need to develop a national strategy was cited by 7 countries and an equal number mentioned the development of skills and training as the main need they wanted to address. This second area seems closely connected to another one mentioned by 5 countries – to strengthen institutional capacities and human resources. Another 5 countries were preoccupied with improving the national statistical system through revealing weaknesses in the system and defining priorities. Other needs for which the countries had received grants to address were strengthening national accounts, special projects in poverty statistics and archiving of micro data from surveys. 38 IA Question 3: How do you judge the quality of the input funded from the TFSCB (consultants, support from Task Team Leader, others)? Responding agencies gave very high ratings to the quality of inputs funded from the TFSCB. They referred to the experts as excellently qualified, professional and supportive. There was also praise for TTLs, for their support and for making significant contributions. In one case the agency expressed satisfaction with the expert but found the resulting NSDS less than satisfactory. In two countries project implementation had not yet begun, hence it was too early to answer this question. IA Question 4: How smooth and timely was the approval process? Generally, the responding agencies found the approval process to be smooth and acceptable. 13 of 29 responding agencies found the approval process to be very timely and smooth. Another 11 found the process satisfactory, but indicated some delays. Of these cases, at least two agencies mentioned that the World Bank country office staff helped facilitate the process. In three cases, the process was referred to as not smooth or too long. One of these agencies reported disagreement with the World Bank over the approach to take and complained that the waiting time for the non-objection was too long. In the two other cases the delays were due to long government approval procedures. Two responses were not applicable. IA Question 5: How smooth and timely was the grant agreement process in your country/agency? Generally, the responding agencies found the approval process to be satisfactory and timely. 10 of 29 respondents found the grant approval procedures to be very smooth and timely, and another 14 indicated that they were satisfactory. Again, several pointed to assistance from the World Bank country offices as very helpful. In one case the agency found the delay was acceptable as it was due to the government‟s approval procedures. Long delays had been experienced in two cases due to the situation in the country at the time, or due to complicated procedures. One case was unexplained. It should be noted that the responses to this question do not reflect whether the project was executed by the recipient or the World Bank. IA Question 6: How smooth and timely was the implementation process (disbursement of funds)? Again, the responses do not distinguish between recipient and World Bank executed projects. About half of the respondents indicated that they found the implementation process timely and smooth, while a large minority (41%) felt that the implementation had suffered undue delays. In one case the agency felt it “disrupted” the project and in another it led to postponing the completion date. The reasons cited were mostly related to difficulties in working with the World Bank rules for procurement and disbursement; the agencies found them cumbersome and time consuming, or didn‟t have sufficient knowledge of the rules. In one case, submission of disbursement requests had to be redone due to the Bank‟s rejection and request for more documentation. 39 Only two respondents referred to problems in the government: in one country the accountant retired and it took a long time to recruit a new one and for her to understand the rules. In another case there was a problem in synchronizing the World Bank rules with the government requirements. IA Question 7: Which other agencies in your country are/were involved in NSDS formulation, and how? A number of countries responding to this question discussed only the financial and technical support received in the formulation of their national strategy for the development of statistics (or equivalent instrument). However, a large majority of the remaining countries indicated that a wide range of agencies were involved in the formulation of the NSDS. In many of these cases it appears that all relevant agencies of central government – essentially, the whole national statistical system – were involved in the formulation process; in some other countries a smaller number of key government institutions participated in the formulation of a strategy, with the outcome shared with other agencies at or towards the end of the process. Agencies most commonly mentioned specifically were the Ministry of Education, Ministry of Health, Central Bank, Ministry of Agriculture, and Ministry of Planning. In a few cases consultations with local or regional (sub-national) bodies were also reported. Several countries made particular reference to the participation of non-governmental actors, including universities, chambers of commerce, non-governmental organizations, and international development partners, either in the strategic formulation process itself or on a consultative basis. In only one case was the national statistical office alone responsible for formulating the NSDS, although even here it was anticipated that all the main producers and users of statistics in the country would be involved in its implementation. Just fewer than half the responding countries described how agencies had been involved in the NSDS formulation process. The commonest means reported was through consultative workshops, validation meetings, seminars and similar gatherings. Unsurprisingly, almost all countries used this method of involving stakeholders; however, in some there was a series of meetings throughout the process, while in others a single seminar to present the final outcome was convened. It was clear that some countries had made considerable efforts to implement a participatory approach at all stages of formulation, including diagnostic work, defining the mission and vision of the national statistical system, drawing up a road map, and elaborating an action plan. Countries used various mechanisms to steer the formulation process, including establishing technical working groups and design teams, in at least one case Cabinet-appointed. In one country, an NSDS coordinating cell was created, while in two others various ministries and agencies developed their own strategic plans, which constituted building blocks for the national statistical development plan. IA Question 8: If the NSDS is still under preparation: 8.a: When do you expect to complete the NSDS? 40 In most cases the NSDS had already been completed. Where the process was ongoing, completion was expected in late 2009 or 2010. The mean time from approval of the TFSCB project to expected completion of the NSDS was some 39 months, underlining the long gestation period of such strategic exercises. 8.b: How do you plan to use it? Where the NSDS was still under construction, countries generally had the intention to use it as the basis for the development and improvement of the national statistical system; and more specifically, to coordinate and harmonize data collection and dissemination within the NSS so as to minimize duplication of efforts and avoid unnecessary waste of resources. It was also envisaged that the strategy would be an instrument to help mobilize funds for statistical activities. IA Question 9: If the NSDS has been completed: 9.a: Are you satisfied with the resulting NSDS? A majority of implementing agencies responding to the questionnaire have completed an NSDS. These implementing agencies were unanimous in expressing satisfaction with the resulting strategy, although in one case there was recognition that a review would be necessary to take account of new government entities and civil society organizations, and in another, problems were foreseen because resource mobilization did not proceed as planned. 9.b: Have you started implementing the NSDS? Of the responding countries which have completed an NSDS, all but two have commenced its implementation, although in a couple of cases the scale of activities was limited. Of the two countries where implementation had not yet started, one was waiting for Cabinet approval while the other cited the need for additional resources before implementation could commence. 9.c: In which stage of implementation are you at this moment? As expected, countries which were already implementing an NSDS were at various stages of the process. Current activities cited by more than one country included advocacy to secure internal and external commitment to the strategy, resource mobilization, the formulation of further projects (intended for either TFSCB or other funding) to advance development of the national statistical system, and the preparation of statistical legislation to underpin the NSDS. 9.d: In your opinion, what are the major factors that might affect implementation of the NSDS? Countries responding described a wide range of factors which might affect, or in some cases already were affecting, implementation of the NSDS. However, there were several recurring themes, the commonest by far being financial resources; two-thirds of the countries that responded cited potential or actual limitations in financial resources as an influencing factor. Needs for additional human resources and in one case a threatened reduction in NSS personnel were also highlighted, while some countries mentioned that NSDS implementation would be affected by the 41 development of human capacities in statistics. Weak statistical infrastructure, for instance in IT, was also cited as a hindrance. In a couple of cases, implementation was seen to depend to some extent on success in approaching partners for technical assistance. Resource-related issues were, however, by no means the only ones which were seen to impinge on the successful implementation of a national strategy for the development of statistics. In this context a number of countries cited the need for commitment and motivation on the part of institutions such as line ministries if the strategy were to be successfully realized. Necessary changes to existing statistical legislation were also seen as important in certain countries. A small number of countries mentioned overall political instability as a factor threatening successful strategy implementation, adversely affecting both domestic statistical activities and support from development partners. IA Question 10: What are the advantages of having a strategy? What were/are some of the positive/negative results from the planning exercise? Responding countries provided a very rich and varied description of the advantages that were seen to flow from the formulation and existence of a national strategy for the development of statistics. Many of these advantages stemmed from the fact that, often for the first time, an opportunity was provided to plan collectively for a national statistical system (NSS) and to create a clear vision of the goals and tasks that that involved. By focusing stakeholders on a single purpose, corporate responsibility for and ownership of the NSS – indeed in some cases the very concept of such a system – was promoted. The existence of a strategy facilitated not just the creation or strengthening of an NSS on paper, but one which was variously described as functioning, credible, well-organized, efficient, capable of producing quality statistics and responsive to national and international needs. As one country put it, the strategy was the cornerstone on which to build the national statistical system. Countries reported that an NSDS served as a concerted and coordinated framework for a programme of statistical activities for the medium term, improving coordination and rationalizing existing capacities. It facilitated the systematic organization of work in terms of particular statistical products. By defining the roles and responsibilities of data providers, it helped ensure that the NSS had comprehensive coverage and that duplication was eliminated. It was also noted that the existence of a strategy facilitated implementation, monitoring and evaluation of activities, helping to measure progress in statistical development as well as tracking of development policy outcomes. Facilitation of prioritization was another theme which ran through the responses. Advantages of the strategic process were seen in the establishment of statistical priorities with the full participation of producers and users of data, and the identification and prioritization of strategic directions for the statistical system, linked to medium-term development targets. An NSDS also permitted the prioritization and identification of the necessary resources to meet identified goals, and helped maximize the utility of available resources. On a practical level, a strategy provided a document which could be used in negotiating for external resources to fund statistical projects, as well as in facilitating the mobilization of domestic resources. 42 Countries also described other tangible benefits from having a strategy, including an improved statistical information system, the production and dissemination of higher quality data, and the availability of reliable and up-to-date indicators. It was also reported that the strategy had resulted in improved public access to the data being produced. Although the linkage between the NSDS and national development priorities was not always clearly articulated in the responses, one country cited as an advantage the fact that the national development programme was based on reliable data. In addition, when the NSDS was conceived of as a strategic action within the national development plan, it was viewed as constituting a political commitment. As to the results flowing from the planning exercise, countries which specifically addressed this question reported in very positive terms. In general, the planning exercise had helped to lay down some ground rules for building the national statistical system, and had strengthened the relationships among its stakeholders and partners. In one instance the planning exercise made it possible to set up a network among statistical units of line ministries. Analysis of strengths and weaknesses during the process of strategy formulation helped in the identification of areas which were subsequently the subject of statistical capacity building, including training and the development of human resources to meet national requirements. The responses also indicated that the planning exercise had provided the opportunity for some national statistical offices to reflect on and strengthen their role, and to increase their credibility within society. When appropriately conducted, the planning process was also seen to make buy-in and support from policy makers virtually guaranteed. IA Question 11: Did the assistance achieve the expected results? Are they sustainable? Of the 29 countries which participated in the survey 27 answered the question on the results achieved. The project had not started in one of the countries and in another it had been concluded but implementation had not begun. The large majority of the countries answering the question stated that the expected results had been achieved. There were several reasons expressed for this: that the NSDS had been achieved, that human resource capacity had been improved, and that the desired strengthening of coordination of or the partnership within the national statistical system had been achieved. Only one country stated that the results had not been achieved and in two cases the answers seemed conditional or not fully clear. Only 17 countries gave explicit answers to the question on sustainability. Of these, 10 answered positively stating that they considered the results sustainable, a few of them referring to tools and capacity, both human and institutional, put in place. One country stated that although good the results were not sustainable, and four countries replied that sustainability was dependent on funding or on support for implementation. 43 IA Question 12: Are you likely to continue seeking assistance from the TFSCB, and if so, what would be the most important areas in which you would seek assistance from the TFSCB over the next 2-3 years? Nearly all the countries participating in the survey stated that they would like to continue to seek assistance from the TFSCB. Two countries stated that they were unlikely to do so and one country did not answer the question. As for the most important areas for further support the countries listed a variety of issues. The areas mentioned can be roughly divided into four classes: implementation of the NSDS or SMP, continuing or reinforcing statistical capacity building, development of infrastructure and subject matter or sectoral statistics. Of the first class, several countries mentioned that they would like to seek assistance, mainly funding but also technical assistance, to implement their new strategies or master plans. Often this was stated as undertaking the actions identified or prioritized by the NSDS or SMP. This is closely related to the previous question on the sustainability of the results achieved; there, several countries thought that the sustainability would depend on whether sufficient funding could be obtained for the implementation of the strategy. Several issues can be placed under the heading of continuing or reinforcing statistical capacity building. The one issue most often mentioned is training or increasing the skills of statistical staff. Management improvement was also mentioned by more than one country. One may also place under this heading the wish to seek funds for updating earlier NSDSs or SMPs. The heading of infrastructural development is quite broad and here the countries mentioned a variety of issues. Thus, legal and organizational issues were thought to be urgent tasks in several countries and institutional reform was also mentioned. Some countries mentioned investment in ICT and there was even a mention made of assistance for acquiring a new office building. As can be expected, under the fourth category the countries listed a number of different subject matter statistics or urgent issues relating to sectoral statistics. Perhaps not so surprisingly, there were some common concerns such as economic statistics, in particular national accounts or price statistics, poverty measurements and the related MDG indicators, and harnessing administrative data sources for statistical purposes. Most of the areas mentioned by the countries belong properly to statistical capacity building but are more related to the regular statistical activities of a NSO. Here, mention can in particular be made of conducting surveys or censuses. 44 Survey among Task Team Leaders (TTLs): Questions and summary of responses With the assistance of the Administration Unit of the Trust Fund, the Evaluation Team distributed a questionnaire to 37 TTLs on the operations and the outcomes of the Fund. Of these, 16 responded (43%), in some cases giving information about more than one project each. The questionnaire contained 12 questions which appear below together with summaries of the responses. TTL Question 1. To what extent do you think the concept of the Trust Fund (TF) meets the needs of statistical capacity building in developing countries? Nearly all responding TTLs (14) were of the opinion that the concept of the Fund meets the needs of the developing countries to a large extent. Two TTLs found the question to be not applicable as the projects they had worked on were global or regional. TTL Question 2. Do you think that the project for which you are task team leader fully reflects the priorities of the country receiving the grant, as expressed in its NSDS (if one exists)? Does implementation of the project involve the country authorities taking reform measures that would not have been undertaken in the absence of TF support? The first part of this question was answered by 13 TTLs who all answered in the affirmative, saying that they thought that the projects they have worked on fully reflected the priorities of the countries. Three TTLs did not find the question applicable in the case of the projects for which they were Task Team Leaders. It was not possible to summarize answers to the second part of the question, as it was interpreted in a variety of ways by the respondents. TTL Question 3. Who originated the idea of the project and who worked up the proposal? The replies of the TTLs indicated that in about an equal number of cases the idea of the project originated with the national authorities (government and NSO) or jointly between World Bank staff and the national authorities. Other replies gave the Bank staff alone and PARIS21 as originators. As for the preparation of the proposals, most of the TTLs who had this information reported that the proposal was developed jointly between the client and the TTL or other World Bank staff. TTL Question 4. How smooth and timely was the grant agreement process? Were other aspects of the process from conception to implementation of the project efficient? The replies were as follows: 45 9 TTLs reported that the grant agreement process had been smooth and timely, while 6 reported that it had been long and cumbersome. 2 cases were not applicable (the TTL did not know or was handling a global/regional project). The replies contained several specific remarks of interest. Thus, several respondents said that client approval of grant agreements took a long time; in one case it was mentioned that the Grant Funding Request system (GFR) had been efficient, and one TTL felt that the process should be simpler for small projects. Further examination of the positive replies revealed that 6 out of the 9 responses referred to Bank-executed projects (including one UNDP-executed project), while 5 out of the 6 negative responses related to recipient-executed projects. One reply referred to several projects. Since the grant activation time was specifically mentioned by several TTLs, the Team was able to identify and compare activation time for 7 projects referred to by TTLs in their responses. Of these 7 projects, 4 were mentioned by TTLs who responded positively to the efficiency of the grant agreement process; the median grant activation time for these 4 projects was 5 months. The remaining 3 projects featured in negative TTL responses; the median activation time here was 10 months. For comparison, the 9 projects reviewed by the Team during country visits, and for which dates were available, had a median grant activation time of 8 months. TTL Question 5. Has the process of TF disbursement gone smoothly? Of the 15 relevant projects which responding TTLs referred to, the process had been mainly positive in 11 cases and generally negative in the remaining 4. The main problems referred to in the replies were lack of capacity by the recipient authority and lack of knowledge of World Bank rules. Correlating the answers with project execution responsibilities revealed that the disbursement process was significantly smoother for Bank-executed projects. TTL Question 6. Has the technical assistance and equipment procured with the grant funds been of appropriate quality? For about two-thirds of the projects (12 out of 17) the TTLs replied that the technical assistance and equipment procured had been of good quality. In four instances the TTLs were not able to say as it was too early in the implementation process and in one case the TTL did not have full knowledge but thought the quality had been appropriate. TTL Question 7. Are there other donors or World Bank funds involved in the project? How easy has it been to coordinate these funds? Other donors or World Bank funds were involved in slightly over half the projects on which Task Team Leaders responded. Moreover, in a few cases where only TFSCB funds were used, the project served as a catalyst for other donors to contribute to follow-up or related activities. 46 Where other donors were involved, experience with the ease of coordination was mixed but positive experiences predominated. The point was made that coordination was more effective where the NSO itself had the capability to lead the coordination effort. TTL Question 8. In your experience, to what extent have the main improvements in the production and use of statistics that the project intended to bring about been achieved? Do you feel that these achievements will be sustainable? In over half the cases the TTLs were unable to respond definitively to this question, either because their projects were essentially NSDS/SMP exercises not aiming directly at the production or use of statistics, or because the projects were still underway. In the latter case, however, a number of TTLs expressed optimism that the intended improvements would be achieved. In relevant projects which had been completed, most TTLs reported (directly or indirectly) that the planned results in terms of production and use of statistics had been largely achieved. TTL Question 9. What are the main sorts of problems that you have been facing in implementing the project? (for example, inadequacy of government budgetary support; lack of coordination among agencies; lack of government commitment to an NSDS; etc) In about a third of the cases the TTLs reported that there had not been any major problems, while the remaining two-thirds replied that there had been a variety of problems, mostly of an administrative nature. Most common of these concerned poor capacities in government to execute the project, including ability to handle financial matters (reported in 6 out of 9 cases); other administrative problems included government budget procedures or lack of government budget for statistics, lack of ability of the project to top up NSO salaries, World Bank rules, and lack of World Bank supervision. Other problems mentioned included lack of human resources and IT staff at the NSO. TTL Question 10. Do you believe that the TF project and grant have contributed to increasing support from other external sources for statistical development in the country, whether directly in support of the TF-assisted project or more generally? The response to this question was quite clear-cut; for 9 out of 13 relevant projects the TTLs reported that the Trust Fund grants had helped to attract more external funding. In the remaining 4 cases the TTLs noted that the NSDS was expected to do that in the future or even that there were funds already lined up. Some TTLs mentioned specific donors like the European Union (EU), African Development Bank (AfDB), United Nations Economic Commission for Africa (UNECA), Japan International Cooperation Agency (JICA) and the Food and Agriculture Organization of the United Nations (FAO), and also funding from other World Bank projects. One TTL said that the TFSCB project had a catalytic effect and helped give legitimacy to proposed projects. 47 TTL Question 11. Do you think that the Trust Fund resources have displaced government spending on statistics? All the TTLs who reported on grants to which this question was relevant stated that the grants had not displaced government spending on statistics. Of comments made, one TTL noted that the government was at this stage not funding statistics in a meaningful manner; another that the grants had been a very small share of government spending on statistics. In other cases it was noted that the grants had not displaced funding but rather filled gaps, while in one instance the project was thought to have promoted increased government spending on statistics. TTL Question 12. On the closure of the project, how has it been evaluated and reported on? Of the 13 projects relevant here, 7 had not yet been closed. In one case there had been a completion report, in three cases the project had been evaluated by government (auditor‟s report, in one case with regard to output rather than impact) and in two cases there had been no evaluation. Of the comments made, two referred to the small size of the projects to explain why there had not been a formal evaluation. 48 Questions for Members of the Internal Management Committee 1. Based on your experience, have there been improvements / changes over time in  the quality of the projects reviewed by the Committee? a) NSDS projects b) non-NSDS projects  the efficiency of servicing of IMC meetings and procedures?  the processes of project implementation and disbursement of funds?  how well country priorities are reflected in the Fund‟s activities?  how well non-NSDS projects adhere to priorities in countries‟ NSDS or Statistical Master Plan?  the ability of the Fund to leverage further funding for statistical capacity building, through STATCAP or from other donors? 2. In which of the following areas, if any, do you think there is room for significant improvement in the Trust Fund‟s operations?  Type of project execution, from the standpoint of both efficiency of project management and administrative burden  Efficient use of resources to achieve outputs  Adequacy of built-in evaluation at project level  Overall adequacy of Fund resources  Broadening of the donor base  Balance of projects among geographic regions and among subject areas 3. The share of funds going to global and regional projects has been increasing. Do you see this trend as in line with the philosophy and Guidelines of the Trust Fund? 4. What are your views about the Trust Fund‟s financing of participation in international conferences and meetings? 5. Do you feel any need for changes in the composition of the IMC or in its working methods? 6. Do you find the work and reports of the Advisory Panel useful? 7. Should the IMC play a greater role in the assessment and follow-up of projects? 8. Given that the Fund has been operating an NSDS window through which a majority of its funds have been disbursed, and that many countries have already embarked on the NSDS process, what are your views on a possible reorientation of the Fund? 9. What impact will the new SRF facility have on the Fund‟s operations? 10. Will the new environment in which the Fund will be operating call for changes in its governance structure? Should the links to PARIS21 be reviewed? 49 Questions for Trust Fund Donors and other Consultative Group members 1. Are you generally satisfied with the evolution of the Trust Fund and its work? 2. Do the financial supporters of the Trust Fund get adequate reports on its activities? 3. Do you think that Trust Fund activities have contributed to improved donor coordination at international or country level? 4. To what extent, in your opinion, have the projects that the Trust Fund has supported contributed to building statistical capacity in the recipient countries? 5. What are your views about the Trust Fund‟s financing of participation in international conferences and meetings? 6. The projects that the Trust Fund supports are intended to be catalytic, with follow- up activities assimilated into donor-funded (or domestically financed) country strategies. To what extent is this happening, in your experience? 7. Is the current governance structure of the Trust Fund appropriate and effective for achieving its objectives? 8. Do the meetings of the Consultative Group have appropriate representation of the stakeholders whom you believe should be involved at that level? [Questions for donors only] 9. Do you feel that your views as a donor are adequately represented through the existing governance arrangements? 10. The current tranche of the Trust Fund (TFSCB III) is scheduled to close on 31 December 2012. Do you envisage that your agency will a) make further contributions to the current tranche b) contribute to any further tranche of the Trust Fund, if one is established? 11. The Trust Fund has been operating an NSDS window through which a majority of its funds have been disbursed. By now, many countries have already embarked on the NSDS process. In addition, the Statistics for Results facility (SRF) has recently been launched. In your view, what impact will the new environment in which the Fund will be operating have on your agency‟s contributions? 50 Annex 2 Reports on Missions undertaken by the Evaluation Team During the course of its work, the Evaluation Team visited five countries and attended the 57th session of the International Statistical Institute (ISI) in Durban, South Africa. The Team‟s travel schedule is shown below, along with a listing of short mission reports. Each country report covers the following:  The background to the statistical environment  A description of the TFSCB grants received  Issues arising in the acquisition and implementation of the grants  The perceived impact of the grants in each country  The likely sustainability of the TFSCB‟s interventions The brief reports from the ISI session in Durban are based on the interviews the Team conducted with representatives of national statistical offices and international and regional agencies. Persons met during the country visits and ISI conference are listed in Annex 3. Page Uruguay 3 - 4 August 2009 52 Paraguay 6 – 7 August 2009 56 ISI, Durban 16 – 22 August 2009 59 Mongolia 3 – 4 September 2009 66 Cambodia 7 – 8 September 2009 69 Indonesia 10 – 11 September 2009 72 51 Uruguay (3–4 August 2009) Background The statistical system in Uruguay had traditionally been considered as advanced by developing country standards. Led by the Instituto Nacional de Estadística (INE) 23, it was regarded as a frontrunner in Latin America in terms of quality, relevance and comprehensiveness. However, from the mid 1990s a combination of budgetary pressures, a public sector hiring freeze, and poor salaries within a compressed salary structure, resulted in an erosion of INE‟s capacity and the loss of many qualified technical staff. The human resource base of the institution deteriorated, along with its capacity to respond to new challenges and its ability to lead and coordinate the statistical system as a whole. By 2005 INE was concentrating on routine data collection activities, to the detriment of research activities, client orientation, infrastructural maintenance, and methodological advances. A change of government in early 2005 provided the opportunity and indeed the imperative to redress the stagnation in statistical development of the previous decade. New governmental programmes in the social field created fresh demands on the national statistical system, including an effective monitoring and evaluation system. INE attempted to respond to the new policy environment, although initially the extent to which changes could be made was hampered by continued budgetary limitations, hiring constraints, and salary restrictions. The weak coordinating role of INE and the fragmented nature of the statistical system also needed to be addressed. TFSCB-funded Projects No. of TFSCB Government Other Purposes projects resources funding funding NSDS, In-kind 2 US$430,840 - SCB contributions 1. Support to Develop a Statistical Master Plan (TF056232, budget $83,640) Against the above background, the first project was drawn up in 2005 to help develop a Statistical Master Plan, presented as the first stage of a national strategy for the development of statistics. The proposal aimed to cover the entire range of statistics produced within the country; it was designed to improve inter-agency coordination and give new direction to the national statistical system. It was to include a detailed assessment of the system, including the legal framework, needs for statistics in various areas, and coordination mechanisms; and the resulting master plan was to identify priority actions to address the main perceived weaknesses, and provide recommendations on redesigning the national statistical system. Apart from the Plan itself, the project was to include a workshop which would provide a broad opportunity to discuss the Master Plan and help clarify responsibilities in its implementation between INE and other agencies within the national statistical system. The project was executed by the World Bank with a TFSCB budget of $83,640 and was completed on schedule on 31 March 2007, with virtually all funds disbursed. 23 National Institute of Statistics 52 2. Foundations for Statistical Development (TF058063, budget $347,200) This second project followed on in very large measure from the first. The Statistical Master Plan was finalized in July 2006; its principal objectives were to improve compliance with demand for statistics through an enhanced information system, expand and improve the statistical production system, and restructure the organization and management of INE and the national statistical system. The second Trust Fund project, drawn up in October 2006, sought to assist the implementation of the SMP in four key areas:  development of the economic statistics of INE  improving educational statistics  consolidating health statistics  strengthening the organization and management of INE The project was executed by INE with a TFSCB budget of $347,200 and significant in-kind government resources, and was scheduled to close on 31 December 2008; it eventually closed six months later, also with a 100 per cent disbursement rate. Process and Management Issues The Team found that the first project had been delivered within its original timetable, while the second had to be extended for a few months. As to process issues associated with the Trust Fund grants, it appeared that broadly speaking, the experience in Uruguay has been very satisfactory. Contracts and procurement worked smoothly, and the Bank had been very responsive and quick in its dealings with INE. For the second project, where the funds had been managed by UNDP, the experience was also good. The consultants who worked on the projects were universally praised as excellent, being knowledgeable, showing good project management and leadership, and playing an important role through links and contacts with the various institutions involved. In many cases the consultants had helped to erode resistance to innovation and improve cooperation within and among the institutions concerned. The Team noted that institutional constraints loomed large throughout the implementation of the TFSCB-funded projects, to some extent blunting the progress made. In particular, institutional obstacles and rigidities in civil service remuneration and personnel policies frustrated the amount of progress that could be made at INE, and forced discontinuation of the exercise to restructure INE as an autonomous institution. Dysfunctional institutional arrangements were also reported in the fields of education and health statistics. The Team heard that there was resistance to change and innovation both at INE and in other institutions. Coordination was problematic between INE and institutions like the Central Bank and the Ministry of Health – indeed the first project revealed weaknesses in the coordination capacity of INE. A delay by the latter Ministry in the recruitment of a suitable consultant held up progress on the health component of the second project, with INE eventually having to step in. The Team judged that overall experience with implementation of the Trust Fund projects in Uruguay has been good, although there have been problems with the education and health components in the second (Foundations for Statistical Development) project. 53 Achievements and Impact The Team felt that the TFSCB projects had registered significant achievements despite the constraints encountered, and indeed had succeeded in ameliorating the situation with regard to areas such as resistance to change and insufficient coordination. There was general agreement that the projects had strengthened INE; statistical capacity had improved; enhancements were noted in the quality of information and in the coverage of the statistical system; more statistics are being published, and utilization of statistics in Uruguay is increasing. Greater demand for statistics has been created, and contacts with users have increased. Micro data are being disseminated for the first time, and the second project had helped INE to commence integrating the results of economic surveys, starting to overcome previous compartmentalization. The first project to develop a Statistical Master Plan was reported to have produced output that was very important, providing an excellent guide map for future developmental work. The Team heard that the preliminary teamwork in preparing the SMP was fundamental, and that as a result of the process a number of institutions felt themselves to be part of the national statistical system for the first time. Apart from the achievements in INE, progress was reported in other fields such as education statistics which would not have been possible without the TFSCB‟s contributions. The Team felt that as a result of the projects INE now seems to be in a much stronger position to lead statistical development in Uruguay, with progress and activities being based on the SMP/NSDS. There had been an increase in professionalism at INE (though this needed consolidating), and staff in the field of economic statistics have been demanding training in order to improve their skills. The Team noted that although the funds granted under the projects were very modest, they sent an important signal to the institutions involved in producing and using statistics in Uruguay, and the projects have caused new thinking about the statistical system. There was consensus that significant improvement had already been made (although much more is needed), and that a framework for further progress had been created. The Team also noted that the Government counterpart contribution for the second project dwarfed the TFSCB input. Moreover the Ministry of Economy and Finance had allotted significant additional funds to INE (from $4 million in 2005 to $10-11 million now), although the Institute had not been able to spend all the allocation due mainly to the institutional constraints it was operating under. Sustainability Although the picture was mixed, the Team was on balance optimistic that the gains made would be largely sustainable. On the positive side, the INE budget has been increased and apparently there is no danger that the process will be reversed. INE itself seems reasonably optimistic of Government funding for SMP/NSDS activities up to 2010, with 2011 less certain. It was already utilizing World Bank funds to put together training courses on time series, econometric analysis etc, and indicated that it would be seeking further assistance from the Bank and other donors in helping it develop coordination capacity (including the legal basis for coordination), and with the 2010 censuses. Outside INE some elements of sustainability are also present, for example the registration of students is continuing after the end of the projects. 54 However, it is recognized that an enlarged national statistical system will need a lot of resources, and outside expertise will be needed to help build further capacity in INE and maintain the momentum of activities. By its own admission, INE still lacks the institutional strength and visibility to fully carry out its coordination role within the national statistical system. Moreover, the Team heard concerns expressed about, inter alia, the Institute‟s ability to retain qualified staff, especially in IT. The Team also understood that problems with civil service salaries and structure were likely to persist. 55 Paraguay (6–7 August 2009) Background Statistics in Paraguay are coordinated by the Dirección General de Estadística, Encuestas y Censos24 (DGEEC), while the Banco Central del Paraguay25 (BCP) leads the work in economic statistics. The statistical system includes some 50 data producing agencies. Statistics production is regarded as fragmented, without common standards or harmonized approaches. The current organizational structure of statistical services was established under a law from 1942, placing the DGEEC under the Technical Secretariat of Planning - an office within the President‟s Office. A new law to establish an autonomous National Institute of Statistics did not materialize. The DGEEC depends to a large extent on external funding for its activities and has successfully used such resources to expand its program and improve its human resource capacity. Its permanent staff strength is about 75 of whom 25 are professional staff. DGEEC also hires some 100 persons for specific ad hoc activities. Paraguay has participated in the General Data Dissemination System (GDDS) since 2001. A population census is planned for 2012. TFSCB-funded projects No. of TFSCB Government Other Purposes projects resources funding funding NSDS, In-kind 3 US$701,450 - SCB contributions 1. Statistical Capacity Building in Economic and Social Data (TF050306, budget $200,000) The first TFSCB project for Paraguay, approved in 2001, was designed to complement the activities of a large-scale regional technical assistance program entitled “Improving the Surveys and Measurement of Living Conditions in Latin America and the Caribbean” known as MECOVI. 26 The project‟s main objective was to enable DGEEC to conduct the 2002 household survey, while simultaneously train DGEEC staff in survey methodologies and other technical topics. A second project component aimed to produce statistical information on emerging social topics by including an additional module to the household survey; and a third component promoted the analysis of these data and their dissemination. The project was executed by DGEEC and completed in March 2005 (some 11 months later than planned) with a disbursement rate of 87 per cent. 24 General Directorate of Statistics, Surveys and Censuses 25 Central Bank of Paraguay 26 Spanish acronym for “Mejoramiento de las encuestas de las condiciones de vida”. The MECOVI program was jointly developed and sponsored by the Inter-American Development Bank (IADB), the World Bank and the Economic Commission for Latin America and the Caribbean (ECLAC), and ran from 1997 to 2004. 56 2. Paraguay National Strategy for the Development of Statistics (TF091481, budget $103,000) This project aims to produce a NSDS for Paraguay. While the project was not yet operational at the time of the Team‟s visit, t he DGEEC expects that the diagnostic consultations among stakeholders will help clarify the needs and interests of producers and users within the statistical system. Likewise, it is hoped that the intended joint formulation process of the strategy will clarify responsibilities and coordination mechanisms resulting in a focused strategy of realistic scope, including formal recognition of DGEEC‟s coordinating role. A steering committee and a technical committee will oversee the process, and the statistical staff of the BCP, whose representatives fully supported the project, will be close partners in its implementation. The NSDS project was approved by the TFSCB Internal Management Committee on 27 November 2007. In accordance with Paraguayan regulations, the project must be approved by the Paraguayan Congress which only happened some 20 months later in July 2009. The DGEEC was planning to start activities in the third quarter of 2009 and complete the NSDS in 2010. Looking ahead to the implementation of the NSDS, DGEEC has already initiated contacts with prospective donors and partners. The DGEEC also contacted the Ministry of Finance on the possibility of STATCAP support. 3. Building Statistical Capacity for Better Economic Evidence-based Policymaking in Paraguay (TF093518, budget $398,450) Approved in 2008, this project aims to improve the national accounts by introducing changes in three areas of the methodology, namely: (i) incorporating the value added of two bi-national hydroelectric power stations into the national accounts,27 (ii) designing an integrated system of household surveys, and (iii) designing institutional accounts consistent with the 1993 System of National Accounts. The project will be implemented by the BCP in cooperation with the DGEEC. Project implementation is pending approval by the Paraguayan Congress. In the meantime, the BCP and the DGEEC are planning an economic census in 2011 mainly funded by a loan from the Inter-American Development Bank (IADB). With regard to regional TFSCB projects, the BCP confirmed receipt of an invitation to a training workshop on national accounts conducted by the Economic Commission for Latin America and the Caribbean (ECLAC). Process and Management Issues Long delays between World Bank approval and start of implementation were experienced for all three projects due to the required Congressional approval of external loans and grants and their subsequent inclusion in the budget documentation. These delays necessitated extensions of the projects. However, once approved, the first project was only a little delayed due to the 2002 population census. The DGEEC reported that valuable assistance was received from the TTL and from the World 27 Estimates suggest that inclusion of the value added generated by the two power stations would increase Paraguay‟s GDP substantially, even at the current low prices received by Paraguay for electricity. 57 Bank country office for all disbursement and procurement procedures. The DGEEC had dedicated a special unit to be responsible for all financial and administrative matters, which they plan to do again for the two other projects. Achievements and Impact The MECOVI program including the TFSCB project was considered highly successful by the DGEEC. It was regarded as bringing about a turning point in the DGEEC‟s development by bu ilding core competencies of some 15 staff members. DGEEC considers its ability to plan, design and conduct surveys to be vastly improved thanks to the MECOVI and TFSCB assistance. The Team was of the view that the TFSCB project was so successful in part because it built on and amplified the results of a large program to which it was associated. Also as a result of the first project, the DGEEC generated surveys of the Guarani indigenous population and the Mennonite colonists, allowing further analysis of the social factors determining the development of their respective communities. Furthermore project-funded national researchers produced a series of articles published in the journal “Economy and Society” and DGEEC published a newsletter on basic statistics targeting members of Congress. However, DGEEC was unable to continue funding of the research and the newsletter. Sustainability The training efforts of the first TFSCB project and the MECOVI program proved sustainable as all 15 trained staff remained in DGEEC‟s service. In fact, all senior staff “graduated” from this program. The attempts to introduce analysis to statistics were useful but not sustainable. Analytical skills could not be transferred to DGEEC ‟s permanent staff, and DGEEC could not mobilize domestic budget resources to retain the researchers or continue the activities with its own staff. The newsletter for Congress members was appreciated but DGEEC also found that a simpler format with more explanation would better meet the needs of the target audience. The Team found that the DGEEC demonstrated a high degree of ownership of the NSDS activities, which is a positive starting point. The good working relationship between the DGEEC and the BCP is conducive to good coordination of national accounts. In the longer term, it is also positive that the BCP intends to gradually transfer some work on the national accounts to the DGEEC. It is evident that domestic budget constraints as well as the outdated organizational structure of statistics hamper the stable development of DGEEC, as reflected in the use of more ad hoc than permanent staff to undertake its regular work program. Both DGEEC and the BCP expressed the view that good statistics is a national responsibility and that reliance on external resources is not sustainable. 58 International Statistical Institute conference, Durban (16 - 22 August 2009) Meetings with country representatives Gabon The Direction Générale des Statistiques et Etudes Economiques (DGSEE) in Gabon has been granted funding by the TFSCB for one project entitled: “Statistical capacity building” with a TFSCB grant of $400,000 and counterpart contribution of $177,000, totalling $577,000, approved in August 2007. The idea of applying for a Trust Fund grant originated from discussions with PARIS21. The project had three components: (a) formulating an NSDS, (b) training, and (c) extension of a database for social data. The formulation of the NSDS followed the guidelines of PARIS21 and up-to-date methodological standards. Currently, there is ongoing work on translating the draft plan of the NSDS into a detailed plan of action for the years 2010-2014. A donors meeting was being planned for late 2009. The political situation is reported to have delayed the process by some months. With respect to the training component, the training is ongoing and more is needed, but the funds for this purpose are exhausted. The third component has been problematic and delayed due to staffing problems (one key IT staff member had left the service and another was undertaking an internship elsewhere). Hence, an extension may be required to complete this work. Regarding the approval process, the country representative explained that the finalization of the grant agreement was delayed as the World Bank had initially felt that the DGSEE could not administer the project funds; after lengthy exchanges to agree on a procedure the World Bank changed its system of management, creating further delays. However, the matter was resolved and the administration of the funds is now handled by an independent accounting company; this arrangement is running satisfactorily. Ghana Ghana received two grants for the NSDS and related activities in 2005 and 2006 totalling almost $400,000. There was supposed to be only one TFSCB project, but it evolved into two as it was easier to get the first one approved. Both projects covered support to the development of a corporate plan for the Ghana Statistical Service (GSS). DFID assisted in preparing the TFSCB submission – at that time the GSS lacked a real programme (including a survey programme) as well as direction. The total project amounted to $4 million of which the TFSCB provided the $400,000 mentioned above. The TFSCB contributions, although only a small part of the total nominal project budgets, reportedly made a lot of difference in, for example, the analysis of data, dissemination, capacity building in economic statistics, and improving the image of the GSS. They had also assisted in informing and orienting the Statistical Service Board whose 7 members were taken on a study tour to see what other countries had done to improve conditions of service. 59 The Corporate Plan has been completed, and discussions are in train with partners to operationalize it – Eurostat, DFID and some other donors are already on board. The attitude in the Ministry of Finance remains sympathetic, but the government is in severe financial difficulties and is currently unable to deliver the counterpart funds. Ghana will be an SRF pilot, to the tune of around $10 million over 5 years. It is also thinking about an IDA credit, and about tapping funds under HIPC (Heavily Indebted Poor Countries Initiative). The real value of the TFSCB was said to be to put the GSS on the path towards statistical development; through the Fund, a unit to deal with program matters had been established. Based on the strategic plan, donors are approaching the Vice President to lobby for regular budgetary support from government. The plan has helped foster good coordination and multi-donor support. The World Bank office has been very helpful. After some initial exchanges before the project started, accessing the funds was straightforward and technical support was good. Lesotho Lesotho had participated in two regional projects, namely: “Statistical capacity building for poverty reduction strategies”, phases I and II, in 2002-2006, implemented by the Southern African Development Community (SADC), and “Building Capacity for Poverty Analysis” for Botswana, Lesotho, Namibia and Swazila nd, in 2006-2008, implemented by the World Bank. 28 The lasting outcome of the SADC-implemented project had been limited. But the second project is reported to have been very useful. Project activities were guided by a Steering Committee. Under the project, a workshop on poverty analysis was conducted, and Lesotho hosted another useful workshop on data archiving. In 2009, a tailor-made workshop was conducted on how to use census and labour survey data. The NSO was fully satisfied with the manner in which the project was implemented and executed by the World Bank TTL. With regard to formulation of an NSDS, the NSO is of the opinion that the AfDB may be better equipped to guide this process than the World Bank. An NSDS had been formulated covering 10 sectors. However, the finalization would need to await the completion of a new National Development Plan, which should integrate NSDS and a new PRSP. In the meantime, the NSO hopes to proceed with implementing parts of the existing NSDS. Malawi Malawi has not availed itself of a TFSCB grant but has received substantial financing from donors, in particular the World Bank, DFID and the Government of Norway (lead donor). The Malawi statistical authorities have drawn up an NSDS, first for the NSO, but then also for the whole national statistical system (NSS). The NSS plan is now being implemented in cooperation between several institutions producing statistics, with the NSO as the coordinating agency. 28 Lesotho was included in the GDDS project, which in its first phase was funded from the TFSCB. 60 Nigeria Nigeria received two TFSCB grants in 2003 and 2006 ($385,000 in total) for NSDS and one grant from the SCB window in 2008 ($325,000). The first grant was used for an NSDS at the federal level and the second for similar strategies at the level of three states. The SCB grant was for strengthening capacity in the three pilot states. Partly as a result of this, the statistical system has undergone some important developments. The National Bureau of Statistics (NBS) was moved from Lagos to Abuja in 2006, which greatly enhanced its operational abilities. This was a part of the NSDS implementation. A new statistical law was also enacted, again as foreseen by the NSDS (or Master Plan as it is called). The NBS coordinates the NSS at the level of ministries (federal level) and at the level of the states. Among activities which have been prioritized are the compilation of a compendium of statistics to be used at all levels (concepts, procedures, classifications, methods etc.), the creation of a common database (access for cooperating institutions to micro data through a portal), training, workshops and awareness courses for ministries, states and other stakeholders. Apart from the World Bank, there have been two major donors contributing to statistical capacity building, DFID and the EU. This year, a development program is in place. Philippines The Philippines has been an active client of the TFSCB, with four grants received to date and another two projects rejected. The first project (grant amount $376,900, approved in March 2001) was on statistical capacity building in the rural sector, implemented by the Bureau of Agricultural Statistics between 2001 and 2005. This followed a World Bank initiative to develop a rural information system and was utilized as an opportunity to produce community-based data down to the barangay level. During this project, problems were experienced with consultancies, which had not been a straightforward process, and with drawn-out procurement procedures which had led to an extension of the project. The project had, however, made some lasting impact, as parts of the National Rural Sector Statistical Information System and Community-Level Statistical Information System created under the project are still functioning. The National Statistical Coordination Board implemented the following two projects focused on poverty estimates. The first of these was entitled “Strengthening Statistical Capacity in the Philippines focusing on Poverty and Socio-Economic Indicators” (grant amount $47,500, approved in October 2002). The other project was entitled “Strengthening Capacity in Priority Sectors” (grant amount $100,000 , approved in December 2005). The results of these are still being used by policy makers. The fourth project, entitled “Improving the Quality and Usefulness of the Philippine System of National Accounts” (grant amount $400,000) is being implemented in close collaboration between the Board and the National Statistical Office. In this project – as with the foregoing ones – efforts are being made to institutionalize the improvements achieved. A good relationship with the TTL has helped obviate any process problems. Possible future applications to the TFSCB might concern monitoring and assessment of the existing Philippine Statistical Development Plan. 61 Russia The Russian authorities and the World Bank have in recent years agreed on cooperation on statistical capacity building. The first tranche of World Bank funding took place in 2000-2006 followed by an agreement that the TFSCB would, against equal domestic funding, provide $100,000 for drawing up a Master Plan (NSDS) for Russian statistics. The second tranche of the funding was launched in 2009. The whole project amount is $100 million of which $10 million is being provided as a loan by the World Bank. The project period is five years. The Russian statistical authorities find various advantages in using World Bank funds and expertise. There is no legislative basis available for using international expertise and therefore the use of the Bank facilities was much appreciated. It is also considered an important stamp of quality. The financial issues are handled by a private Russian institution (Bureau of Economic Analysis) in cooperation with the Federal State Statistical Service (Rosstat) and the World Bank. The TFSCB had been a very useful facility at the preliminary stage. Senegal Senegal has received substantial donor funding for statistical capacity building in the last few years, in particular from the European Commission. Starting in 2004, the statistical system of Senegal has been reorganized and reformed and a new autonomous national statistical office, the Agence Nationale de la Statistique et de la Démographie (ANSD) was established. The second phase of the reform started in 2006 with work on a Statistical Master Plan (Schéma Directeur de la Statistique), which was linked to the PRSP. The work was organized in six sub-committees each representing producers and users of statistics within a main sector. In the process some 113 data producing agencies were identified. A priority in the implementation of the Master Plan was to reduce the number of data producing agencies and streamline and harmonize their activities. The Master Plan, which was formally adopted in early 2007, was rather ambitious – calling for outlays of more than US$100 million – and funding has proved challenging. The ANSD will present a detailed, budgeted action plan to a donors meeting scheduled for the last quarter of 2009. Senegal has benefitted from a regional TFSCB grant to support statistics schools in West Africa, one of which is located in Senegal. 29 The scholarships and short courses offered by the school have provided essential resources for current and new staff of ANSD. Looking ahead, the Senegalese statistical authorities are of the opinion that support for statistical schools should be one of the future TFSCB priorities. They are 29 A regional TFSCB project approved in 2008 for African regional schools of statistics (Ecole Nationale Supérieure de Statistique et d‟Economie Appliquée ( ENSEA) in Abidjan, Institut Sous-Régional de Statistique et d‟Économie Appliquée (ISSEA) in Yaoundé, and the Department of Statistics and Demography of the Ecole Nationale d‟Economie Appliquée (ENEA) in Dakar) for $228,955. 62 also interested in applying for TFSCB funding to support statistical training which is an integral part of the SMP. Uganda Uganda has received three TFSCB grants. The first two concerned poverty monitoring, supporting a workshop on that subject in 2001 (grant amount $43,900) and financing a project entitled “Strengthening and linking quantitative, qualitative and GIS based poverty monitoring in the context of the PRSP” (grant amount $350,000) carried out in 2002-2004. The third TFSCB grant ($138,550) was approved in May 2005 for the project “Development of a National Statistical Development Strategy for Uganda”. The work on the Ugandan NSDS was started in 2005 and it followed the plan presented in the TFSCB funding proposal. Initially, 8 line ministries and institutions were involved (now the number of institutions is 16). The Uganda Bureau of Statistics (UBOS) led and coordinated the work. It involved advocacy, awareness programs and meetings of the institutions involved, training, budgeting and financial management. The TFSCB money came in late but DFID stepped in and provided interim funding (in fact increasing the total amount thereby). The project period was extended which in fact helped the work. For international expertise, UBOS relied on the DFID adviser resident within the Bureau. Since October 2007, UBOS and other institutions have been working on the implementation of the NSDS for which the African Development Bank has provided funding. The implementation has involved strengthening of sector statistics (many are very weak at present), infrastructure, awareness and training, and is now moving on to cross-cutting issues, such as ICT inventory, development of metadata, statistical concepts etc. UBOS has its own budget line and has now got a separate item reserved for support to other ministries (albeit a very small item and more of a token than real support). The use of STATCAP is not feasible. UBOS is interested in having a funding basket from the various donors. West Bank and Gaza The TFSCB approved a grant for West Bank and Gaza in August 2007 in the amount of $96,500 for preparing a national strategy for the Palestinian Central Bureau of Statistics (PCBS) and the Palestinian national statistical system (NSS). The PCBS received a further TFSCB grant for the implementation of the NSDS, $400,000, approved in December 2008. The work on the NSDS was carried out by the PCBS in 2008-2009 in cooperation with the Ministry of Finance, line ministries, the Palestinian Central Bank and other institutions involved in official statistics. The NSDS was formally adopted by the Government in June 2009. There have been a series of workshops between the different institutions within the NSS and a major international conference was held in June launching the NSDS and its implementation. The grants have mainly been used for statistical capacity building in terms of study visits, training courses and technical assistance missions. While the grants made the work on the NSDS possible and funded the different workshops and conferences held to inform on and advocate the NSDS, they are not sufficient for its implementation. The PCBS was the 63 implementing agency for these grants; disbursement was according to schedule and the interactions with the World Bank in project implementation were smooth. Meetings with representatives of international and regional institutions Statistics Division, Food and Agriculture Organization of the United Nations (FAO) FAO has implemented two TFSCB activities. The first one was a workshop on capacity building linked to a meeting of the African Commission on Agricultural Statistics in 2001, and the second in 2003-2005 was a precursor to a $10 million project on capacity building in agricultural statistics. Although the latter project had not eventuated, perhaps because donors were not involved from the outset, the TFSCB contribution had helped pave the way for agricultural census and other activities at country level; many countries had benefitted from the Fund. A third project had not been approved, however. In connection with this project substantial difficulties had surfaced in the financial cooperation between the FAO and the World Bank as the financial rules of the two institutions are not compatible. The poor state of agricultural statistics remains an issue. There is a need to bring the ministries of agriculture, which are responsible for agricultural statistics in most African countries, into the capacity building programs. African Centre for Statistics, UN Economic Commission for Africa (UNECA) UNECA has completed two TFSCB projects and has another one ongoing. The meeting focused mainly on the Fund‟s impact in individual African countries as well as the perceived quality of SMPs/NSDSs in Africa. The TFSCB is thought to be very useful from the viewpoint of African countries to assist in the drafting of SMPs and NSDSs. However, there was criticism of the design of many national plans and strategies because of their reliance on international consultants and lack of local stakeholder involvement and ownership. The TFSCB should be continued, inter alia because some NSDSs needed to be redesigned and improved, and small sums only were sufficient for this purpose. Other areas thought deserving of TFSCB support were sub-regional statistical training centres (which might not be so aware of the Fund), training and capacity building in sectoral statistics, and training for leadership and management of African NSOs. AFRISTAT was seen as a possible channel for TFSCB monies for Francophone countries. UNECA also identified a role for the TFSCB to assist countries in moving from the NSDSs, which were largely in place, to their implementation. Statistics Division, Southern African Development Community (SADC) The Statistics Division of SADC has mainly handled one TFSCB project which was divided up into two phases of close to US$400,000 each. The principal aim was to support poverty reduction strategies in the SADC region. A substantial part of phase 2 64 was to be devoted to providing assistance in drawing up NSDSs in the SADC countries. The project was carried out at two levels, regional and national. The national component was intended to be based on the needs of the individual countries as they defined them. In this respect, the countries varied a lot and some of them had difficulties in coming up with specific plans or wishes for action. Hence, the impact of this part of the program was uneven. For the regional part, it proved heavier to implement than anticipated and there was not enough time to carry it out properly. There had been specific problems that had to be dealt with, in particular the question of the participation of Zimbabwe. The disbursement mechanism had also involved some difficult procedures. The program ended in 2006 but there had not been any evaluation report. As for the future, the NSDSs are necessary as they provide a framework for statistical capacity building. The need for making inventories of the existing capacities prior to planning was emphasized. SADC has worked a lot with training and the provision of training material at three levels; for basic courses, for a program of short courses at intermediate level and advanced diploma courses. The SADC Statistics Division is of the opinion that there is a need for a strong training institution in southern Africa. Statistical Division, UN Economic Commission for Europe (UNECE) The UNECE Statistical Division has planned a program on short term economic statistics financed by the TFSCB. The program is scheduled to be launched in September 2009 and will involve three training workshops for central Asia. The workshops will be conducted in Russian and thus have an impact far beyond the target countries. The process of acquiring TFSCB funding has been quite long-drawn out. The application for a TFSCB grant was made in September 2008, it was approved in April 2009 but the financial formalities had not been concluded as of mid-August and the project was not in place. This was unsatisfactory as the program had been planned to start in September and a postponement would involve considerable cost. The UNECE has been participating in a joint program in gender statistics carried out with the World Bank Institute. This has proved a useful program which has had an impact, not only in the eight core countries but also in several others, through training workshops, a manual on gender statistics, and media material (translated into Russian). World Bank funding was very helpful for the Statistical Division; it is a small institution but a specialized one which can do a lot of work with developing countries. The impact of projects is invariably felt beyond the target countries. 65 Mongolia (3-4 September 2009) Background Official statistics in Mongolia have greatly improved over the last decade. Since being re-established as the National Statistical Office (NSO) in 1994 under a new statistics law, the NSO has sought to improve statistical practices, harmonize survey approaches, make official statistics timely and meet international standards. 30 The NSO has successfully drawn on a range of external support to address these challenges, along with steadily increasing domestic resources. Mongolia has participated in the GDDS since August 2000. Recently, the government approved a STATCAP program to further improve the national statistical system as foreseen in the Statistical Master Plan. 31 The government plans to conduct a population census in 2010. TFSCB-funded Projects No. of TFSCB Other Purposes Government funding projects resources funding NSDS, US$80,150 + in-kind 4 US$1,035,000 US$270,000 SCB contributions 1. Strengthening the Institutional Statistical System (TF028427, budget $367,000) Approved in 2001, the purpose of this project was to develop a strategic plan, improve data collection for supply-use tables, and build a poverty monitoring system. The project helped establish the methodologies for and conduct of the first international standard Living Standards Measurement Study (LSMS) in 2002. NSO had already adopted the 1993 SNA in 1995, but underlying data needed improvement. A set of supply-use tables using 2000 as reference year were also prepared. Finally, the project assisted the NSO in formulating a Corporate Strategic Plan which the NSO later used as a basis for the “Program for Official Statistics Development 2006 -2010”.32 The project was executed by NSO and completed on schedule in June 2004 with a disbursement rate of 97 per cent. 30 The law was subsequently revised in 1997, 2004 and 2008, further strengthening the independent and coordinating functions of the NSO. 31 Called MONSTAT, the program calls for US$6.6 million over 5 years. As of August 2009, US$3.1 million had been secured (IDA/STATCAP: US$2 million, TFSCB: US$400,000, Korean Trust Fund: US$700,000), along with domestic resources. 32 Other external assistance, for example the European Union Technical Assistance to the Commonwealth of Independent States (TACIS) program, also provided inputs to this program, as referenced in NSO: “National Strategy for the Development of Statistics”, 2007, page 2. 66 2. Strengthening the Institutional Statistical System in Mongolia (TF056595, budget $200,000) Building on the 2002 LSMS and the Household Income and Expenditure Survey (HIES) also conducted in 2002, the purpose of this project was to design a new combined LSMS-HIES whereby core LSMS questionnaires were integrated into the annual HIES with different rotating modules year to year. The methodologies and procedures established by the project are now fully used in the annual Survey. The resulting data serve to meet the statistical requirements of the Poverty Reduction Strategy. The project also contained a training component, including domestic as well as overseas training in the form of long courses in general statistics topics, and short courses covering topics such as sampling techniques, surveying, poverty statistics, and gender statistics. Approved in late 2006, the project was executed by the NSO and completed on schedule in June 2009 with a disbursement rate of 90 per cent. 3. Updating of the Statistical Master Plan and Preparation of a Financing Strategy for 2007-2010 (TF057815, budget $68,000) By 2006, the Government had approved the Program for Official Statistics Development 2006-2010, but recognized the need for further strengthening of statistical capacities, triggering the need for revising and expanding the statistical development plan. For this purpose a TFSCB grant of US$68,000 was approved, leading to the National Strategy for the Development of Statistics, 2008 -2012, dated November 2007, with a full-fledged plan of activities and their cost. The project was executed by the World Bank and completed on time at the end of 2007 with a disbursement rate of 98 per cent. 4. Implementation of NSDS, Census Preparation (TF093575, budget $400,000) The project addresses several tasks included in the Statistical Master Plan with respect to the preparation of the 2012 population census, including census mapping, preparation of manuals, and training of supervisors and enumerators. It also includes an equipment component. NSO is coordinating the census preparations, adding its own resources and other external support (United Nations Population Fund (UNFPA) and others), bringing the total budget to US$706,000. While approved in January 2008, the project is still not operational because it will be implemented as part of the MONSTAT program using the same execution modalities. Finally, the NSO confirmed that they had been involved in a multi-country TFSCB project entitled “Measurement of the non-observed economy” implemented by OECD in 2003. Process and Management Issues The implementation process generally went smoothly and on time once the grant agreements were in place, which took from 6 to 12 months. Of the two projects implemented by the NSO, one was executed directly, which was the arrangement the NSO preferred; for the second project the NSO went through the Ministry of Finance, which added a bureaucratic layer and limited the flexibility of the NSO. The project activities appeared to be well coordinated with those supported by other donors, hence maximizing overall efficiency. The NSO attached much importance to the training with the aid of TFSCB and other donor support. The training seems to have been well 67 planned and executed and the training courses selected had good coverage and substantial content. Achievements and Impact According to the findings of the Evaluation Team, the TFSCB achieved its objectives and contributed greatly to the development of statistical capacities in Mongolia. The LSMS methodology was improved to meet international standards, and subsequently combined with the HIES which the NSO is able to continue and adjust according to needs. The first ever supply-use tables completed in 2003 were followed by a second set prepared in 2006-2008 using 2005 as base year. The NSO took ownership of the statistical planning process and is committed to its implementation through the approval of MONSTAT. NSO confirmed that the training of NSO and district staff had improved their competencies for the benefit of the statistical work. Sustainability The adoption of the Statistical Master Plan and the commitment by Government to its implementation through MONSTAT represent a policy framework conducive to coordinated planning within the statistical system. At the institutional level, another factor helping to secure NSO‟s ability to effectively coordinate statistical activities is the existence of Memorandums of Understanding between NSO and district governors on responsibilities in data collection. The methodological improvements achieved in survey methods and structure as well as in data preparation of supply-use tables have been maintained beyond the duration of the TFSCB projects and are most likely to continue. The human resources available to the NSO present a critical factor in maintaining the capacities developed so far. The NSO reported that staff strength has steadily improved from 65 established posts in 1997 to 192 in 2007 with actual funding also increasing. The NSO was also able to upgrade the posts of those staff who had received extensive training as a measure to retain and encourage staff. Likewise, staff going on longer overseas training should make commitments to remain in NSO service for a specified length of time. The training opportunities themselves made NSO an appealing employer. While effective in retaining staff at the moment, this may not hold for future years as employment opportunities in the private sector grow. 68 Cambodia (7-8 September 2009) Background Cambodia‟s statistical infrastructure and activities were practically non-existent up to about 15 years ago, following two decades of war and civil strife. Starting from 1992, however, the quality and coverage of economic and socio-demographic data have gradually improved, and significant efforts have been made in recent years to improve the statistical system. Donors have been very prominent in financing data collection and compilation activities, and donor budgets for statistical activities still dwarf government inputs in financial terms. The Government‟s poverty reduction strategy, overall national development plans and the monitoring of progress towards Cambodia‟s Millennium Development Goals have increased the need for data, especially on poverty and socio-economic development. However, Cambodia‟s capacity to produce, disseminate and analyse statistics remains weak and the use of data for decision-making is at best uneven. The country has a largely decentralized statistical system, with the National Institute of Statistics (NIS) within the Ministry of Planning as the focal point on statistical matters. The NIS compiles and consolidates statistics provided by decentralized offices and also collects primary data through household and establishment surveys and population, agricultural and economic censuses. The Statistics Law of 2005 encouraged the development of an integrated statistical system by giving the NIS responsibilities to coordinate the official statistical activities of the statistical bureaus and sections within the planning and statistics departments of the various Ministries, as well as of provincial and district offices. TFSCB-funded Projects No. of TFSCB Government Purposes Other funding projects resources funding NSDS, US$12,500 + in- US$80,000 + in-kind 1 US$222,000 SCB kind contribution contributions Statistical Master Plan Finalization and Capacity Development (TF055305/TF055438, budget $222,000). The NIS had been working on the development of a longer-term statistical plan and forward work program for some years, with assistance from several donor agencies. The TFSCB proposal, originally drawn up in April 2005 with revisions in 2006 within the same budget, aimed in part to permit the NIS to develop a 10-year Statistical Master Plan (SMP), along with implementation and funding arrangements, and finalize the SMP based on the results of a stakeholder workshop. The project also planned to develop capacity through the training of NIS and other staff in areas such as survey methods, data dissemination skills, and statistical analyses and report writing, as well as to support the preparation of a new Cambodian Socio-Economic Survey. The SMP element of the project (TF055305) was executed by the World Bank with a budget of $53,500 and closed on schedule on 31 December 2007, with a 69 disbursement rate of 93 per cent. The training component (TF055438) was executed by the NIS; the TFSCB budget of $168,500 was supplemented by donor and government resources of $92,500 plus in-kind contributions, and this element of the project also closed as scheduled on 31 March 2008, again with 93 per cent of the project budget disbursed. Process and Management Issues Within the time at its disposal, it proved difficult for the Team to ascertain the exact situation in terms of process and project implementation because of the turnover of staff in the World Bank office. In particular, the background to the bifurcation of the original project and the execution of the SMP component by the Bank itself were unclear. Nevertheless, there appeared to have been no major issues in the process of acquiring and disbursing the TFSCB project funds. Tight control of the Bank‟s projects in Cambodia had resulted in on-time closure of both components of the project and a disbursement rate of 93 per cent, considered fully satisfactory. Goods and consultant services had been procured in accordance with guidelines and procedures agreed by the Bank and the Ministry of Economy and Finance. Achievements and Impact Although the Team heard that some agencies were not involved as concretely as they might have been in the redesign of the Statistical Master Plan, awareness and recognition of it were widespread among both national agencies and donors, and its profile was sufficiently high for it to be referred to by the Prime Minister during the presentation of the 2008 Population and Housing Census results. The SMP was seen as an important milestone in the development of official statistics, and together with the Statistics Law represented a significant advance over the earlier state of affairs. It could in principle be used to avoid overlapping and duplication, and was viewed as a reference document that the NIS could cite in its discussions with other agencies and with donors. Importantly, the SMP sets out clearly a schedule for the main censuses and surveys in the plan period and thus presents a clear set of priorities. This prioritization in the SMP, which had been agreed by the institutions involved and decided with a view to international requirements, was seen as considerably reducing the risk that donor priorities would supplant national priorities. The training element of the TFSCB grant has also been seen as very important and seems to have been used with care, addressing areas like report writing and analysis which are acknowledged to be weak. A number of government agencies as well as the NIS indicated that they had benefited from the training. Overall, the Team noted that the use of statistics in Cambodia was clearly increasing and that they were receiving heightened attention. The official statistics were being compiled on a professional basis and appear to be fully independent. In contributing to this situation, the TFSCB grant has been of value to the NIS out of proportion to its small size; it has brought important recognition and encouragement to the Institute, as well as expertise which is seen as having been very useful, in particular for finalizing the SMP. 70 Sustainability The SMP has been well received by donors and there are clear indications that it will be respected as the framework for basic statistical activities and hence donor project funding in the years to come. The Team noted that the donor coordination currently being undertaken by the NIS will however need to be strengthened for full effectiveness, and extend to the harmonization of supplementary payments. Based on the SMP, plans and finance are already in hand for a number of future statistical operations including a Demographic and Health Survey (DHS) and economic census. Moreover, some agencies like the Ministry of Economy and Finance and the National Bank of Cambodia have their own budgets for statistical activities. There continues to be much interest in seeking additional funds for continued training in the next few years, as well as for updating the SMP. Nevertheless, the SMP‟s implementation will inevitably be hampered due to grossly insufficient Government funding for the NIS, staff retention difficulties due to uncompetitive salaries (alleviated to some extent by the introduction of a merit based remuneration scheme for key NIS staff), diversion of staff to donor-funded fieldwork, lack of technical know-how, weak information sharing and coordination, and low levels of analytical and report-writing skills, among other factors. It is abundantly clear that effective statistical capacity building in Cambodia will require a very long- term commitment. 71 Indonesia (10-11 September 2009) Background Indonesia has a well-established, mainly centralized national statistical system comprising (i) basic statistics with a broad range of governmental and societal uses which are produced by the Badan Pusat Statistik (BPS – Statistics Indonesia), (ii) sectoral statistics which are typically generated from administrative records of government agencies, and (iii) so-called specific statistics produced by other organizations. BPS maintains the metadata for all three types of statistics. Compared with many other Asian developing countries, Indonesia has a strong history of statistical development planning. Although the process of decentralization of government over the past decade did not encompass BPS as such, the effects of decentralization have had a major impact. The increased roles and responsibilities of regional and local government have created a great demand for sub-national and local statistics, which BPS has had some difficulties coping with owing to inadequacies and untimely data supply on the part of regional and local authorities. Other challenges include maintaining capacity to capture new forms of economic activity, and continuing to improve cooperation with data producers, data users and respondents. TFSCB-funded Projects No. of TFSCB Other Purposes Government funding projects resources funding NSDS, US30,000 + in-kind 3 US$823,600 - SCB contributions 1. Increasing access to statistical data for local policy decision making (TF050609, budget $356,400) This project was approved in early 2002 against the background of decentralization of many aspects of government, which had just started. It aimed at enhancing access to household survey and village census data and increasing their use at the district level for policy purposes. The main modalities were the delivery of training based on needs assessment seminars and the production of a CD-ROM, along with some computer hardware. The project was executed by Statistics Indonesia which made significant inputs of staff time, and was completed on schedule on 31 December 2004, with a 94 per cent disbursement rate. 2. Streamlining Data Flows from Regions to the Center under Decentralization (TF055498/TF057574, budget $392,200) Approved in mid 2005, this second project had a twofold objective, one of which again related directly to strengthening the statistical system under the policy of decentralization. The quantity and quality of statistics in several key sectors at the regional level had deteriorated with the transfer of responsibilities to sub-national agencies, and the transmission of sectoral data to central and provincial governments 72 had become problematic. The project aimed to design an improved sectoral data collection system, using the structure of BPS at all levels of government to channel data up towards the centre. This component of the project was to include a stocktaking of sub-national sectoral statistics, the holding of a workshop, and design and execution of a pilot project. A further component aimed to address the more general question of developing a fresh strategic approach for BPS in the new institutional environment of decentralization, in essence a national strategy for the development of statistics. Because of timing issues related to Indonesian budget procedures, this NSDS component (TF055498) with a budget of $117,450 was executed by the Bank, and was completed on schedule on 31 December 2007 with a 94 per cent disbursement rate. The decentralization component (TF057574) with a budget of $274,750 is being executed mainly by BPS, and is still underway. 3. Development of a Statistical Master Plan for Indonesia (TF090799, budget $75,000) With assistance from the previous TFSCB project, the BPS had prepared and published the 2005-2009 BPS Strategic Plan for Statistical Development (Renstra). This project, approved in August 2007, aimed to assist in the translation of the Renstra into a statistical master plan, in particular to allow the preparation of detailed investment plans for critical components of the SMP. Given the tight time schedule associated with the Government‟s expression of interest in applying for a STATCAP loan to help implement the SMP, it was decided that the project should be executed by the Bank. The TFSCB budget of $75,000 was supplemented by government contributions of $30,000, and the project was completed on schedule on 31 December 2008 with a disbursement rate of 70 per cent. Process and Management Issues The Evaluation Team noted that while most of the project components had run smoothly and been completed on time, the „decentralization‟ component (TF0575 74) of the second project was a notable exception. It was explained to the Team that Indonesian regulations required the project to be included in the Government‟s budget, but the timing of the project‟s approval on 9 June 2005 just missed the May budget deadline and the project could only be included in the 2007 rather than 2006 budget. In 2007/8, the BPS successfully implemented the initial project activities on stocktaking of sub-national statistics, conducting a workshop, and drawing up the terms of reference for a pilot project to be undertaken by a consulting firm. The BPS first followed Indonesian procurement rules in compliance with government regulations, but the Bank issued an objection and the process had to be started again. BPS found the Bank procurement procedures for recruiting a consulting firm (as opposed to individual consultants) difficult, cumbersome and time-consuming, and reported that while the TFSCB guidelines provided a starting point, they were found to be too general to cover all the required forms and procedures. While, at the time of the Team‟s mission, the project seemed about to move forward again, there had overall been a very significant delay, almost certainly necessitating a further extension of the closing deadline to June 2010 – some 18 months after the scheduled completion of the project, and five years after the project had been approved. The Team found that the World Bank Jakarta office had done its utmost to facilitate and expedite the procedures; it gathered that in cases such as this, Bank execution would have sped up the process and would in fact have been welcomed by BPS. 73 Achievements and Impact The Team noted that even though the activities under the first grant had been completed nearly five years ago, their impact was still felt to some extent. Increasing the accessibility and use of data at local level had been very important owing to the lack of understanding of the significance of statistical information at the level of regions and localities. The Team heard that the first project and the activities carried out so far under the second one had produced results, in that data flows between central and regional offices had been reorganized and improved. Moreover, there was clear evidence from the BPS standpoint of a heightened appreciation of statistics among local politicians and officials. Government agencies were generally aware of the 2005-2009 Renstra but less so of the Statistical Master Plan which the third TFSCB project had helped develop. The SMP was generally seen as an important tool by the relevant agencies, although its detailed nature called for a degree of flexibility with regard to changing priorities. It was clearly central to the future work and orientation of the BPS. Moreover the SMP has also paved the way for a substantial STATCAP loan for statistical capacity building with significant government counterpart funds. From its meetings with the representatives of the statistical and planning authorities, the Team has noted that the TFSCB grants have had an impact considerably larger than might be supposed from their amount, given the size of an economy like Indonesia. The Indonesian authorities have appreciated the TFSCB support not only for the funds involved but no less for the spirit in which the grants are extended, the input of knowledge involved and the encouragement they bring. Sustainability With the occasional exception, cooperation and coordination between BPS and other Government agencies appeared to be good, and this, along with understanding and interest in official statistics at high levels, augurs well for the development of the new Renstra starting in 2010. The STATCAP loan currently under elaboration would be aligned to the Renstra; plans for utilizing the STATCAP facility, which could be in the order of $93 million, were largely in place. Sustainability of the results achieved by the TFSCB inputs to date therefore appears to be very well assured. 74 Annex 3 List of persons met World Bank, Washington, 25 June – 1 July 2009 Statistical Development and Partnership Team, Development Data Group (DECDG) Mr Misha V. Belkindas Manager, IMC member Mr Neil Fantom Senior Statistician (Team leader), IMC member and TTL Mr Graham Eele Senior Statistician, IMC member and TTL (met in Durban, 18 August) Mr Mustafa Dinc Senior Economist/Statistician, TFSCB Grant Administrator Ms Barbro Hexeberg Senior Economist Mr Haeduck Lee Senior Economist/Statistician, IMC member and TTL Mr Ronald Luttikhuizen Senior Economist/Statistician, TTL (met in Durban, 21 August) Ms Ingrid Ivins Statistical Officer Mr Omar Hadi Senior Program Assistant Ms Lisa M. Burke Program Assistant TFSCB Internal Management Committee (IMC) Members (not mentioned above) Ms Ghislaine Delaine Senior Statistician, African Department (and TTL) Mr Antoine Simonpietri Senior Statistician, AFT-Results and Learning Mr Jose R. Molinas Vega Senior Economist, PREM, LAC (and TTL) – by phone TFSCB Task Team Leaders (TTLs) (not mentioned above) Ms Maria Beatriz Orlando Senior Economist and Gender Coordinator, LAC (met 21 July) Mr Johan Mistiaen Economist/Statistician, DECDG (met 25 September) TFSCB Advisory Panel Mr Chandrakant Patel Member Mr Jean-Louis Bodin Member (met in Durban, 21 August) TFSCB Consultative Group Mr Pieter Everaers Director, External Cooperation, Eurostat (met in Sarajevo, 31 August) Uruguay, 3 – 4 August 2009 National Institute of Statistics (Instituto Nacional de Estadística -- INE) Ms Alicia Melgar Technical Director Ms Susana López Deputy Director Mr Álvaro Fuentes Acting Director, Economic Statistics Division 75 Centro de Investigaciones Económicas (CINVE) Mr Fernando Lorenzo Senior Researcher Ministry of Education Mr Andrés Peri Director de la División de Investigación, Evaluación y Estadística, Administración Nacional de Educación Primaria Ministry of Economy and Finance Mr Michael Borchardt Director, Macroeconomic and Financial Advisory Department Ms Cecilia Marconi Economist, Macroeconomic and Financial Advisory Department Central Bank of Uruguay Mr Daniel Dominioni Head of Macro-Economic Analysis Mr Jorge Basal National Accounts Section Ms Rozanna Fernandez-Castro National Accounts Section Ms Loreley Molinari National Accounts Section Ministry of Public Health Ms Rosario Grove Director, Health Information Unit, National Directorate of Health Consultants, TFSCB projects Ms Rosa Grosskoff Former Project Coordinator Mr Edgardo Greising Former Project IT Consultant World Bank Ms Maria Inés Ferrés Team Assistant Paraguay, 6 - 7 August 2009 General Directorate of Statistics, Surveys and Censuses (Dirección General de Estadística, Encuestas y Censos – DGEEC) Ms Zulma Sosa de Servín Director General Ms Nimia Torres Director, Department of Economic Statistics Central Bank of Paraguay (Banco Central del Paraguay – BCP) Mr Jorge Corvalan Mendoza President Mr Carlino Velazquez Head, Economics Studies Department Mr Miguel Mora Department of National Accounts and Internal Market Mr Rodi Ozuna Department of National Accounts Mr César Daniel Yunis Chief, Division of Monetary Studies 76 United Nations Mr Julio Fernández Frutos Chief Technical Advisor, UNDP Former Director of DGEEC Mr Andres Osorio Official, UNICEF Inter-American Development Bank Mr Eduardo Feliciangeli Specialist in Modernization of the State World Bank Ms Gloria Dure Executive Assistant The 57th Session of the International Statistical Institute, Durban, South Africa, 16 - 22 August 2009 Country representatives Gabon Mr Paul-Henri Directeur Général Adjoint, Direction Générale de la Nguema Meye Statistique et des Etudes Economiques (DGSEE) Mr Daniel Edjo Mve Statistician, DGSEE Ghana Ms Grace Bediako Government Statistician Lesotho Ms Liengoane Lefosa Director, Bureau of Statistics Malawi Mr Charles Machinjili Commissioner of Statistics Mr Jameson Ndawala Assistant Commissioner of Statistics Nigeria Mr O.F. Nwaboku Deputy Director, National Bureau of Statistics Philippines Mr Romeo S. Recide Director, Bureau of Agricultural Statistics, Department of Agriculture Ms Jessamyn O. OIC-Director, Social Statistics Office, Encarnacion National Statistical Coordination Board Russian Federation Mr Sergey Egorenko Deputy Director, Foreign Statistics and International Cooperation, Rosstat (Federal State Statistical Service) Senegal Mr Mamadou Falou Directeur Général Adjoint, Agence Nationale de la Mbengue Statistique et de la Démographie Mr Mamadou Niang Directeur du Management de l’Information Statistique 77 Uganda Ms Norah T. Madaya Principal Officer, Statistical Coordination, NSDS Coordinator, Uganda Bureau of Statistics (UBOS) Mr Ben Paul Mungyereza Director, Statistical Coordination Services, UBOS West Bank and Gaza Ms Ola Awad Acting President, Palestinian Central Bureau of Statistics Organization representatives FAO Mr Hiek Som Chief, Country Statistics Service, Statistics Division Mr Naman Keita Principal Statistician, Statistics Division IMF Mr Oliver Chinganya Regional Advisor, General Data Dissemination System (GDDS) PARIS21 Mr Christophe Duhamel Deputy Manager, PARIS21 Secretariat Mr Richard Roberts Evaluator of PARIS21 Ms Claudine Voyadzis Evaluator of PARIS21 Southern African Development Community (SADC) Mr Ackim T. Jere Senior Programme Manager, Statistics UN Economic Commission for Africa Mr Ben Kiregyera Former Director, African Centre for Statistics Mr Dimitri Sanga Senior Statistician/Acting Director, African Centre for Statistics UN Economic Commission for Europe Ms Lidia Bratanova Director, Statistical Division Mr Petteri Baer Regional Advisor in Statistics, Statistical Division Ms Jessica Gardner Statistician, Statistical Division Mongolia, 3 - 4 September 2009 National Statistical Office Mr Dagvadorj Chilkhaajav Director General Ms Oyunchimeg Dandar Head of Population and Social Statistics Department Ms Badamsetseg Batjargal Director, Macro-Economic Statistics Department Ms B. Tserenkhand Head of Information Technology Department Ms D. Lkhagvatseren Programmer, Data Processing Technology Department 78 City Government of Ulaanbaatar and District Statistical Offices Ms Baigalmaa Namsrai Senior Officer in Charge of Department of Statistics Mr T. Altansukh Senior Statistician, Department of Statistics Ms G. Battsetseg Statistician, Songino Khairkhan District National Development and Innovation Committee Mr Jargalsaikhan Jamba Director, Development Policy and Strategic Department World Bank Ms Altantsetseg Shiilegmaa Economist/Public Administration Specialist Ms Maya Razat Program Assistant Cambodia, 7 - 8 September 2009 National Institute of Statistics (NIS) H.E. San Sy Than Director General Mr Heang Kanol Director of Analysis Department Mr Nor Vanndy Deputy Director, Department of National Accounts Mr Lim Penh Director, Standard and Analysis Statistics Ministry of Agriculture, Forestry and Fisheries H.E. Kith Seng Under Secretary of State Ms Meas Sotheavy Chief of Statistics Office, Department of Planning and Statistics Mr Meach Yady Vice Chief of Agricultural Marketing Office Ministry of Economy and Finance H.E. Hang Chuon Naron Secretary General Ministry of Education, Youth and Sports Ms Kuy Phalla Deputy Director of Planning Department National Bank of Cambodia Mr Tann Sokhann Deputy Director, Economic Research and Statistics Department Mr Sim Sothearith Monetary Statistics Division Ministry of Health Dr Sao Sovanaratnak Deputy Director of Planning and Health Information Department Dr Khol Khemerary Chief, Bureau of Health Information UNICEF Mr. Peter Leth Monitoring & Evaluation Officer Ms Bossadine Uy Monitoring & Evaluation Officer 79 Swedish International Development Cooperation Agency (SIDA)/ Statistics Sweden, Institutional Capacity Building Project, NIS Ms Birgitta Mannfelt Chief Advisor Ms Agneta Sandqvist Senior Advisor, Household Surveys UNFPA Mr Vanna Sok Programme Manager, Population & Development Mr N. Rama Rao Census Consultant Mr Gouranga Dasvarma Census Consultant Mr Harry Lode IT Consultant JICA Mr F. Nishi Statistician, Statistical Research and Training Institute, Ministry of Internal Affairs and Communications, Japan Ms Minori Terada Project Formulation Advisor (Good Governance), Cambodian Office World Bank Mr Tim Conway Senior Poverty Specialist Mr Samsen Neak Poverty Specialist Mr Tan Peng Seng Program Assistant Indonesia, 10 - 11 September 2009 Statistics Indonesia Mr Rusman Heriawan Chief Statistician Mr Eri Hastoto Head, Program Development Bureau Mr Arie Sukarya Head, STATCAP-CERDAS Project Management Unit Mr Sentot B. Widoyono Head, Planning Development Division Ms Fatimah Head, Planning Development for Integrated Activity Subdivision Mr Wynandin Imawan Director, Social Resilience Statistics Mr Teguh Pramono Head, Social Vulnerability Statistics Sub-Directorate Ms Nurma Midayanti Ministry of Planning – Bappenas Mr Bambang Prijambodo Director for Macro Planning Ms Wismana Adi Suryabrata Director for Allocation of Development Funding Ministry of Finance Mr Askolani Director, Public Expenditure Policy, Fiscal Policy Office Mr Kunta Fiscal Policy Office Ms Ria Fiscal Policy Office Bank Indonesia Mr Triono Widodo Director, Economic and Monetary Statistics 80 Ms Widi Agustin Economic Research Associate, Real Sector Statistics Team Association of Southeast Asian Nations (ASEAN) Secretariat Mr Agus Sutanto Head, Statistics Unit Mr John Frederick de Guia Senior Officer for Statistics World Bank Ms Vivi Alatas Senior Economist/Poverty Team Leader - PREM Ms Noriko Toyoda Statistical Capacity Building Coordinator 81 Annex 4 Project Cycle Proje Project stage Who Activity Milestone Documentation Remark Client agency,* Discuss scope of Use TFSCB WB Country activities. on-line Office, WB brochure, Idea Appoint TTL. Project concept available in statistics focal points in area English only. dept., TFAU Formulation and Project proposal Use TFSCB agreement on project (incl. Guidelines, - Preparation Same proposal. Use of GFR endorsements as mostly for system. required) WB Staff. Any time for Project Internal Review proposals against Decision Memo. NSDS approval, Management established applications projects. 2 Approval conditional Change status in Committee conditions and appraisal times /year approval, GFR. (IMC) criteria. for other rejection projects. Determine execution responsibilities: recipient Grant 1-20 months, Implementing or in exceptional cases, agreement averaged 10 Agency (IA), WB. signed. months. Activation Grant agreement MoF, TTL, WB Observe IA Legal Service Prepare grant agreement. Release of approval Await govt. formal funds procedures. approval. Advisory Report to (Quality Review all approved Consultative Post-facto control) panel project proposals Group IA (or Organize project executing Contracts activities: procure Implemen- agency on Technical and Apply WB individual experts, firms, tation/ behalf of IA), financial reports procurement organize study tours, Execution TTL TTL Back-to- rules. workshops. Supervise. WB Country Office reports Use GRM, IRIS. Office Report progress twice a Progress reports. Results Monitoring TTL, TFAU year in GRM. Request Closing date reporting not extension as required. changed. systematic. Completion IA, TTL, TFAU Finalize documents. Close Project Completion Follow-up accounts. Change status in account memorandum. activities GRM. closed. depend on TTL. IA (Recipient Review project Evaluation Evaluation Govt./regional Very seldom. achievements. report body) projects. Note: * Client Agency or Implementing Agency (IA) include: NSO, other national client agency, regional organizations, United Nations Departments, Agencies and Regional Commissions. 82 Annex 5 Financial status of the TFSCB33 TFSCB I (TF021846) Established: April 1999 TFSCB I is operationally closed Donor contributions: UK (US$6.7 million), Netherlands (US$4.4 million), Germany (US$1.3 million), Switzerland (US$200,000), France (US$141,300). Receipts in million US$ Expenditures in million US$ Donor contributions 12.7 Total project expenditures 11.3 Investment income 0.7 Fees (9% of TF principal) 1.2 Total 13.4 Total 12.5 Carry forward to TFSCB III 0.9 TFSCB II (TF053421) Established: April 2004 Planned closing: December 31, 2010 Donor commitments:34 UK (£3.6 million), Netherlands (€2.6 million), Germany (€1.2 million), France (€300,000), Switzerland (US$200,000). Approximate planned contributions based on commitments: US$15.3 million. Allocations as of Oct. 31, 2009 in million Receipts as of Oct. 31, 2009 in million US$ US$ Allocations to Donors contributions 11.7 11.9 projects35 Fees (5% of TF Investment income 0. 8 0. 6 principal) Total 12.5 Total 12.5 Unallocated 0.047 33 The sources of information for these tables are: internal World Bank documents for establishing the trust funds, TFSCB Progress Reports of May and October 2009, and data provided by the TFSCB Administration Unit. 34 Commitments refer to agreed contributions as per signed memorandums of understanding. 35 Allocations to projects refer to the sum of all project budgets approved by the IMC. 83 TFSCB III (TF070561) Established: August 2006 Planned closing: December 31, 2012 Donor commitments: UK (£6.4 million), Canada (Ca$6 million), Netherlands (€1.3 million). Donor interest expressed from France, Germany, and Switzerland. Approximate planned contributions based on indicative commitments: US$19.2 million. Allocations as of Oct. 31, 2009 in million Receipts as of Oct. 31, 2009 in million US$ US$ Allocations to Carry-over from TFSCB I 0.9 9.8 projects36 Estimated admin fee Donor contributions 15.1 0.4 (2% of TF principal) Estimated oversight Investment income 0.8 0.8 fee Estimated project 0.4 supervision fee Total 16.8 Total 11.4 Available for programming 5.4 Combined TFSCB I+II+III First established: 1999 Planned closing: December 31, 2012 Donor contributions and commitments: UK (US$6.7 million + £10 million), Netherlands (US$4.4 million + €3.9 million), Canada (Ca$6 million), Germany (US$1.3 million + €1.2 million), France (US$141,300 + €300,000) and Switzerland (US$400,000). Allocations as of Oct. 31, 2009 in million Receipts as of Oct. 31, 2009 in million US$ US$ Donor contributions 39.5 Allocations to projects 33.0 Investment income 2.3 TFSCB fees 37 3.4 Total 41.8 Total 36.4 Available for programming 5.4 36 The allocations include some US$3 million allocated to projects approved in September 2009. Because of the timing of their approval, these projects are not covered in the body of this report. 37 Including TFSCB administrative fee, estimated TFSCB oversight fee, and estimated project supervision fees. 84 Annex 6 Terms of Reference for an Evaluation of the Trust Fund for Statistical Capacity Building (TFSCB) Overview of the Trust Fund Background 1. The Trust Fund for Statistical Capacity Building (TFSCB) was set up in November 2000 by the Development Data Group of the World Bank to strengthen the capacity of statistical systems in developing countries. It is closely coordinated with the work of the PARIS21 partnership and is part of the worldwide effort to reduce poverty by strengthening the evidence base for decision making at all levels. In conjunction with national governments, the aid programs of bilateral donors as well as other international initiatives, TFSCB provides a practical mechanism to achieve the PARIS21 vision, that is, to develop effective and efficient national statistical systems and to promote a culture of evidence-based decision-making. In line with the priorities identified by the contributing donors, the emphasis is on strengthening statistical systems in support of national poverty reduction strategies and other national and international development goals. In particular, TFSCB helps developing countries to set out a medium to long-term strategic vision for their statistical systems, to prepare programs and plans to put this vision into effect and then to implement specific capacity building projects. The current project framework for the TFSCB is provided in Attachment 1. 2. TFSCB provides an important mechanism to implement the Marrakech Action Plan for Statistics (MAPS), which provides the overall framework for the World Bank‟s efforts to improve the availability and use of statistics to support development and poverty reduction. In particular, the Trust Fund supports the first and third MAPS actions: to mainstream strategic planning of statistical systems and prepare national strategies for the development of statistics (NSDS) for all low-income countries by 2006; and to increase financing for statistical capacity building. Projects 3. TFSCB finances two main kinds of projects. The first type supports the preparation of NSDSs. In order to implement MAPS, TFSCB works closely with PARIS21 to ensure that all countries have an integrated and comprehensive plan for the strategic development of their national statistical systems. The purpose of an NSDS project is to define the best national long-term strategy to develop an efficient and sustainable statistical system in the recipient country. In general projects are expected to support the preparation of an NSDS in line with PARIS21 guidelines and recommendations. 4. The second type of project (non-NSDS projects) supports capacity building activities in specific priority areas. For instance, these projects may target the implementation of one or more critical components of an NSDS. All non-NSDS 85 projects are expected to be based on a formal assessment of the statistical system and must demonstrate how they will address the main capacity weaknesses. If a country does not have an NSDS it is strongly encouraged to prepare one before applying for a non-NSDS project. Project proposals are required to set out specific targets for capacity development. The resources provided must be additional and cannot be used simply to replace national budgetary resources. While it may be possible to use funds to meet some of the costs of data collection activities, especially where new activities are being tried out, projects that are just concerned with maintaining existing systems are unlikely to be approved. 5. In all proposals, recipients are required to demonstrate their commitment to the project by providing some contribution to the overall costs, which may be in kind or in cash. Given that the preparation of NSDS is a priority for developing countries TFSCB attempts to allocate up to 60 per cent of its resources for this purpose. The remaining funds are then to be allocated to non-NSDS projects. 6. The maximum value of any one grant is $400,000, and the maximum duration of a project is two years, although an extension of up to 12 months can be approved where this is justified. There are also limitations on what the grant may finance, with no more than 20 per cent for goods and equipment. All projects are normally executed by an appropriate national or regional agency. They may also be implemented by third parties such as donor agencies or, in exceptional circumstances, by the World Bank itself. All executing agencies must be able to sign legal contracts and are required to demonstrate financial soundness and a capacity to manage and account for funds. Executing agencies are required to follow World Bank procedures for procurement, disbursement and accounting. For each project the Bank nominates a Task Team Leader, who takes responsibility for liaison with the implementing agency, supervision and ensuring compliance with trust fund procedures. Management 7. The overall governing body of the Trust Fund is the Consultative Group (CG). This is chaired by the Director of the Development Data Group of the World Bank and includes representatives from all contributing donors as well as representatives from the United Nations, the European Commission, OECD, the IMF and the PARIS21 Secretariat. The CG usually meets once a year in conjunction with the PARIS21 Steering Committee, reviews operations and sets policy, in line with the agreed Trust Fund prospectus. 8. Day to day operations are managed by the TFSCB Administration Unit and all proposals are reviewed by the Internal Management Committee (IMC). The IMC reviews NSDS proposals virtually, while a meeting is held twice a year to review the non-NSDS projects. Projects can be approved as submitted or with minor revisions; if necessary that can also be returned to the Task Team Leader with requests for more substantial changes, or may be rejected outright. The CG is advised by an Advisory Panel consisting of two eminent development statisticians, who provide an annual review of operations. A progress report is provided to the CG every six months. 86 9. All projects are now managed through the World Bank‟s e -trust fund system and are monitored using the web-based Grant Reporting and Monitoring (GRM) mechanism. Since mid-2008, applications have also been submitted on line using the Grant Funding Request (GRF) application. Task team leaders are required to complete GRM reports twice a year and prepare an Implementation Completion Memorandum (ICM) when the project closes. This process is monitored on a regular basis by the Administration Unit. Financing 10. Between November 2000 and August 2004, donor contributions were received into a parent trust fund, TFSCB I. A total of $12.3 million was received and $11.8 million was committed to 55 projects. Of these one was eventually not proceeded with, 37 were substantive projects and 17 were approved under a fast-track procedure then operating. This second group consisted of small projects with a commitment of $50,000 or less. 11. In October 2004, a new parent trust fund, TFSCB II, was set up. The guidelines and procedures were changed to incorporate recommendations from a first independent evaluation carried out earlier in the year. The changes were also designed to bring the trust fund in line with MAPS. The main change was to introduce two financing windows. The first, for NSDS projects operated on a fast- track process and projects have been reviewed by the Internal Management Committee as they are received. The second window is for all other projects and these have been reviewed through a twice yearly competition. 12. Further changes were introduced in 2007 to implement recommendations from the Consultative Group (CG) and also to incorporate changes in the way that the World Bank manages trust funds generally. The main change was to make some resources available for program management as well as for the supervision of individual projects. TFSCB-III has the same objectives and the same governing structure and administration arrangements as earlier, but with a reduced administration fee and a separate allocation for project management and supervision. The changeover from TFSCB-II took place in 2007/08 and from the beginning of 2008 on all new donor contributions and all new projects are being managed through TFSCB-III. 13. TFSCB II has received contributions from donors amounting to $11.6 million and has allocated $11.6 million to 71 projects. Of these, 39 were approved under the NSDS window, 26 under the non-NSDS window and 6 provided for the participation of developing country representatives at important international statistical meetings. 14. TFSCB III remains active and so far has received contributions of the order of $15.9 million. As of January 31, 2009, 27 projects had been approved and about $6.6 million committed. Since its inception, therefore, TFSCB has received contributions of about $41.6 million and has committed about $30.0 million to some 153 statistical capacity building projects. At present, TFSCB III is scheduled to close on December 31, 2012 87 Previous evaluations 15. TFSCB has been subject to two evaluations that have been managed by the World Bank and one managed by the UK‟s Department for International Development, one of the main donors. In 2003, the trust fund was evaluated jointly with PARIS21; the report of this evaluation can be accessed here. In 2006/07, an internal review was completed. The results of this exercise are not publicly available, but will be made available, if required to the selected team of consultants. In 2007, DFID sponsored an evaluation of both the Trust Fund and an associated project being implemented by the Bank to support statistical capacity building in sub-Saharan African countries. This report is not yet publicly available, but will be made available, if required to the selected team of consultants. 16. In addition to these direct evaluations, the Trust Fund has also been evaluated as part of wider programs. In 2008 an independent evaluation of the Marrakech Action Plan for Statistics included, inter alia, TFSCB support to national strategies for the development of statistics. If required, copies of the report of this evaluation will be made available to the selected team of consultants. There are also some evaluations of individual projects financed by TFSCB. The Evaluation Context 17. The Marrakech Action Plan for Statistics provides the context for World Bank support to statistical systems in developing countries and related activities until the end of 2010. A number of the key elements of MAPS, including PARIS21 and the Trust Fund will be subject to independent evaluations in 2009, with a view to reviewing progress so far and to developing a coordinated strategy for the period 2010 to 2015. The evaluation proposed in these terms of reference is part of this process and will be carried out in close coordination with the parallel evaluation of PARIS21. Both will build on the results of the earlier evaluations as well as the 2008 evaluation of MAPS as a whole. Purpose 18. The purpose of the evaluation is:  To assess the impact of the Trust Fund in supporting the preparation of national strategies for the development of statistics and in helping to strengthen the capacity of national statistical systems;  To identify what changes might be needed in the design of the Trust Fund or in the ways in which it operates to improve its effectiveness and impact in the future; and  In the light of this evaluation, the MAPS evaluation and the related evaluation of PARIS21, to recommend what changes might be needed to enable the Trust fund to support not only the preparation of strategies, but also their implementation. Scope 19. The following list of topics is intended as a guide for the evaluation team, other issues may also be considered. 88  Coverage  The number of projects by country, topic and other characteristics and the extent to which coverage has reflected TFSCB guidelines  Implementation  The time taken for projects to be implemented and for activities to be completed as compared with the original timetables  Problems encountered during implementation  Actual and budgeted expenditure  Activities planned and those actually completed  Achievements and impact  Outputs proposed and achieved  Project ratings  Whether or not projects were independently evaluated  The actual or expected impact of a sample of projects  Follow-up and sustainability  To what extent have projects been followed up and to what extent individual project outputs and outcomes have been and are expected to be sustained?  Process  How easy or otherwise was it for implementing agencies to access the Trust Fund.  What problems were encountered in getting the grant approved and then getting activities started?  What problems were encountered in dealing t he Bank‟s procurement, financial management and disbursement guidelines?  To what extent were agencies discouraged from access in the trust fund and what were the main constraints they faced Management of the review 20. The World Bank will hire a team of three individual consultants as outlined below. Terms of reference for each member of the team are set out in Attachment 2. The team will be responsible to Misha Belkindas, Manager, DECDG and will liaise with the TFSCB Administration Unit. The Administration Unit will be responsible for providing access to relevant project documentation. 21. The World Bank and the PARIS21 will set up a joint Reference Group that will be responsible for responding to the outputs from the team and for providing guidance as required. The Reference Group will have members from the PARIS21 Bureau, Donor representatives, the OECD Evaluation Department, the World Bank and PARIS21. Timing 22. It is anticipated that the evaluation will require about three months to complete and each member of the team will be expected to work for about 50 days. The work is expected to start in June and be completed by the end of September 2009. 89 Outputs 23. The consultants will be required to prepare an inception report within 10 days of the start of the assignment setting out how the tasks will be carried out, what information will be required and what kind of information will be prepared. The inception report should also contain an outline of the final report 24. A draft final report will be prepared after 45 days of work. This will be reviewed by the World Bank who will provide comments and feedback within 14 days. A final report will then be provided within a further fifteen days. One or more members of the consultants team may be required to make a presentation on the final report to a Consultative Group meeting that is expected to take place together with a PARIS21 Steering Committee meeting in November Work Program 25. It is expected that the work program for the team will be as set out below. Timing Activities Location Weeks 1 and 2 Initial review of documentation, consultations and Washington DC preparation of the inception report Week 3 Preparation of instruments to be used to collect To be confirmed information Weeks 4 to 8 Consultations with implementing agencies, donors Field trips to a sample of up and PARIS21 to 10 projects Weeks 9 to 10 Analysis of the information collected from field trips To be confirmed and questionnaires etc Weeks 11 to 12 Drafting of the final report To be confirmed Week 13 Presentation of the draft final report Washington DC Week 16 Presentation of final report To be confirmed To be confirmed Presentation of the final report to the Consultative To be confirmed Group 26. It is anticipated that a variety of methods will need to be employed to collect information and get feedback from grant recipients, donors, government officials and others, including the PARIS21 Secretariat. The team will be responsible for the design, implementation and analysis of questionnaires, reporting forms and other instruments. 27. The team will be expected to make two visits to Washington DC and also to undertake field trips to a sample of recipient countries and projects. It is expected that up to 10 projects will need to be visited in two or three different continents. The program for field trips will be agreed with the World Bank and set out in the Inception Report. The World Bank will be responsible for making arrangements for the field trips. Consultants 28. The World Bank will hire a team of three suitably qualified consultants. It is anticipated that the team will be made up of:  An experienced statistician or manager of statistical capacity building projects and programs  An evaluation specialist  A project management specialist with a background in the financial management and procurement aspects of donor funded projects and programs. 90 29. One member of the team will be designated team leader. 30. Those bidding for the work are asked to set out how they propose to undertake the evaluation as well as submitting a financial proposal. 91 Attachment 1: Logical Framework Narrative Summary Objectively verifiable Means of verification Assumptions indicators Goal: Developing a culture of 1. By 2010 reduce by 1. MDG monitoring evidence-based policy 75% the number of progress reports. WDI making and monitoring in countries unable to report database support of economic on progress to the MDGs. 2. JSA reports on PRSPs development and poverty 2. By 2006 identified and PRSP progress reports reduction improvements in PRSP monitoring in at least 20 countries Purpose: Improved capacity of 1. By 2006 all developing 1. PARIS21 reports 1. NSDSs lead to an national statistical systems countries to have national 2. GDDS metadata improvement in capacity in developing countries, strategies for the published on IMF’s DSBB and output and national based on a nationally development of statistics 3. Monitoring of the policy makers use better owned and developed 2. By 2010 all developing Marrakech Action Program information to develop and national strategies for the countries to have detailed on Statistics (MAPS) implement pro-poor development of statistics capacity building programs 4. IMF ROSC reports policies. in operation 2. National Governments 3. By 2006 improved provide acceptable level of capacity and effectiveness resources to sustain of statistical systems in 30 improvements. developing countries Outputs: 1. Projects in place to 1. At least 30 projects to 1. TFSCB progress and 1. Projects are well support the preparation and develop NSDSs in place or annual reports designed and properly adoption NSDSs in poor successfully completed by 2. Project evaluation implemented and agreed countries the end of 2006. reports outputs are delivered on 2. Specific improvements 2. At least 25 other time and within budget. in the capacity of statistical projects to improve 2. Countries are aware systems in selected statistical capacity in place of, and make use of the countries or successfully completed Trust Fund. by the end of 2006 3. NSDSs lead to funded implementation programs Activities: Inputs: 1. Management, of the Total fund: $4-8 million per 1. Donors are willing to Trust Fund by the Bank, year over lifetime of project, make regular contributions and implementation of depending on the level of 2. WB has sufficient some projects if necessary. donor contributions capacity to provide 2. Assistance to poor Bank contribution about appropriate technical countries to help prepare $0.25 million per year gross assistance as required well-designed projects ($0.13 million per year after 3. Bank is able to provide 3. NSDS projects overheads) adequate project 4. Non-NSDS projects Project supervision about supervision and monitoring $0.5 per year 92 Attachment 2: Terms of Reference for Team Members Statistician or statistical manager The statistician should be a qualified and experienced statistician or economist with at least 15 years experience in the management and leadership of official statistical agencies and systems. Experience in the design and implementation of statistical capacity building programs and projects, especially in developing countries is required. Experience of international statistical programs and the organization and management of the international statistical system would be an advantage. He or she should be fully fluent in English; the ability to communicate in French and/or Spanish would also be an advantage. In consultation with the other team members, the TFSCB Administration Unit and the Reference Group, the Statistician will be required to carry out the following tasks. (i) Based on the TFSCB Logical Framework and the description of the scope set out above prepare a program for the evaluation. (ii) Identify the main information needs for the evaluation and the sources to be used. (iii) Design different instruments to collect information from different groups of stakeholders, including executing agencies, national statistical agencies, the donors, representatives of recipient governments and data users. (iv) Analyze the data derived from secondary sources and that collected during the evaluation itself. (v) Review background documentation and other relevant material. (vi) Review the design of a sample of projects and make assessments of their quality at the design stage, the extent to which the design was implemented and, as far as possible, their cost effectiveness, efficiency, outcome and impact. (vii) Assess the extent to which the technical design of TFSCB supported projects has been in line with the Guidelines and the overall requirements of the Marrakech Action Plan for Statistics. (viii) Make recommendations on the organization and management of the Trust Fund following the evaluation, including, if required, changes to the Guidelines and operating procedures. (ix) Contribute to the Inception Report, the Draft Final report and the Final Report and, if required, participate in meetings to present the results of the evaluation to the Consultative Group and World Bank management. Evaluation specialist The evaluation specialist should be a senior and an experienced evaluator with at least 15 years experience in the design and implementation of evaluations of projects and programs, with substantial experience in the evaluation of global development programs. Specific experience of the evaluation of programs and projects in developing countries would be an advantage. He or she should be fully fluent in English; the ability to communicate in French and/or Spanish would also be an advantage. 93 In consultation with the other team members, the TFSCB Administration Unit and the Reference Group, the Evaluation Specialist will be required to carry out the following tasks. (i) Based on the TFSCB Logical Framework and the description of the scope set out above prepare an overall design and a program for the evaluation. (ii) Identify the main information needs for the evaluation and the sources to be used. (iii) Contribute to the design of the different instruments to be used to collect information from different groups of stakeholders, including executing agencies, national statistical agencies, the donors, representatives of recipient governments and data users. (iv) Review background documentation and other relevant material. (v) Review the design of a sample of projects and make assessments of their quality at the design stage, the extent to which the design was implemented and, as far as possible, their cost effectiveness, efficiency, outcome and impact. (vi) Assess the extent to which the technical design of TFSCB supported projects has been in line with the Guidelines and the overall requirements of the Marrakech Action Plan for Statistics. (vii) Make recommendations on the organization and management of the Trust Fund following the evaluation, including, if required, changes to the Guidelines and operating procedures. (viii) Contribute to the Inception Report, the Draft Final report and the Final Report and, if required, participate in meetings to present the results of the evaluation to the Consultative Group and World Bank management. Project management specialist The project management specialist should be an experienced project manager with at least 15 years of experience in the design, implementation and supervision of donor funded aid projects in developing countries. Experience of different aspects of project design, including financial management, procurement and disbursement is required. Specific experience of the design and management of World Bank funded projects would be an advantage. He or she should be fully fluent in English; the ability to communicate in French and/or Spanish would also be an advantage. In consultation with the other team members, the TFSCB Administration Unit and the Reference Group, the Project Management Specialist will be required to carry out the following tasks. (i) Based on the TFSCB Logical Framework and the description of the scope set out above prepare an overall design and a program for the evaluation. (ii) Identify the main information needs for the evaluation and the sources to be used. (iii) Contribute to the design of the different instruments to be used to collect information from different groups of stakeholders, including executing agencies, national statistical agencies, the donors, representatives of recipient governments and data users. (iv) Review background documentation and other relevant material. (v) Review the design of a sample of projects and make assessments of their quality at the design stage, the extent to which the design was implemented 94 and, as far as possible, their cost effectiveness, efficiency, outcome and impact. (vi) Assess the extent to which World Bank requirements for financial management, procurement and disbursement, had any effect on the length of time required for the project to be approved and become effective and whether implementing agencies were able to access the advice and support needed to put these requirements into effect. (vii) Make recommendations on the organization and management of the Trust Fund following the evaluation, including, if required, changes to the Guidelines and operating procedures. (viii) Contribute to the Inception Report, the Draft Final report and the Final Report and, if required, participate in meetings to present the results of the evaluation to the Consultative Group and World Bank management. 95