43160 Evaluation of the ECA Innovation Grant Program, FY05-FY07 Heidi S. Zia Antonieta Romero-Follette Evaluation of the ECA Innovation Grant Program, FY05-07 Heidi S. Zia Antonieta Romero-Follette WBI Evaluation Studies No. EG08-137 World Bank Institute The World Bank Washington, D.C. January, 2008 Acknowledgments This report has been written under the supervision and leadership of Heidi Zia, task manager. The author wishes to thank Ms. Zia for her advice and guidance; to ECA staff for participating in the interviews that helped complement information on the effectiveness of the ECA Learning Innovative Grants Program; and the assistance of Humberto S. Diaz in formatting and graphics. The World Bank Institute Evaluation Group (WBIEG) evaluates learning by staff of the World Bank and activities of the World Bank Institute (WBI). The findings, interpretations, and conclusions expressed in WBI Evaluation Studies are entirely those of the author and do not necessarily represent the views of the World Bank Group, including WBI, for which this study was completed. WBI Evaluation Studies are available at http://www.worldbank.org/wbi/evaluation Suggested citation: Heidi S. Zia and Antonieta Romero-Follette. 2008. Evaluation of the ECA FY05-07 Innovative Learning Grants. Report No. EG08-137 Washington, DC: World Bank Institute. Vice President, The World Bank Institute Rakesh Nangia, Acting Manager, WBI Evaluation Group Karin C. Millet, Acting WBI Task Manager Heidi S. Zia ii Abbreviations and acronyms ESW Economic and Sector Work FSAP Financial Sector Assessment Program CPS Country Program Strategy CT Country team ECA Europe and Central Asia IEG Independent Evaluation Group of the World Bank TA Technical Assistance TF Trust fund WBI World Bank Institute WBIEG World Bank Institute Evaluation Group iii Table of contents ABBREVIATIONS AND ACRONYMS...................................................................................iii 1. BACKGROUND, OBJECTIVES AND METHODOLOGY................................................... 1 A. Background....................................................................................................... 1 B. Evaluation objectives......................................................................................... 1 C. Methodology...................................................................................................... 2 2. DESK REVIEW OF GRANT APPLICATIONS AND POST TRAINING REPORTS.............. 3 A. Findings on the applications............................................................................. 5 B. Findings on post training reports....................................................................... 5 C. Issues on application forms, post training reports and documentation.............. 6 D. Recommendations to improve application forms, post training reports and documentation ........................................................................................................... 7 3. STAFF INTERVIEWS.................................................................................................... 9 A. Effectiveness, uses and impact of the grants program...................................... 9 B. Impact and replicability of learning events ..................................................... 12 C. Value of the innovation grants......................................................................... 14 4. CONCLUSIONS AND RECOMMENDATIONS ............................................................... 17 Appendix 1: ECA region innovation grant fund guidelines for FY05 and FY06 announcement.................................................................................... 19 Appendix 2: WBI's suggested modifications to ECA region innovation grant application form................................................................................. 22 Appendix 3: Questionnaire to recipients of an ECA region innovation grant ....... 24 Appendix 4: Grants and staff interviewed.............................................................. 26 iv 1. BACKGROUND, OBJECTIVES AND METHODOLOGY A. BACKGROUND 1.1 In FY05, the ECA Learning Committee introduced the Innovation Grant Fund to increase the contribution of knowledge-sharing and strategic learning to business results. The purpose of the Innovation Grant Program (IGP) is to "encourage greater knowledge- sharing with clients or other agencies on innovative solutions, particularly where these can support business development in the short- to medium-term."1 The fund is to be used according to specific guidelines (Appendix 1) and applicants can submit applications twice a year (Round I and Round II). In FY06, the guidelines included a new `use-it or lose-it' rule, by which all grants awarded must be expensed by the end of each Round.2 In FY05, the IGP disbursed funds for 5 learning events. In FY06, the number rose to 7 events, and in FY007, again it rose to 9 events. Between FY05 and FY07, the amount of awards evolved as follows: $124,600 in FY05; $304,200 in FY06, and $241,400 in FY07. The average grant size is $31,914 for the FY05-07 period. One quarter of the approved grants did not get implemented, mainly due to unforeseen external factors (e.g. coups d'état, governments' indecisions) or to staff's work program time constraints.3 For this period, grants not implemented amount to $192,050.4 The Innovative Grant Program has up to $300,000 available annually. 1.2 An independent evaluation of previous ECA grants is mentioned in the FY05 guidelines. Two lessons stemming from that evaluation are incorporated in the guidelines: (1) management commitment to grant outcomes is a critical element in sustaining learning gains, and (2) learning events that meet Bank norms and staff preferences while contributing directly to Bank operational work achieve greater success. The current evaluation is the first one to focus on ECA's Innovation Grant Program value. B. EVALUATION OBJECTIVES 1.3 This evaluation for the ECA Knowledge and Learning Committee identifies benefits of the Innovation Grant Program and provides recommendations on how the administration of the IGP grants may be improved. 1ECA Region Innovation Grant Fund Guidelines, see Appendix 1. 2However, delayed activities of two grants approved in FY05 (National Development Plan Study Tour to Spain and Ireland) and in FY06 (Framework for GEF Mediterranean Strategic Partnership) were allowed to be resubmitted and moved to the next fiscal year. 3Consequently, midway through the program, the Regional Learning Coordinator reallocated funds. 4Figures calculated (Appendix 4) based on FY05 and FY06 ECA's Learning Program Innovation Grant Fund Recipients, list posted in the Bank's intranet, and from FY07 grantees' application forms. 1 1.4 The objective of the evaluation is to answer two questions: Did the grantees meet the requirements by increasing Bank staff knowledge, skills or understanding? Did the grant activity either support business development or contribute to the identification or application of solutions to development challenges?5 Specifically, we determined whether grantees: · Complied with the grant requirements, as specified in the application prerequisites. In FY06, the requirements were to: o increase the knowledge, skill or understanding of Bank staff; o engage multi-sector teams in knowledge-sharing activities; o identify goals and intended outcomes and a mechanism to monitor and report results; and o include cost-sharing from unit learning funds or outside partners. · Contributed to one or both of the criteria for selection. In FY06, the criteria for selection were to: o support business development through innovation in the short to medium term o contribute to the identification or application of solutions to development challenges. C. METHODOLOGY 1.5 The methodology of the evaluation consisted of a combination of desk reviews of applications and post training reports coupled with ECA staff interviews. Interviews aimed at obtaining insight about the extent of relevance and usefulness of knowledge, information or skills acquired versus expectations. Interviews also aimed at identifying ideas that may surface about learning events that should be pursued afterwards. The questionnaire used is presented in Appendix 3. 1.6 While it is desired to have some concrete evidence of business development or application of solutions to development challenges in the post training reports, it was recognized that not sufficient time has passed to have produced such outcomes. Therefore, when evaluators interviewed members of the country unit they asked more about outcomes and aimed their questions at capturing extent of fulfillment of outcomes or identification of ideas that should be pursued, e.g., find out about where the learning activity was focused. 5In each year, the wording was slightly different in the grant application. Evaluators took this into account to determine if the grantees met the objectives of the grant in the year of application. 2 2. DESK REVIEW OF GRANT APPLICATIONS AND POST TRAINING REPORTS 2.1 The desk review consists of a detailed study of 18 applications and 16 post training reports (not necessarily for the same learning activity) and related documents that are available.6 As already mentioned, the objective of the desk review is to find out whether grants: · Comply with requirements, and · Support business development or contribute to the identification or application of solutions to development challenges. 2.2 Table 1 provides a summary of desk review findings with regard to compliance7 of the grants with pre-determined prerequisites and reports' criteria. The desk review reveals that out of a total of 21, 18 (86%) grants had available applications and comply with the following requirements: · Increase knowledge or skill understanding, · Engage multi-sector teams in knowledge sharing activities, and · Identify goals and intended outcomes. 2.3 Ten of 11 of the grants (52%) fully comply with: · Identifying a mechanism to monitor and report results. 2.4 And 15 of the grants (71%) comply with: · Including cost-sharing from unit or outside partners. 2.5 The comparison between application forms and post training reports (15 sets) reveals that: · 13 of 15 (87%) grants have expected-final-outcomes that fully coincide with intended outcomes. 6The applications and reports correspond to 15 sets of applications and reports for the same lending activity, 3 application forms without a report, 1 report without an application, and 2 grants with neither application or post-activity report. (see Table 2 for details). 7Based on the number of grants with available application forms and/or a post-activity report. 3 Table 1. Compliance of grants with applications' prerequisites and reports' criteria1 Innovation Grants data Application form Post-activity reports Criteria orlli nis d itnu de la edct dev e sk e,g eamtr seitivit en ot st fin th ac nti msi sul omfr ret ed de hie ac edlw cto ngi sei-t ngi ar por er th ex ar tori sltuserfosno nok gni sh ndasal ngir s ncii han pexeta e ha ectp mec re mon d nd mul e- go a d -sstoc rtn coyl ts es pa nda ful atiic se ta d es rs ge ed desi ed es ecfler rt ons ppla FY #Grant Grant Title ea crnI denu ngaE dglewonk edfiit edfiit anro ud omctuol mr endI com out endI out end com monit Incl or Int out Repo fina fonI alu act FY05 1 Improve Bank effectiveness in the wave of EU x x accession countries FY05 2 EU fiduciary harmonization n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. FY05 4 Communications clinic on the regional study on x x poverty, growth and inequality FY05 5 Risk management workshops n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. FY05 6 Human development sector swaps in Croatia p x n.a. n.a. n.a. (preparation work done only, no report issued) FY06 10 Design of regional/local development policies p x based on EU experience2 FY06 12 National development plan study tour to Spain and n.a. n.a. n.a. n.a. n.a. x x Ireland FY06 13 Competitive environmental infrastructure p x investments, EU financial support (Spain) FY06 14 Kyrgyz team learning on SWAP3 x n.a. n.a. n.a. FY06 15 Sharing the lessons of reform in the dual-system of p x x training FY06 18 Forum on job creation in Eastern Europe and p x Central Asia FY06 19 Transit corridors performance measurement in p x Central Asia FY07 20 Monitoring and evaluation framework for the GEF p Mediterranean Strategic Partnership FY07 21 Consumer protection in financial services and develop CPFS good practices FY07 22 Trade policy decision-making and maximization of x multilateral trade gains FY07 24 Job creation based on European models--Ireland & Spain study tours FY07 26 Replicability of innovation finance experience of x x n.a. n.a. n.a. Israel in ECA4 FY07 27 EU8 experience and new financial framework for x x x EU structural funds FY07 28 Ukraine communications for carbon finance x advocacy FY07 30 State-of-the-art judicial systems x FY07 31 Poland study tour EU funds absorption x Total #: 21 Documents that fully comply with criterion: 18 18 18 11 15 13 14 3 Fully complies with criterion. "p" Partially complies with criterion. "x" Does not address criterion. "n.a." Not available 1Report criteria were set by evaluators. ECA did not establish specific post training report prerequisites. 2Explicit post training report that summarizes knowledge/lessons learned and is linked to application expected results. 3Evaluation of the application originally proposing a study tour to Nepal. 4Report not requested; but training results were incorporated into a conference. 4 2.6 Concerning results, the desk review reveals that: · 14 of 15 (93%) reports clearly reflect that expected final outcomes are achieved, and · Only 3 grant applications explicitly explain the mechanisms to monitor or comment on the process of monitoring achievement/application of results. 2.7 Finally, the desk review also reveals some issues related to the documentation process, as well as the quality of the post training reports. A. FINDINGS ON THE APPLICATIONS 2.8 The completed applications provide a clear action plan for the learning activities. All of the 18 FY05-07 grant applications that were reviewed provide well-rounded descriptions of the knowledge/information gap or skill expected to be acquired, of the actual learning event, and of the intended benefits to be reaped from funding the proposed training event. Budget details are clear and grantees provide adequate information on "reporting results." However, some applications do not provide a description of the proposed monitoring mechanism. Among those that do provide a description, some proposed monitoring the activity while others proposed to monitor using/applying results. Apparently, it is not clear to the applicants whether the "monitoring aspects" refers to the execution of the learning activity or to the incorporation of the expected outcomes in Bank's work. B. FINDINGS ON POST TRAINING REPORTS 2.9 Post training reports are not uniform and not linked to application forms. To date, the 16 post training reports reviewed come in all forms, sizes and degrees of completeness. This reflects the fact that the ECA Innovation Grant Program does not require a specific format for the post training report from applicants.8 Grantees provided related documents­­such as workshop agendas, symposium reports, power point presentations or special papers on lessons learned and recommendations­­as post training reports. Only a few reports explicitly link proposed achievements with actual results. For example, the FY06 Designing of regional and local development polices based on EU experience (grant #10) proposed to "exchange information and experiences on best practices with institutions involved and to prepare a memo summarizing the knowledge gained." Subsequently, a post training report was submitted explaining how and what parts of the expected learning was achieved. Many reports, however, provide scant evidence on whether activities achieved the expected results. Nevertheless, the ECA Lead Knowledge 8Any document is acceptable­e.g. a policy note to a client, a project paper, a report on a dialogue with clients, or a conference­as long as it reflects that the new learning has been incorporated into the Bank's work. 5 and Learning Officer seems to be aware of how grants learning gained is being applied or used.9 And for some of the late FY06 and the FY07 events, it is still premature for results to materialize. Thirteen of the 16 reports (81%) either omit or unclearly explain whether specific monitoring benchmarks are accomplished.10 These reporting shortcomings hinder transparency, accountability, written institutional memory, and evaluation of grant compliance with the originally proposed terms. C. ISSUES ON APPLICATION FORMS, POST TRAINING REPORTS AND DOCUMENTATION 2.10 Table 2 describes the current status of grants' documentation­as provided by ECA­, which reveals that there is room for improvement in the documentation of grants' applications, post training reports and related documents. Going forward, documentation of grants needs to be improved. Details on implementation, postponement, cancellations or modifications to grants' delivery status (which were provided verbally) also need to be documented in paper.11 It would be useful to also document in writing how each grant's learning contribution is been used or applied. 9Ms. Olive clarified that: "In particular, I have not required a BTOR from a Director because I know he is applying what he learned in the forthcoming Knowledge Economy Forum­the results will thus be in a major conference." 10For example, the Forum on job creation in Eastern Europe and Central Asia grant application states that "results would be monitored by both the process and outcome indicators, which would be: Process indicators: (i) the successful completion of the learning events, (ii) broader presentation of key conclusions from these events through GDLN. Outcome indicators: a draft document for a multi-sectoral operational strategy to support employment in depressed regions." However, (i) the mere completion of the training activities does not provide an idea of the success of the learning, measurable indicators of outcomes or impacts. (ii) The post-training draft paper summarizes the conference conclusions, it indicates that staff is working on developing an "operational approach," but it does not mention whether the development of a multi-sectoral operational strategy is being prepared. The draft does not mention whether the GDLN presentation is planned or actually took place. It does not describe whether the other intended outcome of achieving a broader consensus between the Bank and counterparts on policy was actually achieved. In contrast, results are expressed more clearly by the grantee in his interview statement: "ECA's main development challenge is jobs creation. The learning enriched our region's approach to job creation and how we analyze the problem and think of solutions: 1) although we have used the MILES framework, the training deepened our understanding of it. 2) The training widened the circle of people who became aware of this job creation framework, both within the Bank and our counterparts." 11For reason explaining non implementation, see Appendix 4, B. Grants not implemented, last column. 6 Table 2. ECA Innovative Grants: Total number, data status and interviews Grants FY05 FY06 FY07 Total % Approved1 9 10 12 31 100% Minus: Postponed to next FY 1 2 --- 3 12% Did not happen 3 --- 3 6 23% Not funded --- 1 --- 1 4% Implemented 5 7 9 21 38% Of which: 100% We have application and report 2 6 7 15 71% Only application 1 1 1 3 14% Only post training report --- --- 1 1 5% Neither 2 --- --- 2 10% Actual grants 5 7 9 21 100% 1 Listed online Interviews as of October 25*: FY05 FY06 FY07 Total % Done 4 6 7 17 81% Pending --- --- 0 0 0% Total 4 6 7 17 81% * No more interviews were conducted after October 25, 2007. D. RECOMMENDATIONS TO IMPROVE APPLICATION FORMS, POST TRAINING REPORTS AND DOCUMENTATION 2.11 Based on the above, WBIEG findings and recommendations are as follows: · Application forms: Must be revised to include two separate sections­­one for describing monitoring mechanisms and another for outcomes­­and also to include an additional section to document grant approval and implementation status. The latter section would be reserved for managing, monitoring and documentation purposes­­for the exclusive use of ECA Learning Committee and should include the following fields: title of the grant, a record number for filing purposes, initiation and expected date of delivery of activities, expected and actual date of post training report submission, and a field for indicating status of deliverables (cancellation, postponement and modifications of grant implementation, and post training report status). Applicants could be asked to identify potential risks that may affect delivery or accomplishing expected results. WBIEG proposed changes are presented in Appendix 2. · Post training reports: WBIEG recommends issuance of standardized post training report forms. WBIEG also advises ECA to create a simple, but uniform template to be used for the presentation of Post Training Reports, with a structure similar and related to that of the Application Form. By providing a simple template that 7 guides the structure and content of the post training report, grant managers would improve report completeness. Some of the information that was gathered in the interviews could be requested to be included in post training reports. Grantees could report in a couple of sentences a summary, for example, on the specific outcomes gained, their use, and the immediate impact of the learning gained. Having a post training template would ease monitoring and evaluation of grant compliance, and would assure timely submission of activities and deliverables. · Documentation: Eventually, WBIEG recommends the use of a database application for documentation purposes. The database would help store and manipulate grant data, and would allow quick identification of individual grant status. 8 3. STAFF INTERVIEWS 3.1 As of October 25, 2007, WBIEG interviewed the 17 ECA grantees that implemented grant learning activities.12 The objective of the interviews was to gain insight about the effectiveness, usefulness, impact and value of the grant program. Results are presented in this section. A. EFFECTIVENESS, USES AND IMPACT OF THE GRANTS PROGRAM 3.2 Based on the interviews, all of the grantees meet ECA Innovation Learning Grants' requirement of increasing Bank staff knowledge, skills or understanding on a specific topic. The grant program was effective in generating new capacity for ECA staff in terms of increased number of projects, improved dialogues with clients, enhanced team work among Bank staff, and increased solutions to development challenges: · Increased number of projects. Half of the grantees interviewed indicated that their newly acquired knowledge or skills contributed to increasing the number of projects.13 Specifically, the learning acquired translated into contributions to the CPS, ESW, FSAPs, or TA work, into concrete project interventions. Another grantee highlighted that the grant enabled development of a set of good practices on the subject of consumer protection in financial services in Slovakia. Additionally, through the email summarizing post-activity results, another grantee (not interviewed) informed that "the new skills have been applied and the results of the activities carried out under the grant have been incorporated into the communications strategy and the ECSSD carbon mitigation program that defines opportunities for future Bank work in the Ukraine." · Stronger dialogues and mutual understanding between WB staff and country clients. Eight of the interviewees also used the newly acquired capacity in building or strengthening dialogue with client countries' authorities, in improving Bank's 12On December 19, 2007, ECA provided clarifications indicating that four additional grants­originally listed as "not happening"­had actually taken place. (This is another example of the problems with documentation management and the effect of delays in event's implementation). At that point WBIEG could no longer undertake further interviews. 13For example, in the particular case of the Transit corridors performance measurement in Central Asia FY06 Grant, the new capacity led the government of Kazakhstan to request the Customs Development Project from WB. In other cases, the new capacity generated the early planning stages of new lending operations. If it was too early to see concrete new projects, staff indicated that there is potential or expectations that the new capacity will contribute to developing future operations. Similarly, the new knowledge/experience derived from the State-of-the-art judiciary systems FY07 program was used in the following Bank's work: the Bulgaria judicial expenditure review ESW (which will lead to a proposed project), the Macedonia judiciary project, the Turkey judiciary policy note, and the preparatory work for the Turkey judiciary modernization project. 9 knowledge about their clients or in developing a new management or coordination approach within a project or a multi-donor partnership (e.g. the GEF Mediterranean Strategic Partnership).14 One grantee who moved to LAC, mentioned that a presentation on lessons learned was given to Peruvian government.15 Another one mentioned that the workshop financed by the grant provided the dialogue for country stakeholders to come to an agreement and enabled the Government to present to Parliament a program with national and international support. · Promoted team building and mutual understanding among ECA staff. Several grantees underscored the positive value of learning activities that enhanced team building within ECA staff from different offices. These grantees highlighted that "the training activity helped ECA staff get acquainted with each other." · Increased solutions to development challenges.16 Again, the majority of interviewees (16 of 17) considered that the learning event contributed to identifying or applying solutions to ECA development challenges. In a couple of learning events­­e.g. the Risk Management Workshops17 and Russia Trade Policy Decision-making­­the objective was to disseminate previously-identified solutions to development challenges or learn how a specific country addressed them. Box 1 describes (in words of the grantees) how the individual learning programs contributed to identifying or applying solutions to ECA development challenges. Box 1. Contributions to identifying or applying solutions to ECA development challenges, according to grantees, FY05-07 "Learning helped us in positioning the Bank in a group of countries looking into EU accession. It sharpened the value-added brought by the Bank in ECA countries." "The development challenge is to align support we provide with that provided by the EU." "If we talk about how to be clear in communicating policy recommendations and key messages stemming from research studies, the ECA innovation grant had an indirect contribution." "Overall, it has assisted us in our dialogue related to programming of loans in our client countries and to propose solutions." 14Having stronger dialogues is considered a valuable skill because ECA's strategic direction going forward "places a premium on developing reliable partners, if we are to minimize programmatic reductions and appropriate burden sharing." (Strategy and Performance Contract, SPC Corporate-Final as of October 10, 2006, Planning quarter Q1 FY2007). 15In his own words: "In my subsequent assignment I used the learning experience gained from the Ireland study tour to give a presentation about social partnerships in Ireland to the Peruvian authorities interested in learning about the role of social pacts." 16Question 7, see Appendix 3. 17This Turkey workshop had already taken place in Colombia and Romania, with adaptations and customized to country's situation, using even the same speakers. 10 "Adapted the coastal cities project for Croatia and developed a new project in Bulgaria." "The main development challenge that ECA faces is the creation of jobs. The learning enriched our region's approach to job creation and how we analyze the problem and think of solutions." "If we consider contribution to financing a challenging project with partners, the grant was a small tool used to set in motion a partnership. That is how it is a contributor to a solution." "In the context of Russia, the client developed a trade strategy discussed by government that is likely to be approved." "The grant contributed very much to our work. Israel is a prime example of a public sector led innovation finance program that too often we disregard in conventional Bank practice or thinking." "The learning event has illuminated, to several ECA clients, the risks involved in trying to emulate the German dual system of training." "New techniques of trade facilitation are very important for the land-locked and poor countries of Central Asia." "This activity brought together colleagues from different parts of the Bank and allowed them to think together on a non-formal setting to see what could work, how Bank could be more effective." "Another impact is that it generated a greater understanding of colleagues work and strengths, building bridges across sectors within ECA" "The program enabled the team to lead better and more informed dialogue with Government on operations under preparation." "The international workshop held last June provided the opportunity for the Slovak finance ministry, the central bank and the bankers' association to come to an agreement on three critical areas, where they had previously disagreed. Then the Government was able to present to Parliament a program for financial consumer protection." 3.3 Table 3 describes the possible outcomes gained from the training events as reported by grantees. In summary, the majority of grantees agreed that the two most valuable outcomes were: · Learning new lessons or skills, and · Identifying solutions to development challenges, some of which relate to satisfying country needs or gaining background information. 11 Table 3. Outcomes gained from learning events funded by the Innovation Grant Program Outcomes gained (most valuable first) 17 Interviewees Answers % 1. Learned new lessons/skills 14 82 % 2. Identified solutions to a development challenge 12 71 % 3. Enriched studies, projects, operations, best practices 8 47 % 4. Identified new entry points for Bank's T.A. or new work 6 35 % 5. Identified causes to a problem 4 24 % 6. Other 4 24 % 7. Identified country needs, background information 2 12 % 8. Complemented an ongoing operation 1 6 % 9. Tested applications to solve a problem 1 6 % 10. Continuation or dissemination of previous research/work 0 0 % B. IMPACT AND REPLICABILITY OF LEARNING EVENTS 3.4 With regards to the immediate impact of the learning events, 16 (94%) interviewees identified a positive impact in enhancing work aspects or addressing development challenges; and the remaining one described the impact as leading to a future dissemination paper or BBL of knowledge gained. With respect to the ECA Strategy and Performance Contract, themes of learning events comply with the strategic objectives pursued. Table 4, below, summarizes the impact of each grant in the words of the grantees. Table 4. Impact of learning events according to grantees, FY05-07 FY Grant and Country Impact 05 1. Strengthening WB assistance in "Creation of action plans in key areas that were being tracked for Bulgaria, Croatia and Romania in one year. We gained better understanding of Bank's value- maximizing benefits of EU added to meet client needs in EU accession countries. We accession articulated our approach in CPS driven by ideas developed in this workshop, mainly in Bulgaria." 05 2. Workshop on EU fiduciary "The event had a direct operational impact: it helped ECA staff harmonization understand the EU system better; operationally linked to learning activity and strengthen partnerships. The event gave staff knowledge that they in turn promoted to teams. We see an increase on the number of projects implemented using EU systems." 05 3. Communications clinic on the "Authors were better able to communicate key messages of their regional study on poverty, growth sector economic work, improved their presentation skills, and and inequality had a better-defined dissemination strategy. We all saw results right away. Trainees were videotaped before and after they received couching and training. They were able to do a better job and a remarkable difference in their performance." 05 4. Risk management workshops, "We raised awareness, participation in FSAPs, exposure to Turkey seminar­job creation for literature and interaction with clients." Turkey and the South Caucasus 12 FY Grant and Country Impact 06 5. Designing regional and local "Staff involved became highly motivated with regional development policies based on EU development. New impetus gained in one company town experience operations, specifically BOR (Serbia)." 06 6. National development plan "It gave us more insight for our dialogue. It provided us with study tour to Spain and Ireland enhanced and new knowledge, we learned lessons." 06 7. Competitive environmental "A project in Bulgaria, application in Croatia and better infrastructure investments, EU coordination among regional offices." financial support (Spain) 06 8. Sharing the lessons of reform "We discussed the findings with education officials in Slovakia, in the dual-system of training Ukraine, and Lithuania." 06 9. Forum on job creation in "Impacts: 1) although we have used the MILES framework, the Eastern Europe and Central Asia training deepened our understanding of it. 2) The training widened the circle of people who became aware of this job creation framework, both within the Bank and our counterparts." 06 10. Transit corridors performance "Four countries of the Central Asia agreed to conduct a transit measurement in Central Asia corridor performance measurement study in the Central Asian region (ongoing currently)." 07 11. Development of a monitoring "Agreements reached during the meeting and reflected in UNEP and evaluation framework for the proposal to GEF. That is what we were after." GEF Mediterranean Strategic Partnership 07 12. Trade policy decision-making "In the context of Russia, the client developed a trade strategy and maximization of gains from discussed by government that is likely to be approved." multilateral trade 07 13. Job creation approaches based "The immediate impact of the job creation study tour is that it on European models--Ireland & improved our understanding of the experience of peripheral Spain study tours countries of Europe that in the past faced similar challenges as do some ECA countries. Another impact is that it generated a greater understanding of colleagues work and strengths, building bridges across sectors within ECA." 07 14. EU8 experience and new "The program enabled the team to lead better and more informed financial framework for EU dialogue with government on operations under preparation ­ structural funds virtually in all sectors that have active operations, as everything is related one way or another to EU funds." 07 15. State-of-the-art judicial "1) Strengthen our team and brought us together from PRIV, systems PREM, WBI, field office and other staff from HQ. 2) Feeding into policy dialogue and operational country dimension. 3) We need to keep our knowledge at the forefront. The program brought us the importance of learning as a continuous process." 07 16. Replicability of innovation "We will soon have a dissemination paper and intend to host a finance experience of Israel in brown-bag lunch." ECA 13 FY Grant and Country Impact 07 17. Consumer protection in "Based on the WB report and international workshop, the Slovak financial services and develop government was able to present to Parliament a program for CPFS good practices financial consumer protection that had the support of the international community and the European commission as well as the local financial community. In addition, the workshop provided a valuable opportunity for the Irish government to provide TA to Slovakia--through the presentations of the Irish financial ombudsman and the head of banking supervision for the Irish financial supervisory agency. The Irish ombudsman stayed for an additional two days to provide further detailed discussions for the Slovak finance ministry and the central bank." 3.5 In terms of the replicability of the learning acquired in other countries (e.g. the extent to which the original idea is transferable to another country), 15 of 17 (88%) grantees indicated that their learning events are replicable. More precisely, grantees indicated that projects are replicable in ECA region countries, specifically in new EU member countries (9/17), countries seeking access to EU membership like Turkey and Macedonia (2/17), or countries outside the ECA region (5/17). The latter projects could be replicated in either multi-partner projects, multi-country projects, the Bank network, African landlocked countries, some Latin American countries, or globally (e.g. the MILES job creation framework and the GEF Mediterranean Strategic Partnership). Two grantees saw replicability of parts of the learning event in all middle income countries (Israel Innovation Finance Structure and Schemes) and with adaptations and country customizing (Risk Management Workshops).18 Another grantee did not specify where his/her project was "highly replicable." C. VALUE OF THE INNOVATION GRANTS 3.6 On the positive side, generally, over 15 of 17 (88%) grantees remarked that to a high extent learning activities provided them with needed new knowledge and that this knowledge was relevant and applicable to their day-to-day work or work in general. These grantees thought that the learning event fulfilled their expectations. In three instances, grantees indicated that participants who initially were highly skeptical­ particularly about study tours­expressed that they found the event very useful and productive and that they liked it a lot. One grantee also pointed out that, in his particular case, he learned to appreciate the value of learning from past experience. He encourages ECA to promote among staff considering the wealth of lessons stemming from the economic history of OECD members that have successfully overcome similar challenges that some ECA countries are currently facing.19 Along these lines, another grantee (not 18This is in line with ECA VPU mission: "The Region offers customized knowledge and innovative financial products to address regional and global challenges in timely manner." (ECA Strategy and Performance Contract, October 10, 2006). 19In the words of the grantee: "The idea of studying the experience of more advanced countries is a relevant benchmark. Economic history benefits have not received enough attention, when it should be the natural place for WB colleagues to look at when one works with ECA countries that nowadays show similarities to 14 interviewed) in the post-training report also recommended that, for countries in the start- up period of absorbing EU funds, "the Bank should work to promote learning from the examples of older EU members." 3.7 Grantees welcome the Innovation Grant Program and give it high marks as a learning tool. Several grantees exhort the grants to continue and even increase. When asked about the extent to which they found the program innovative (Appendix 3, question 1), all interviewees gave high ratings of 4 and 5 (on a Lickert scale of 1-5). Staff indicated that they were pleased with having such learning funds available at a time when ECA is facing a learning budget constraint. The staff also appreciated the possibility of acquiring the new knowledge necessary to meet ECA strong and changing client demand for both traditional and new emerging product lines.20 Some staff considered putting together the application and actually preparing for the learning event as a quite useful exercise and discipline. 3.8 Across the board, grantees specifically indicated that the innovative aspects of the program provided them with high to very high value, underscoring the following synthesis of virtues of the Innovation Grants Program, in their words: · "This newly available mechanism is an effective and motivating way to acquire knowledge thanks to its innovative focus; it also prompted teams to think about innovative solutions." · "It is a program that is proactively managed; and Marsha M. Olive21 has provided very helpful assistance in shaping the event by sharing her expertise with grantees that may need it." · "The program is also innovative in its flexibility, as shown by the broad range of activities supported by the program." · In sum, the program reinforces the notion that "we [at the Bank] need to keep knowledge needs at the forefront, and recognize the importance of learning as a continuous process." former experiences of more advanced countries. I would recommend that looking into past experiences of more advanced countries be systematized. This type of drawing lessons from historical experience can provide much light. This has not been done because staff tends to focus on current or more recent situations of "peer" countries, mainly because historical memory tends to get lost. But historical experiences­not always documented on paper­could be very enlightening." 20The ECA Strategy and Performance Contract, October 10, 2006, highlights the need for knowledge in a budget restricted environment as follows: "The emergence of Eastern European democracies beginning in 1989 and the dissolution of the Soviet Union and Yugoslavia in 1990s have placed inordinate demands on the ECA Region. [...] Even though a number of ECA countries have joined the EU and two countries have formally graduated, demand for Bank services remains strong and the environment in which we engage our clients continues to evolve rapidly. While demand for Bank support remains strong, our ability to respond has been curtailed by budget constraints; in fact, there are some requests for support that we cannot meet." 21In September 2002, ECA appointed Marsha McGraw Olive as its first Regional Learning Coordinator to help manage and supervise the region's learning program. Promoted to Lead Knowledge and Learning Officer in FY05, Marsha reports to the Director, Regional Operations and Renewal and is an advisor to the Regional Leadership Team (RLT). She has managed the Innovation Learning Grant Program since FY05. 15 3.9 Interviewees described the innovative nature of the grants as ones that: · Promoted new ways of sorting problems on specific policy themes. · Provided insight into conceptual frameworks. · Allowed discourse on difficult topics in the most supportive possible environment22 or in non-formal setting conducive to learning. · The content of some events per se was innovative because the Bank did not have the information, there was not much information available on the topic in question, or the available information was very expensive to obtain through other methods. · Raised awareness among staff that activity events, in particular study tours, are a productive, highly targeted, interactive and customized learning mechanism. · Provided just-in-time and unique funding support, which sometimes served as "seed money" with a very important multiplier. 3.10 On the negative side, several grantees pointed out that the main problem with the program's operation was the timing of fund disbursement, and the "use-it-or-lose-it" clause. They remarked that the clause especially affects more the second round of grants because staff has to carve time out when work demands are most pressing. To address the challenge, these grantees recommended having application deadlines earlier in the fiscal year than later, when bunching occurs. 22Meetings are considered innovative because they connected WB colleagues with counterparts of other countries in a setting conducive to sharing knowledge or raising awareness about a specific topic. 16 4. CONCLUSIONS AND RECOMMENDATIONS 4.1 The desk review and staff responses to WBIEG questionnaire confirmed that ECA Innovation Learning Grants proved to be a valuable learning tool that: · Increases the knowledge, skills or understanding of Bank staff. · Engages multi-sector teams within the Bank and with counterparts in knowledge- sharing activities and in thinking on how the Bank could be more effective. · Supports business development through innovation in the short- to medium-term. · Contributes to identifying or applying solutions to development challenges, and also to developing best practice documents. · Is replicable in ECA region countries, and EU current and aspiring members. In some cases, it would be replicable in other regions, the Bank network, in middle- income countries, and even globally. · Is considered by grantees to be a valuable program that is simple in its application and straightforward in its procedure. They would like it to continue. · Has demonstrated to be highly hands-on and beneficial training. It increased motivation and morale, and showed the value of "study tours" as effective learning tools. 4.2 Based on interview results, WBIEG encourages the continuation of the ECA Innovation Grants Program. WBIEG also encourages ECA to consider ways to address grantees' suggestion about moving to an earlier date the application cycles so that staff can take advantage of the summer slack time to plan learning activities. 4.3 Based on the desk review of all available grant documentation provided by the ECA region, WBIEG makes the following recommendations for improving the program administration and accountability: · Application Forms: Must be revised to include a separate box for describing outcomes and monitoring mechanisms for applying learning. Also the application forms should include an additional section­­reserved for managing, monitoring and storing purposes­­that could be added for the exclusive use of ECA Learning Committee to document status of grant and deliverables. Applicants could also be asked to identify potential risks that may affect delivery or accomplishing expected results. 17 · Post training reports: WBIEG also advises ECA to create a simple, but uniform template to be used for the presentation of Post Training Reports, with a structure similar related to that of the Application Form. By providing a simple template that guides the structure and content of the post training report, the template would improve report completeness, ease monitoring and evaluation of grant compliance, and assure timely submission of activities and grantees' deliverables. · Documentation: Eventually, WBIEG recommends the use of a database application for documentation purposes. The database would help store and manipulate grant data, and would allow quick identification of the status of each grant. 18 APPENDIX 1: ECA REGION INNOVATION GRANT FUND GUIDELINES FOR FY05 AND FY06 ANNOUNCEMENT ECA REGION INNOVATION GRANT FUND FY05 GUIDELINES23 Deadlines: July 31 and December 15, 2005 A. INTRODUCTION In FY05, the ECA Learning Committee introduced a $300,000 fund to increase the contribution of knowledge-sharing and strategic learning to business results. Continuous learning, by staff and with clients, improves development outcomes through product, service and process innovation. The Region is continuing this program in FY06. The Innovation Grant Fund will encourage greater knowledge-sharing with clients or other agencies on innovative solutions, particularly where these can support business development in the short- to medium-term. Last fiscal year [FY05], several grant awards were not used, often due to scheduling problems. To improve the utilization of funds in FY06, all grants awarded in Round I (July 31 deadline) must be expensed by December 30, 2005. Unspent award funds will be reallocated in the second round. Grantees may reapply but there will be no guarantee of approval. B. APPLICATION PREREQUISITES Applications are now being invited for FY06 innovation grants. All grants must: · increase the knowledge, skill or understanding of Bank staff; · engage clients and/or external agencies in knowledge-sharing activities; · identify goals and intended outcomes and a mechanism to monitor and report results; · include cost-sharing from unit learning funds or outside partners. C. EVALUATION CRITERIA Applications containing the prerequisites will be judged by the ECA Learning Committee on the degree to which the grant-funded program will: · support business development through innovation in the short to medium term; and 23Source: FY05 Innovation Grant Fund, ECA's Learning Program, World Bank Intranet URL: http://go.worldbank.org/FS47LBFFZ0. 19 · contribute to the identification or application of solutions to development challenges. Lessons of Experience: The Committee encourages applicants to take the following lessons into consideration in designing their learning programs. An independent evaluation of previous ECA grants found that management commitment to grant outcomes is a critical element in sustaining learning gains. In addition, learning events that meet Bank norms and staff preferences while contributing directly to Bank operational work achieve greater success. Finally, when facilitators are invited, teams must set clear and feasible expectations for their services. (Rarely do facilitators have technical expertise on Bank topics; their skill lies in increasing the productivity and success of complex group discussions). More generally, the Region has considerable experience using GDLN to enrich knowledge-sharing through guest experts and clients, at significant cost savings. D. BUDGET AND FUNDING ISSUES There is no funding ceiling but grant amounts should be commensurate with expected outcomes. As is the case more generally, scarce learning funds should not be used as a BB supplement. The grant will support variable costs, cross-support, and participation costs of non-Bank experts. Until the Learning Board finalizes funding guidelines for joint staff-client learning events, client travel and subsistence should not be included as a budget line item. Applicants should also follow Bank policy on honoraria. All funds should be utilized by December 30 for Round I and by June 30, 2006 for Round II. FY06 INNOVATION GRANT FUND ANNOUNCEMENT24 The ECA Learning Committee has allocated $300,000 for client-team learning described in the attached guidelines. Each application will be screened according to the following prerequisites, and eligible applications will be judged by the ECA Learning Committee on the basis of the listed selection criteria. Deadlines for the two rounds are July 31 and December 15, 2005. Prerequisites · increase the knowledge, skill or understanding of Bank staff; · engage clients and/or external agencies in knowledge-sharing activities; · identify goals and intended outcomes and a mechanism to monitor and report results; and · include cost-sharing from unit learning funds or outside partners. 24Source: ECA's Learning Program, World Bank Intranet URL: http://go.worldbank.org/3R12UGFFY0 20 Selection Criteria · support business development through innovation in the short to medium term; and · contribute to the identification or application of solutions to development challenges. Note new `use it or lose it' rule: All grants awarded in Round I must be expensed by December 30, 2005. Attachment: Innovation Grant Fund Application 21 APPENDIX 2: WBI'S SUGGESTED MODIFICATIONS TO ECA REGION INNOVATION GRANT APPLICATION FORM Innovation Grant Application Form Round I or II (For submission to Marsha McGraw Olive, cc: Ian Conachy, by July or December year deadline) Name of Applicant(s): Unit: Title of Learning program: Date: 1. What are the goals and intended outcomes of the grant-funded learning program? 2. What monitoring mechanism of learning events will be used? How will results be reported? How will the application of results be monitored? 3. What activities are planned and who will participate? 22 4. How will the learning program contribute to business development in the short-to- medium term? What specific solution to development challenges will it help identify or apply? 5. List major budget categories and sources of cost-sharing: ____________________________________________________________________ ____________________________________________________________________ For the ECA Learning Committee Fund's use only Grant Number: # FY: Round: Grant Award Status: Date: Reason: Approved Not funded Postponed Cancelled Activity Delivery Status: In preparation Expected date of delivery Delivered Post Training Report Status: In preparation Expected date of submission Submitted 6. List the possible risks and impediments to realizing the intended outcomes of the learning program: 23 APPENDIX 3: QUESTIONNAIRE TO RECIPIENTS OF AN ECA REGION INNOVATION GRANT Person interviewed:__________________________________________ Date:____________ Grant Title:_______________________________________________________FY:_______ Low High 1. On a scale of 1 (low) to 5 (high), please rate the extent to which you found the program was innovative? 2. Please identify the 3 most valuable outcomes gained from your training? a. Learned new lessons/skills f. Complemented an ongoing operation b. Identified new entry points for g. Identified country needs, Bank's T.A. or new work background information c. Enriched studies, projects, h. Identified solutions to a operations development challenge d. Identified causes to a problem i. Tested applications to solve a problem e. Continuation or dissemination of j. Other, explain previous research/work 3. On a scale of 1 (low) to 5 (high), please indicate the extent to which the learning activities: Low High a. Have provided you with new knowledge b. Have been relevant to your day-to-day work c. Have been applicable to your work d. Have fulfilled your expectations 4. How replicable is the learning acquired in other countries, (e.g. can the original idea be transferable to another country)? 5. In 2 or 3 sentences, please explain how you specifically used the new knowledge, skills and experience that you gained from the learning program. 24 6. Has the learning program contributed to Yes No Give specific new business development in your unit? examples: 7. Has the learning program contributed to Yes No Give specific identifying or applying solutions to ECA examples: development challenges? 8. Please describe the immediate impact of your learning event. 9. What are your suggestions for improving the effectiveness of the ECA Innovation Learning Grants in the future? 10. Any other comment? 25 APPENDIX 4: GRANTS AND STAFF INTERVIEWED A. IMPLEMENTED GRANTS AND STAFF INTERVIEWED, FY05-07 FY05- # per Fiscal Year and US$ Interview 07 FY Grant Number Amount1 Grant Title Applicant Date 1 1 FY05--Grant #4 3,100 Communications clinic on the Ms. Merrell Tuck- 7/2--Yes Complete regional study on poverty, Primdahl, Senior growth and inequality Communications Officer 2 2 FY05--Grant #1- 35,000 Strengthening WB Anand Seth, 5/8 --Yes Complete assistance in Bulgaria, Country Director Croatia and Romania in Oscar Bruyn Cops maximizing benefits of EU accession 3 3 FY05--Grant#2-- 48,500 EU fiduciary harmonization John Hegarty, 4/30--Yes Application and Manager report missing 4 4 FY05-Grant #5 30,000 Risk management Fernando Montes- 4/30--Yes No report workshops Negret, Sector Director 5 5 FY05--Grant #6- 8,000 Human development sector Ms. Shiyan Chao No (Only preparation swaps in Croatia interview work) ­only preparation work. scope reduced, happened in FY06 FY05 #: 5 FY05 SUM: $124,600 FY05 AVERAGE: $24,920 6 1 FY06 #12- 65,000 National development plan Originally Bernard 5/2--Yes Complete (FY05-- study tour to Spain and Funck (FY05); Satu Grant #8, Ireland Kahkonen, lead postponed) economist (FY06) 7 2 FY06-Grant #10-- 51,600 Designing regional and local Gerardo M. 4/30 -- Yes Complete development policies based Corrochano, sector on EU experience manager 8 3 FY06-Grant #13-- 42,100 Review and learn from the Manuel Marino, 5/8 -- Yes Complete subnational and regional Lead Water and experience in Spain on Sanitation Specialist designing environmental (Peter Thompson, infrastructure investments to manager) be competitive for EU financial support 9 4 FY06-Grant #14 9,000 Kyrgyz health and education Michael Mertaugh No (Partially teams (Nepal-Kyrgyz team interview happened, learning on SWAP application implementation­did not available, report happen) pending) 10 5 FY06-Grant #15 37,500 Sharing the lessons of Michael Mertaugh, 5/9 -- Yes (Complete) reform in the dual-system of lead education training economist 11 6 FY06-Grant #18- 75,000 Forum on job creation in Gordon 5/3 -- Yes Complete Eastern Europe and Central Betcherman, Lead Asia Economist 12 7 FY06--Grant #19- 24,000 Transit corridors Aslan Sarinzhipov, 5/11-- Yes Complete performance measurement Operations Officer in Central Asia FY06 #: 7 FY06 SUM: $304,200 FY06 AVERAGE: $43,457 13 1 FY07-Grant #20 20,000 Development of a monitoring Emilia Battaglini, 5/1 -- Yes (originated in and evaluation framework for Senior Operations FY06)--Complete the GEF Mediterranean Officer 26 FY05- # per Fiscal Year and US$ Interview 07 FY Grant Number Amount1 Grant Title Applicant Date Strategic Partnership 14 2 FY07-Grant #26 40,000 Replicability of innovation Fernando Montes- 4/30 -- Yes Application, no finance experience of Israel Negret, Sector report in ECA Director­Gerardo Corrochano 15 3 FY07-Grant #21- 36,000 Consumer protection in Sue Rutledge, Sr. No Application financial services and Private Sector Dev. interview available, report develop CPFS good Spec. missing practices 16 4 FY07-Grant #22-- 20,000 Trade policy decision-making Asad Alam, Sector 5/1 -- Yes Complete and maximization of gains Manager from multilateral trade 17 5 FY07-Grant #24 36,400 Job creation for Turkey and Organized and 9/17 -- Yes Complete the South Caucasus conducted by Oscar Calvo-Gonzalez. 18 6 FY07-Grant #27 33,000 EU8 experience and Florian Fichtl 9/17 -- Yes Complete absorption of EU structural ­Bulgaria Country funds­Bulgaria Manager, Peter Pojarski­Senior Operations Officer 19 7 FY07-Grant #28 15,000 Ukraine communications for Yulia Snizhko, No Complete carbon finance advocacy Dmitro Derkatch interview 20 8 FY07-Grant #30 21,000 State-of-the-art judicial Amitabha 9/18 -- Yes Complete systems Mukherjee, Lead Public Sector Specialist 21 9 FY06-Grant #31- 20,000 Poland study tour EU funds Samir Suleymanov Did not former FY06 absorption (formerly, happen Grant #11 financial cooperatives in rural postponed regions­expanding outreach) Complete FY07 #: 9 FY07 SUM: 241,400 FY07 AVERAGE: $26,822 FY05- 07 #: 21 FY05-07 SUM: 670,200 FY05-07 AVERAGE: $31,914 B. GRANTS NOT IMPLEMENTED, FY05-07 FY05- # per US$ 07 FY Fiscal Year Amount Grant Title Applicant Reason 1 1 FY05-Grant#3- 50,000 New approaches to Hossein Razavi Did not (Cancelled) infrastructure investments take place 2 2 FY05-Grant #8 4,000 National development plan Originally Bernard Postponed, (postponed) study tour to Spain and Funck (FY05); Satu due to work Ireland Kahkonen (FY06) program time constraints 3 3 FY05-Grant #9 20,000 Economic challenges of Kamer Karakurum WBI (cancelled) Turkey EU accession Ozdemir followed- up in FY05 4 4 FY05-Grant #7 5,000 Seminar on comparative Akshay Sethi Did not (cancelled) growth models: Ireland and happen Finland FY05 #: 4 FY05 SUM: $79,000 FY05 AVERAGE: $19,750 27 FY05- # per US$ 07 FY Fiscal Year Amount Grant Title Applicant Reason 5 1 FY06-Grant #16 $50,550 Meeting Infrastructure Peter Thompson Not funded (not funded at the financing needs through at the end end) investment funds FY06 #: 1 FY06 SUM: $50,550 FY06 AVERAGE: $50,550 6 1 FY07-Grant #23 $10,000 Supporting Ukraine TB Shiyan Chao Difficulties policy implementation with government who kept on changing its mind 7 2 FY07-Grant #25 $32,500 Development of a common Ruslan Yemtsov, Did not (conflict with work approach/strategy for senior economist happen program) collection of poverty monitoring data and its assessment 8 3 FY07-Grant#29 20,000 Banking sector risk Sophie Sirtaine Conflict (work time assessment training with work constraints) program FY07 #: 3 FY07 SUM: $62,500 FY07 AVERAGE: $20,833 FY05- FY05-07 SUM 07 #: 8 Grants Not $192,050 FY05-07 AVERAGE $16,004 Implemented: Grants Not Implemented: C. PROJECTS THAT ACTUALLY HAPPENED1 FY Fiscal Year and US$ Interview 05-07 Grant Number Amount1 Grant Title Applicant Date 1 FY05-Grant #6- 50,000 Human development sector Ms. Shiyan Chao Croatia (Cancelled) swaps in Croatia SWAP ­preparation work collapsed 2 FY06-Grant #11 52,500 Financial cooperatives in Samir Suleymanov Postponed (postponed to rural regions­expanding FY07) outreach 3 FY06-Grant #14 43,050 FY06­Kyrgyz health and Michael Mertaugh Coup in (Coup d'Etat) education teams­Nepal Nepal, ­Kyrgyz team learning on partially SWAP implementation happened 4 FY07-Grant #28 15,000 Communications for carbon Yulia Snizhko, Conflict (conflict with work finance advocacy Dmitro Derkatch with work program) program 1On December 17, 2007, Ian Conachy (Senior Program Assistant, ECAVP) informed that the projects actually took place. 28