RPR
57637
THE ASSOCIATION FOR THE
DEVELOPMENT OF EDUCATION
IN AFRICA
REGIONAL PROGRAM REVIEW
Volume 2 Issue 2
THE WORLD BANK GROUP
WORKING FOR A WORLD FREE OF POVERTY
The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development
(IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the
Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment
Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi-
ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and
private sectors.
THE INDEPENDENT EVALUATION GROUP
Improving Development Results Through Excellence in Evaluation
The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group.
IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on
assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA
guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General,
Evaluation.
The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the
Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group
work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn
from evaluation findings.
The Association for the Development of
Education in Africa
March 3, 2008
Corporate and Global Evaluations and Methods
http://www.globalevaluations.org
�2008 Independent Evaluation Group, The World Bank Group
1818 H Street NW
Washington DC 20433
Telephone: 202-458-4497
Internet: http://www.globalevaluations.org
E-mail: grpp@worldbank.org
All rights reserved
This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group.
It is part of an ongoing series that reviews global and regional partnership programs in which the World
Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this
volume do not necessarily reflect the views of the Executive Directors of The World Bank or the
governments they represent.
IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors,
denominations, and other information shown on any map in this work do not imply any judgment on the
part of IEG concerning the legal status of any territory or the endorsement or acceptance of such
boundaries.
Rights and Permissions
The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits
these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring
body.
Cover photo: Children at a school in Burkina Faso. Photo by Ray Witlin, 1976, courtesy of the World Bank
Photo Library.
ISBN-13: 978-1-60244-106-4
ISBN-10: 1-60244-106-5
Printed on Recycled Paper
Independent Evaluation Group
Knowledge Programs and Evaluation Capacity
Development (IEGKE)
E-mail: grpp@worldbank.org
Telephone: 202-458-4497
IEG Mission: Improving Development Results
Through Excellence in Evaluation
The Independent Evaluation Group (IEG) of the World Bank reviews global and regional
partnership programs (GRPPs) in which the Bank is engaged as one partner among many for two
main purposes: (a) to provide accountability in the achievement of the program's objectives by
providing an independent opinion of the program's effectiveness, and (b) to identify and disseminate
lessons learned from the experience of individual GRPPs. The preparation of a global or regional
program review (GPR) is contingent on a recently completed evaluation of the program, typically
commissioned by the governing body of the program.
The first purpose includes validating the findings of the GRPP evaluation with respect to the
effectiveness of the program, and assessing the Bank's performance as a partner in the program. The
second purpose includes assessing the independence and quality of the GRPP evaluation itself and
drawing implications for the Bank's continued involvement in the program. Assessing the quality of
GRPP evaluations is an important aspect of GPRs, since encouraging more consistent evaluation
methodology and practice across Bank-supported GRPPs is one of the reasons why IEG embarked on
this new product in 2005.
IEG annually reviews a number of GRPPs in which the Bank is a partner. In selecting
programs for review, preference is given to those that are innovative, large, or complex; those that are
relevant to upcoming sector studies; those for which the Executive Directors or Bank management
have requested reviews; and those that are likely to generate important lessons. IEG also aims for a
representative distribution of GPRs across sectors in each fiscal year.
A GPR is a "review" and not a full-fledged "evaluation." It assesses the independence and
quality of the relevant evaluation; provides a second opinion on the effectiveness of the program;
assesses the performance of the Bank as a partner in the program; and draws lessons for the Bank's
engagement in global and regional programs. The GPR does not formally rate the various attributes of
the program.
A GPR involves a desk review of key documents, consultations with key stakeholders, and a
mission to the program management unit (secretariat) of the program if this is located outside of the
World Bank or Washington, DC. Key stakeholders include the Bank's representative on the
governing body of the program, the Bank's task team leader (if separate from the Bank's
representative), the program chair, the head of the secretariat, other program partners (at the
governance and implementing levels), and other Bank operational staff involved with the program.
The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP.
Each GPR is subject to internal IEG peer review, Panel review, and management approval.
Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat
of the program. Comments received are taken into account in finalizing the document, and the formal
management response from the program is attached as an annex to the final report. After the
document has been distributed to the Bank's Board of Executive Directors, it is disclosed to the
public on IEG's external Web site.
ii
Abbreviations and Acronyms
ADEA Association for the Development of (Association pour le D�veloppement de
Education in Africa l'Education en Afrique)
AfDB African Development Bank (Banque Africaine de D�veloppement)
AU African Union (Union Africaine)
CIDA Canadian International Development Agency (Agence Canadienne de D�veloppement
International)
COMEDAF Conf�rence des Ministres de l'Education de (Conference of Education Ministers of the
L'Union Africaine African Union)
DEA Donors to African Education (Bailleurs de fonds pour l'�ducation en Afrique)
DGF Development Grant Facility (World Bank)
ECCAS Economic Community of Central African (Comit� Economique des Etats de l'Afrique
States Centrale)
ECOWAS Economic Community Of West African Communaut� Economique des Etats de l'Afrique
States de L'Ouest
EFA Education for All (Education pour Tous)
FAWE Forum for African Women Educationists (Forum des �ducatrices africaines)
ERNESA Education Research Network in Eastern and
Southern Africa
ERNWCA Education Research Network for West and
Central Africa
FTI Fast Track Initiative (Initiative pour la mise en oeuvre acc�l�r�e)
GPR Global Program Review (IEG) (Revue Globale de Programme)
GRPP Global and Regional Partnership Programs (Programmes globaux et r�gionaux de
partenariat)
HEDCO Higher Education for Development (Education Sup�rieure pour le D�veloppement
Cooperation de la Coop�ration)
HDNED Human Development Network Education (R�seau des Resources Humaines pour
(World Bank) l'Education)
IDRC International Development Research Centre (Centre de Recherche pour le D�veloppement
(Canada) International-Canada
IEG Independent Evaluation Group, formerly (Groupe d'Evaluation Ind�pendent, Banque
OED (World Bank) mondiale)
IIEP International Institute for Educational (Institut international pour la planification
Planning �ducative)
ILO International Labor Organization (Organisation Internationale du Travail)
IOA Institutional and Organizational Assessment (Evaluation Institutionnelle et
Organisationnelle)
NEPAD New Partnership for Africa's Development
NGO Non-Governmental Organization (Organisation Non-Gouvernementale)
RPR Regional Program Review (IEG) (Revue de programme r�gional)
SADC Southern African Development Community (Communaut� de D�veloppement de l'Afrique
Australe)
SC Steering Committee (ADEA) (Comit� de Coordination)
TTL Task Team Leader (Charg� de Projet)
WG Working Group (ADEA) (Groupe de travail)
Director-General, Independent Evaluation Mr. Vinod Thomas
Director, Independent Evaluation Group (World Bank) Ms. Cheryl Gray
Manager, Corporate and Global Evaluations and Methods Mr. Mark Sundberg
Global Programs Coordinator Mr. Chris Gerrard
Task Manager Mr. H. Dean Nielsen
iii
Contents
ABBREVIATIONS AND ACRONYMS................................................................................................... II
PROGRAM AT A GLANCE: ASSOCIATION FOR THE DEVELOPMENT OF EDUCATION IN
AFRICA ..................................................................................................................................................V
KEY BANK STAFF RESPONSIBLE DURING PERIOD UNDER REVIEW .........................................VI
GLOSSARY..........................................................................................................................................VII
PREFACE..............................................................................................................................................IX
EXECUTIVE SUMMARY.......................................................................................................................XI
1. PROGRAM OBJECTIVES, ACTIVITIES, AND COSTS........................................................... 1
Objectives................................................................................................................................. 1
Activities................................................................................................................................... 1
ADEA Vehicles ......................................................................................................................... 3
Governance .............................................................................................................................. 5
Costs and Financing ............................................................................................................... 6
2. THE 2005 INDEPENDENT EVALUATION OF ADEA.............................................................. 9
Evaluation Process ................................................................................................................. 9
Monitoring and Evaluation Framework ................................................................................. 9
Main Findings and Recommendations of the Evaluation ................................................. 10
ADEA's Response ................................................................................................................. 10
3. THE EFFECTIVENESS OF THE PROGRAM......................................................................... 12
Relevance ............................................................................................................................... 12
Efficacy ................................................................................................................................... 13
Objective: Improving Educational Policies.......................................................................................13
Objective: Enhancing Development Agency Coordination ..............................................................16
Program: Promoting Policy Dialogue...............................................................................................17
Program: Promoting Analytical Work...............................................................................................19
Program: Strengthening Capacities in African Countries.................................................................21
Program: Promoting Regional and Sub-Regional Networking .........................................................22
Conclusion.......................................................................................................................................23
Efficiency................................................................................................................................ 24
Governance and Management ............................................................................................. 24
4. WORLD BANK'S PERFORMANCE AS A PARTNER........................................................... 27
iv
5. LESSONS LEARNED............................................................................................................. 29
ANNEX A. EVALUATION FRAMEWORK FOR GLOBAL PROGRAM REVIEWS............................ 31
ANNEX B. KEY DATES IN ADEA HISTORY ..................................................................................... 38
ANNEX C. ADEA WORKING GROUPS ............................................................................................. 39
ANNEX D. ADEA BIENNIAL MEETINGS, 1991�2006....................................................................... 41
ANNEX E. ADEA ORGANIZATIONAL CHART.................................................................................. 42
ANNEX F. TECHNICAL AND SUB-REGIONAL MEETINGS, 2003�2007......................................... 43
ANNEX G. USES OF ADEA FUNDS, 2001�2006 .............................................................................. 44
ANNEX H. IEG ASSESSMENT OF THE QUALITY OF ADEA'S ANALYTICAL WORK .................. 45
ANNEX I. PERSONS CONSULTED ................................................................................................... 46
ANNEX J. OFFICIAL RESPONSE OF ADEA TO IEG'S GLOBAL PROGRAM REVIEW ................ 48
Boxes
Box 1. ADEA's New Vision Statement, Mission Statement, and Strategic Objectives ................. 2
Box 2. ADEA and African Policies on the Language of Instruction ............................................. 14
Box 3. "The Culture of Quality" and National Education Plans: A Disconnect?......................... 17
Tables
Table 1. External Evaluation Recommendations and ADEA Response....................................... 11
Table 2. World Bank Task Team Leader Views of ADEA: Frequencies ....................................... 14
Table 3. ADEA Published Studies, by Working Group and Time Period ..................................... 20
Annex Table 1. Assessing the Independence and Quality of the Evaluation .............................. 31
Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program......... 32
Annex Table 3. Assessing the Bank's Performance as a Partner in the Program...................... 37
Figures
Figure 1. Annual Contributions to ADEA (Restricted and Unrestricted)........................................ 7
Figure 2. ADEA Use of Funds 2001�2006 .......................................................................................... 8
Figure 3. Sample of ADEA Analytical Work..................................................................................... 20
v
Program at a Glance: Association for the Development of
Education in Africa
Start date ADEA, originally called the Donors to African Education, was founded in 1988.
It changed its name in 1997 to the Association for the Development of
Education in Africa (ADEA).
Original � Improving educational policies;
Objectives � Enhancing development agency coordination.
Activities ADEA addresses the original set of implicit objectives though:
� The loosely coordinated actions of working groups;
� Regular biennial meetings of the agency and ministry partners;
� Meetings of the Steering Group and Bureau of Ministers;
� Special or sub-regional meetings;
� Training or capacity building activities,
� Publication and information dissemination (including databases).
WBG During ADEA's first three years (1988�1990), in-kind contributions by the WBG
contributions and budget support amounted to about $445,000 (about 18 percent of total
development agency support of $2.5 million). After 1991 Bank financial support
came largely through the Special Grants Program (renamed Development
Grant Facility in 1998). By the early years of the current decade (2000�2005),
DGF funding amounted to about $300,000�350,000 per year (or about 6�14
percent of total development agency support). In 2006 DGF funding increased
from $350,000 to $1.0 million, although $400,000 of this $650,000 increase
was for activities already financed through other channels (a $250,000 on-grant
to FAWE and $150,000 for the Working Group on Higher Education).
Other See above. All development agencies participating in ADEA in recent years
development have been expected to contribute at least $50,000 annually as a membership
agency fee (many contribute more). In addition, partners earmark contributions to
contributions specific activities or working groups through restricted funds. Total contributions
to ADEA in the current decade have increased from $2.5 million in 2000 to
$6.5 million in 2006. During that period restricted funds have exceeded
unrestricted (membership) funds by about 2�3.5 to 1.
Location An informal Secretariat was housed at the World Bank in Washington, DC, from
1988�91. Then a formal Secretariat was formed and hosted by UNESCO's
International Institute of Educational Planning (IIEP) in Paris. Beginning in 2008
the Secretariat will be hosted by the African Development Bank in Tunis,
Tunisia. Since 1997 all Biennial Meetings have been held in Africa.
Governance and ADEA is governed by a Steering Committee consisting of 21 development
management agencies and 10 African ministers of Education (the Bureau of Ministers). Its
day-to-day operations are managed by its Secretariat, consisting of an
Executive Secretary and 5�6 staff members. World Bank officials (Task Team
Leaders, TTLs) have also held a place on the Steering Committee; in 2005 a
World Bank official (Sector Manager) was elected by the Steering Committee to
be its Chair.
Latest program- Evaluation of the Association for the Development of Education in Africa
level evaluation (ADEA), December 2005, conducted by the private firm, Universalia of
Montreal, Canada.
vi
Key Bank Staff Responsible during Period under Review
Position Person Period
Task Team Leader Peter Moock FY90�FY95
Avril Van Adams FY96�FY98
Adriaan Verspoor FY99�FY00
Birger Fredriksen FY01�FY04
Dzingai Mutumbuka FY05�FY08
Jee-Peng Tan FY08 � present
Director of Human Maris O'Rourke February 1997 � September 2000
Development Network, Marlaine Lockheed September 2000 � June 2001
Education Team Ruth Kagia March 2001 � present
Vice President of HD Network Eduardo A. Doryan June 1999 � August 2005
Jean-Louis Sarbib July 2003 � July 2006
Joy Phumaphi February 2007 � present
Trust Fund Operations Arif Zulfiqar, Director June 1999 � present
Global Programs & Margret Thalwitz, Director May 2004 � present
Partnerships
Program Manager
Position Person Period
Program Manager (Executive Peter Moock 1988�1992
Secretary) Chris Shaw 1992�1995
Richard Sack 1995�2001
Mamadou Ndoye 2001 � present
vii
Glossary
Devolution or exit A proactive strategy to change the design of a program, to devolve some of its
strategy implementation responsibilities, to reduce dependency on external funding, or to phase out
the program on the grounds that it ha achieved its objectives or that its current design in no
longer the best way to sustain the results which the program has achieved.
Donor Any organization or entity that makes a financial or in-kind contribution to a program that is
reflected in the audited financial statements of the program. Therefore, this includes not only
"official donors" but also developing countries that contribute annual membership dues,
seconded staff, or office space, provided that these are formally recognized in the financial
statements of the program. Another term, often used synonymously but more inclusive, is
"development agency."
Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives,
taking into account their relative importance. The term is also used as a broader, aggregate
measure -- encompassing relevance and efficiency as well -- of the overall outcome of a
development intervention such as a Global and Regional Partnership Programs (GRPP).
Efficiency The extent to which the program has converted or is expected to convert its resources/inputs
(such as funds, expertise, time, etc.) economically into results in order to achieve the
maximum possible outputs, outcomes, and impacts with the minimum possible inputs.
Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or
project, its design, implementation, and results. The aim is to determine the relevance and
achievement of its objectives, and its developmental effectiveness, efficiency, impact, and
sustainability.
Governance The structures, functions, processes, and organizational traditions that have been put in place
within the context of a program's authorizing environment to ensure that the program is run
in such a way that it achieves its objectives in an effective and transparent manner. It is the
framework of accountability and responsibility to users, stakeholders and the wider
community, within which organizations take decisions, and lead and control their functions,
to achieve their objectives.
Impacts Positive and negative, primary and secondary long-term effects produced by a development
intervention, directly or indirectly, intended or unintended.
Independent evaluation An evaluation that is carried out by entities and persons free from the control of those
involved in policy making, management, or implementation of program activities. This
entails organizational and behavioral independence, protection from interference, and
avoidance of conflicts of interest.
Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to
measure achievement, to reflect the changes connected to an intervention, or to help assess
the performance of a development actor.
Legitimacy As a criterion for assessing governance and management, the way in which governmental
and managerial authority is exercised in relation to those with a legitimate interest in the
program -- including shareholders, other stakeholders, implementers, beneficiaries, and the
community at large.
Logical framework or A management technique that is used to develop the overall design of a program or project,
logframe to improve implementation monitoring, and to strengthen evaluation, by presenting the
essential elements of the program or project clearly and succinctly throughout its cycle. It is a
"cause and effect" model which aims to establish clear objectives and strategies based on a
results chain, to build commitment and ownership among the stakeholders during the
preparation of the program or project, and to relate the program's or project's interventions to
their intended outcomes and impacts for beneficiaries.
Management The day-to-day operation of the program within the context of the strategies, policies,
processes, and procedures that have been established by the governing body.
viii
Monitoring The continuous assessment of progress achieved during program implementation in order to
track compliance with a plan, to identify reasons for noncompliance, and to take necessary
actions to improve performance. Monitoring is usually the responsibility of program
management and operational staff.
Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development
intervention.
Oversight One of the core functions of the governing body of a program: Monitoring the performance
of the program management unit, appointing key personnel, approving annual budgets and
business plans, and overseeing major capital expenditures.
Partners Stakeholders who are involved in the governance or financing of the program (including the
members of the governing, executive, and advisory bodies).
Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy
the good at the same time) and non-excludable (it is difficult to prevent people who do not
pay for the good from consuming it). If the benefits of a particular public good accrue across
all or many countries, then the good is deemed a global or international public good.
Relevance The extent to which the objectives and design of the program are consistent with (a) the
current global/regional challenges and concerns in a particular development sector and (b) the
needs and priorities of beneficiary countries and groups.
Shareholders The subset of donors that are involved in the governance of the program. Therefore, this does
not include individual (particularly anonymous) donors who choose not to be so involved, or
who are not entitled to be involved if their contribution does not meet the minimum
requirement, say, for membership on the governing body.
Stakeholders The parties who are interested in or affected, either positively or negatively, by the program.
Stakeholders are often referred to as "principal" and "other", or "direct" and "indirect".
While other or indirect stakeholders -- such as taxpayers in both donor and beneficiary
countries, visitors to a beneficiary country, and other indirect beneficiaries -- may have
interests as well, these are not ordinarily considered in evaluations unless a principal
stakeholder acts as their proxy.
Sustainability When the term is applied to the activities of a program, the extent to which the benefits
arising from these activities are likely to continue after the activities have been completed.
When the term is applied to organizations or programs themselves, the extent to which the
organization or program is likely to continue its operational activities over time.
Transparency As a criterion for assessing governance and management, the extent to which a program's
decision-making, reporting, and evaluation processes are open and freely available to the
general public. This is a metaphorical extension of the meaning used in physical sciences --
a "transparent" objective being one that can be seen through.
Value for money The extent to which a program has obtained the maximum benefit from the outputs and
outcomes it has produced with the resources available to it.
Source: Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and
Standards. Independent Evaluation Group � World Bank, 2007.
ix
Preface
ADEA was established by the World Bank in 1988 in response to an influential policy paper
issued the previous year on education in sub-Saharan Africa, Education in Sub-Saharan
Africa: Policies for Adjustment, Revitalization, and Expansion. Originally the Association
was created as an informal network of development agencies in Africa (Donors to African
Education, DAE) having an informal secretariat at the World Bank, but by 1994 (when its
Principles of Association were formalized) it grew into a professional association of both
development agencies and African ministries of education. In a further evolution, in 1997 it
changed its name to the Association for the Development of Education in Africa (ADEA).
With this evolution came institutionalization: whereas in the early years administrative
support was provided by the World Bank, by 1992 a free-standing Secretariat was created for
this purpose, placed in Paris where it was linked to UNESCO's International Institute for
Educational Planning (IIEP).
At the request of the World Bank members of the Steering Committee, an external evaluation
was undertaken 2004�05, given the fact that the Association had been financially supported
by Development Grant Facility (DGF) funds for more than five years. The program
commissioned a private company, Universalia, headquartered in Montreal, Canada, to
undertake the evaluation, which submitted its final report to ADEA in December of 2005.
This Regional Program Review (RPR) assesses the quality and independence of the 2005
evaluation of ADEA, provides a second opinion on the effectiveness of the program, assesses
the performance of the Bank as a partner in the program, and draws lessons for the future
operation of the program. IEG chose ADEA for an RPR because of the importance of the
sector in the Bank's mission of poverty reduction, because IEG has done relatively few
reviews of regional programs, and because IEG had not conducted a global or regional
program review in the field of education before.
This Regional Program Review included:
A review of ADEA documents, reports, newsletters, proceedings (of meetings),
databases, and published materials (books, and pamphlets);
A visit to the ADEA Secretariat and interviews with all ADEA professional staff, plus
relevant officials at its host institution IIEP;
Interviews with representatives from development agency partner institutions:
UNESCO, AfDB, UNICEF, and the Rockefeller Foundation;
Interviews with Bank team leaders and members of the Steering Committee past and
present;
Interviews with academics and NGO representatives who have conducted research on
education in Sub-Saharan Africa;
Telephone survey of 9 World Bank TTLs covering education in sub-Saharan Africa
(5 who have attended ADEA events and 4 who have not), distributed by sub-region.
Interviews with 4 ministers or ex-ministers of education of selected African countries
(including the current and a past chair of the Bureau of Ministers).
x
IEG gratefully acknowledges all those who made time for interviews, in particular ADEA
governance members, management, and staff. A list of people consulted can be found in
Annex I.
Copies of the draft RPR were sent for comments to the ADEA Secretariat, to the Bank's
global program task team leader, to AFTHD (which is responsible for the Bank's
engagement with ADEA), and to other Bank units that have responsibility for the Bank's
involvement with global and regional programs more generally. Their observations have
been taken into account in finalizing this GPR. The formal response received from ADEA
management can be found in Annex J.
xi
Executive Summary
Program Objectives, Activities, Governance, and Costs
1. When ADEA was created in 1988 as one response to the Bank's landmark policy
paper on the crisis in sub-Saharan African Education, it established the general goals of
stimulating discussion of education policies in Sub-Saharan Africa and improving donor
coordination in the sector. During the next 17 years the Association frequently articulated
more specific goals and aims, but never established a formal set of objectives and related
outcomes measures. This became an issue during the 2005 external evaluation, which
recommended that ADEA develop "a strategic plan and business plan to operationalize
ADEA's vision, set clear expectations, and monitor progress towards results." The ADEA
Secretariat drafted such plans during 2006 and the ADEA Steering Committee formally
adopted a new vision statement, mission statement, and five strategic objectives in October
2007 (Box 1). Since this occurred after the present review was conducted, this review has
assessed the effectiveness of ADEA against its two original goals, with one modification.
The review will substitute "improvement of educational policies" for "discussion of
educational policies" since it is clear from ADEA's documents that policy strengthening is
what the discussion (dialogue) has been all about. Thus, the present review has regarded
(a) improving educational policies and (b) enhancing development agency coordination as
the imputed strategic objectives of ADEA before October 2007.
2. From the outset the Association aspired to influence educational policy and
development agency coordination in the region by stimulating policy dialogue informed by
research (analysis of existing problems and needs, and innovative solutions). Gradually it
also became aware of the need to build greater capacity for policy formulation and
implementation and to promote enduring networks. In 2003 ADEA released a "visioning
document," in which the following four challenges were set out as the work of the
Association:
(a) Promoting policy discussions (or dialogue): informed discussion across and within
countries, between government and non-government organizations, and between
country representatives and development agencies with the purpose of identifying
common problems, formulating regional and national priorities, demonstrating and
comparing innovative solutions, and building consensus;
(b) Promoting analytical work: applied research using the "praxis approach" to promote
understanding of educational conditions and constraints; identify, evaluate and
compare solutions to problems; and analyze educational outcomes, typically
undertaken within working groups or in preparation for a regional or sub-regional
meeting;
(c) Strengthening capacities in African countries: explicit or implicit efforts to
improve local capacity for strategic planning and related technical skills, generally
undertaken by working groups, or in the context of anchoring ADEA to African
institutions;
xii
(d) Promoting regional and sub-regional networks: efforts to build new self-standing
regional education development organizations (e.g., through graduating working
groups), to link ADEA working groups to African institutions; and to partner with
and contribute to regional networks and movements, such as those connected to the
African Union.
3. The vehicles through which this work was undertaken have over the years consisted
of the following: (a) working groups (originally 9, now 11 plus 4 ad hoc groups, considered
to be the "laboratories" of ADEA; (b) biennial meetings (the premier regional event
involving all partners every other year); (c) Steering Committee and Bureau of Ministers
meetings (held every six months to provide guidance and decision making); (d) technical or
sub-regional meetings (3�4 day events at request of Ministers on urgent topics); (e) capacity
building (usually done by working groups, or through exchanges and study tours); and (f)
publications and information dissemination (including management of data bases).
4. ADEA's Steering Committee is comprised of ten African ministers of education and
representatives of most multilateral, bilateral and private development organizations that
work in the education sector in Africa (currently 21). The Steering Committee is both
ADEA's governing body and its primary facilitator of coordination among partners. The ten
ministers of education who sit on the Steering Committee constitute the ADEA Bureau of
Ministers. They are chosen by the Caucus of Ministers which consists of all the ministers
of education in sub-Saharan Africa. Administrative support for ADEA is provided by the
ADEA Secretariat, which since 1992 has been hosted physically and administratively by the
International Institute for Educational Planning (IIEP) in Paris, a UNESCO affiliated
organization. In 2008 the Secretariat will move to Tunis, Tunisia, where it will be hosted by
the African Development Bank. When the Steering Committee is not in session, decisions
about ADEA are made by the Bureau of ADEA (also referred to as the Bureau of the
Steering Committee), which consists of the Steering Committee Chair, the Alternate Chair
(also head of the Bureau of Ministers), the Vice-Chair (ex-officio and head of the host
agency), and the Executive Secretary of ADEA.
5. ADEA is funded almost entirely through its 20�22 (the number has fluctuated
recently) development agency partners, with some support coming from the African
ministries. In recent years a minimum agency contribution of $50,000 has been required for
joining and remaining in the partnership. These amounts are "unrestricted" or core funding
and can be allocated by ADEA according to its needs. Contributors also pledge "restricted"
funds, which are generally earmarked for particular working groups. Restricted funding has
exceeded unrestricted by as much as 3 to 1 in recent years. Funding (especially restricted)
fluctuates from year to year, which places constraints on long-term planning. Some countries
(namely, Canada, the Netherlands, Finland and Sweden) have worked to moderate this by
signing multiyear funding agreements, but this has not become the norm. The estimated
funding for calendar year 2006 -- about $6.6 million, some of which is pledged for
subsequent years -- is the highest level of funding ever (a 150 percent increase in funding
over that for the year 2000 in nominal terms), reflecting an expansion in ADEA programs
and donor support.
xiii
The 2005 Independent Evaluation of ADEA
6. The ADEA Steering Committee commissioned an independent evaluation in 2005,
which was undertaken by Universalia, a Canadian management consulting firm based in
Montreal. This was the third evaluation of ADEA, the first being in 1994, when the
Association was still known as the DAE, and the second in 2000, which focused explicitly on
the ADEA working groups. The cost of the evaluation was covered by exceptional funds
provided by the World Bank (DGF) and CIDA.
7. The present IEG review found the 2005 evaluation to have been executed
independently and objectively. While the evaluation sought and responded to feedback from
ADEA stakeholders on factual matters, there was no sign of manipulation of findings or
recommendations by them. One of the findings of the Universalia evaluation was that ADEA
had never formulated a well defined strategic plan nor established a clear framework for
monitoring and evaluation (with performance indicators to track progress). This made the
evaluation less results-oriented than it might have been. Also, on some points the evaluation
simply reports stakeholders' perceptions of Associations' organizational effectiveness instead
of contextualizing such inputs and using them together with other evidence to draw
conclusions (triangulation).
8. Universalia presented 23 findings and eight main recommendations, which ADEA
partners interviewed for this review generally considered to be fair and constructive. The
eight recommendations covered strategic planning, improvement of management and
governance, successor planning for the Executive Secretary, and a feasibility study for the
relocation of the Secretariat to Africa. ADEA took the recommendations seriously, having
drafted a new strategic plan for 2008�2012 during 2006 and completed succession planning
and the feasibility study. The governance improvements are still pending, since some of these
were dependent upon first adopting the new strategic plan, which occurred in October 2007.
The Effectiveness of the Program
9. Relevance. The founding aims of ADEA of improving education policies and
enhancing donor coordination, taken by this review as its strategic objectives, were derived
from the 1987 Education Sector Policy Paper and thus highly relevant to the conditions
portrayed in it (inadequate education policies, and growing but uncoordinated and often
conflicting donor influence). However, over the years perceptions and circumstances have
changed, creating a need for revisiting the continuing relevance of these objectives; for
example the need to make the association more country-driven, to create a more specific set
of evaluable objectives, and to respond and contribute to the agendas of new regional
organizations, such as those sponsored by the African Union. In general ADEA was
appropriately designed to address its original strategic objectives. However, its design has
not included a clear monitoring and evaluation framework for tracking how well these
objectives have been met, and management structures are becoming inadequate for the
growing size and complexity of ADEA (see also Governance and Management).
10. Efficacy. With respect to the strategic objective of improving educational policies,
there is some anecdotal evidence that ADEA programs have had an impact at the country
level, especially on girls' education and other areas where working groups have been
xiv
especially active. However, since ADEA has not set up systems to assess the attainment of
such outcomes, it is not possible to make a full judgment about the achievement of this
objective. Similarly, on development agency coordination, there are indications of
improvement, including more interaction among donors, more co-sponsoring of regional
programs and the availability of an ADEA database covering agency projects (PRISME), but
no systematic evidence. Reinforcing a recommendation from the 2005 external evaluation,
this review indicates the need for better strategic planning, including the specification of
objectives, and tracking of outcomes indicators.
11. With respect to programs, ADEA has had a strong record in promoting policy
dialogue, and has become the region's premier forum for evidence-based policy discussions
-- possessing unmatched "convening power." The policy dialogue space that it has carved
out (its "social capital") is characterized by high levels of trust, informality and openness,
and the lack of political and ideological constraints. Its promotion of analytical work is
evidenced by increasing volumes of studies for its biennial meetings and the output of its
working groups. The volume and breadth of the work has clearly increased, but there are
questions about its quality. Some working groups have produced few publications, calling
into question whether they are contributing to regional education development. On
strengthening capacities in Africa countries, there is evidence that ADEA has made
substantial contributions, especially through the efforts of the most vigorous working groups,
through regional and sub-regional meetings, and through the anchoring of ADEA programs
in African institutions. Nevertheless, the reach of ADEA's capacity building has been
limited, and mainly available to those directly involved in WG activities or attending regional
and sub-regional meetings. In promoting regional and sub-regional networking ADEA's
accomplishments have also been notable, as a "networker of networks," but not as strong as
had been hoped given that only one of its working groups (FAWE) has "graduated" into a
self-reliant regional entity. Also, new ad hoc working groups have less potential for
spawning sustainable regional networks than the original variety.
12. Efficiency. There have been no cost-benefit analyses of ADEA products, given the
difficulty in placing a monetary value on Association outcomes, and no cost-effectiveness
ratios, given the lack of quantitative measures of results. However, analysis of ADEA
financing has led to some concerns about efficiency. Of the two forms of ADEA funding,
unrestricted and restricted (earmarked for specific programs like working groups), the latter
has predominated, particularly in recent years. This restricts the capacity of ADEA
management to allocate funds according to performance. Also, there are ongoing concerns,
initially raised in the 2000 evaluation, that working groups are not held accountable for their
funds. Solutions to these endemic problems are needed if ADEA is to operative efficiently.
13. Governance and Management. There are strong indications that ADEA has a
legitimate status as a regional entity, both among development agencies and government
ministries. Its governance model has a mix of shareholder (for development agencies) and
stakeholder (for ministries) elements, which works to enhance legitimacy. The fact that
agencies have twice as many seats on the Steering Committee than the ministries could raise
equity issues, but in recent years priorities have been set by the ministries, suggesting that
this imbalance does not undercut ministry influence. The bigger issue is the endemic problem
of frequent turnover among ministry representatives, which needs to be addressed. Still
ADEA's 31 member Steering Committee provides vigorous leadership all on a voluntary
xv
basis. However, it is overstretched, forcing it to focus more on procedural issues than
strategic planning and programming. In addition, decision making by consensus constrains
the Steering Committee's capacity to make difficult decisions. As noted in the 2005 external
evaluation, the Secretariat is also overloaded, given dramatic increases in ADEA's scope and
programs, suggesting the need for improved managerial resources and efficiency.
World Bank's Performance as a Partner
14. The World Bank has played numerous roles in ADEA, from co-founder, to initial
host institution, to lead financial supporter, to source of leadership (SC Chair and some
Executive Secretaries) and major consultants. These crucial roles, often intersecting in
complex ways, have made the Bank an indispensable partner. The Bank's analytical work
and seed money leveraged other agency support for ADEA (DAE), and its presence
continues to encourage partner involvement. Conversely, ADEA programs and positions
have often reinforced (added value to) the Bank's programs at the country level in Sub-
Saharan Africa. Such added value is not universal, however; it appears mostly in countries
where a Minister and/or Bank team has been deeply and personally engaged with ADEA
programs.
15. When backstopping of ADEA moved from Bank headquarters to its Secretariat in
Paris, Bank involvement shifted to the oversight mode. Over the 17 years of oversight this
function has been handled by five task team leaders, all recognized for their professionalism
and vision. The present task team leader has also been selected by the SC as its Chair. This
raises potential conflict of interest issues (not uncommon with global and regional programs),
which need to be acknowledged and mitigated. Bank disengagement from ADEA is not
being seriously discussed because of ADEA's unique position in fostering regional
cooperation and progress in reaching Education for All (EFA) and the Millennium
Development Goals. In addition, the Association is entering a crucial transition period
(moving its Secretariat to Africa, adopting of the new strategic plan, and replacing a retiring
Executive Secretary in 2008) during which the Bank's financial and professional support will
be important to keep up momentum and to leverage continued partner agency support.
However, exit strategies for some underperforming working groups are needed and should
figure into the new strategic plan.
Lessons Learned
16. The following lessons have emerged from the Bank's partnership with ADEA. Some
of these are previously identified lessons which have not yet been heeded, some are in the
process of being acted upon, some are modifications of previous lessons, and others are new.
� Strategic planning for ADEA's future, which has focused mostly on desired processes
and programs, also needs to specify organizational objectives in terms of desired
outcomes (e.g., EFA outcomes, such as improved access or learning outcomes). This is
a main point in the 2005 external evaluation which has not yet been heeded. In addition
to desired outcomes, there is a need for measurable performance indicators. Even some
of the so called "intangibles" that are prized in ADEA could be tracked in some way to
show how ADEA is measuring up to its ideals. This calls for an agreement among
ADEA partners as to the type of results-oriented monitoring and evaluation framework
xvi
that would be reasonable and credible, given ADEA's current and emerging objectives,
resources, and organizational structure.
� ADEA working groups need to be realigned with the Association's new strategic plan
and subject to a new set of evaluation criteria which the Association needs to apply
with firmness (a work already in progress). This would presumably lead to phasing
out or graduation of some current working groups. The new pattern of forming ad hoc
working groups could also be reviewed and modified as appropriate, given that its
acknowledged benefits are accompanied by certain disadvantages, such as the groups
not being anchored in Africa and their placing an administrative burden on the
Secretariat.
� Analytical work supported by ADEA is growing in volume but is of uneven quality.
As ADEA moves into new realms of applied research it needs to set higher quality
standards, and to concentrate as much on the quality (usefulness) of the research as on
quantity (output). This recognizes the important capacity and ownership-building
benefits of drawing new, local researchers into the process as co-investigators with
more senior researchers. For research used as background documents in biennial and
regional meetings and those published by ADEA, it would be desirable to establish a
formal policy and procedures for document review.
� Creative solutions to chronic overburdening of the ADEA Steering Committee must
be found so that the Committee has time to exercise more substantive leadership,
quality oversight, and identification of cutting edge issues. The formation of a
smaller, constituency-based Executive Board would be a step in this direction.
� Rapid turnover of Steering Committee members, particularly those from the Caucus
of Ministers, is undermining the strength, confidence, continuity and institutional
knowledge of their inputs into policy dialogue and decision making. Ways need to be
considered to bring more permanent ministry officials into ADEA decision making
circles and to find roles for emeritus committee members who are still prominent and
influential in education circles.
� ADEA programs and products are having a significant impact on those in the ADEA
inner circle, but do not reach very far beyond this. Ways need to be found to extend
the reach of ADEA programs and outputs, using country focal points or other
innovative vehicles.
1
1. Program Objectives, Activities, and Costs
Objectives
1.1 When ADEA was created in 1988 as one response to the Bank's landmark policy
paper on the crisis in sub-Saharan African Education, it established the general goals of
stimulating discussion of education policies in Sub-Saharan Africa and improving donor
coordination in the sector. During the next 17 years the Association frequently articulated
more specific goals and aims, but never established a formal set of objectives and related
outcomes measures.1 This became an issue during the 2005 external evaluation, which
recommended that ADEA develop "a strategic plan and business plan to operationalize
ADEA's vision, set clear expectations, and monitor progress towards results." The ADEA
Secretariat drafted such plans during 2006 and the ADEA Steering Committee formally
adopted a new vision statement, mission statement, and five strategic objectives in October
2007 (Box 1).2 Since this occurred after the present review was conducted, this review has
assessed the effectiveness of ADEA against its two original goals, with one modification.
The review will substitute "improvement of educational policies" for "discussion of
educational policies" since it is clear from ADEA's documents that policy strengthening is
what the discussion (dialogue) has been all about. Thus, the present review has regarded (a)
improving educational policies and (b) enhancing development agency coordination3 as the
imputed strategic objectives of ADEA before October 2007.
Activities
1.2 From the outset the Association aspired to influence educational policy and
development agency coordination in the region by stimulating policy dialogue informed by
research (analysis of existing problems and needs, and innovative solutions). Gradually it also
became of aware of the need to build greater capacity for policy formulation and
implementation and to promote enduring networks. In 2003 ADEA released a "visioning
document," in which the following four challenges were set out as the work of the Association:
(a) Promoting policy discussions (or dialogue)
(b) Promoting analytical work
(c) Strengthening capacities in African countries
(d) Promoting regional and sub-regional networks.
1. In its 2003 "visioning document" ADEA described its reluctance to commit to a limited set of objectives as
an asset ("institutional intelligence"), justified by the association's need to maintain an informality plus an
openness to diversity and the changing needs of African education (ADEA, 2003).
2. In the March 2007 Steering Committee meeting, some suggestions were made for revisions. The strategic
plan was revised and then adopted during the October 2007 Steering Committee Meeting. This undertaking was
not without controversy. Many of those interviewed preferred to keep the ADEA charter informal and open, and
worried that naming specific objectives and performance indicators would reduce its flexibility and
responsiveness. The ADEA Executive Secretary responded to this by committing to build flexibility into the
annual work plans.
3. The present review uses the term "development agency" instead of "donor" (the term used at ADEA's
founding), since many of the agencies contributing to development in Africa (e.g., the World Bank and UN
agencies) are not, technically speaking, donors.
2
Box 1. ADEA's New Vision Statement, Mission Statement, and Strategic Objectives
The ADEA Steering Committee formally approved the following at its most recent meeting in
October 2007.
Vision statement: ADEA becomes a leader in the processes of dialogue, sharing, and learning for the
qualitative transformation of education in support of Africa's development.
Mission statement: To act as a catalyst for innovative policies and practices conducive to change in
education through the pooling of thinking, experiences, lessons learned, and knowledge.
Strategic objectives:
1. Foster the development of a shared understanding for coordinated and effective action by the
main stakeholders to tackle the major challenges facing educational development in Africa.
2. Contribute to the achievement of continental and regional integration objectives in the education
sector, as defined by the AU and its NEPAD program through the Plan of Action for the Second
Decade of Education for Africa (2006�2015) and Africa's Science and Technology
Consolidated Plan of Action.
3. Produce the knowledge and lessons learned from successful experiences in areas that are critical
to the qualitative transformation of education in Africa in order to reinforce the policies and
capacity for reform.
4. Disseminate as widely as possible the messages and findings accumulated by ADEA from
policy dialogue, analytical research and exchanges of experience in order to support knowledge
sharing, inter-learning, and capacity building.
5. Improve ADEA's performance in the areas of management, planning, evaluation, and
accounting and financial reporting while at the same time strengthening its institutional capacity
and organizational mechanisms
1.3 Promoting policy discussions. ADEA was built on the premise that educational
policies can be improved through informed discussion among development partners. At the
regional level this involves dialogue across countries, among government and non-government
organizations, and between country representatives and development agencies. At the national
and sub-national levels it involves the all-important discussions among actors and stakeholders
(government, non-government and development agency). The purposes of such dialogue are to
identify common problems, to formulate regional and national priorities, to demonstrate and
compare innovative solutions, and to help build consensus at least at the national level, but
often at the regional level and across agency types (e.g., development agencies and ministries
of education). Such dialogue is expected to build partnerships across organizations at the
regional and country level. It can be especially constructive for the (mostly) small countries in
Africa, which cannot possibly try out a full range of solutions to serious educational problems,
and for which learning from others makes sense. In ADEA policy dialogue takes place within
thematically oriented working groups, during regional and sub-regional meetings (most
importantly the biennial meeting); during Steering Group and Ministers' Meetings; and
through newsletters and publications. Given that most countries in the region have prioritized
basic education ("Education for All") it is at that level that most ADEA policy dialogue has
taken place, but interest in post-basic education has been building in recent years.
3
1.4 Promoting analytical work. One of the founding principles of ADEA is that policy
dialogue needs to be informed by rigorous analytical work, work that provides and
understanding of the nature of problems and constraints, identifies solutions and their
effectiveness, and compares strategies and outcomes. ADEA has come to promote a "praxis"
approach to research which consists of cycles of action, grounded research, reflection, and
improved action (including policy making). The Association promotes analytical work within
its working groups, which are expected to act as laboratories in identifying and testing new
solutions, and as preparation for thematic discussions in regional meetings at which research
reviews and case studies are presented. ADEA makes the results of analytical work available
in these forums and through its communications programs (archiving, publication and
dissemination).
1.5 Strengthening capacities in African countries. This was not an explicit concern at the
founding of ADEA, but by the mid-1990s partner governments began to note the serious
constraints that poor institutional capacity were placing on policy reform efforts and policy
implementation. Thus, at the request of the ministries, ADEA adopted capacity building as one
if its core challenges, a position reinforced by a 2002 needs assessment in which ministries
placed capacity building high on their list of needs. There is no specific ADEA program and
budget for this, but in many or most of the working group and regional meetings there are
explicit or implicit efforts to improve local capacity for both strategic programming and related
technical skills (e.g., initiating and managing reform, creating management information
systems, conducting research, and managing finances).4 Most of the technical capacity building
that it has undertaken has been by or in connection with various working groups, with those on
Finance and Education, Communications for Education and Development, and Education
Statistics being particularly active. Improved capacity for strategic programming and reform
management has often been facilitated by regional/sub-regional meetings and cross-national
exchanges. In addition, ADEA's earnest efforts to shift coordination of working groups to
African institutions are also a form of institutional capacity building, the best case of this being
the Forum for African Women Educationists (FAWE).
1.6 Promoting regional and sub-regional networks. This activity or "challenge" has
been implicit in ADEA since the beginning, especially in the concept of working groups and
their eventual "graduation" to the status of self-standing regional organizations. During the
mid-1990s a concern for anchoring ADEA activities in African education spurred an effort to
link ADEA working groups to African institutions, and more recently there are ADEA efforts
to partner with and contribute to other regional networks, such as those connected to African
Union and various sub-regional economic communities and research associations.
ADEA Vehicles
1.7 From the early years of the DAE to the present the main vehicles used by ADEA to
advance its work have been: (a) loosely coordinated working groups; (b) Steering Group and
Bureau of Minister meetings; (c) biennial meetings involving all agency, ministry and civil
society partners; (d) special or sub-regional meetings; and (e) publication and information
4. Since ADEA is primarily a policy and knowledge network, it does not have the charter and/or capacity to
undertake wide-spread capacity building, and if it did, it would be in danger of encroaching on other institutions
that have this function.
4
dissemination (including databases). Each feeds into one or more of the activity areas above,
and is organized as follows.
1.8 Working Groups. At the outset nine working groups were formed, covering topics
considered by the agency partners to be highest priority in Africa, many directly connected to
findings and recommendations in the Sub-Saharan Africa policy paper. (See Annex C for a
chronology and description of ADEA working groups.) Initially, working groups were all
sponsored by one or more of the development agency partners, and given an open agenda.
With the push towards more anchoring of ADEA in Africa beginning in 1994, increasing
numbers of working groups (starting with the one on Female Participation) forged
connections to other institutions in Africa.5 By 2007 over half of the working groups were
linked to African institutions. New working groups have been created along the way and a
few closed or retired (the 2000 evaluation recommended that a large number be either
dismantled or graduated but few have been) and that no new ones be created (one has been).
Instead, a new kind of working group, now called an ad hoc working group, has been set up
to deal with urgent cross-cutting themes over a limited time period.
1.9 Steering Committee and Bureau of Ministers Meetings. ADEA is governed by a
Steering Committee consisting of partner development agencies plus a group (Bureau) of
African ministers of education (see the ADEA Organizational Chart in Annex E). Steering
Committee meetings are held twice a year to provide general guidance and decision making
on ADEA directions, work plans, and budget allocation. The Bureau of Ministers generally
meets twice a year adjacent to a Steering Committee meeting and uses this time to consider
emerging issues, plan sub-regional activities, and launch regional programs such as inter-
country quality nodes and peer reviews of national education programs. These events are
intense, covering a large array of management, policy and budgetary issues, but are also
expected to provide opportunities to critically reflect on emerging problems, strategies and
promising new practices in African education.
1.10 Biennial Meetings. The major event in the ADEA calendar is the biennial meeting.
This event convenes all donors to ADEA and their teams, most ministers of education in Sub-
Saharan Africa, and a variety of stakeholders from academia and civil society groups. Its
purpose is to stimulate open and frank policy dialogue among the partners, generally around a
particular theme for which papers and case studies have been prepared in advance. (A list of
biennial meetings, their dates, venues and themes, is provided in Annex D.) The first of these
meetings was held in Manchester England with a focus on reports and papers from one of the
working groups (that on Textbooks and Libraries). In early years such meetings were called
"plenary meetings," to distinguish them from meetings having a more limited scope, like
Steering Committee or working group meetings. The next two bienniales (as they began to be
called) were held in France. After that, in keeping with the movement towards more African
ownership of ADEA, bienniales have been held in African national capitals. Products from the
bienniales are published proceedings, the case studies and research presented, an ADEA
newsletter summarizing biennial reports and discussion highlights, and follow-up activities.
5. The Working Group on Female Participation became connected to the Forum for African Women
Educationists (FAWE) and eventually "graduated" from ADEA (although it has maintained connection as an
associate member).
5
1.11 Publications and Information Dissemination. ADEA produces, publishes, and
promotes a wide range of articles, books, databases, papers, and reports that relate to the
development of education in Africa. The complete catalogue of ADEA publications and
documents contains more than 200 titles. Many of these are available online, and others can be
ordered from the ADEA Secretariat and from the working groups. These publications include:
� Quarterly ADEA Newsletters reaching 12,000 addresses and covering topics of the
day (results from biennial or Steering Committee meetings, summary of WG group
findings and recommendations, topics of special interest);
� Occasional ADEA News Briefs, covering breaking news on African education and
ADEA happenings
� A new (2005) series of books covering country case studies in education;
� Books and materials reporting research and proceedings from case studies and
bienniales.
1.12 In addition ADEA maintains three databases relevant to the development of education
in Africa, including (a) Profile: a database of ADEA working groups and programs in 48
countries of sub-Saharan Africa; (b) PRISME, a descriptive database of foreign aid to
education in sub-Saharan Africa; and (c) CV Experts, an online curriculum vitae information
system of African specialists in education. Finally, the ADEA communications section
manages the annual Africa Education Journalism Award, created by ADEA in 2001 to
recognize the best articles on education in Africa written by African journalists and published
in the African press, and invites an increasing number of receptive journalists from Africa
and other parts of the world to its gatherings.
Governance
1.13 ADEA's Steering Committee composed of ten African ministers of education and
representatives of most multilateral, bilateral and private development organizations that
work in the education sector in Africa (currently 21). The Steering Committee is both
ADEA's governing body and its primary facilitator of coordination among funding agencies,
among African ministries of education, and between these two components. The ten
ministers of education who sit on the Steering Committee constitute the ADEA Bureau of
Ministers. They are chosen by the Caucus of Ministers which consists of all the ministers
of education of sub-Saharan Africa. Bureau Ministers are selected by region (two each for
Western Africa, Eastern Africa, Central Africa, Southern Africa and the Indian Ocean) on the
basis of alphabetical rotation by country. They serve for a four-year period. Both the Bureau
of Ministers, which meets twice a year, and the Caucus of Ministers, which meets during
ADEA's biennial meetings, are forums for dialogue and consultation on the challenges and
opportunities for the development of education in Africa. The Steering Committee meets
twice a year to decide on budgets and work programs, including those proposed by Working
Groups. Each Working Group in turn has its own steering committee.
1.14 Administrative support for ADEA is provided by the ADEA Secretariat, which since
1992 has been hosted physically and administratively by the International Institute for
Educational Planning (IIEP) in Paris, a UNESCO affiliated organization.6 In 2008 the
6. According to the Revised Principles of Association of the "Association for the Development of Education in
6
Secretariat will move to Tunis, Tunisia, where it will be hosted by the African Development
Bank.7 The ADEA Secretariat facilitates the coordination of the working groups, promotes
information exchange and communication, organizes the biennial and Steering Committee
meetings, develops and updates databases on education in Africa, and coordinates programs
and activities such as the intra-African exchange program. The Secretariat consists of an
Executive Secretary and 5�6 full time professionals and part-time consultants. When the
Steering Committee is not in session, decisions about ADEA are made by the Bureau of
ADEA (also referred to as the Bureau of the Steering Committee), which consists of the
Steering Committee Chair, the Alternate Chair (also head of Bureau of Ministers), the Vice-
Chair (ex-officio and head of the host agency), and the Executive Secretary of ADEA.
Costs and Financing
1.15 ADEA is funded almost entirely through contributions from its 20�22 development
agency partners (the number has fluctuated recently), and African Ministries of Education
through in-kind and cash offerings.8 A minimum agency contribution of $50,000 has been
required for joining and remaining in the partnership. These amounts are "unrestricted" or core
funding and can be allocated by ADEA according to needs. Contributors also pledge
"restricted" funds, which are generally earmarked for particular working groups. Restricted
funding has exceeded unrestricted by as much as 3 to 1 in recent years. As can be seen in
Figure 1 funding (especially restricted) fluctuates from year to year, which places constraints
on long-term planning. Some countries (namely, Canada, the Netherlands, Finland and
Sweden) have worked to moderate this by signing multiyear funding agreements, but this has
not become the norm. Given such multi-year pledges, ADEA annual budgets generally show
carry-overs from year to year. The estimated funding in calendar year 2006 -- about $6.6
million, mostly covering that year, but for some donors stretching to subsequent years -- is the
highest level of funding ever (a 150 percent increase in funding over that for year 2000 in
nominal terms), reflecting an expansion in ADEA programs and development agency support.9
Africa" (Paris: 1995), IIEP acts as host agency for the ADEA Secretariat, so that ADEA funds are administered as a
"funds-in-trust project" within the UNESCO special account for IIEP, operating under the responsibility of the IIEP
Director and Governing Board. The Director of the host agency serves as the ex-officio Vice Chair of ADEA's
Steering Committee. As host agency, IIEP serves in a custodial role, monitoring the policies and programs defined by
the Steering Committee. The ADEA Secretariat operates under the administrative regulations of the host agency, with
the Secretariat staff to be hired a personnel of the host agency. (In practice, Secretariat members have been hired as
short-term staff members.) Over the years IIEP has accorded ADEA virtual autonomy over its affairs and budget.
7. The feasibility study was conducted to consider various alternatives for the Secretariat's move to Africa (R.
Sack, Kinyanjui, K., and Kinzounza, K., "Feasibility Study on the Transfer of the ADEA Secretariat to Africa,"
December, 2006). This commented on AfDB's strong commitment to education (based the number of projects
0and professional staff members in the education sector), but the IEG mission to the AfDB in Tunis was informed
by AfDB staff that its education program was being downsized, given the AfDB's prioritization of infrastructure
under its new leadership, and that there might be few or no projects in the pipeline for education in the near future.
ADEA has benefited from its being hosted by an agency (IIEP) with a strong commitment and reputation for
excellence and leadership in educational research and policy studies. ADEA may not receive the same kind of
support from AfDB if the latter moves towards a relatively weak commitment to the education sector.
8. African ministries of education have been expected to contribute $1000 a year, but in 2007 the Bureau of
Ministers recommended raising that to $2,500.
9. It can be seen from Figure 1 that the increase in funding for 2006 was almost completely in the form of
unrestricted funds, which is a positive development, given the recurrent "structural deficit" that ADEA has faced,
7
Figure 1. Annual Contributions to ADEA (Restricted and Unrestricted)
US$ Millions
7.0
Restricted
6.0
Unrestricted
5.0
4.0
3.0
2.0
1.0
0.0
2000 2001 2002 2003 2004 2005 2006
Audited Estimate
Source: ADEA, 2006 Report of Activities.
1.16 The use of funds in ADEA (Figure 2) appears to be consistent with its charter. A
majority (47�55 percent in recent years) of the funds are devoted to the operation of the
working groups. Funding support for policy dialogue fluctuates between 7 and 21 percent
depending upon whether it is a biennial meeting year. Funding for publications and
communications have in recent years been around 5�6 percent of the total, except in 2004 --
the year following the report-rich bienniale on education quality -- when it rose to over
12 percent. During the past three years (the only years in which this has been broken out) the
percent of budget allocated to the Secretariat has been 16�17 percent.
1.17 ADEA has generally been known for sound financial and budget management. For
example, in 2004 it executed its initially approved budget plus a 9 percent overrun which was
approved by the Steering Committee and covered by the mobilization of additional funds.
However, the financial situation deteriorated unexpectedly in 2005, when IIEP presented ADEA
claims for a significant payment arrears, based on its 2002�04 accounting.10 An independent
audit found that the arrears was not as great as originally presented, but this still put a strain on
ADEA programming, especially by the Secretariat, whose funds were especially curtailed, during
2005 and 2006. A large increase in development agency pledges of unrestricted funds in 2006
(especially from the World Bank's Development Grant Facility) seems to be improving the
situation, but results will only be known for sure when audited financial accounts are available.
in which there is less funding than needed for programs covered by the association's "general" (discretionary)
account. ADEA's campaign to increase unrestricted funds and correct its structural imbalance is expected to be
given a boost by the new strategic plan (providing the basis for longer-term commitments by agencies) and by the
ministries' decision in 2007 to increase their membership fees from $1,000 per year to $2,500.
10. This was to cover an apparent debt incurred by ADEA over 2002�2004 but not perceived by IIEP because
of UNESCO's transition to SAP and other delays in reconciling accounts for spending in Africa.
8
Figure 2. ADEA Use of Funds 2001�2006
US$ Millions
3.5
3.0
2001 2002 2003 2004 2005 2006
2.5
2.0
1.5
1.0
0.5
0.0
Ad hoc Publications Secretariat Working Coordination Overhead Policy Regional &
Working & Communi- Groups & Facilitation & Reserve Dialogue Sub-regional
Groups cations Cooperation
Kind of Activity
Source: ADEA, 2006 Report of Activities
1.18 Financial support from the World Bank has been significant over the entire nineteen-
year period of ADEA's existence. In the first three years, when the support services resided
at the Bank, its contributions were largely in kind (i.e., staff time). One analysis estimated the
Bank funding during that period to be about $445,000 compared to the overall development
agency support of $2.5 million, or about 18 percent of the total. After 1992, Bank support
came largely through grants from its Special Grants Program (renamed the Development
Grant Facility in 1998). By the early years of this decade (2000�2002), DFG funding
amounted to about $300�350,000 per year, or about 12�14 percent of the total. In the middle
years of the decade, financial support for ADEA from other agencies increased, but DGF
support remained the same (reducing its share to less than 6 percent). In 2006, the DGF
agreed to an increase in DGF funding (from $350,000 to $1.0 million), although $400,000 of
this $650,000 increase was for activities previously funded though other channels (a
$250,000 on-grant to FAWE and $150,000 for the Working Group on Higher Education).
The additional increase was to help ADEA support new ad hoc working groups mainly
linked to EFA, for example, on HIV/AIDS and education, and on the quality of education,
the latter as a follow-up to a very successful biennial meeting on this topic and the
preparation for a second one in 2006. The Bank's task team leader advocated these increases
on the basis of the increased value added of ADEA programs to the Bank's educational
sector objectives in the region. Depending on the estimates used for total commitments to
ADEA in 2006, this represented an increase in the proportion contributed by the Bank to
around 12�15 percent -- a return to the share provided in the early years of the current
decade.
9
2. The 2005 Independent Evaluation of ADEA
Evaluation Process
2.1 The 2005 independent evaluation of ADEA was undertaken by Universalia, a
Canadian management consulting firm based in Montreal. This was the third evaluation of
ADEA, the first being undertaken in 1994, when the Association was still known as the
DAE, and the second in 2000, which focused explicitly on the ADEA working groups. The
2005 evaluation was undertaken under the authority of the ADEA Steering Committee at the
request of the World Bank's Development Grant Facility, which requires programs receiving
its financial support to be evaluated every 3�5 years.
2.2 The terms of reference for the evaluation were drafted by the ADEA Secretariat and
reviewed by the Steering Committee. They were finally approved by the Bureau of ADEA
(Steering Committee Chair and Alternate Chair, the Director of the IIEP, and the ADEA
Executive Secretary). The contractor was selected from among candidate groups that were
nominated by Steering Committee Members. The short-listed firms were sent Request for
proposals (RFPs) which were reviewed by the Bureau of ADEA. Universalia was judged to
have the strongest proposal. The evaluation cost about US$100,000, which was covered by
exceptional funds provided by the World Bank (DGF) and CIDA.
2.3 According to the current Executive Secretary of ADEA, the Association had never
had contact with this contractor before. Moreover, none of Universalia investigators had ever
been involved with ADEA in the past. All conclusions in the evaluation report were arrived
at independently; there was no manipulation of the findings or recommendations by ADEA
stakeholders in any way. After the evaluation report was drafted it was reviewed by the
ADEA Secretariat and Steering Committee over a two-month period, and revisions were
made in response to corrective inputs.
Monitoring and Evaluation Framework
2.4 One of the main findings of the Universalia evaluation was that ADEA did not have a
well defined strategic plan and therefore had no clearly stated objectives or expected results and
related strategies, and no clear framework for monitoring and evaluation (with performance
indicators to track progress). The use of a logical framework has been somewhat antithetical to
ADEA's work, which has been characterized as informal and responsive to changes in the African
education scene. Consequently, much of ADEA's work and outputs have not been systematically
evaluated for their value added. This made the external evaluation less results-oriented than it
might have been. Universalia did acknowledge that some useful results had been produced, but
even in its evaluation report, these were hard to find. For example, the main operational arm of
ADEA is its working groups, but the evaluation's review of the working groups' contributions and
products was only the sixth of six items in its section on program effectiveness.
2.5 Universalia's evaluation adopted a methodology known as the Institutional and
Organizational Assessment (IOA) framework, developed jointly by itself and the
International Development Research Centre of Canada (IDRC). The framework focused the
evaluation on three main features of organizational performance, relevance, effectiveness,
and efficiency (similar to IEG's approach). The evaluation also covered governance and
10
resource mobilization and management (in its section on efficiency), and touched upon exit
strategy, especially for working groups. Sustainability and risk were not explicitly covered.
2.6 In undertaking its assessment of relevance, effectiveness, and efficiency it used a mix
of quantitative and qualitative methods, including:
Document review;
Questionnaire (administered to a potential pool of 177 stakeholders -- 74 of whom
responded);
Interviews (45 face to face); and
Field visits to Africa (including Benin, Burkina Faso, Niger, Senegal, Uganda, and
Kenya).
Main Findings and Recommendations of the Evaluation
2.7 The Universalia evaluation presented 23 findings, from the confirmation of ADEA's
success in building "social capital" around educational change to the diverse perspectives on
the ideal geographical location for ADEA's Secretariat. About a third of the findings dealt
with the Association's effectiveness and another third with its efficiency. Some of the
findings on efficiency covered funding and financial management, including the relative
proportions of restricted and unrestricted funding, but the predominance of the former was
not highlighted as much as it could have been.
2.8 This review notes that many of the evaluation's findings document stakeholder
judgments on various questions, rather than the evaluator's own independent judgments, based
on evidence collected. For example, one finding is that "ADEA is perceived to be effective in
facilitating policy change" (italics added). This finding is largely derived from questionnaire
results in which three-quarters of the stakeholders surveyed endorsed this position, many of
whom have some stake in a positive ADEA image. A more nuanced assessment would have at
least differentiated between the stakeholders having insider (producer) and outsider (consumer)
status. It would have also been good to provide more evidence of how ADEA events or programs
explicitly influenced a policy or program at the national level. Similarly stated findings are:
"ADEA is regarded as an effective medium for building partnerships," and "ADEA's Secretariat
is generally considered to be very effective at ... disseminating relevant information."
2.9 Also, given the importance ADEA attaches to the conduct of analytical work and its
use in informing practice, it was surprising that the evaluation did not highlight or catalogue
the reports/publications that had been produced, including the main messages and
conclusions from the 2003 bienniale, which would soon appear as The Challenge of Learning
(2005). In addition, nothing was written in the evaluation about the quality of the studies,
reports, and reviews that have been sponsored by ADEA.
2.10 Universalia's findings led to the formulation of 8 recommendations, presented in
Table 1, along with ADEA's responses.
ADEA's Response
2.11 Interviewees asked about the external evaluation generally found it to be fair and
constructive, some saying that it brought up important issues that had been just below the
11
Table 1. External Evaluation Recommendations and ADEA Response (before the
Steering Committee meeting in October 2007)
Recommendations ADEA Response
1. Based on the principles set out in the 2003 Vision 1. Some changes in organization are underway (see
Document, the Steering Committee should clarify point 2 below), but this will not be fully undertaken
what type of organization structure is most until the new strategic plan has been finalized.
appropriate for ADEA's future development.
2. The Steering Committee should revisit its mode of 2. The Association has proposed that its finance sub-
operation in order to address important matters committee fuse with the Steering Committee
such as management oversights, decision-making, Bureau11 to become an Executive Board in charge
ownership, and fiduciary responsibility. of making certain decisions for the Association. A
consultant has been appointed to work with the
Secretariat in working out the mandate and the
boundaries of the new Executive Board.
3. In order to provide a greater sense of direction and 3. The Secretariat spent much of 2006 drafting a new
focus, the Steering Committee should ensure the strategic plan which was reviewed and commented
development of a strategic plan and business plan by the Steering Committee during its October 2006
to operationalize ADEA's vision, set clear and March 2007 meetings. There has been some
expectations, and monitor progress towards results. controversy in ADEA over the creation of a
formalized strategic plan, given the tradition of
operating with informality, but the Executive
Secretary has indicated his intension to build some
flexibility and spontaneity into the plan. The plan will
be finalized before the second SC meeting in 2007.
4. The Secretariat should be strengthened both in 4. ADEA is looking at ways to strengthen its
terms of its analytical contributions and logistical Secretariat commensurate with its expanding
oversight functions. Resources attributed to the mission. This will not be fully actualized until after
Secretariat should be commensurate with the the new work plan is finalized and approved but at
expectations of ADEA's constituency. least there is a decision to strengthen the financial
and administrative management post in the
Secretariat and to recruit a senior person to fill it in
2008.
5. The Steering Committee, with the support of IIEP, 5. Given the fact that ADEA will be disassociating itself
should modify its human resource procedures in with IIEP during the next year, addressing this issue
order to regularize the status of Secretariat staff. is untimely. It would be appropriate to address it
with the new back-up agency.
6. Over the course of next year (2006), the Steering 6. Succession planning for successor to the current
Committee should plan for the succession of the Executive Secretary who will retire in 2008 is going
Executive Secretary. ahead. There is now a ten step plan, and, as of
January 2007, five had been completed. The
process should complete by summer '08.
7. The Steering Committee, in collaboration with the 7. A new communications plan developed by the
Secretariat, should develop policy guidelines to Secretariat was briefly presented at the March `07
improve dissemination of ADEA's work. SC meeting but there was no discussion of it.
8. The Steering Committee should carry out a 8. The feasibility study concerning relocation was
feasibility study for the relocation of the Secretariat presented at the March 2007 Steering Committee
and a detailed cost-benefit analysis of the current meeting and a decision reached to relocate to Tunis
arrangements with IIEP. in association with the African Development Bank. A
cost-benefit analysis of arrangements with IIEP was
not undertaken, given ADEA's impending separation
from IIEP.
11. The Steering Committee Bureau consists of the SC Chair, Alternate Chair, Ex-Officio Vice-Chair, and the
Executive Secretary of ADEA.
12
surface. The ADEA Steering Committee has taken its recommendations seriously and has
initiated many changes in response to it, especially with respect to recommendation #3 on the
drafting of a strategic plan; #6 on planning for a successor to the current executive secretary;
#7 on the creation of a new communications plan; and #8 on the conduct of a feasibility
study for relocating the Secretariat to Africa.
2.12 With respect to the other recommendations -- #1 on changing the organization
structure, #2 on changing the Steering Committee's mode of operation, #4 on strengthening
of the Secretariat, and #5 on changing the human resources procedures -- much preliminary
planning and debate has gone into them, but these could not be completed until after the new
strategic plan was finalized in October 2007 or, in the case of human resources, detailed
arrangements with the new host organization (AfDB) are worked out.
3. The Effectiveness of the Program
Relevance
3.1 Relevance of Objectives. The founding aims of ADEA of improving education
policies and enhancing donor coordination, taken by this review as its strategic objectives,
were derived from the 1987 Education Sector Policy Paper and thus highly relevant to the
conditions portrayed in it. For example, the paper had demonstrated how inadequate policies
in education were behind much of the poor performance and stagnation of the education
system that were apparent in the late 1980s. Also, it revealed a crisis of underfunding in basic
education, which had created a growing dependency on external funding in many countries.
This had led to a chaotic mix of development agency requirements and preferences, often
creating more problems than solutions. Bringing development agencies together in
partnership with ministries of education to discuss needed policy reforms in education was a
much needed step in addressing these problems.
3.2 However, over the years perceptions and circumstances have changed, creating a need
for revisiting the continuing relevance of these objectives. For example, a growing awareness
of the donor-driven origins of many educational initiatives in the early 1990s led to ADEA's
determination to anchor the Association more in African institutions and in country-defined
needs -- out of which grew a stronger emphasis on institutional capacity and regional network
building, and less explicit emphasis on donor coordination.12 Other perceptions focused on the
vagueness of the ADEA agenda, which had produced a highly divergent set of expectations for
the Association, and widely varying perceptions on how effective the Association had been
(see 2005 external evaluation) -- problems which the new strategic plan for 2008�2012 is
presumably intended to address.13 Finally, new regional initiatives launched by the African
Union -- e.g., the Decade for Education in Africa (2006�2015) and the New Economic
12. Donor coordination in Africa, as it other parts of the world, is increasingly being seen in relation to the
creation and support of country or regional sector plans and not as a separate issue.
13. The 2005 evaluation's recommendations that a strategic plan be produced "outlining clear expectations" and the
monitoring of "progress towards results," was one of the factors leading to ADEA's decision to draft a new strategic
plan in 2006. But the new strategic objectives approved by the Steering Committee in October 2007 are still
mostly about what ADEA expects to do (activities and outputs) rather than what it expects to accomplish (outcomes).
13
Partnership for Africa's Development (NEPAD) -- have set out ambitious regional goals for
education that ADEA needs to respond or adjust to or face growing irrelevance.14
3.3 Relevance of design. In general ADEA was appropriately designed to address its
original strategic objectives of improving policies and enhancing donor coordination.
Especially relevant were its policy dialogue and analytic components which were to operate
through its semi-autonomous working groups and regional/sub-regional meetings. However, as
noted in the 2005 external evaluation, its design has not included a clear monitoring and
evaluation framework for tracking how well these objectives have been met.15 Also, ADEA has
failed to establish accountability systems vis-�-vis its working groups to assure their
productivity and to document their outcomes and policy impacts. This is in part a consequence
of its outdated financing design, which allows most donors to contribute far more earmarked
funds than unrestricted ones.16 Finally, the management structures are becoming inadequate for
the growing size and complexity of ADEA, a point brought up in the 2005 external evaluation
and more fully elaborated in the Governance and Management section of this review.
Efficacy
3.4 To assess the efficacy of the Association, this review examined outcomes on the two
strategic objectives of (a) improving educational policies and (b) enhancing donor
coordination. It also reviewed the outputs of the programs which contribute to the fulfillment
of these objectives. This sub-section will examine the efficacy of both the objectives
(outcomes) and the programs (outputs).
OBJECTIVE: IMPROVING EDUCATIONAL POLICIES
3.5 As mentioned in the previous section, there is not enough systematic evidence to show
a clear pattern of ADEA impacts on the objective of education policy improvement in Sub-
Saharan Africa. However, testimonies of positive impacts abound. For example, one World
14. ADEA contributed to the development of AU's "Decade for Education in Africa" and has put it, along with
NEPAD priorities, into the background section for its new strategic plan. One difference between the draft
ADEA objectives and those under these AU initiatives, is that the latter are quite outcomes oriented (e.g.,
calling for use of African languages of instruction, systems to monitor and evaluate learning achievement,
achieving UPE by 2015, among other goals), whereas those of ADEA are more process oriented.
15. Responding to the external evaluation recommendation calling for a framework for assessing the achievement of its
objectives has created a dilemma for ADEA. Since ADEA is not a conventional investment project like its development
agency partners finance at the country level, it does not have easily measured inputs, processes, outputs and outcomes.
Indeed, as a regional forum pressing for policy dialogue and voluntary change at the national level, its outcomes can be
quite evasive and effuse. This is not to say that tracking ADEA-stimulated policy changes in various countries would be
impossible, but given the complexity of doing so (such as dealing with the issue of attribution) and given existing
methodologies, this has so far been viewed as a less worthwhile use of its relatively modest regional program funding.
Responding to this recommendation in a cost-effective way will require some creative thinking.
16. It cannot be assumed that if partner agencies were prevailed upon to reduce their restricted funding that they
would shift the difference into unrestricted funding, since for some agencies one of the main reasons for
supporting ADEA is to have the theme of their favorite working group on the agenda. The rebalancing of the
two main forms of financing may need to come primarily from partners increasing their subscription fee
$50,000 (an amount set over a decade ago) to $100,000 (an amount already contributed by some). Also, there
appears to be a new commitment in ADEA (in the new strategic plan) to making the working groups more
accountable to the SC, which will make it less possible for them to get funding irrespective of their productivity.
14
Box 2. ADEA and African Policies on the Language of Instruction
For decades primary school children in African countries have been required to study in colonial
language which most of them do not use at home or even understand. Propelled by the growing
awareness of how little children learn under such circumstances, the Ad Hoc ADEA Working Group on
Quality of Education introduced the themes of mother tongue and bilingual education into its agenda for
the 2003 ADEA Biennial Meeting, "Improving the Quality of Basic Education in Sub-Saharan Africa."
Participants in the meeting concluded that "African languages were a necessary choice for the new
century," and proposed further "stock-taking" research on the subject and more intensive discussions.
As a response, ADEA commissioned a set of state-of-the-art reviews and case studies in 2005, aimed at
presenting evidence-based recommendations for language-in-education policies for use by policy
makers and other stakeholders. The result was an ADEA volume, Optimizing Learning and Education
in Africa -- the Language Factor: stock-taking review on mother-tongue and bilingual education in
Sub-Saharan Africa, which contained critical reviews of the existing literature by six experts and new
case studies from 13 African countries on effectiveness (in relation to learning outcomes) of bilingual
education programs (mother tongue plus a metropolitan language), efficiency, program design elements,
and reasons for successes and failures. This volume became the focal point of a three day thematic
conference co-sponsored by ADEA (together with UNESCO and GTZ) in Windhoek, Namibia in
August 2005, attended by over 60 international experts and African Ministries of Education from about
20 countries. Once revised in response to discussions in that conference the report was presented during
the 2006 bienniale in Gabon, where it had a prominent place on the agenda.
This visibility and evidence-based advocacy is making an impact on the African education
community, at least at the rhetorical level. For example, the action plan adopted by the Conference of
Ministers of Education of the African Union under its 2006�2015 new Decade for Education in
Africa includes an item directly or indirectly influenced by these efforts, namely: "strengthening of
curricula and teaching materials, with special emphasis on mathematics and technology, the use of
African languages in bilingual education, and the gearing of curricula to local contexts."
Nevertheless, evidence of policy change at the national level is hard to find. Only one instance of
country level follow-up is mentioned in the 2006 ADEA Report of Activities: expert advice to
Burkina Faso in formulating a national language policy. Some interviewees for this report indicated
that enthusiasm for bilingual education may be cooling down these days, perhaps as a result of some
well publicized negative experiences with bilingual education and/or the stock-taking report's strong
advocacy of an excellent but demanding "late exit" bilingual education model (requiring 6�7 years of
instruction in the mother tongue), which some governments may find too demanding.
3.6 Bank report indicated that as a result of participating in ADEA many countries had
improved their record in undertaking evidence-based policy adjustment, had recognized the
need for national political commitment as a pre-requisite for reaching EFA goals, had increased
their commitment to girls participation in schooling and girl-friendly schooling, had recognized
the need for a new "culture of quality" to enhance basic education outcomes, and had grasped
the need for closer government cooperation with NGOs, teachers' unions, and parents'
associations in reforming educational policy and practice.17 Written for a management review,
these statements were not supported by a strong body of evidence. Some impacts were reported
to occur with a time-lag, such as the changes in rural education since 2000 following from
visits by African Ministers of Education to Guatemala and El Salvador in 1999.18
17. See World Bank, 2005, Partnership Approval and Tracking System, Global Programs and Partnerships,
"Association for the Development of Education in Africa."
18. Communication from the ADEA Secretariat in December 2007.
15
3.7 The 2005 external evaluation cited anecdotal evidence in its review of ADEA's
impact on improved policies. The prime example was the widespread view of how the WG
on Female Participation (now FAWE) influenced the commitment to girls' basic education
across the continent. Other fields where ADEA programs were said to have influenced
policies at the national level are public finance and education, teacher policies (e.g., use of
contract teachers), improved statistical systems for education, national policies on early
childhood education, and policies on language of instruction and bilingual education (Box 2).
3.8 All the Ministers interviewed for this review could point to policy changes in their
countries that were stimulated by ADEA discussions, but also suggested that impact was
greatest where a minister or official had direct involvement in an ADEA policy dialogue
(e.g., as a contributor to a biennial event). ADEA appears to have had less success in
influencing ministers and ministerial agendas among those not directly involved in ADEA
events. Reasons for this are not fully clear, but they appear in part to be a consequence of
inadequate dissemination of ADEA information and products (especially within countries),
and widespread failure to establish recommended ADEA focal points in ministries and
national coordinators for working groups.19
3.9 For this review nine World Bank education Task Team Leaders (TTLs) covering all
sub-regions of Sub-Saharan African (about half having direct experience with ADEA) were
interviewed (Annex H). Of the 9 TTLs, only two indicated that ADEA objectives and
activities were relevant to their country's current development priorities (Table 2). Those two
also indicated that they were highly familiar with ADEA, buttressing the point that
perceptions of ADEA's impacts and relevance are, at least in part, a function of direct
experience with the Association.20
3.10 Looking across country programs more generally, there are signs of a disconnect
between the rhetoric and lofty intentions of ADEA gatherings on the one hand and the
realities on the ground on the other (Boxes 2 and 3). If ADEA had been tracking policy
impacts more systematically, it could presumably have become more aware of such
disconnects and worked to overcome them.
19. In addition to disseminating ADEA information and products at the national and sub-national levels, a
country focal point or coordinator could also track the extent to which ADEA agenda items and products
(including those from working groups) are being used in the country and what their impact has been, thus
becoming part of a relatively low-cost monitoring framework.
20. Given the alleged high value added of ADEA to the World Bank's Education Sector Work in Africa (see
World Bank, 2005), it is curious that more TTLs are not well versed in ADEA's purposes and programs. This
may be in part a consequence of the small amount of the Bank's administrative budget allocated for ADEA
activities ($50,000 per annum), which has gone mostly to support the Working Group on Higher Education
(personal communication from Birger Fredriksen). The Education Sector in Africa might consider giving higher
priority to briefing its African task team leaders about ADEA and involving them in its activities in some way.
16
Table 2. World Bank Task Team Leader Views of ADEA: Frequencies (n=9)
A A A
A great Don't
substantial modest negligible
deal know
Question amount amount amount
1. Familiar with ADEA /1 4 3 2
2. Used ADEA products or services in
3 2 4
country operational work
3. Country counterparts show interest in
2 4 3
ADEA products and services
4. Objectives and activities relevant to
country's current development 2 3 3 1
priorities
Don't
Very Moderately A little Not at all
know
5. Aspects of ADEA considered most
useful:
a. Facilitating communications
4 2 1 2
among practitioners
b. Generating and disseminating
2 2 2 1 2
information & knowledge
c. Improving donor coordination 1 2 4 2
d. Advocating approaches to
2 2 1 4
development in the sector
e. Supporting national level policy,
2 1 2 4
institutional & tech reforms
f. Strengthening capacity and
1 2 4 2
training
/1 For this item the response categories were: Very, somewhat, heard of it, never heard of it.
OBJECTIVE: ENHANCING DEVELOPMENT AGENCY COORDINATION
3.11 For this objective, there is no systematic information on outcomes. The overarching
assumption appears to be that improved interagency coordination will result from all major
agencies being ADEA partners, that are sitting together on the Steering Committee,
interacting during biennial and other regional/sub-regional meetings, reaching a consensus
on some reform agendas, and co-sponsoring working groups and ADEA events. ADEA's
promotion of government-owned, sector-wide, education development plans, supported by
development agencies in a coordinated manner, can be assumed to have positively
influenced agency coordination. In addition, donors have often joined hands in co-sponsoring
various events and gatherings with ADEA (see section on Regional and Sub-Regional
Networking). And a database of development agency education projects in Africa has been
maintained by ADEA (PRISME) which has been used as a tool for enhanced coordination.21
21. PRISME is currently downloadable to remote sites, but not yet available on line. The entries were updated
in 2005 and include projects with start-up dates as recent as 2004. Information includes only project title,
planned start-up and completion dates, and planned amounts. There are no project descriptions or actual
disbursement data.
17
Box 3. "The Culture of Quality" and National Education Plans: A Disconnect?
The 2003 ADEA Biennial Meeting, entitled "The Challenge of Learning: Improving the Quality of
Basic Education in Sub-Saharan Africa," ended with a pledge to create a new "culture of quality" for
education on the continent. Enthusiasm for this agenda was such that pro-quality policies and
programs emerged as the top priority in ADEA and among its partners. This is demonstrated by the
fact that quality improvement returned again as the main theme of the 2006 bienniale. This has meant
increased attention to how and how well students are learning, since as an ADEA newsletter article
states, "Systems that adopt a culture of quality as the basis for their operational practice, focus on
learning as the central goal of their education system..." (Jan�March, 2004, p. 8). Given this one
might expect to see learning outcomes as a major concern in government plans and programs.
As a test of this, a new set of government poverty reduction programs were examined to see to what
extent they included a focus on education, and in particular on improved learning outcomes. The set
consisted of new Poverty Reduction Support Credits (PRSCs), produced by African governments
subsequent to the 2003 bienniale and funded by the World Bank. Of the 11 PRSCs in Africa
approved in the Bank's fiscal years 2004 and 2005, only 3 mention poor learning outcomes as a
problem, and only two had improving learning outcomes as an objective (both from the same
country). To be sure, seven revealed intentions to cover some aspect of quality improvement (teacher
training, books etc.), but without a focus on learning outcomes, it is unclear whether such inputs will
lead to "the central goal" of the education systems.
3.12 Thus, there is fragmentary evidence that agency coordination has improved, but
neither ADEA nor the 2005 external evaluation provided any systematic evidence to
document it. When TTLs were asked if they noted an ADEA influence on donor coordination
at the country level (Table 2), only one gave a positive response while six indicated the
influence to be little or nil, leaving some doubt as to whether coordination often extends
beyond ADEA gatherings.22 If ADEA takes this objective seriously, explicit efforts are
needed to track improved coordination both at the regional level (reaching consensus on how to
deal with the region's most pressing educational problems) and at the country level (working
together on country-led efforts in a harmonized fashion).
PROGRAM: PROMOTING POLICY DIALOGUE
3.13 Based on key informant interviews and other evidence, it is clear that ADEA has
become the premier forum in Africa for educational policy dialogue. Evidence for this is its
drawing power during key policy dialogue events, especially its biennial meetings. The last
biennial meeting, held in Libreville, Gabon in March 2006 was attended by over 500
participants (the largest number ever), including 60 education ministers and deputy ministers.
In such meetings ministers are brought into dialogue with representatives (some high level)
of all major development agencies and NGOs active in education on the continent. Most of
the 21 agencies contribute regularly to ADEA's budget, which includes policy dialogue as its
second largest line item (after working groups). Increasingly in recent years policy dialogue
has also taken place during the growing number of thematic and sub-regional meetings (see
Appendix F for a listing.)
22. In fairness, ADEA never formally articulated an aim to promote development agency coordination at the
country level, although this could be expected as a by-product of the agencies' meeting with African Ministries
of Education every six months. Another avenue for country-level agency coordination would be through
ADEA's growing cooperation with FTI, which has agency coordination at the country level as an explicit goal.
18
3.14 Many informants, including the 2005 external evaluator of ADEA, have commented
on the unique features of the policy space that the Association has carved out, its openness
and informality, the high level of trust that delegates feel, the lack of political or ideological
constraints, and the frequent use of research and experience-based reflection to back up
policy positions. Such intangibles (sometimes referred to as ADEA's "social capital") are
both difficult to measure and to replicate in other settings.23 A final confirmatory point is
ADEA's "convening power." Individual actors (agencies or networks or countries) may have
difficulty calling attention to a particular issue without creating some suspicion as to their
motives, but according to the ministers of education interviewed for this review, if an issue
makes it onto the ADEA agenda, it is taken seriously by the education community in Africa.
3.15 ADEA's important role in policy dialogue was also noted by World Bank TTLs
(Table 2). Seven of the nine TTLs responded to a question about what aspects of ADEA they
found most useful. Four of the seven indicated that "facilitating communications among
practitioners in the sector" (i.e., policy dialogue) was very useful (more than for any other
category). These were the same four that indicated that they were "very familiar" with
ADEA. In other words, those having a less positive view of ADEA's usefulness in
facilitating communications were those less familiar with the Association.
3.16 As mentioned above, the main ADEA vehicle for policy dialogue on specific topics
has been its working groups (WGs). It is difficult to generalize about the efficacy of the
WGs, since they vary greatly in their activity levels and impact on policy discourse. The year
2000 external evaluation, which focused on WGs, concluded that 5 of the 11 then active
WGs were having high or medium/high impact, one a medium impact, and four low or no
impact (the most recent one was too new to rate). The more recently created ad hoc WGs
have a stronger record, since their work feeds directly into the bienniales and has a limited
scope and time line.24 The 2000 evaluation concluded with 14 recommendations on how the
output of the regular WGs could be improved, including closing some, graduating others, and
enhancing WG team building, funding flexibility and reliability, support to coordinators, and
the mobilization of country-level coordinators and focal points. According to the 2005
external evaluation, only two of these recommendations were fully implemented, five were
partially, and 6 were not implemented. In fact, ADEA has always had difficulty making hard
decisions about its WGs.25 Results of the latest ADEA Steering Group Meeting (March 2007)
23. Even so, it is conceivable that some of the intangibles could be tracked in some way, giving stakeholders a
clearer indication of ADEA stature in the region, and how it might wax and wane.
24. As can be seen in Annex C, ad hoc working groups were created in the fields of HIV/AIDS and Education,
Quality Education, Policy Dialogue, and Post-Primary Education, over the period 2001-2004 (one per year). This
was done at the initiative of the ministries of education. The first three have been coordinated by the ADEA
secretariat and the fourth by the World Bank. These groups are expected to deal with cross-cutting issues and have
a limited time span. For example, the one on quality education produced studies and preparations for the two
bienniales on education quality (2003 and 2006) and is now slated for dissolution, and another is focused on
preparing for the Biennial on post-primary education in 2008. It is not clear whether this kind of group will be the
format of the future, since ad hoc working groups do not have the potential for creating permanent networks or
institutions (like FAWE). There may continue to be a mix of standing and ad hoc groups, but this will need to be
addressed as WGs are restructured, now that the 2008�2012 strategic plan has been adopted.
25. Among the reasons for this are that the ADEA Steering Committee makes decisions by consensus and that
most of the funding for WGs comes through specific earmarks (restricted funds) from sponsoring development
agencies, which may be determined to keep their preferred programs on life support.
19
may change that. WGs will now be asked to align their activities with the new strategic plan
and their continued existence will be reviewed against a new set of criteria which are being
drawn up and applied before the next SG meeting in the 4th quarter of 2007. This could also
be an opportunity for creating a more systematic program of progress monitoring and self-
evaluation among the working groups.
PROGRAM: PROMOTING ANALYTICAL WORK
3.17 ADEA sponsors analytical work mainly as background material for biennial meetings
and within working groups. Over the years ADEA has come to support a participatory and
applied variety of analytical work undertaken at the national or sub-national level, but
embedded in a regional and comparative framework, in order to identify similarities and
differences across countries and to create the basis of reflection and innovation (a kind of
analytical work often referred to as "praxis").
3.18 The most productive time for such research has been just before biennial meetings,
and their research-based policy dialogue sessions. It is beyond the scope of this review to
review ADEA's productivity of such analytical work from the beginning. Instructive would
be to consider the analytic work, quantity, quality and impact during the past two bienniales,
those held in 2003 and 2006. The bienniale in 2003, "The Quest for Quality: Learning from
the African Experience," brought forth a record amount analytical work, including 22 case
studies, 3 reviews of development agency experiences, 15 background papers, and 4 thematic
syntheses, all of which were included in one form or another in the large conference volume
(the main basis for policy discussions), The Challenge of Learning: Improving the Quality of
Education in Sub-Saharan African (edited by Adriaan Verspoor).
3.19 The 2006 bienniale, which was also mainly focused on effective learning ("More and
Better Education"), saw even higher numbers of studies presented (near 60), including 17 on the
theme of improved literacy, 18 on the conditions and factors of effective schools in Africa, and
19 on the scaling up of early childhood education.26 Among the papers presented at this bienniale
were case studies and synthesis reports on the characteristics of effective schools in various
African countries. In many cases, relatively inexperienced local researchers were partnered with
experienced researchers, giving the undertaking of a capacity building sub-agenda.
3.20 One of the prominent researchers interviewed for this review commented on the
uneven quality of the studies undertaken as inputs into biennial policy dialogues, warning
that some studies were "not worth reading." Prompted by this, this review took a near
20 percent random sample of studies presented at the 2006 bienniale and subjected them to a
quality review, using a rating scale adapted from an instrument used by the World Bank in a
"quality assurance" review.27 Figure 3 shows the overall rating outcomes for the 10 studies
selected (listed by number). The results of this analysis do reveal an unevenness in the
26. The total number of studies "produced" for the bienniale was said to be over 80, but around 60 were listed
for presentation in various sessions.
27. The dimensions covered in the instrument were: (a) task objective (clearly stated? value added nature of the
problem); (b) quality of the analysis (methods appropriate to objectives? quality of evidence presented?);
(c) quality of conclusions (appropriately specific and consistent with analysis/evidence?); and (d) quality of
recommendations (clearly stated? flow well from conclusions? implementable and feasible?). On the 1�4 rating
scale, 1 was high and 4 was low.
20
Figure 3. Sample of ADEA Analytical Work
Average Rating
3.5
3
2.5
2
1.5
1
0.5
0
#22 #47 #24 #15 #35 #43 #7 #36 #50 #10
Report Number
quality of the papers, with a handful of papers (3 of 10) receiving high rating on all or most
dimensions, a few (3 of 10) getting mostly weak ratings, and the rest (the majority)
somewhere in between. In many of the papers a mention was made of the modest resources
available for the studies and brief time available. For the next bienniale, ADEA might want
to consider putting more funds into the preparation of fewer papers, and the establishment of
a peer review process.
3.21 Concerning analytical work by ADEA working groups, the record is unclear, given
the lack of comprehensiveness in ADEA's data base. All of the working groups have
sponsored a considerable amount of work, but only a small fraction of it has been published
and archived by the Secretariat. Nevertheless, an examination of ADEA's publications
archives does provide a rough estimate of productivity across working groups (or at least
productivity of materials considered by the Secretariat and/or the WG to be worthy of
publishing). Table 3 shows the number of study titles, by both WG and time period.28 Of the
89 studies listed, nearly two-thirds were completed in the last six years (since 2000),
indicating either higher research productivity in recent years or recent improvements in
ADEA effectiveness in archiving products or both.
3.22 Clearly the most productive WG in terms of documenting analytical work has been
FAWE (now graduated). Its studies comprised almost 1/3 of the total, and were both early
(before 2000) and more recent. The Finance and Education WG, often also mentioned as a
candidate for graduation given its strong ties to the Council for the Development of Social
28. This list does not include the analytical work produced for the 2006 bienniale, which included a large
number of new studies on nonformal and early childhood education.
21
Table 3. ADEA Published Studies, by Working Group and Time Period
(in descending order by Working Groups)
ADEA Working Group Before Year 2000 Year 2001�2006 Total
Female Participation (FAWE) 14 14 28
Finance and Education 2 9 11
Books and Learning Materials 3 7 10
Education Sector Analysis 4 6 10
Nonformal Education 2 7 9
Higher Education 5 2 7
Early childhood education 0 6 6
Distance Ed and Open Learning 0 3 3
Teaching Profession 3 0 3
Education Statistics 0 2 2
Total 33 56 89
Source: ADEA Publication Archive
Science Research in Africa registered the next highest number of titles (12 percent), almost
all in recent years. Three other WGs -- Books and Learning Materials, Education Sector
Analysis, and Nonformal Education -- had around 10 titles (or 10 percent of the total) each.
Three WGs showed very little published research productivity, namely those on Distance
Education and Open Learning, the Teaching Profession, and Educational Statistics.29 Higher
Education (showing only 7 study titles) was lower than might be expected, given the fact that
universities thrive on research and publications. The general overview is that about two-
thirds of the publications archived by ADEA stem from the output of four working groups.
The productivity of many of the others is low enough to call into question whether they are
making a contribution to African education development.
PROGRAM: STRENGTHENING CAPACITIES IN AFRICAN COUNTRIES
3.23 Capacity strengthening was not an original ADEA objective, but various initiatives,
starting in the mid 1990s put it on the Association's agenda. The visioning paper in 2003 called
capacity strengthening one of ADEA's continuing challenges. This was reinforced by the results
of a 2003�04 needs assessment conducted by ADEA among Ministers of Education, in which
ministers showed particularly strong demand for capacity-building activities. Unfortunately, there
has been no clear statement of what capacity building or strengthening means in this context.
This has led to varying expectations of what ADEA can do and equally varying assessments of
its effectiveness, as mentioned in the 2005 external evaluation. The present review will briefly
consider three kinds of capacity building mentioned in ADEA documents: (a) technical capacity
of ministry officers, (b) institutional capacity of African educational agencies, and (c) country
capacity to enter and influence the policy dialogue in education.
3.24 ADEA has been active in providing technical capacity building in the region,
especially in fields covered by its most active working groups (e.g., those on Finance and
29. The WG on Education Statistics was never oriented towards research productivity but more towards model
and capacity building, and thus, strictly speaking, should not be compared with the others.
22
Education; Communications for Education and Development; and Education Statistics).
Training and technical assistance have helped many ministries to strengthen relationships
with ministries of finance (increasing the priority that the latter give to education); to build
national plans and/or offices for communications about education; and improve the capacity
of education system/managers to produce and effectively use more timely and reliable school
data. One of the ministers interviewed for this review was particularly appreciative of what
her staff gained from an ADEA exchange with a neighboring country and from sub-regional
workshops; another cited ADEA as having helped his country create a national policy on
early childhood education. Nevertheless, the reach of ADEA's technical capacity building
has been limited, mainly available just to those directly involved in ADEA WGs or attending
regional/sub-regional meetings (see 2005 external evaluation). This is likely what led all but
one of Bank TTLs to conclude that capacity building by ADEA was of little or no benefit in
the countries where they work (Table 2).
3.25 Institutional capacity building became an important theme in ADEA during the mid
to late 1990s with its emphasis on anchoring and institutionalization of WG activities in
African institutions, and this has been gaining ground. By 2006 seven of eleven active
working groups were coordinated from institutions based in Africa.30 This has in most cases
led to institutional strengthening and new continental focal points for relevant research and
policy dialogue. In addition, as a follow-up to the 2003 bienniale in Mauritius the Ad Hoc
Working Group on Quality of Education set up inter-country quality "nodes" to encourage
deepening of research and exchange of experiences on various themes. Some of these started
to take off in 2004�05 but activities were put on hold in 2006, given the need for financial
austerity, and it is not clear how much new support will be forthcoming, given the shift to the
activities defined by the new strategic plan.
3.26 The effort to anchor ADEA in Africa has also meant relying more heavily on the
Caucus of Ministers to set its agenda. This meant a stronger representation of African
Ministers on the ADEA Steering Committee and proactive efforts (e.g., peer review activities
at Ministries) to assure that country needs are driving ADEA programming. In addition, an
increasing amount of ADEA analytical work is being done by African researchers, and where
research skills are not fully developed capacity building efforts are made. One limitation in
such capacity building is the high turn-over among ministers. Typically, by the time a
minister becomes accustomed to ADEA decision-making processes and has developed "a
voice," the minister is replaced and the induction process has to start again. (For more about
this issue see the section on Governance and Management.)
PROGRAM: PROMOTING REGIONAL AND SUB-REGIONAL NETWORKING
3.27 ADEA is considered by its current and past executive secretaries to be a "network of
networks." Mission interviews revealed this having come about in two ways: (a) through the
creation of working groups which have grown into networks, (b) through partnering with
existing networks, and (c) through sponsoring or co-sponsoring regional or sub-regional
exchanges. The best example of network creation is the working group on Female
Participation, which merged with FAWE, and has become one of the continent's most
30. This includes the working groups on Finance and Education, Communication for Education and
Development, Higher Education, Distance Education and Open Learning, Mathematics and Science Education,
Education Statistics, and Books and Learning Materials.
23
influential networks (and continues to be an ADEA associate). Two other working groups,
those on Finance and Education and Educational Statistics, both anchored in Africa and
mentioned for "graduation," have created acknowledged networks of specialist and animators
in educational finance and statistics. (There does appear to be a positive relationship between a
WG's being anchored in an African institution and becoming a recognized and enduring
regional network.) The WGs on Higher Education and Communications for Education and
Development, both now anchored in Africa, also have the capacity to spawn regional networks.
If the "quality nodes" grow and mature over the next few years, they also have the capacity to
do so in their specialty areas. One irony connected with the highly acclaimed ad hoc working
groups is the following: Given that they are set up for specific tasks with a limited life span,
and are mostly coordinated by the ADEA Secretariat, they may not have the right make-up for
spawning sustainable networks. In sum, considering the fact that ADEA has only spawned one
self-reliant network so far (FAWE), its success in networking is not as strong as it could be.
3.28 ADEA's second approach to networking networks has been its partnering with existing
networks. The clearest example of this is in the growing ties (even a possible merger) between
the ADEA Caucus of Ministers and COMEDAF, which is promoting the second Decade of
Education for Africa (2006�15). Other regional and global networks to which ADEA has
contributed include the New Partnership for Africa's Development (NEPAD), the various
regional/sub-regional economic communities (the Economic Community Of West African
States (ECOWAS), the Economic Community of Central African States (ECCAS), and the
Southern African Development Community (SADC)), the Education Research Network in
Eastern and Southern Africa (ERNESA), the Education Research Network for West and
Central Africa (ERNWCA), and the EFA Fast Track Initiative (FTI).
3.29 The third approach has been networking in the form of exchanges among educators in
the region. As mentioned under "promoting policy dialogue," this takes place every time
ADEA sponsors or co-sponsors a regional or sub-regional meeting. The simplest form of
exchanges are professional visits between neighboring countries, which ADEA has financed
over the years -- they are a line item in the ADEA budget -- but given recent budget
tightening, could not take place in 2005 and 2006. Closely related are "peer reviews" (ADEA-
sponsored professionals critically reviewing country educational plans, coordinated by the WG
on Education Sector Analysis), which have recently piloted reviews in Mauritius, Gabon and
Nigeria. More significant are the growing number of sponsored and co-sponsored thematic and
sub-regional meetings where professional networking takes place (mentioned in the section on
policy dialogue and listed in Annex F.)
CONCLUSION
3.30 For the strategic objective of improving educational policies, there is some anecdotal
evidence that ADEA programs have had an impact at the country level, especially on girls'
education and other areas where working groups have been especially active. However, since
ADEA has not set up systems to evaluate the attainment of such outcomes, it is not possible
to make a full judgment about the achievement of this objective. Similarly, for development
agency coordination, there are indications of improvement but no clear evidence. Reinforcing
a recommendation from the 2005 external evaluation, this review indicates the need for better
strategic planning, including the specification of objectives, and the tracking of outcome
indicators.
24
Efficiency
3.31 The 2005 external evaluation covered many aspects of ADEA efficiency. It did not
undertake a cost-benefit analysis, given the difficulties in placing monetary values on
ADEA-promoted outcomes, nor was it able to compute cost-effectiveness ratios, given the
fact that outcomes have not been specified or measured. It did cover a set of issues that are
still relevant today, namely, the efficiency of: (a) the modes of ADEA financing; (b) the
operation of the working groups; (c) the management and oversight by the Steering
Committee, and (d) management support by the Secretariat. (The latter two will be covered
in the section on Governance and Management below).
3.32 As mentioned in section 1 above, agency funding for ADEA comes in two forms:
unrestricted and restricted (which is mainly earmarked for the various working groups).
Although ADEA prefers to receive unrestricted funds, a preference supported by recent
evaluations, pledges of restricted funds have in recent years always exceeded those of
unrestricted funds (by as much as 3� times), and the gap between the two modes of funding
has been widening (Figure 1). This has negatively affected ADEA's efficiency, since it restricts
that capacity of the Association to allocate funds on the basis of performance. (Even though the
absorptive capacity of many working groups is low, they still receive independent funding.)
ADEA has continuously pleaded with agencies to increase their basic contribution (see
footnote 16), and some agencies are now beginning to respond to the call.31 (There was an
improvement in the proportion of unrestricted funding in 2006, but it is not clear if this will be
sustained). Another sign of improvement was the 2007 resolution by the Bureau of Ministers to
increase the membership fees for African ministries from $1,000 to $2,500 per annum.
3.33 Working groups receive about half of the ADEA budget each year, but it is not clear
how accountable the WGs are for the funds they receive, an issue raised in the 2005 external
evaluation. Whereas their annual plans and budget requests are reviewed by the Steering
Committee each year, their performance, in terms of how well they have met their goals, has
not been carefully monitored. Recently there has been a move to require annual WG
reporting on performance indicators, but this has not received prominent attention in the
latest (2006) ADEA annual activities reports.
Governance and Management
3.34 At its creation in 1988 ADEA (then DAE) was an informal association of development
agencies, coordinated by the World Bank, having the twin goals of discussing and improving
educational policies and donor coordination. Initially, legitimacy was based on urgency and
relevance of the policy paper messages that had stimulated DAE's formation, the World
Bank's strong presence in Africa, and the limited participation of a few African governments.
However, it was only after the mid-1990s, when the Association's name was changed to
ADEA and the governance structure was formally revised to include a larger block of African
Ministries of Education, that its legitimacy as a regional association became firmly established,
both among agencies and ministries of education. Legitimacy was further strengthened in the
31. Encouragingly, there was some improvement in the proportion of funding being unrestricted in 2006, and
three agencies announced their intentions to shift their support entirely to unrestricted funding in 2007 and
2008.
25
1990s by a conscious effort to anchor ADEA in Africa through holding all Biennial and most
Steering Committee meetings in Africa and linking as many working groups as possible to
African institutions. Finally, the 2005 external evaluation confirmed the reality that ADEA had
become the forum for regional policy dialogue and education innovation in Africa.
3.35 ADEA's governance model has a mix of the shareholder and stakeholder elements. The
Steering Committee comprises both shareholders (the development agencies), who are required
to pay an annual subscription fee ($50,000 a year), and stakeholders (the African Ministries of
Education) who are represented in their entirety though the Caucus of Ministers and the 10
Ministries that it selects to sit on the SC on a rotating basis.32 The representation of
shareholders on the SC has enhanced legitimacy since virtually all of the major development
agencies and foundations active in African education have been subscribers over the past ten
years.33 The representation of stakeholders has also enhanced the legitimacy of ADEA, but has
however raised some issues of efficiency, as discussed in the next paragraph.
3.36 The ADEA Steering Committee, its prime governing body, is a large body consisting
of 21 contributing agency members and the ten member Bureau of Ministers, all providing
their services on a voluntary basis. Given the growing reach and complexity of ADEA, this
body has an enormous volume of work to do, all packed into three or so days of meetings
every six months. The 2005 external evaluation noted that it was not possible for the SC to
cover all items on the agenda with the level of detail that is required, "forcing members to
focus on procedural obligations such as the review and approval of budgets and workplans at
the expense of more in-depth discussions of the actual effectiveness and relevance of
activities undertaken by ADEA" (page 26). In addition, given the Steering Committee's
consensus approach to decision making, it is often not able to make difficult decisions
quickly, if at all. Many important decisions, such as what to do with underperforming
working groups, or how to streamline the SC itself, have been deferred for months or years.34
3.37 The balance of power between the development agencies and the ministries of
education on the Steering Committee has also raised the issue of equity. In terms of sheer
numbers, the agencies have twice as many seats on the Steering Committee as the ministry
representatives. In addition to this, there is a reported tendency for development agencies to
dominate the discussions often due to structural inequalities -- ministries being reluctant to
rock the boat since they are generally the recipients of financial support from the agencies.
Others contend that ministries contribute less than they might because of frequent turnover of
ministers and because so many members are inexperienced in ADEA. There appears to be no
32. Legitimacy would presumably be strengthened even further if ADEA follows through on a serious proposal
that its Caucus of Ministers be replaced by the African Union's Council of Ministers of Education
(COMEDAF). A possible downside of this could be an increase in formality and political correctness among
Ministers and a decrease in openness and frankness.
33. One recent exception, presenting a slight challenge to legitimacy, is the Swedish International Development
Agency, which surprisingly withdrew from ADEA in 2006, contending that COMEDAF (of the African Union)
was a more legitimate regional education body. It is not clear what SIDA will do if ADEA replaces its Caucus
of Ministers with COMEDAF (see previous footnote).
34. Ironically, one of the recommendations of the 2005 external evaluation was that an "executive board" be
formed to assist the SC in covering some of the more routine tasks, leaving the SC to focus on oversight, quality
assurance, and strategic direction; but the decision to create such a board, its membership and function is itself
taking time.
26
simple formula for rebalancing: doubling the ministry members would make the SC even more
unwieldy, but halving the number of agency members would put legitimacy in jeopardy. Also,
the 2005 external evaluation found that in recent years the African ministries have been setting
the Association's priorities, suggesting that the need for rebalancing may not be crucial. What
is crucial is finding ways to deal with the endemic problem of turnover among ministry
delegates.35 One informant suggested bringing in as non-voting SC members or associate
members long-serving permanent secretaries or directors general, who have a long history of
policy reform experience, or prominent ministers "emeritus" (no more than one per country),
who could add some depth to the discussions.
3.38 The day-to-day management of ADEA lies with the Secretariat, led by an Executive
Secretary. As mentioned by the 2005 external evaluation, the Association has been blessed
with dedicated, visionary, and highly respected Executive Secretaries, including the
incumbent; many staff members have also given extraordinary service. Similar to the SC, the
Secretariat's volume of work has expanded faster than the resources available (funds, time,
space and skills) and some of the more substantive tasks (management support to working
groups, synthesizing and disseminating cutting-edge messages, or gathering and analyzing
monitoring and evaluation data, have taken a back seat to administrative ones. The creation a
new post of deputy executive secretary or manager has been proposed as a way of increasing
office efficiency. Another way would be to elevate the stature of current administrative and
financial team.36 The 2005 evaluation also found that increasingly tense relations between the
ADEA Secretariat and its host institution, IIEP, were putting a drag on institutional
efficiency.37 The next few months will be crucial in ADEA's efforts to improve managerial
efficiency: (a) it will need to recruit a new Executive Secretary, given the retirement of the
incumbent in 2008; (b) it will realign its budget and its secretariat functions -- perhaps
involving changes in job descriptions and personnel -- to the new medium term strategic
framework (2008�2012); and (c) it will move its Secretariat to Africa, shifting its
management partner from IIEP to AfDB. All of these changes can breathe new life into the
organization, but can also put it at some risk of losing momentum.
3.39 According to this review, the main shortcomings in ADEA governance and
management, and areas for future improvement, are: (a) the rapid turnover among SC
members and its influence on quality of discussions and decision making; (b) the
predominance of restricted funds within ADEA's budgeting system; (c) the continuation of
unproductive working groups and the slow pace of "graduating" productive ones; (d) the
shortcomings in making ADEA processes and products known beyond those directly
involved (outreach); (e) the under-emphasis on evidence-based self-evaluation, and (f) the
overloading of both the Steering Committee and the Secretariat.
35. This also appears to be a problem with some development agency delegations. Solutions for them would be
quite different from those for ministries.
36. In the past three years the proportion of budget allocated to the Secretariat has been stable at 16-17 percent,
a share in line with that of other global programs. Thus, it may not be that increasing budget is the answer, but
perhaps reorganizing tasks and streamlining procedures. The impending move to Africa (the AfDB in Tunis)
will present a chance for reorganization.
37. Tensions have arisen over some budget issues (see paragraph 1.18), contractual terms for Secretariat staff,
and space available for ADEA staff and interns at the IIEP premises.
27
4. World Bank's Performance as a Partner
4.1 The World Bank has played numerous roles in ADEA: producer of the policy paper
on African education that led to founding of ADEA; co-founder of the Association; initial
host organization; a lead financial supporter; holder of a seat on Steering Committee;
frequent source of leadership for the Secretariat and (at least once) the Steering Committee;
source of professionals for many technical assistance tasks; agency coordinator for Working
Groups (Higher Education and Post-Primary Education); and managing the Donor
Information System (which eventually became a function of the Secretariat)). These crucial
roles, often interacting in complex ways, have made the World Bank an indispensable partner
in ADEA. This section will cover the following four features of Bank's performance as a
partner which are generally covered in Global and Regional Program Reviews:
(a) comparative advantage at the global and country levels; (b) exercising effective and
independent oversight; (c) identifying and managing risks; and (d) disengagement strategy.
4.2 As mentioned earlier, a major WB policy paper documenting the mounting crisis in
African education in the late 1980s provided the foundation and rationale for a regional program
on African education, which became ADEA. After launching the paper, the Bank also convened
the first gathering of development agencies active in the sector in Africa along with African
educational officials to consider new ways of stimulating policy reform and inter-agency
cooperation. Thus the Bank exercised its comparative advantage as a convener and as the largest
provider of external funding to education in Africa to play a leading role in the creation of this
new regional initiative. The Bank also provided seed money and staff time for backup support
once the agencies involved decided to create a backstopping body. The Bank's leadership and
recognized stature was helpful in leveraging support from other development agencies -- in the
early years at a ratio of 1:5 (one part Bank funding to five parts by other agencies), moving to
about 1:15 in the middle of the current decade, and back to about 1:7 after 2006.
4.3 ADEA has also added value to the country programs supported by the Bank. Many
current and former Bank officials interviewed for this report pointed out how the ADEA's
convening power brought legitimacy and weight to Bank-supported reforms at the country
level. Championing such ideas by itself could have left the impression of the Bank's trying to
impose its will, but there is less suspicion when the policy positions are taken over and
furthered by ADEA (although still some risk, see paragraph 4.5). This has allowed many Bank-
supported programs to bear fruit at the country level, such as evidence-based policy
adjustment, the need for increased national political commitment as a prerequisite for reaching
EFA goals, the need for a new "culture of quality" to enhance basic education outcomes, a firm
commitment to further girls participation in education and girl-friendly schooling, and closer
government cooperation with NGOs, teachers' unions and parent associations in reforming
educational policy and practice. Changes in these areas have been found in many African
countries, as a result of country involvement in ADEA. Instances of such value added are not
universal, however. Instead they are patchy and concentrated in those countries where a
Minister and/or the Bank team have become deeply and personally engaged with ADEA (e.g.,
during the time of ADEA bienniales). Of the nine TTLs of education sector projects
interviewed for this review, three indicated that ADEA products and services were used in their
country's operational work a substantial amount but four indicated that these were used "a
28
negligible amount" (Table 2). The value added of ADEA still seems to vary according to the
level of engagement of the government and the Bank's country team.
4.4 When the ADEA Secretariat was created in Paris in 1992, Bank involvement shifted
to the oversight mode. Over the 18 year period from 1990 to 2007 this oversight was handled
by five Bank managers -- Peter Moock, Avril Van Adams, Adriaan Verspoor, Birger
Fredriksen, and Dzingai Mutumbuka. All have been recognized for their professionalism and
vision. Additional quality assurance support has come in the form of technical assistance
from two prominent, retired Bank education task managers who helped to create background
papers for the two recent bienniales on education quality. Bank oversight has also included
financial support for the external evaluations that were conducted in 1994, 2000 and 2005.
All three are considered to have been objective and constructive. In 2005, in a break with
precedent, the SC elected a World Bank Sector Manager as the co-chair of the Steering
Committee.38 The World Bank Education Advisor for the Africa Region took up the IDA
seat on the Committee. In late 2007, on the Sector Manager's retirement from the Bank, the
co-chair position rotated to someone outside the Bank.
4.5 The unprecedented situation in 2005�2007, when a Bank Sector Manager for
education in Africa was also the co-chair of ADEA, put the Bank in a possible conflict of
interest position, for example, when he was called upon both to judge ADEAs' performance
(as Sector Manager) and advocate for continued funding for ADEA (as co-chair). Such
conflict of interest situations are becoming increasingly prevalent where the Bank is an active
participant in global and regional programs. In this case, it should have been acknowledged
(and if it happens in the future, should be acknowledged) so that steps to address it can be
taken.39 Another risk is that the Bank could be seen as having an undue influence over
ADEA's agenda. In the early years of the DAE, the agenda did spring mainly from the
Bank's policy paper and its conclusions. Over the years, however, the Association has
transformed itself into a forum for enacting an African-owned agenda, at the regional and
country level. Still, with the chair of the SC being a Bank official and the Executive
Secretary being a former Bank employee, there is the possibility of decisions being biased in
the direction of the Bank's goals and preferences. While this review came across no concerns
about this risk, either in partner agencies or among Ministers of Education, the Bank could
get a neutral party to sound this out periodically in order to mitigate this risk.40
38. There had been an unwritten rule that the leadership of the SC should be given to a representative of one of
the "smaller agencies." In 2005 no candidate from these agencies was available for the position, given the fact
that most were now providing mostly program support (e.g., through sector-wide programs) and thus had a
reduced number of education experts on staff.
39. One way to mitigate the such a risk would be for the Sector Manager to recuse himself from Sector Board
discussions of ADEA performance and budget decisions.
40. World Bank Management has acknowledged the existence of such conflicts of interest in the Bank's
oversight and management of global and regional programs and, in its formal response to IEG's Phase 2 Report
on the Bank's involvement in global programs, made a commitment to develop TORs for Bank staff serving on
the governing bodies of global programs. That such Bank-wide guidelines have not yet been developed is not
the fault of the Education Sector Manager or ADEA. Rather it is to their credit that they appear to have
managed these potential conflicts of interest in a pragmatic way in the absence of any Bank-wide guidelines for
this purpose. However, the need to develop such guidelines still remains.
29
4.6 There has been no serious discussion of a World Bank disengagement strategy from
ADEA, despite its having sponsored the Association for almost two decades. The Bank's
financial support through the Development Grant Facility is classified as "window 1," which
signals a long-term engagement, at least through 2010. However, task managers have argued the
need for an even longer engagement, given that the Millennium Development Goals in
education, a key ADEA focal point, will culminate in 2015, and more than half of SSA countries
are still off-track for reaching them. In addition, the next few years are seen as a crucial transition
period for ADEA, given its new medium-term strategy (2008�2012), and the move of its
Secretariat to an African country, starting in 2008, and the need to replace the current Executive
Secretary when he retires in 2008. Bank financial and professional support are considered as
crucial for keeping up momentum during these transitions and leveraging continued support by
partner agencies. This is not to deny the importance of improved exit strategies with respect to
the ADEA working groups. Only one group, FAWE, has graduated so far, whereas others have
been on the brink of this for years or are almost dormant (already mentioned in the 2000 external
evaluation), suggesting that some WGs are being carried for too long. It is crucial that this issue
be taken up as the new strategic plan is put into place.
5. Lessons Learned
5.1 Over the course of almost two decades, ADEA has grown from a small consortium of
donor agencies and a few ministries of education to become Africa's premier forum for
educational policy development and agency cooperation. Its influence is largely a
consequence of its growing legitimacy (it now includes as partners all African ministries of
education and all major development agencies active in African education) and its relevance
(the focus of its work has evolved along with the region's educational issues). However, such
a high level of legitimacy has had costs in terms of efficiency, as decisions have not been
arrived at quickly and insufficient attention has been given to strategic planning and
performance assessment. These concerns are now being addressed with the formulation of a
new strategic plan for 2008�2012 and the establishment of an Executive Board, which will
manage the operation leaving the Steering Committee to focus on more strategic and
substantive matters. ADEA's core strength has been its convening power, which allows it to
promote policy dialogue through a variety of forums such as biennial meetings, technical
gatherings, and the activities of its 15 working groups. It also promotes country and regional
analytical work on educational problems, constraints and solutions; the strengthening of local
capacity for educational planning and related skills; and the creation of regional and sub-
regional professional networks. In each of these areas ADEA has shown both strengths and
weaknesses, making it possible to identify a number of important lessons. Some of the
following lessons are previously identified lessons which have not yet been heeded, some are
in the process of being acted upon, some are modifications of previously identified lessons,
and some are new.
� Strategic planning for ADEA's future, which has focused mostly on desired processes
and programs, also needs to specify organizational objectives in terms of desired
outcomes (e.g., EFA outcomes, such as improved access or learning outcomes). This is
a main point in the 2005 external evaluation which has not yet been heeded. In addition
to desired outcomes, there is a need for measurable performance indicators. Even some
of the so called "intangibles" that are prized in ADEA could be tracked in some way to
30
show how ADEA is measuring up to its ideals. This calls for an agreement among
ADEA partners as to the type of results-oriented monitoring and evaluation framework
that would be reasonable and credible, given ADEA's current and emerging objectives,
resources, and organizational structure.
� ADEA working groups need to be realigned with the Association's new strategic plan
and subject to a new set of evaluation criteria which the Association needs to apply with
firmness (a work already in progress). This would presumably lead to phasing out or
graduation of some current working groups. The new pattern of forming ad hoc working
groups could also be reviewed and modified as appropriate, given that its acknowledged
benefits are accompanied by certain disadvantages, such as the groups not being
anchored in Africa and their placing an administrative burden on the Secretariat.
� Analytical work supported by ADEA is growing in volume but is of uneven quality.
As ADEA moves into new realms of applied research it needs to set higher quality
standards, and to concentrate as much on the quality (usefulness) of the research as on
quantity (output). This recognizes the important capacity and ownership-building
benefits of drawing new, local researchers into the process as co-investigators with
more senior researchers. For research used as background documents in biennial and
regional meetings and those published by ADEA, it would be desirable to establish a
formal policy and procedures for document review.
� Creative solutions to chronic overburdening of the ADEA Steering Committee must
be found so that the Committee has time to exercise more substantive leadership,
quality oversight, and identification of cutting edge issues. The formation of a
smaller, constituency-based Executive Board would be a step in this direction.
� Rapid turnover of Steering Committee members, particularly those from the Caucus
of Ministers, is undermining the strength, confidence, continuity and institutional
knowledge of their inputs into policy dialogue and decision making. Ways need to be
considered to bring more permanent ministry officials into ADEA decision making
circles and to find roles for emeritus committee members who are still prominent and
influential in education circles.
� ADEA programs and products are having a significant impact on those in the ADEA
inner circle, but do not reach very far beyond this. Ways need to be found to extend
the reach of ADEA programs and outputs, using country focal points or other
innovative vehicles.
31 Annex A
Annex A. Evaluation Framework for Global Program Reviews
Note: This evaluation framework is a general framework that has been designed to cover the wide
range of such programs in which the World Bank is involved, encompassing policy and knowledge
networks, technical assistance programs, and investment programs. It is not expected that every
global program review will cover every question in this table in detail.
Annex Table 1. Assessing the Independence and Quality of the Evaluation
Evaluation Questions
1. Evaluation process
To what extent was the GRPP evaluation independent of the management of the program, according to the following
criteria:
� Organizational independence?
� Behavioral independence and protection from interference?
� Avoidance of conflicts of interest?
Factors to take into account in answering these questions include:
� Who commissioned and managed the evaluation?
� Who approved the terms of reference and selected the evaluation team?
� To whom the evaluation team reported, and how the evaluation was reviewed?
� Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions
on access to information, travel, sampling, etc.?
2. Monitoring and evaluation framework of the program
To what extent was the evaluation based on an effective M&E framework of the program with:
� Clear and coherent objectives and strategies that give focus and direction to the program?
� An expected results chain or logical framework?
� Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the
program?
� Systematic and regular processes for collecting and managing data?
3. Evaluation approach and scope
To what extent was the evaluation objectives-based and evidence-based?
To what extent did the evaluation use a results-based framework -- constructed either by the program or by the evaluators?
To what extent did the evaluation address:
� Relevance � Governance and management
� Efficacy � Resource mobilization and financial management
� Efficiency or cost-effectiveness � Sustainability, risk, and strategy for devolution or exit
4. Evaluation instruments
To what extent did the evaluation utilize the following instruments:
� Desk and document review � Consultations/interviews and with whom
� Literature review � Structured surveys and of whom
� Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being
implemented or completed
� Case studies � Other
Annex A 32
Evaluation Questions
5. Evaluation feedback
To what extent have the findings of the evaluation been reflected in:
� The objectives, strategies, design, or scale of the program?
� The governance, management, and financing of the program?
� The monitoring and evaluation framework of the program?
Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program
Every review is expected to cover the first four criteria in the following table: (a) relevance,
(b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover
(e) resource mobilization and financial management and (f) sustainability, risk, and strategies for
devolution or exit if the latter are important issues for the program at the time of GPR, and if there is
sufficient information available on which to base an independent opinion.
Evaluation Criteria and Questions
Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional
challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and
groups.
1. Supply-side relevance -- the existence of an international consensus that global/regional collective action is
required.
To what extent does the program reflect an international consensus on the need for action, on the definition of the
problem being addressed, on priorities, and on strategies for action?
Is the original consensus that led to the creation of the program still present? Is the program still needed to address
specific global/regional public concerns?
Take into account the origin of the program in answering these questions:
� Is the program formally responsible for implementing an international convention?
� Did the program arise out of an international conference?
� Is the program facilitating the implementation of formal standards and approaches?
� Is the program primarily donor-driven? Did donors establish the program with little consultation with developing
countries?
� Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners?
2. Demand-side relevance -- alignment with beneficiary needs, priorities, and strategies.
To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as
articulated in the countries' own PRSPs, and in donors' strategies such as the World Bank CASs, and the UN
Development Assistance Frameworks?
To what extent has the voice of developing and transition countries been expressed in the international consensus
underlying the program?
3. Vertical relevance -- consistency with the subsidiarity principle.
To what extent are the activities of the program being carried out at the most appropriate level -- global, regional,
national, or local -- in terms of efficiency and responsiveness to the needs of beneficiaries?
To what extent are the activities of the program competing with or substituting for activities that individual donors or
countries could do more efficiently by themselves?
Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or
local public goods.
33 Annex A
Evaluation Criteria and Questions
4. Horizontal relevance -- the absence of alternative sources of supply.
What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with
similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or
technical capacity that is otherwise unavailable to meet the program's objectives?
To what extent are the good and services being provided by the program in the nature of public goods? Are there
alternative ways of providing these goods and services, such as by the private sector under regular market conditions?
5. Relevance of the design of the program
To what extent are the strategies and priority activities of the program appropriate for achieving its objectives?
What are the major activities of the program:
� Policy and knowledge networking?
� Financing country and local-level technical assistance?
� Financing investments to deliver national, regional, or global public goods?
Has the program articulated an expected results chain or logical framework, along with assumptions that relate the
progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the
achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to
what extent are these assumptions valid?
For programs providing global or regional public goods, is the design of the program consistent with the way in which
the individual efforts of the partners contribute to the collective outcome for the program as a whole -- whether "best
shot", "summation", or "weakest link?"
Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their
relative importance.
6. Achievement of objectives
To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made
towards achieving these objectives?
To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the
program should also be held accountable?
To what extent are there any positive, unintended outcomes of the program that have been convincingly document?
To what extent have these assessments by the program or the evaluation been evidence-based?
7. Progress of activities, outputs, and outcomes.
To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes?
How did the program or the evaluation aggregate its outputs and outcomes at all levels -- global, regional, national,
and local -- to provide an overall summary of its results?
To what extent have factors such as changes in the location of the program, its legal structure, or governance
processes affected the outputs and outcomes of the program?
To what extent have there been outcomes that can be uniquely attributed to the partnership itself -- such as the scale
of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host
organization?
8. Linkages to country or local-level activities.
To what extent has the program established effective operational linkages with country-level activities, taking into
account that:
� The desired nature of these linkages will vary according to the objectives, design, and implementation of each
program?
� Positive outcomes at the country or local level are generally a joint product of both global/regional and county-
level activities?
Annex A 34
Evaluation Criteria and Questions
Efficiency or cost-effectiveness:
Efficiency -- the extent to which the program has converted or is expected to convert its resources/inputs (such as
funds, expertise, time, etc.) economically into results.
Cost-effectiveness -- the extent to which the program has achieved or is expected to achieve its results at a lower
cost compared with alternatives.
9. Efficiency
To what extent is it possible to place a monetary value on the benefits arising from the activities of the program?
To what extent has the program or the evaluation conducted impact evaluations of representative program activities?
To what extent has the program or the evaluation analyzed the program's costs in broad categories (such as overhead
vs. activity costs), and categorized the program's activities and associated benefits, even if these cannot be valued in
monetary terms?
10. Cost-effectiveness
To what extent is the program measuring up against its own business plans:
� Has the program cost more or less than planned? How did it measure up against its own costing schedule?
� Have there been any obvious cases of inefficiency or wasted resources?
To what extent is the program delivering its activities cost-effectively in comparison with alternatives:
� How do actual costs compare with benchmarks from similar programs or activities?
� Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the
objectives and activities of the program?
How does the program compare with traditional development assistance programs:
� For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions
costs compared with traditional development assistance programs?
� For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing
efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and
evaluation)?
Governance and management:
Governance -- the structures, functions, processes, and organizational traditions that have been put in place within
the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its
objectives in an effective and transparent manner.
Management -- the day-to-day operation of the program within the context of the strategies, policies, processes, and
procedures that have been established by the governing body. Whereas governance is concerned with "doing the right
thing," management is concerned with "doing things right."
11. Compliance with generally accepted principles of good governance.
To what extent are the governance and management structures and processes well articulated and working well to
bring about legitimate and effective governance and management?
To what extent do governance and management practices comply with the following seven principles:
� Legitimacy -- the way in which governmental and managerial authority is exercised in relation to those with a
legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and
the community at large?
� Accountability -- the extent to which accountability is defined, accepted, and exercised along the chain of
command and control within a program, starting with the annual general meeting of the members or parties at the
top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in
some cases, to the beneficiaries of the program?
� Responsibility -- the extent to which the program accepts and exercises responsibility to stakeholders who are
not directly involved in the governance of the program and who are not part of the direct chain of accountability in
the implementation of the program?
� Fairness -- the extent to which partners and participants, similarly situated, have equal opportunity to influence
35 Annex A
Evaluation Criteria and Questions
the program and to receive benefits from the program?
� Transparency -- the extent to which a program's decision making, reporting, and evaluation processes are open
and freely available to the general public?
� Efficiency -- the extent to which the governance and management structures enhance efficiency or cost-
effectiveness in the allocation and use of the program's resources?
� Probity -- the adherence by all persons in leadership positions to high standards of ethics and professional
conduct over and above compliance with the rules and regulations governing the operation of the program?
12. Partnerships and participation
To what extent has the program identified a complete list of stakeholders, or "stakeholder map", including the agreed-
upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine
programmatic function, updated regularly, and transparently available?
Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is
limited to financial and other contributors), or a stakeholder model (in which membership also includes non-
contributors)?
To what extent, if any, is the program's legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa?
13. Programs located in host organizations
To what extent is the location of the program in the Bank or other partner organization adversely affecting the
governance, management, or other aspects of the program, such as compliance with the principles of transparency
and fairness?
For which functions is the program manager accountable to the host organization and the governing body of the
program, respectively? Are conflicts of interest being managed appropriately?
To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of
other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the
program objectively?
Resource mobilization and financial management:
Resource mobilization -- the processes by which resources are solicited by a program and provided by donors and
partners.
Financial management -- the processes that govern the recording and use of funds, including allocation processes,
crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In
cases where funds accumulate over time, this would also include the management of the cash and investment
portfolio.
14. Resource mobilization
To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from
what sources -- the Bank, bilateral donors, foundations, etc.?
To what extent has the program succeeded in diversifying its funding beyond a small number of donors?
To what extent are the sources of funding for the program (including donor restrictions on the use of resources)
affecting, positively or negatively:
� The strategic focus of the program?
� The outputs and outcomes of the program?
� The governance and management of the program?
� The sustainability of the program?
15. Financial management
Are there any issues that have emerged during the course of the review in relation to:
� The quality of financial management and accounting?
� The methods, criteria, and processes for allocating funds among different activities of the program?
� Financial management during the early stages of the program?
Annex A 36
Evaluation Criteria and Questions
Sustainability, risk, and strategy for devolution or exit:
Sustainability -- When applied to the activities of a program, the extent to which the benefits arising from these
activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to
which the organization or program is likely to continue its operational activities over time.
Devolution or exit strategy -- a proactive strategy to change the design of a program, to devolve some of its
implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the
grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results
which the program has achieved.
16. Sustainability of the benefits of the program's activities
What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will
not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental
to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these
changes actually materialize?
17. Sustainability of the program
This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its
continuity of effective management, and its ability to withstand changing market or other conditions.
To what extent is there still a sufficient convergence or accommodation of interests among the major partners to
sustain the program financially? To what extent has the program developed institutional capacity such as performance-
based management, personnel policies, learning programs, and knowledge management that help to sustain a
program?
In what areas could the program improve in order to enhance its sustainability, such as better marketing of the
program's achievements in order to sustain its reputation?
18. Prospects for continuation and strategies for devolution or exit
To what extent should the program be sustained?
Is the continuation of the program the best way of sustaining the results achieved?
Should the design of the program be modified as a result of changed circumstances, either positive or negative?
What other alternatives should be considered to sustain the program's results more cost-effectively, in the light of the
previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability:
� Reinventing the program with the same governance?
� Phasing out the program?
� Continuing country or local-level activities with or without devolution of implementation?
� Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency
on external sources?
� "Spinning off" from the host organization?
37 Annex A
Annex Table 3. Assessing the Bank's Performance as a Partner in the Program
Evaluation Questions
1. Comparative advantage at the global/regional level.
To what extent is the Bank playing up to its comparative advantages at the global/regional level -- its global mandate
and reach and convening power?
To what extent is the Bank's presence as a partner in the program catalyzing other resources and partners for the
program?
2. Comparative advantage at the country level.
To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the
program?
To what extent has the Bank's country operations established linkages to the GRPP, where appropriate, to enhance the
effectiveness of both?
3. Oversight.
To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as
appropriate, whether the program is housed in the Bank or externally managed?
To what extent is the Bank's oversight independent of the management of the program?
To what extent does the Bank's representative on the governing body have a clear terms of reference?
4. Risks and risk management. To what extent have the risks associated with the program been identified and are being
effectively managed?
For example, IEG identified the following risks in its global review:
� Bank bears a disproportionate share of responsibility for governing and managing in-house programs?
� Confusion at the country level between global program activities, Bank activities, and Borrower activities?
� Representation of NGOs and the commercial private sector on program governing bodies?
� Unclear role and application of Bank's safeguards?
� Trust-funded consultants and seconded staff representing the Bank on some program governing bodies?
5. Disengagement strategy.
To what extent is the Bank engaged at the appropriate level in relation to the Bank's new strategic framework:
� Watching brief?
� Research and knowledge exchange?
� Policy or advocacy network?
� Operational platform?
To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in
relation to the Bank's objectives for its involvement in the program:
� The program declares "mission accomplished" and closes?
� The program continues and the Bank withdraws from all aspects of its participation?
� The program continues and the Bank remains engaged, but the degree of the Bank's engagement in some or all
aspects (such as financing) declines over time?
Annex B 38
Annex B. Key Dates in ADEA History
1988 � Donors to African Education (DAE) is founded.
1992 � DAE moves from the World Bank (Washington) to the IIEP (Paris).
� The Steering Committee expands to include a Bureau of Ministers.
� The Association evolves into a partnership between development agencies and ministries
of education.
1995 � DAE changes its name to ADEA (Association for the Development of Education in Africa).
1997 � The Bureau of Ministers expands to ten ministers. It is decided hereafter to hold all
Biennial Meetings in Africa.
2007 � The Steering Committee decides to move the ADEA Secretariat to Tunis, Tunisia in
Association with the African Development Bank.
� It is decided that ADEA will cover all of Africa and not just sub-Saharan Africa.
� A new Strategic Plan for ADEA is drafted by the Secretariat (yet to be fully ratified by the
Steering Committee.
39 Annex C
Annex C. ADEA Working Groups
Working Group Agency Regional Coordinator Year Year closed
Coordinator established or graduated
School Examinations HEDCO � Ireland 1988 Closed
Vocational Education ILO 1988 Dissolved
and Training 1995
Book Sector (Books ODA (DFID) 1988
& Learning Materials)
Higher Education World Bank Association of African 1988
Universities
Accra, Ghana
Education Statistics SIDA UNESCO 1988
BREDA
Teaching Profession Commonwealth W. Africa: Togo 1988
Secretariat E. Africa: Uganda
Indian Ocean: Seychelles
Southern Africa: Zambia
Mobilization of French 1988 ??
Resources Government/
World Bank
Donors Information World Bank 1988 Became
System function of
ADEA
Secretariat
Action Group on UNESCO 1988
Exchange of Sector
Studies
Educational IDRC 1989 Dissolved
Research and Policy 2000
Analysis
Female Participation Rockefeller Forum for African Women 1989 Graduated
Foundation Educationists (FAWE)
Evaluation of French 1990 Closed
Education Systems Government
Finance & Education CIDA CODESRIA 1994
Dakar, Senegal
Non-formal Swiss Agency for 1996
Education Development and
Cooperation;
UNESCO (UIE)
Early Childhood Netherlands UNICEF Ghana 1997
Development Government
Distance Education UNESCO BREDA Mauritius Ministry of Ed & 1997
and Open Learning Research
Communication for NORAD Wanad Center Benin ??
Education and
Development
Annex C 40
Working Group Agency Regional Coordinator Year Year closed
Coordinator established or graduated
Mathematics and Center for Math, Science 2004
Science Education & Technology in Africa,
Nairobi, Kenya
Ad Hoc WG on HIV/ ADEA Secretariat 2001
AIDS and Education
Ad Hoc WG on ADEA Secretariat 2002
Quality Education
Ad Hoc WG on ADEA Secretariat 2003
Policy Dialogue
Ad Hoc WG on Post- World Bank 2004
Primary Education
41 Annex D
Annex D. ADEA Biennial Meetings, 1991�2006
Date Location Theme Ministers Present
October 28 � Manchester, United Textbook Provision and 30 ministers or other
November 1, Kingdom Library Development in Africa Senior officials
1991
October 22�24, Angers, France Improving the Implementation Over 30 ministers
1993 of Educational Projects in
Africa through Ownership
October 18�22, Tours, France Formulating Educational Policy
1995 in Sub-Saharan Africa
October 14�18, Dakar, Senegal Partnership for Capacity
1997 Building and Quality
Improvements in Education in
Africa
December 5�9, Johannesburg, S. What Works and What's New
1999 Africa in Education: Africa Speaks!
October 7�11, Arusha, Tanzania Reaching Out, Reaching All:
2001 Sustaining Effective Policies
and Practices for Education in
Africa
December 3�6, Grand Baie, The Quest for Quality:
2003 Mauritius Learning from the African
Experience
March 27�31, Libreville, Gabon More and Better Education: 26 countries
2006 What makes Effective Learning represented by
in Schools and in Literacy ministers of education
Programs and Early Childhood
Development Programs?
Annex E 42
Annex E. ADEA Organizational Chart
STEERING COMMITTEE
Chair: Mr. Dzingai Mutumbuka (World Bank); Alternate Chair: Hon. Rosalie Kama-Nyamayou (Congo);
Vice-Chair (ex-officio): Mark Bray (IIEP), all as of 2006*
Development Agency Members Bureau of Ministers (as of 2006)
African Development Bank West Africa :
Austria Senegal, Burkina Faso, Mali
Carnegie Corporation
CIDA (Canada) Southern Africa:
Denmark Angola, Mozambique
DfID (United Kingdom)
European Commission Central Africa:
Finland Central African Republic, Congo
France
GTZ (Germany) Eastern Africa:
Gulbenkian Foundation Tanzania, Rwanda
Ireland
JICA (Japan) Indian Ocean:
Netherlands Mauritius
Norway
Rockefeller Foundation
SDC (Switzerland)
UNESCO
UNICEF
USAID (United States)
World Bank
SECRETARIAT
Executive Secretary: Mamadou Ndoye
Senior Program Specialist: Hamidou D. Boukary
Publications/Communications Officer: Thanh Hoa Desruelles
Program Specialist: Maud Seghers
Administrative and Finance Officer: Sabine Ayeh
Senior IT and Database Specialist: Beedeeanun Conhye
*These officers plus the Executive Secretary make up what is referred to as the Bureau of ADEA.
43 Annex F
Annex F. Technical and Sub-Regional Meetings, 2003�2007
Topic Location Date
Effective Responses to the HIV/AIDS Pandemic in the Libreville, Gabon May 27�29,
Education Sector: From Analysis to Action (co-sponsored 2003
by UNAIDS)
Education for Lusophone and Hispanic Countries (co- Luanda, Angola Nov 29 �
sponsored by IBE and the World Bank Institute) Dec 4, 2004
Teaching in a World with AIDS (co-sponsored by Nov 29�Dec 1,
UNESCO) 2004
Education in Countries in Crisis or Post-Conflict Mombasa, Kenya June 2�4,
2004
Integration of ICT in Education: Issues, Challenges, and Abuja, Nigeria July 26�30,
Infrastructure in Africa 2004
Employment and Use of Contractual Teachers in Primary Bamako, Mali Nov 21�23,
Education (co-sponsored by the World Bank, Education 2004
International, and the Government of Mali)
3rd African Conference on Early Childhood Development Accra, Ghana May 30 �
(co-sponsored by the Government of Ghana, World Bank, June 3, 2005
UNESCO, the Consultative Group on Early Childhood
Development, and the Virtual University)
Bilingual Education and the Use of Local Languages in Windhoek, Namibia August 3�5,
Educational Systems (co-sponsored by GTZ, UNESCO 2005
Institute for Education, and the Agence Intergouvere-
mental de la Francophonie)
Assessment and Certification of Competences for the Grand Baie, Mauritius May 2�4, 2005
Indian Ocean Countries (co-sponsored by the MOES
Mauritius and IIEP)
Education for Rural People in Africa (co-sponsored by Addis Ababa, Ethiopia Sept 7�9,
FAO, the French Ministries of Foreign Affairs and 2005
Agriculture and the Italian Ministry of Foreign Affairs)
Financing of Literacy Programs Morocco Dec 19-20,
2006
Secondary Education in Africa (co-sponsored by the � Kampala, Uganda Jn 9�13, 2004
World Bank) � Dakar, Senegal Jn 6�9, 2004
� Accra, Ghana Apr 1�4, 2007
Conference on School Fee Abolition (co-sponsored by Bamako, Mali June 19-22,
UNICEF and the World Bank) 2007
Annex G 44
Annex G. Uses of ADEA Funds, 2001�2006
2001 2002 2003 2004 2005 2006
PROGRAM Audited Audited Audited Audited Audited Estimated
Policy Dialogue 748,033 257,641 817,788 512,276 593,721 811,196
Regional and Sub-
0 47,985 204,383 460,862 317,948 87,517
regional Cooperation
Ad hoc Working Groups 0 0 861,563 171,380 184,646 168,629
Support to Working
1,464,775 1,930,388 1,998,653 2,322,774 3,109,975 3,034,236
Groups
Complementary Programs 282,492 504,677 merged with other programs
Coordination, Facilitation
& Knowledge 138,986 120,980 76,981 205,789 182,307 247,826
Management
Publications &
179,960 189,634 408,006 651,481 381,572 347,472
Communications
Secretariat 0 0 0 923,415 977,967 1,064,370
Overhead and Reserve 625,070 674,064 829,787 0 295,351 831,963
TOTAL COSTS 3,439,316 3,725,369 5,197,161 5,247,977 6,043,487 6,593,209
45 Annex H
Annex H. IEG Assessment of the Quality of ADEA's Analytical
Work
Report Number
#7 #10 #15 #22 #24 #35 #36 #43 #47 #50
Dimensions
1. Task Objectives
a. Clearly stated? 2 3 3 1 1 1 2 2 1 3
b. Relevant to ADEA mission 1 2 1 1 1 1 3 1 1 2
2. Quality of analysis
a. Methods appropriate 2 3 3 1 2 2 3 3 1 4
b. Quality of the evidence 3 3 2 1 1 2 3 3 2 3
3. Quality of conclusions (appropriately
specific and consistent with analysis 3 3 2 1 1 2 3 3 1 3
4. Quality of recommendations
a. Clearly stated? 2 4 1 1 1 3 1 2 1 2
b. Flow well from conclusions 2 4 1 1 1 3 4 1 1 3
c. Likely to have an impact
(feasible)? 3 4 1 1 2 2 1 3 1 2
Note: Dimensions rated on a four point scale with 1 being the highest
Annex J 46
Annex I. Persons Consulted
Members of the ADEA Secretariat
Mamadou Ndoye, Executive Secretary
Hamidou Boukary, Senior Program Specialist
Thanh-Hoa Desruelles, Publications and Communications Officer
Maud Seghers, Program Specialist
Beedeanun Conhye, ADEA Database and IT Specialist
Richard Sack, former Executive Secretary
Partner Agency Representative
Alice Hamer, African Development Bank
Cream Wright, UNICEF
Ann-Therese Ndong-Jatta, UNESCO, Director Pre-Primary to Secondary Education
Joyce Moock, Rockefeller Foundation (retired)
Mark Bray, Director, IIEP
Robert Prouty, Deputy Secretary, EFA-Fast Track Initiative
World Bank Staff Members (active and retired)
Dzingai Mutumbuka, Education Sector Manager, and Co-Chair ADEA Steering Committee
Jee-Peng Tan, Education Advisor, Africa Region
Birger Fredriksen, former Senior Education Advisor for Africa (2001�04) and Sector
Director for Human Development (Africa) (1997�2001)
Peter Moock, First and founding ADEA Executive Secretary and former,
Adriaan Verspoor, consultant AFTHD, former Lead Education Specialist,
Keiko Inoue, Operations Officer, AFTHD
World Bank TTLs in Africa:
Olatunde Adetoyoyese Adekola, TTL for Nigeria
Jacob Bregman, TTL for Cameroon and Nigeria
Rest Barbabas Lasway, TTL for Tanzania
Xiaoyan Liang, TTL for Mozambique
Mmantetsa Marope, TTL for Malawi
Susan Opper, TTL for Rwanda
Mercy Tembon, TTL for Mauritania
Gary Theisen, TTL for Ethiopia
Academics and NGO Representatives
Martial Dembele, Associate Professor, University of Montreal
Penelope Bender, Plan USA
47 Annex I
Minister and Ex-Ministers of Education
Rosalie Kama Niamayoua, Minister of Education, Republic of Congo Brazzaville, and Chair,
ADEA Bureau of Ministers
Margareth S. Sitta, Minister of Education and Vocational Training, Tanzania
Louis Steven Obeegadoo, Ex-Minister of Education, Mauritius (and ex-Chair of the ADEA
Bureau of Minsters)
Ann-Therese Ndong-Jatta, Ex-Minister of Education, The Gambia
Amadou Wade Diagne, Director, Cabinet d'appui � l'Education, Senegal
Annex J 48
Annex J. Official Response of ADEA to IEG's Global Program
Review
January 29, 2008
ADEA welcomes the World Bank IEG review of ADEA. The timing of the report is particularly
opportune as ADEA is completing its first strategic planning exercise following a
recommendation stemming from the external evaluation of ADEA conducted in 2005.
Comments on Overall Lessons Learned
As a result of the strategic planning exercise and ADEA's new Medium-Term Plan (2008-
2012), some of the recommendations spelled out in section 5 listing "lessons learned" are
already being acted upon. In particular, we are pleased to inform that ADEA has addressed
issues raised in the bulleted lessons learned N�1, N�2, N�4 and N�6 (page 28 of review):
� ADEA has re-formulated its strategic objectives (there are five) � now officially
approved by the Steering Committee � and has developed a Performance Evaluation
Framework (PEF)for monitoring and evaluating ADEA's progress in achieving the 5
strategic objectives. The PEF includes expected outcomes, outputs and performance
indicators.
� The process of realigning ADEA's Working Groups to the overall Strategic Plan has
started and the ADEA Steering Committee has proposed the merging and graduation
of some WGs. As of today, the merging of three long standing Working Groups
(Education Statistics, Education Sector Analysis and Finance and Education) is
underway, with the view of having one unified Working Group that will bring its
expertise to the setting up of the African Union's Observatory for the Second Decade
of Education for Africa.
� ADEA has decided to create an Executive Committee in charge of operational and
management oversight that will relieve the Steering Committee (SC) in order to
enable it to focus on more substantive policy, organizational and strategic matters.
� ADEA's new Strategic Plan also addresses lesson learned N�6 (the need to extend
the reach of ADEA programs and outputs) by putting greater emphasis on outreach
activities towards key partners and stakeholders in education in Africa.
In response to lesson learned N�3 related to analytical work, it should be stressed that
ADEA � which considers itself as a learning organization committed to social transformation
� is first and foremost engaged in fostering a culture of analysis. For the Biennales for
example, ADEA tries to get as many countries as possible to engage in a learning process
through the analysis of their policies and practices, the learning from their experiences and
the sharing of lessons learned with other countries. The ultimate objective sought is the
qualitative transformation of policies and practices. For ADEA, the process activated is more
important than the actual result and the "quality" of what is produced is not the only element
that counts. The process continues after the Biennale as countries are invited to participate
in "thematic inter-country quality nodes" where countries interact with other countries and
experts/organizations having an edge in specific thematic areas. It should be added that in
addition to studies conducted by countries, ADEA also commissions background papers and
syntheses from experienced researchers and experts.
49 Annex J
Effectiveness of the Program
Other comments on the review findings are the following:
� Networking (paras. 3.25 and 3.26) goes beyond what is described in the Review. It is
an essential element of ADEA's strategy and activities and ADEA defines itself as a
network of networks for a host of reasons: as the Review mentions, ADEA WGs
have contributed to building networks of African specialists in their specific thematic
area (networks of education statistics and information systems specialists, education
finance specialists, specialized education journalists and communicators, etc.),
ADEA maintains strong links with African research networks such as ERNWACA
(Education Research Network for Western and Central Africa) and ERNESA
(Education Research Network for Eastern and Southern Africa) and it is developing
relations with the Regional Economic Communities (RECs) and engaging them to
support the implementation of AU's Second Decade of Education for Africa. In
carrying out activities, ADEA also partners with a number of organizations with which
it has common areas of interest: in 2007 for example, the Conferences on School
Fee Abolition, Secondary Education and Education for Rural People and the High
Level Meeting of RECs were organized in collaboration with many partners
(UNICEF, the World Bank, FAO, the French Ministries of Foreign Affairs and
Agriculture and the Italian Ministry of Foreign Affairs, AU, UNESCO and ADB).
� Several mechanisms exist within ADEA to ensure that ADEA's program and activities
are country-driven (para. 8):The Bureau of Ministers composed of 10 Ministers of
education representing the five sub-regions of Africa voices its concerns and guides
ADEA in defining its priorities and activities; the Caucus of Ministers, meeting every
two years during the ADEA Biennale, also ensures that ADEA is responsive to
country needs and priorities. For instance, the last meeting of the Caucus of
Ministers recommended that ADEA facilitate education peer review exercises, which
it has done through its Working Group on Education Sector Analysis. Working
Groups also make sure that African stakeholders are represented on their Steering
Committees and they are increasingly anchored in Africa and coordinated by African
organizations, thereby ensuring that their agendas are African-driven and that
synergies are developed with African bodies (the WG on Higher Education is
coordinated by the AAU based in Accra; the WG on Communication for Education
and Development by the West African News Media Organization (WANAD) in
Cotonou, the WG on Maths and Science Education by SMASSE-WECSA in Nairobi
and the Working Group on Distance Education and Open Learning by the Mauritius
Ministry of Education and BREDA). The Review could also mention that ADEA has
developed a questionnaire addressed to Ministries of Education to assess their
needs on a regular basis and to make sure that ADEA programs and activities
respond to these needs.
� Regarding development agency coordination (para. 3.10) while there is no
systematic outcomes information specifically related to ADEA, there is no doubt
today that there is greater coordination among agencies than there was when ADEA
was founded and that coordination is increasingly sought. A number of mechanisms
specific to the education sector have developed in the last twenty years and are
evidence of this: the EFA Movement, the FTI, Sector wide approaches, the Rome
Declaration on Harmonization, the EPDF funding mechanism. Although improved
coordination cannot be solely attributed to ADEA, it can be stated that ADEA was the
Annex J 50
first organization to set this prominently on the agenda of educational cooperation
and therefore one of the main instigators for greater coordination among agencies.
The existence of PRISME should be mentioned in the Review as a database of
externally funded projects and programs on education in Africa, specifically
developed by ADEA as a tool for enhanced coordination.
� Within ADEA, which defines itself as a learning organization, the notion of "capacity-
building" (paras. 3.21 � 3.24) is broad and directly linked to the central importance of
learning in ADEA's methodological approaches and strategic objectives. Capacity-
building goes beyond the technical capacity-building and shifting of WGs in African
institutions described under "Strengthening capacities in African countries" (para1.5).
Capacity-building within ADEA operates as a learning process building on the
exchange of experiences, the sharing of knowledge, the pooling of expertise,
strategic partnerships with expert institutions, networking, and, most importantly
learning by doing (the "praxis" approach). Capacity-building occurs throughout
ADEA's activities e.g. in preparation of ADEA's Biennales when countries are asked
to document and to analyze country initiatives, within ADEA's inter-country quality
nodes (networks of countries and expert organizations set up to ensure that
recommendations coming out of the ADEA Biennales and other major meetings are
translated into concrete actions in African countries), within ADEA Working Group
activities and through study tours organized for African Ministers and senior Ministry
representatives.
� Regarding communication, publications and dissemination (para 1.11), many efforts
are being made to extend ADEA's reach to education stakeholders. The ADEA
Newsletter reaches some 12 000 addresses, in Africa and other regions and
readership includes professionals in Ministries of Education, development agencies
and organizations, universities, civil society organizations, NGOs and other African
and non-African organizations involved in education. ADEA also has a web site
containing comprehensive information on ADEA's programs and activities, a
Members Only Area to facilitate exchanges between ADEA members and a number
of ADEA publications on-line. Media relations � which the review does not mention �
have also developed considerably over the past 6-7 years. ADEA now systematically
invites the media to all major ADEA events and meetings with two main objectives in
mind: extending its outreach beyond the education community of practitioners and
specialists; and contributing to the development of a network of African journalists
specialized in education reporting. In 2007 for example, journalists were invited to
the following ADEA meetings: the Regional Conference on Secondary Education in
Accra; the International Conference on School Fee Abolition on Bamako; the
Workshop on Education for Rural People in Africa in Rome and a fringe meeting
organized during the ADEA Steering Committee in Brazzaville.
� Regarding support to ADEA from African countries (Para 1.15), the phrase "ADEA is
funded almost entirely through its 20-22 development agency partners, with marginal
support coming from the African countries" should be modulated. Indeed, countries
provide significant contributions when meetings are organized in their countries (they
appear as in-kind contributions in ADEA reports). Amounts going towards the
organization of Biennial Meetings and conferences have been particularly significant,
averaging roughly 200 000 US Dollars for the Biennales and 100 000 US Dollars for
major conferences.
51 Annex J
� Concerning ADEA's financial and budget management (para 1.20), the paragraph
should be re-worded and the footnote deleted. In 2005, ADEA's financial situation
deteriorated unexpectedly, when IIEP claimed a significant amount of payment arrears
from ADEA, based on its 2002�04 accounting. An independent audit found that the
claim was unjustified as it was established that ADEA had been debited twice for the
payment arrears claimed. The amount claimed has since been returned to ADEA, which
thereby recovered its healthy financial condition.
� On the impact of ADEA in countries (paras. 3.5 � 3.9), while ADEA takes note of the
TTL views and the need to track policy impacts more systematically, one needs to be
cautious and to take into account two factors: (i) the fact that the impact of ADEA's
efforts are long-term and outcomes are often observed years later; and (ii) the
difficulty of linking outcomes/results directly to ADEA. A case in point is the impact of
the study tour organized in 1999, which invited African Ministers of Education to visit
Guatemala and El Salvador. Years later, we are observing the direct impact ADEA
has had on country policies and programs: Senegal is implementing a students'
government scheme in secondary schools; Niger has implemented a rural education
program inspired by Guatemala's Escuela Nueva; Madagascar has put in place
school/community contracts. In some cases, outcomes are only observed several
years after, as is the case for Senegal, which will only start to apply the student's
government scheme as of 2008.
� Finally, regarding the balance between restricted and unrestricted funding (para
3.29), since the Review was drafted several agencies have either moved towards
unrestricted funding (CIDA, Netherlands and Irish Aid) or have indicated that they
would be contributing unrestricted funding starting 2008 (SDC).
Overall, we find the IEG Evaluation pertinent and a useful perspective as ADEA is refining
its strategies as well as its governance and management instruments and tools. We thank
the evaluation team for the review which will be circulated to all ADEA constituencies.
Yours Sincerely,
Mamadou Ndoye
ADEA Executive Secretary
The Global Program Review Series
The following reviews are available from IEG.
Volume #1, Issue #1: ProVention Consortium
Issue #2: Medicines for Malaria Venture
Issue #3: Development Gateway Foundation
Issue #4: Cities Alliance
Volume #2, Issue #1: Critical Ecosystem Partnership Fund
Issue #2: Association for the Development of Education in Africa
Issue #3: Population and Reproductive Health Capacity Building Program
Issue #4: International Land Coalition
Volume #3, Issue #1: Consultative Group to Assist the Poor
RPR
Over the course of almost two decades, the Association for the Development of
Education in Africa (ADEA) has become the premier forum for educational policy develop-
ment and agency cooperation in Africa. Its influence is largely a consequence of its growing
legitimacy (it now includes as partners all African ministries of education and all major
development agencies active in African education) and its relevance (the focus of its work
has evolved along with the region's educational issues). However, such a high level of legiti-
macy has had costs in terms of efficiency. These concerns are now being addressed with
the formulation of a new strategic plan for 2008�12 and the establishment of an executive
board to oversee the operation, thereby allowing its steering committee to focus on more
strategic and substantive matters. ADEA's convening power has enabled it to promote policy
dialogue and analytical work on African educational problems through a variety of forums,
such as at biennial meetings and technical gatherings and in 15 working groups.