ICRR 14380
Report Number : ICRR14380
IEG ICR Review
Independent Evaluation Group
1. Project Data: Date Posted : 06/30/2014
Country : Bolivia
Project ID : P083965 Appraisal Actual
Project Name : Bo-secondary US$M ):
Project Costs (US$M): 10.00 9.97
Education
Transformation
L/C Number : C4365 Loan/ US$M):
Loan /Credit (US$M): 10.00 9.97
Sector Board : Education Cofinancing (US$M):
US$M ):
Cofinanciers : Board Approval Date : 10/09/2007
Closing Date : 12/31/2010 03/31/2013
Sector (s): Secondary education (79%); Primary education (12%); Sub-national government
administration (9%)
Theme (s): Education for all (50% - P); Decentralization (25% - S); Administrative and civil service
reform (25% - S)
Prepared by : Reviewed by : ICR Review Group :
Coordinator :
Susan Ann Caceres Judyth L. Twigg Lourdes N. Pagaran IEGPS2
2. Project Objectives and Components:
a. Objectives:
The appraisal document (p. ix) states that the project development objective was "to support the Municipal
Government's education strategy by : increasing access to secondary education for adolescents and young
people and improving their permanence in the education system; improving quality and relevance of primary and
secondary education; and strengthening the decentralized education management capacity of the Municipality
of La Paz." The same objective was noted in the Financing Agreement (p. 5).
On April 25, 2012 during a Level 2 Restructuring, four project development objective -level indicators were
revised (see Section 4).
b.Were the project objectives/key associated outcome targets revised during implementation?
Yes
If yes, did the Board approve the revised objectives /key associated outcome targets?
No
c. Components:
The project had three components :
Access and Permanence (appraisal US$ 4.90 million; actual US$ 6.72 million) was intended to expand
coverage of secondary (grades 9 through 12) and upper primary (grades 7 and 8) education to unattended
areas and overcrowded urban center districts . It was to support the national strategy of prioritizing equal
opportunities to marginalized children by building two schools in the peri -urban areas out of the total of four
planned, which was to create new places for 7,600 students. New classroom construction in existing
schools was to provide an additional 12,000 new places, if two shifts were used . An Incentive Program for
at-risk students was to be established to provide pedagogical support through after -school supplementary
academics.
Quality of Secondary Education (appraisal US$ 3.73 million; actual US$ 2.58 million) was to improve the
quality of service delivery by enhancing the learning environment in three dilapidated schools (63
classrooms) and implementing an experimental maintenance program with three options depending on the
size and complexity of the civil works . It was to strengthen educational communities and social
organizations to promote the participation of these schools in the design and implementation of School
Improvement Plans and grants. It was also to develop training materials related to the topics of democracy,
gender equity, health and sexuality, the environment, multiculturalism, and life skills . Seven Pedagogic
Resource Platforms (science and computer labs) were to become a central hub for workshops and
pedagogical reinforcement for at -risk students.
Management and Institutional Strengthening (appraisal US$ 0.86 million; actual US 0.64 million) was to
finance activities to strengthen the Municipal Office of Human Development (OMDH), such as enhancing its
financial and accounting practices, revising its procurement standards, purchasing equipment /furniture, and
training OMDH staff. It was to finance the adaptation of an existing monitoring and information system for
the Municipality and OMDH. An impact evaluation was planned to inform the design of the Municipality's
future strategy in education and determine the impact of project interventions . The component was also to
support assessments of student learning progress for the Municipality to inform the government on the
status of learning (twice a year for all students in their second year of secondary schooling in math and
language) and provide in-service training to teachers .
The components were revised at an April 2009 restructuring as follows:
Component One, Access to Education : primary schools were added as beneficiaries, modifications to the
at-risk program were made to reflect the hiring of experienced teachers to develop remedial courses, and
the eligibility criteria of the at-risk students were changed .
Component Two, Quality of Education : the scope of the school infrastructure maintenance program was
refocused to develop a framework to clarify roles and responsibilities within the regular maintenance
program and participation of parents, teachers, and students in the school councils (Juntas Escolares); and
a post-graduate program for teachers, principals, and municipal staff was set up .
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates:
Project Cost : Costs for component one (access) were much higher than planned (137%), while costs for
component two (quality) and three (management) were lower. The ICR explains that funds were reallocated
among components due to increases in the cost of construction, equipment, and furniture .
Financing : The project was financed solely by a credit from the International Development Association (IDA).
Borrower Contributions : The Borrower did not make any contributions, nor were any planned .
Dates :
Level two restructuring of the project occurred four times : (1) in April 2009 to include primary schools as
beneficiaries of the infrastructure activities (to recognize that different levels of schools shared one building ); (2)
in September 2009 to modify the definition of operational costs to include teacher payments for the pedagogical
activities; (3) in December 2010 to revise the scope of project activities to reflect municipal realities; and (4) in
April 2012 to revise four PDO-level indicators.
The closing date was extended twice . The first extension was approved on December 13, 2011, which changed
the date from December 31, 2010 to September 30, 2012. The second extension was approved on May 7, 2012
and changed the closing date to March 31, 2013. The ICR does not state why the extensions were needed .
3. Relevance of Objectives & Design:
a. Relevance of Objectives:
Substantial under both the original and revised indicators /targets .
At the time of appraisal, La Paz faced a growing demand for primary and secondary education . As enrollment
increased, there was accompanying overcrowding in schools and an increasing student -teacher ratio. Schools
operated in several shifts. As a result of these factors, quality was low and drop -out and repetition rates were
high. The Municipality's 2020-2040 Development Plan focuses on increasing access to secondary education
and improving quality of education at all levels . The Plan sets education as one of the main pillars within human
development, particularly at the preschool and secondary levels . At the national level, the Government also
includes human development as one of its main focal areas .
The Bank's Country Partnership Strategy (FY12-15) outlines the Bank's role in supporting the country's efforts to
improve the coverage and quality of education services . This municipal project was consistent with the broad
areas of Bank engagement described in the Partnership Strategy, such as decentralization processes and youth
engagement. Thus, the objectives of the project are substantially relevant for the Government and Bank;
however, the ICR (p. 7) notes that the objectives were quite broad . The objectives remained the same over the
life of the project.
b. Relevance of Design:
Modest under both the original and revised indicators /targets .
There was a logical link between the project's objectives, activities, and outcome measures in relation to access
and permanence. The at-risk program was coordinated with the regular schools to permit monitoring of
students' learning and provide additional support to improve their retention . Civil works emphasized
marginalized areas, as well as those with unmet demand to increase access and enrollment .
The results chain was less developed for the objectives of quality, relevance, and strengthening decentralized
management capacity. Design did not sufficiently address these aspects . In addition, outcome measures were
not adequate to demonstrate achievement of these objectives (see Section 10a for shortcomings related to
indicators and the Results Framework ). The ICR (p. 4) indicates that there was a need for better definition of
some sub-components and activities.
The restructuring did not change the design but revised the PDO indicators, and these changes did not alter the
Results Framework in a substantive way . The restructuring scaled back the project's scope and to respond to
municipal and country realities (Restructuring Paper, p. 1).
4. Achievement of Objectives (Efficacy):
Increase adolescents' access to secondary education : Substantial under the original indicator /targets and
Substantial under the revised indicators /targets .
Outputs:
145 classrooms were constructed or rehabilitated within 12 schools (including provision of furniture ), which
exceeded the revised target of 137 classrooms. This created 15,840 new spaces, which met the revised
target (15,840), as well as original target (12,000). The ICR suggests that the construction and equipping
of schools and classrooms was done in areas with overcrowding or unmet demand, specifically in
peri-urban areas or marginal urban districts . Beneficiaries reported that students' travel time decreased
with the greater proximity of the schools, which created cost savings related to transportation for parents .
63 sanitary facilities were constructed, which exceeded the revised target of 49. This intermediate outcome
indicator was added during the April 2012 Level 2 Restructuring.
79 administrative offices were constructed and provided with furniture and equipment, which exceeded the
revised target of 73. This intermediate outcome indicator was added during the April 2012 Level 2
Restructuring.
Outcomes under the original indicators /targets:
The original PDO indicator was to increase the gross secondary enrollment rate from 67.7% in 2005 to
77.1%. The change in the gross secondary enrollment rate in La Paz was not reported in the ICR, even
though this was a key performance indicator to measure attainment of the objective for most of the life of
the project. The Restructuring Paper (p. 6) explains that the original indicator measured enrollment rates
throughout La Paz even though the project focused only on some of the districts . The revised indicator was
changed to an output (number of students), rather than tracking the enrollment rate in project -supported
schools, as the number of inhabitants in La Paz would not be available until the 2012 census was collected
and analyzed (Restructuring Paper, p. 6). In the region's reply to the draft ICR, it was reported that the
secondary gross enrollment for the four grades classified as secondary at the time of project appraisal
increased from 98.3% in 2005 to 121.4% in 2012, based on 2012 census data. This exceeded the target of
9.4 points improvement. The enrollment rate for the six grades of secondary, the new education structure,
was not provided.
Outcomes under the revised indicators /targets:
A new PDO-level indicator was implemented during the April 2012 Level 2 Restructuring. Secondary
education enrollment increased from 3,416 students in 2008 to 6,311 students in 2013, which exceeded the
revised target of 5,383 students.
Improve the permanence of adolescents in the education system : Substantial under both the original and
revised indicators /targets .
Outputs:
Curricular adaptations in mathematics and language were carried out for years 7 and 8 in primary and 1 and
2 in secondary.
156 teachers were trained in pedagogical support related to the Incentive Program . Training was carried
out related to curriculum adaptation and evaluation techniques, and teaching tools .
10,131 students in grades 1 to 4 in secondary received support in math and language, which exceeded the
original target of 7,075 students and the revised target of 9,768 students. The students served by the
program had low grades and were located in the most populated macro -districts. Parents felt their children
benefitted from the program, based on surveys (ICR, p. 12). Students expressed more motivation to
continue their studies, as their skills improved with the support .
Outcomes under the original indicators /targets:
The promotion rate in secondary education in La Paz increased from 79.4% in 2005 to 89.9% based on
2011 census data, meeting the original target (84.2%).
The secondary education repetition rate in La Paz decreased from 12.4% in 2005 to 6.7% in 2013, which
met the original target (10.7%).
The primary education repetition rate decreased from 4.2% in 2007 to 3.6% in 2010 (latest year where data
were available). The ICR points out that these rates are for the entire municipality, not just project -funded
schools.
Outcomes under the revised indicators /targets:
The promotion rate in secondary education in La Paz increased from 79.4% in 2005 (or 86.2% in 2008) to
89.9% based on 2011 census data, which (nearly) met the upwardly revised target of 90.9%. The original
PDO indicator and target were changed during the April 2012 Level 2 Restructuring to include grade 6,
consistent with the new education law (Restructuring Paper p. 6).
The secondary education repetition rate in La Paz decreased from 12.4% in 2005 to 6.7% in 2013, which
did not meet the revised target of 5.7%. The ICR reports that the baseline was revised to 8.3%, based on
revised data (p. iii). The Restructuring Paper (p. 6) further reports that baselines and targets were revised
to include grade 6, consistent with the new education law . The Implementing Agency (ICR, p. 38) clarified
that the Ministry of Education adjusted its data in 2012, which resulted in differing baseline rates at
preparation, restructuring, and preparation of the ICR . The ICR points out that these rates are for the entire
municipality, not just project-funded schools.
Additional Outcomes specific to project -funded schools:
The promotion rate in the schools reconstructed by the project increased from 82.3% to 89.6%.
The ICR reports high retention among students (99.3%) in the four new schools built by the project .
Improve quality and relevance of primary and secondary education : Modest under both the original and
revised indicators /targets .
Outputs:
33 classrooms were built and equipped with audiovisual or computer equipment, which met the target . The
ICR also reports that science and computer labs, multi -purpose rooms, libraries, and recreational spaces
were established in 19 schools.
1,061 teachers, administrators, and members of the school councils (Juntas Escolares ) received training in
civic values, which exceeded the target of 883 persons. Teachers used participatory methods and content
from the Civic Values Program. The content of the training included : gender equity, multiculturalism, health
and sexuality, and emotional intelligence . 85,029 secondary school students between grades 3 and 6
became involved in civic values activities, which exceeded the target of 20,000 students. 1,564 classroom
projects were prepared on civic values . 96 student dialogues were conducted and 200 school fairs were
held. 503 teachers, principals, and municipal staff received diplomas in civic values and active citizenship .
One Education High Performance Center was established, which met the revised target of one . The Center
complemented the resources provided in schools with learning materials, computers, and libraries . 11,000
beneficiaries (teachers, students, members of the community ) used the support of the Center in relation to
management and curriculum planning (ICR, p. 26). The Center was used to provide training to teachers
participating in the Civic Values Program and teachers in the Incentive Program . Staff from the Center also
produced educational materials to support teachers (ICR, p. 26).
PISA test content (from other countries) was analyzed to examine critical skills of students .
Outcomes:
The ICR reports that learning skills only slightly improved in math and language for students enrolled in the
Incentive Program, based on pre and post test results (pp. 10 and 13). Comparative results for math in 2011
showed a slight increase in the percentage of students reaching a level of good (from 1% to 7% for grade 1;
from 2% to 3% in grade 2; from 0.1% to 4% in grade 3; and from 0.38% to 3% in grade 4) and a level of
satisfactory (from 5% to 8% in grade 1; from 5% to 8% in grade 2; from 4% to 10% in grade 3; and from 3%
to 6% in grade 4) (ICR, p. 13).
Strengthen the decentralized education management capacity of the Municipality of La Paz : Substantial
under both the original and revised indicators /targets .
Outputs:
2,068 Juntas Escolares (school councils) and 191 teachers and principals in secondary schools were
trained. While the ICR reports increased participation, it does not describe the results attained from the
plans or increased participation of management teams .
An information management and monitoring system was put in place during the first year of the project by
adapting the existing information system from another project . The system monitored the project's physical
and financial implementation.
OMDH was equipped and strengthened through the provision of furniture and computers .
A study on roles and responsibilities in school maintenance was conducted, and the roles and
responsibilities were defined.
Outcomes under the original indicators /targets:
113 functioning education management teams were established . 92 school improvement plans were
prepared by trained school management teams, which met the target . These plans covered 183 primary,
secondary, and night schools, representing 40% of the schools in La Paz.
Outcomes under the revised indicators /targets:
In the April 2012 Restructuring the key performance indicator was changed to "strengthen capacity for
decentralized education administration ." The revised performance indicator removed language to clarify
that resources are handled at the central level, rather than by school committees . It also clarified that
school committees help each of the schools using the building, as they can be used by more than one
school. The ICR reported that 113 education management teams were established and 92 school
improvement plans were prepared, meeting the target . In follow-up visits by OMDH, it was made clear that
the goods that were acquired through the improvement plans were used . The ICR (p.15) reports that
"directors, teachers, parents, and students of the schools using the same building collaborated on the
maintenance and care of the school premises ....to identify issues related to infrastructure, learning
materials, and goods."
The ICR reports more efficient administrative procedures at the municipal level . For example, the time to
process project payments was reduced from 90 to 60 days.
5. Efficiency:
The ICR notes an internal rate of return (IRR) of 4.5%, while the appraisal document estimated IRR to be 5.9%.
The ICR explains that the difference is attributable to increased construction costs (for example, the cost per
classroom increased from US$ 5,392 to US$ 7,733), which meant that the project created fewer classrooms
than anticipated. Similar methods were used to calculate the rates of return in the appraisal document and the
ICR. Both documents estimated benefits from age -earnings and age working probability profiles obtained from
household survey data.(ICR, p. 15 and p. 28). Studies confirm that individuals completing secondary education
earn more than those only with primary education .
The ICR reports high construction quality and efficiencies in building supervision over the life of the project . The
Implementing Agency states that the new facilities were improved in aspects such as noise, waste disposal, and
handicapped accessibility (ICR, p. 40). The number of beneficiaries reached through the quality and relevance
activities (i.e. Incentive Program and Civic Values ) was higher than anticipated at appraisal .
The project closed more than two years later than anticipated . As noted in the Restructuring Paper 2012 (p. 2),
"construction, furniture, and equipment have been more costly than anticipated due to delays in project
implementation and a significant increase in the cost of materials and labor in the years following project
approval. This has resulted in reduced budget for the education quality and institutional strengthening aspects
of the project." Project effectiveness delays also impacted implementation, and more recently, a second series
of delays took place due to new institutional arrangements and infrastructure regulations issued by La Paz in
May 2010, which established a Municipal Infrastructure Office (OMI) independent from the Municipal Office of
Human Development.
Efficiency : Modest
ERR )/Financial Rate of Return (FRR)
a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the
re -estimated value at evaluation :
re-
Rate Available? Point Value Coverage/Scope*
Appraisal Yes 5.9% 100%
ICR estimate Yes 4.5% 100%
* Refers to percent of total project cost for which ERR/FRR was calculated.
6. Outcome:
Original: Relevance of objectives was substantial, while relevance of design was modest . There was substantial
achievement of three objectives related to access and permanence and of adolescents in the education system
and capacity strengthening, but modest achievement related to quality /relevance. Efficiency is rated modest.
Revised: Relevance of objectives was substantial, while relevance of design was modest . There was
substantial achievement of three objectives related to access and permanence and of adolescents in the
education system and capacity strengthening, but modest achievement related to quality /relevance. Efficiency is
rated modest.
The overall project outcome is weighted by the amount of the total loan disbursed before (74%) and after (26%)
the April 2012 Restructuring, which changed the PDO outcome indicators, as follows :
Outcome, original objectives = Moderately Satisfactory (4) x .74 = 2.96
Outcome, revised objectives = Moderately Satisfactory (4) x .26 = 1.04
Total: 4 or Moderately Satisfactory
The At-Risk program supported the academic skills of 10,131 students; student promotion and retention
increased in project schools . There were improvements in the secondary enrollment rate in La Paz, as well as
an increase in the number of students enrolled in project -supported schools. However, there were only modest
changes in quality and relevance,
a. Outcome Rating : Moderately Satisfactory
7. Rationale for Risk to Development Outcome Rating:
There are several factors that mitigate the risk to the development outcome . The Government benefits from
grant financing for its national education priorities . It is expected that the infrastructure built by the project will be
maintained, as maintenance roles and responsibilities have been clarified . The Municipal Government is
promoting a culture of respect and responsibility in relation to school facilities, furniture, and learning materials . It
has also launched a competition to reward schools that properly maintain their infrastructure . With the new
schools and classrooms, enrollment is expected to be sustained . The Center for Higher Education
Performance was established and is functioning, as the Municipality finances the salaries of the Center staff .
Civic content has been integrated into ongoing municipal programs . However, some of the programs begun
under this project have not continued (e.g. Incentive Program and Civic Values Program ). It is not clear whether
the capacity of School Management Teams will weaken as newly elected members come on board and need
training. Similarly, without follow-up training, teachers may not continue to use some of the pedagogical skills
that they learned. Thus, there is a moderate risk to the development outcome .
a. Risk to Development Outcome Rating : Moderate
8. Assessment of Bank Performance:
a. Quality at entry:
Preparation began for a national -level (rather than municipal-level) project and engaged the Government of
Bolivia via dialogue and technical work on the challenges of education, which identified the transformation of
secondary education as a critical priority . During this time, there were a number of political changes at the
national and local levels that created some uncertainties in the education sector . The new government
decided to pursue a Bank-financed project at the municipal level, instead of the national level, because of the
availability of other donor resources for the national level (ICR, p. 1). The Bank, with support from the
national government, engaged with the municipal government . The Bank took the earlier national dialogue
and background work into its preparation of the project without fully understanding the municipal realities .
Thus, preparation lacked a deep knowledge of the La Paz education context and municipal norms . As a
result, unfeasible strategies were developed for the Civic Values Program, student assessment, and School
Improvement Plans by the School Management Team . For example, multiple institutions were responsible
for schools: the Ministry of Education was responsible for curriculum; the Municipality was to maintain
infrastructure; and District Offices were responsible for personnel . Also, school committees were limited in
their ability to receive financial resources, making autonomy at the school level more difficult (among other
factors).
Other preparation shortcomings reported in the ICR include : lack of definition of the content and reach of the
Civic Values Program, and lack of alignment in the implementation strategies with municipal realities . The
latter included few firms or universities to provide pedagogical training for the Incentive Program; School
Management Teams could not receive and manage funds; inadequate allocation model for the School
Improvement Plans; and unfeasible proposal to use service agreements with NGOs to run the Pedagogical
Resource Platform.
at -Entry Rating :
Quality -at- Moderately Unsatisfactory
b. Quality of supervision:
Supervision was active and benefited from staff in the field office . Bank staff supported the Implementing
Agency's staff in the areas of financial and procurement management and safeguard protection . At least two
supervision missions were conducted each year . Solutions to implementation bottlenecks were provided by
the Bank team. Supervision addressed many of the design shortcomings . The Bank team worked with the
Implementing Agency to revise project activities related to the Incentive program, Civic Values Program,
Pedagogical Resource Platform, School Committees, and infrastructure maintenance . Some of the
weaknesses in the indicators in the Results Framework were rectified, while others were not (no indicators
were added in to track outcomes related to quality and relevance ). Restructuring of the project took
considerable time and was implemented late in the project . Also, the Bank missed the opportunity to
address the overly ambitious development objectives (ICR, p. 20).
Quality of Supervision Rating : Moderately Satisfactory
Overall Bank Performance Rating : Moderately Satisfactory
9. Assessment of Borrower Performance:
a. Government Performance:
The Ministry of Education decided not to pursue a national secondary education project . When the project
was refocused to the municipal level, the Ministry of Planning and Development approved the subsidiary
agreement (with some delay) to enable the Municipality to assume the credit . During the early stage of the
project, the Ministry of Education and Ministry of Planning and Development backed the Municipality's
efforts. However, delays in the actions of the ministries negatively impacted project implementation . There
was a delay in effectiveness for 14 months, given the time it took the Congress and Ministry of Finance to
approve the agreements. This delay resulted in disbursement lags during the project . The OMDH was
unable to access MOE data (including age information to select participants in the Incentive Program and
information on students' grades to assess project outcomes ).
The Ministry of Education was responsible for curriculum and placement of teachers in newly constructed
classrooms, which were provided in a mostly timely manner . School Councils had to work with departmental
offices of the Ministry to address unmet needs in specific subjects . The Ministry was slow in working with the
Municipality to find legal solution to enable implementation of the School Improvement Plans and in
approving the project's restructuring .
Government Performance Rating Moderately Unsatisfactory
b. Implementing Agency Performance:
The Municipal Human Development Office (OMDH) was the Implementing Agency, instead of a separate
unit. There was strong municipal level leadership and engagement during preparation . OMDH hired a
coordinator and three specialists to strengthen its capacity and ensure adequate attention to supervision . It
monitored activities closely and showed creativity to overcome design shortcomings . OMDH obtained
regular feedback from stakeholders to improve implementation . Construction activities were well supervised,
and improvements in technical specifications were incorporated in subsequent phases . A clear set of criteria
was used to select project -supported schools. The Implementing Agency worked closely with the Bank to
revise project activities as construction costs increased . The ICR notes improvements in administrative
practices such as approval and payment of contracts, and supervision of civil works . Qualified staff were
recruited, which strengthened the Implementing Agency's performance over the lifetime of the project .
Fiduciary management was adequate and complied with covenants .
The municipal bureaucratic process negatively impacted the project, extending the steps for approvals and
thus lengthening the time to implement activities .
Implementing Agency Performance Rating : Moderately Satisfactory
Overall Borrower Performance Rating : Moderately Satisfactory
10. M&E Design, Implementation, & Utilization:
a. M&E Design:
The ICR (p. 7) suggests that the project's outcome indicators were broad and difficult to monitor, as they
required Ministry of Education Data . Indicators were not derived from school data, but instead from across the
Municipality, which was beyond the scope of the project's interventions . There were no indicators to measure
improvements in quality and relevance in upper primary level for grades 7 and 8, but there were indicators to
demonstrate improved permanence . An impact evaluation was proposed .
b. M&E Implementation:
Key performance indicators and targets were changed during the project . These changes were the result of new
baseline data, as well as addressing some of the underlying M&E weaknesses . For example, a new indicator
was added to measure increases in enrollment in project schools . However, during restructuring, no indicators
were added to measure improvements in quality and relevance in upper primary level (grade 7 and 8).
OMDH was responsible for project monitoring and evaluation . The information system that was revised during
the project was used by OMDH for project monitoring purposes . OMDH prepared reports for the Incentive
Program, the Civic Values Program, and the School Improvement Plan . Student learning results were included
with the Incentive Program Report.
The impact evaluation was not conducted due to difficulties in accessing data from the Ministry and the high cost
to carry out the field work to gather student information, in comparison to the program cost . A final report of
project activities was prepared . This report provided descriptions of the project's achievements and the
satisfaction of beneficiaries .
The planned student assessment of learning was not conducted; however, the Ministry examined PISA results
from other countries and then assessed a sample of students from the Incentive Program with items from the
2006 PISA to examine the performance of its students .
c. M&E Utilization:
The ICR reports that OMDH regularly used progress reports and other types of data to evaluate project
implementation. The results from the student assessment were used by the Implementing Agency to identify
specific skill weaknesses of students .
M&E Quality Rating : Modest
11. Other Issues
a. Safeguards:
The project was classified Category B for Environmental Assessment (OP 4.01), since it financed the
construction of schools and rehabilitated classrooms or other facilities in existing sites . The ICR reports that
environmental standards were followed during project implementation . Cards outlining the potential
environmental impacts and mitigation measures were created and reviewed by the Municipal Environmental
Unit. An environmental license was issued . The cards were used by the construction company and civil works
supervisors to guide the construction process to ensure compliance with environmental safeguards .
Given the high concentration of indigenous people living in La Paz (60 percent of the population), the project
was developed in accordance with the Indigenous Peoples Policy (OD 4.20). The entire project is part of an
Indigenous People's Development Plan . During preparation, there was consultation with stakeholders (in May
2007), and their feedback was incorporated into project design . However, the ICR does not specify the group of
stakeholders, their feedback, or subsequent design changes .
b. Fiduciary Compliance:
There were complex municipal payment requirements that created several approval steps . This created delays
in payments (up to 90 days). The delays were associated with the whole municipal payment administration
system, not just that related to the project . The ICR also reports that revised procurement and financial
management guidelines were implemented, which reduced the payment timeline from 90 days to 60 days. The
ICR (p. 8) reports that external audits were conducted annually and were noted without observation .
Procurement was consistent with the arrangements noted in the appraisal document . The Implementing Agency
performed well, and capacity increased over the course of the project . The ICR indicates that the Implementing
Agency had to process many more contracts than were envisioned in the appraisal document . OMDH legal and
procurement teams were strengthened and were able to manage the large number of contracts (ICR, p. 15).
Audits pointed to shortcomings in monitoring some goods acquired through the School Improvement Plans,
which was due to sharing of goods among multiple schools . The Implementing Agency addressed this concern
by adequately documenting the presence of the goods in the schools .
c. Unintended Impacts (positive or negative):
Since the buildings were used by more than one level, there were benefits to students in pre -primary and night
schools.
d. Other:
None
12.
12. Ratings : ICR IEG Review Reason for
Disagreement /Comments
Outcome : Moderately Moderately
Satisfactory Satisfactory
Risk to Development Moderate Moderate
Outcome :
Bank Performance : Moderately Moderately
Satisfactory Satisfactory
Borrower Performance : Moderately Moderately
Satisfactory Satisfactory
Quality of ICR : Satisfactory
NOTES:
NOTES
- When insufficient information is provided by the Bank
for IEG to arrive at a clear rating, IEG will downgrade
the relevant ratings as warranted beginning July 1,
2006.
- The "Reason for Disagreement/Comments" column
could cross-reference other sections of the ICR
Review, as appropriate.
13. Lessons:
The ICR provided several lessons (pp. 19-20), which IEG has synthesized to the following :
Well defined results chains (objectives, activities, and outcome measures ) are crucial . In this case, the
results chain was weak, which meant that there were insufficient activities to achieve some of the
objectives, as well as indicators that were inadequate to measure the objective . The project successfully
met its targets, while using fewer resources than initially planned, but because of design weaknesses, the
project could not attain all of its objectives .
Understanding local legal context and jurisdictional issues is needed during project preparation . The
project's implementation strategies were not adequate for the municipality and required changes from the
start. At entry, the Bank would have benefitted from the views and knowledge of local counterparts and
staff in La Paz, which would have permitted identification of more appropriate strategies than what were
designed.
14. Assessment Recommended? Yes No
15. Comments on Quality of ICR:
The ICR is concise and well written. It provides good analysis of the factors that impacted project
implementation. The ICR is frank on data limitations, which reduced the evidence to judge achievement of
objectives. The ICR's description of project design is scant, as it mainly reports issues in preparation or entry .
The ICR did not report data on one of the project's key performance indicators . In relation to the other key
performance indicators, the ICR reported both original and revised targets, as well as actual values . The ICR
did not weight the ratings to determine the outcome rating, even though the project's key performance outcome
targets were revised. Thus, the ICR has a number of shortcomings, which make it only marginally satisfactory in
terms of quality.
a.Quality of ICR Rating : Satisfactory