Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Report Number: ICRR0023337
1. Project Data
Project ID Project Name
P088966 BR Municipal APL3: Teresina
Country Practice Area(Lead)
Brazil Water
L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD)
IBRD-75230,IBRD-85860 31-Jul-2013 51,914,061.38
Bank Approval Date Closing Date (Actual)
27-Mar-2008 15-Apr-2022
IBRD/IDA (USD) Grants (USD)
Original Commitment 31,130,000.00 0.00
Revised Commitment 51,914,061.38 0.00
Actual 51,914,061.38 0.00
Prepared by Reviewed by ICR Review Coordinator Group
Avjeet Singh Fernando Manibog Ramachandra Jammi IEGSD (Unit 4)
2. Project Objectives and Components
DEVOBJ_TBL
a. Objectives
The Project Development Objective (PDO) is: "To modernize and improve the management capacity of the
Borrower in the financial, urban, environmental, service-delivery, and economic development fields and to
improve the quality of life of the low-income population of the Lagoas do Norte Region." (Schedule 1, page 6
of the Loan Agreement dated July 24, 2008). This was identical to the wording of the PDO in the Project
Appraisal Document (PAD, p.7), with the exception that the “Borrower” was originally stated as the
“Municipality of Teresina.” The PDO did not change, in essence, with this modification.
Page 1 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Based on this statement, this review considers two objectives
i. Modernize and improve the management capacity of the Teresina Municipal Government in the
financial, urban, environmental, service delivery, and economic development fields; and
ii. improve the quality of life of the low-income population of the Lagoas do Norte region of the city.
A split rating was applied to derive the project outcome. Although the PDO remained the same throughout
implementation, the scope of the project and associated PDO indicators and targets were revised to reduce
the level of ambition of the project at the first restructuring in July 2013. The impact of the subsequent
restructurings/ AF does not merit a split evaluation as it aimed to scale up the project scope. The wording of
some intermediate indicators was improved to capture the impact of project interventions better. Therefore,
only one split assessment will be done.
b. Were the project objectives/key associated outcome targets revised during implementation?
Yes
Did the Board approve the revised objectives/key associated outcome targets?
No
c. Will a split evaluation be undertaken?
Yes
d. Components
The project, as appraised included the following three components.
Component 1: Municipal Management Modernization, City Development and Project Management
(US$4.37 million estimated at appraisal; US$19.27 million with Additional Financing; US$8.37 million actual
cost)
This component was intended to: (i) consolidate the Municipal Government of Teresina’s (PMT - Prefeitura
Municipal de Teresina) recent efforts to improve its financial and budget management processes and
capacity; (ii) improve service delivery, with a focus on water supply and sanitation and education (for the
former by bringing PMT into compliance with the new federal WSS law; for the latter through the
development and implementation of a new education policy and strategy for the city); (iii) improve urban and
environmental management capacity and processes; (iv) lead the process and dialogue of the city’s local
economic and social development vision and strategy; and (v) support Project management.
Component 2: Integrated Urban-Environmental Improvement in Lagoas do Norte (US$36.07 million
estimated at appraisal; US$191.02 million with Additional Financing; US$92.43 million actual cost)
This component was intended to undertake urban-environmental improvement interventions in the Lagoas
do Norte (LND) region, including water supply, wastewater collection and treatment, macro and micro-
drainage, recuperation of the natural environmental, urban upgrading, rehabilitation and implementation of
parks and leisure spaces, local road network improvements, resettlement of beneficiary families from areas
of risk, and implementing housing improvements.
Page 2 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Component 3: Social and Economic Development in Lagoas do Norte (US$1.91 million estimated at
appraisal; US$7.81 million with Additional Financing; US$7.29 million actual cost)
This component was intended to support social and economic development interventions in the LDN region,
including strengthening community associations and groups, provision of improved social services (health,
education services, etc.), environmental education, employment, and wealth creation programs, and
community mobilization and participation for works interventions.
e. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project Cost. The original project cost at appraisal was US$44.47 million (US$42.35 million baseline costs
and US$2.12 million physical and price contingencies) with IBRD contribution of US$ 31.13 million. At
project closing, the actual cost was US$108.09 million based on the comments received from the ICR team.
Additional Financing (AF) of US$ 176 million (with IBRD and counterpart financing of US$88 million each)
increased the project costs to US$218.3 million. The actual IBRD disbursement was US$51.91 million.
Financing. The project was financed by two IBRD loans, specifically the original project (P088966,
US$31.13million) and the AF (P146870, US$ 88 million). Respectively, the actual disbursements were
US$31.13 million for the original loan and US$20.78 for the AF, for a total IBRD disbursement of US$51.91
million.
Borrower Contribution. The Borrower committed US$13.34 million under the original project, and US$88
million under the AF. The actual contribution of the Borrower at project closing was US$56.18 million based
on the comments received from the ICR team.
Dates. The project took 14 years to implement. It was approved on March 28, 2008 and became effective
on September 3, 2008. The midterm review was conducted on September 26, 2011. The original closing
date was July 31, 2013 but the project actually closed nearly nine years later on April 15, 2022.
Restructuring. The Project was restructured eight times during implementation, with one additional
financing, four extensions to the closing date, two project scope and results revisions, and one reallocation
between disbursement categories.
First restructuring: (July 15, 2013 - disbursed amount US$ 22.35 million): The restructuring reduced
the project scope, revised the indicators and targets in the Results Framework, and extended the closing
date.
In terms of project scope, a total of six activities were canceled due to limited funding, although
deemed important for modernizing the municipal management and quality of life in LDN. The ICR
notes (para 15) that the cost of most project activities was higher than anticipated at appraisal, which
was further exacerbated by inflation and changes in market conditions that affected Brazil in the first
years of project implementation. Between project appraisal and the 2013 restructuring, there was a
significant variation in the exchange rate between the Brazilian Real (BRL) against the Dollar,
representing a 20 percent reduction in BRL funds in the project. Under Component 1, the activities
relating to the modernization of Teresina’s education management, the development and
implementation of a Local Economic Development (LED) strategy for the Municipality, the
Page 3 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
preparation of a solid waste management master plan, the development of strategies for enforcing
existing legal and administrative mechanisms, and activities related to strengthening the
Municipality’s environmental management capacity were canceled. For component 2, the
restructuring dropped the planned wastewater improvements and reduced the scope of the
drainage, restoration of the lagoons, urbanization around the restored lagoons (including leisure
areas), and resettlement interventions in certain areas. Component 3 was not changed.
The Results Framework was revised as it was deemed overly ambitious, with indicators that were
not directly related to the project interventions. The PDO indicators were reduced from 15 to five.
Three outcome indicators related to municipal legislation and policy development on education,
urban development, and environmental management were dropped, and a new indicator was added
to measure the increase in municipal revenues as a result of the modernization activities
implemented under the Project. Eleven outcome indicators measuring the quality of life in LDN,
including the urbanization index, satisfaction with urban services, access to housing and social
services, and income generation, were dropped. A new indicator on direct project beneficiaries was
added. The intermediate indicators were reduced from 29 to 13, of which three original indicators
were retained, five revised, five new added and 21 dropped. Some of the targets of indicators that
remained were also adjusted to reflect the new scope of the project components.
The loan closing date was extended by 18 months to complete the remaining project activities that
were affected by delays shortly after effectiveness.
Second restructuring: (Nov 26, 2014 - disbursed amount US$28.18 million): The closing date was
extended by 12 months to ensure funds are available for the continuity of the contracting of works and
resettlement processes until the AF was signed and became effective.
Third restructuring: (May 6, 2015 - disbursed amount US$30.57 million): The Results Framework was
revised in anticipation of the AF which aimed to scale up the project scope. The restructuring was
processed separately from the AF due to legal constraints in Brazil preventing the simultaneous processing
of an AF and the restructuring. One indicator related to families resettled was revised and moved from
intermediate to PDO level. The wording of nine intermediate indicators was improved to better capture the
impact of project interventions, four additional indicators were added and three dropped. The targets of
some indicators were also revised to reflect the increased scope of the project activities.
Fourth restructuring: (Nov 18, 2015 - disbursed amount US$30.96 million): The closing date was
extended by 6 months for the timely completion of the procurement and resettlement processes to enable a
successful implementation of the AF operation.
Fifth restructuring/ Additional Financing: (Feb 16, 2016- disbursed amount US$31.05million): The
restructuring provided additional financing of US$88 million in World Bank financing to the project as well as
counterpart financing of US$88 million in order to cover a financing gap in the ongoing loan and enable the
completion of some of the originally planned activities.
For Component 1, the following activities were reinstated as per the original project scope (i)
Strengthening the Municipality’s environmental management capacity and (ii) consolidating and
improving the efficiency of the Municipality’s financial and budget management.
For Component 2, the scope reverted to what was planned under the original project design,
including the implementation of comprehensive and integrated interventions (drainage, sanitation,
lagoon restoration, urban requalification including roads, health center, schools and markets, and
resettlement).
Page 4 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
For Component 3, based on the lessons learned during the implementation of the original loan,
activities related to the prevention of crime and violence, with a focus on vulnerable populations
were incorporated.
The project closing date was extended by 70 months to complete the planned activities.
Sixth restructuring: (Oct 19, 2016- disbursed amount US$31.99million): The funds were reallocated
among the categories of the original loan to increase funding for goods, consulting, and non-consulting
services, and reducing for civil works based on the expenses actually incurred under the loan.
Seventh restructuring: (April 9, 2020 - disbursed amount US$47.82 million): The funds were
reallocated between disbursement categories to finance the costs for cash compensation for resettlement.
Eighth restructuring: (Dec 14, 2021 - disbursed amount US$50.59 million): At the time, the overall
disbursement rate was 43 percent. As a result, most of the works and related resettlements aimed at
reducing flooding and restoring the lagoons had not been carried out. Given the age of the project, the Bank
assessed another restructuring was not viable, and the PDO could no longer be fully achieved. The closing
date was extended by four months to allow the Municipality to orderly close the Project and significantly
reduce the pending safeguards commitments. This was the last extension.
3. Relevance of Objectives
Rationale
Country and Sector Context: At appraisal, the city of Teresina had an estimated population of 800,000
residents and had the country's third-highest poverty rate. The city also ranked the third lowest under the
Human Development Index. Support to Teresina was strategically important for the Bank given it was the
capital of State of Piauí, one of Brazil’s poorest and least developed states. The city historically attracted
migrants, generally poor, that settled in low-lying areas on the city's outskirts which lacked basic
infrastructure and services, thus leading to a poor quality of life. The project’s focus area, the Lagoas do
Norte (LDN) region of Teresina was particularly vulnerable since it was located in a hydrologically sensitive
area and inhabited by low-income settlements. The empty spaces between the lagoons in the region had
been gradually occupied over the years, with drainage channels being covered over and new landfills
created. The lack of basic infrastructure, such as housing, water supply, wastewater collection and
treatment, drainage and solid waste collection had a negative impact on the local environment.
The frequent flooding and precarious socio-economic conditions in this lagoons region had set it apart from
the rest of the city. The diagnosis undertaken during Project preparation showed that PMT has been unable
to deal efficiently with the challenges posed in this region. Peri-urban areas with high population growth had
high demand for fundamental education. The rate of capacity utilization of municipal schools was relatively
low. A robust public financial management system was deemed essential for the implementation of policies
to support fiscal discipline, strategic allocation of resources, and efficient service delivery.
Country Strategy: The Project design was responsive to the Government of Brazil’s multi-year plan (2004-
2007), which focused on social inclusion and universal access to high-quality public services,
macroeconomic stability, greater equity among regions through regional and local development, and better
Page 5 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
integration between development and environment [PAD, para 18]. The Project also supported the Growth
Acceleration Plan (PAC) 2007-2017, which prioritized the modernization of public sector management,
urban upgrading, and restoring macro-fiscal sustainability. The project objectives were also aligned with the
Government of Brazil’s most recent National Urban Development Policy (2021) that aims to promote social
equality and sustainable urban development in the municipal space [ICR, para 24].
Bank Strategy: The PDOs were aligned to the Bank’s 2004-2007 Country Assistance Strategy (CAS) –
recognizing the importance of the sub-national level in economic growth and social equity, as well as
relevant to Bank Country Partnership Strategies (CPS, 2008-2011 and 2012-2015), that called for:
continued lending to sub-national governments; improved quality and expanded provision of public services
for poor households; increased efficiency of public (and private) investments; promoting regional economic
development and improving sustainable natural resources management. The PDOs remained relevant to
the Country Partnership Framework (CPF) for Brazil for FY18–FY23 and contribute to (i) Focus Area 1
(Fiscal consolidation and government effectiveness), objective 1.1 on strengthening fiscal management
(including public financial management) at all levels of government, (ii) Focus Area 2 (Private Sector
Investment and productivity growth), objective 2.3 on mobilizing greater investment in infrastructure to
improve services, including through Public Private Partnerships and (iii) Focus area 3 (Inclusive and
Sustainable Development), objective 3.2 on providing more inclusive and sustainable urban services.
Bank experience: The Project was the third in a series of eight similar Bank-supported municipal
operations under the Brazil Municipal Lending Program APL and designed to implement local level policies
to improve the lives of the urban poor and promote sound local governance and economic competitiveness.
This included direct lending to municipalities to support their increased autonomy with capacity building,
infrastructure, and service delivery investments. The Project reflected three of the four pillars of the Bank’s
Municipal and City Strategy for Brazil: improving municipal, fiscal, and administrative management;
increasing competitiveness through local economic development; and strengthening municipal service
delivery. In addition, the Bank had an active portfolio in Brazil at appraisal largely aimed at maintaining a
focus on access to urban services at the subnational level in times of fiscal strain. This growing portfolio of
projects had contributed to a broad understanding of the issues facing municipalities and the identification
of appropriate mechanisms to assist them to realize their role in contributing to the national objective of
socially equitable economic growth and development.
The PDOs were directly relevant to the past and current strategies of the Bank and the
Government. However, a significant shortcoming was that the PDOs were over-ambitious relative to the
institutional capacity of the Municipality. The capacity building objective spanned five sectors of government
– urban, fiscal, environmental, service delivery (water, education), and economic development. The scale
and type of activities could not capture the high level of the capacity-building objective. Moreover, the ICR
noted that there was a lack of clarity on the definition of some of the PDO outcomes. Despite the reduction
in original ambition and scope during project implementation, the objectives were not revised accordingly.
While acknowledging the multiple, interrelated problems of urban poverty, a series of operations with
focused objectives (for example, improve access to services, strengthen flood risk management, improve
fiscal management, etc.) consistent with the long-term vision of the Municipality would have been a more
realistic approach in the Teresina context
Rating Relevance TBL
Rating
Modest
Page 6 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
4. Achievement of Objectives (Efficacy)
EFFICACY_TBL
OBJECTIVE 1
Objective
Modernize and improve the management capacity of the Teresina Municipal Government in the financial,
urban, environmental, service delivery, and economic development fields.
Given the interlinkages between ‘modernization’ and ‘improvement of management capacity’, and in the
absence of a clear definition in the PAD, these are evaluated concurrently.
Rationale
Theory of Change: Since a theory of change (TOC) was not required for the PAD, the ICR constructed an
ex-post TOC by analyzing the interlinkages between the activities, and the Results Framework. The first
objective was to be attained through TA and capacity building interventions across the five sectors in the
PDO. First, the support to financial management included activities to implement information technology
platforms and databases and to provide training for the PMT. With these interventions, it was expected that
the financial system would be more reliable, allow for faster internal processing, and better financial and
budget planning. This would likely lead to an increase in the collection efficiency of municipal revenues.
Second, the support to urban management included activities to contribute towards the preparation of the
city’s municipal urban master plan, transport master plan, drainage master plan, and solid waste
management plan, as well as to update the laws to accompany the implementation of the urban planning
instruments. Third, the environmental management activities included strengthening the capacity to
introduce environmental licensing and enforcement. Fourth, the support to service delivery included TA to
develop the municipal WSS plan, operationalize the Municipal Public Services Regulatory Agency (ARSETE),
and develop a service provision contract consistent with federal law. These planning and regulatory tools
were expected to establish clear performance targets and management mechanisms including the signing of
a contract between the private player and the municipality. The contracting to a private entity responsible for
WSS infrastructure investments and service delivery was expected to lead to improved WSS services. The
service delivery aspect also included the modernization of education management by supporting PMT in the
elaboration and implementation of a new education policy that aimed to rationalize and bring more efficiency
to education services. Fifth, the local economic development included activities to assist PMT in the
development and implementation of local economic development strategy for Greater Teresina. The adoption
and implementation of urban, environmental, education, and local economic development policies/ planning
instruments were expected to contribute towards strengthening municipal effectiveness for efficient and
sustainable service delivery.
The ToC has significant shortcomings. While the modernization of the FM systems and WSS service delivery
focused on actual capacity building and the causal pathway towards the outcomes was clear, other services
included in the PDO (urban, environment, education, and local economic development) only focused on the
policy level and were closer to the output level, rather than the outcome level. Given there was no sequencing
of activities, it is not evident that the capacity building activities directly supported the achievement of the
second objective which focuses on investments for improved quality of life. The project’s TOC was weak as
the project was intended to advance on all fronts simultaneously, and the causal link presented in the TOC
Page 7 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
was less evident. There were no critical assumptions highlighted in the ICR. Another shortcoming in the TOC
was that the PDOs were presented as ‘Higher-level outcomes’ in the TOC rather than as outcomes.
OUTPUTS
Financial
A management information system for administrative, financial, and human resource management
was implemented
Internal control systems were partially implemented
A unified system for the management of human resources was implemented
An electronic invoice and receipts system to facilitate tax revenue collection was implemented
Service delivery
A service delivery framework compliant with the Federal Water Supply and Sanitation (WSS) Law was
established. This included the development of the WSS plan, restructured regulatory agency, and
formalization of service delivery contractual arrangement with a private operator. This was reflected as
an outcome indicator under the PAD.
A service delivery contract between the PMT and Piaui Water and Sanitation Company (AGESPISA)
was signed initially. Subsequently, a concession agreement was signed with Águas de Teresina, a
private entity for expanding and improving WSS infrastructure services in 2017. The new operator
further invested in updating the city's WSS master plan and introduced some innovative approaches to
the design of the wastewater treatment plants to enhance their efficiency.
The development and implementation of the new Municipal Education Policy were dropped.
Urban
A municipal drainage master plan was developed, approved, disclosed, and adopted
The following outputs were dropped (i) Development of Municipal Urban Master Plan; (ii) Updating of
urban legislation; (iii) Development of Municipal Solid waste management plan
Environment
Restructuring plan for the municipal environmental secretariat was developed and adopted. Priority
interventions implemented by project closure included the upgrading of the office and field equipment
and administrative restructuring of the secretariat. The expansion of the secretariat offices and the
establishment of software for operational procedures were yet to be completed at project closing.
The updating of Environmental legislation was dropped.
Economic Development
The elaboration and implementation of the Local Economic Development Strategy was dropped.
Page 8 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
OUTCOMES
Financial
Municipality aggregate PEFA score increased from C+ to B as against the original target of B+. A
higher PEFA indicated a stronger public financial management system, with effective planning and
efficient budget allocation and use (ICR, para 27). The ICR notes that despite the decrease in the
budget predictability (in part due to the context of COVID-19 pandemic) and inspection and external
auditing arrangements indicators, PMT improved in comprehensiveness and transparency, Public
Financial Management aspects, including budget and revenue management, and accounting
framework, records, and reporting.
Service Delivery
Improved financial and operational efficiency of WSS services – the indicator was dropped. The ICR
team shared with IEG that discussions with beneficiaries indicate improvements in customer services
and hours of water supply. With regards to financial efficiency, the PMT staff reported improvements
but the team clarified that there was no financial data received from the service provider to validate the
findings.
The new operator connected more than 1500 households in LDN to the sewerage system (ICR, para
42).
Urban
The Urban Drainage Master Plan developed is being used by the PMT as the guiding instrument for
urban drainage interventions in the city, including procedures for land allocation and use, and
construction norms. (ICR, page 38)
Environment
No outcome was reported. The indicator on ‘Decrease in time to obtain PMT environmental approvals
was dropped’
Economic Development
No outcome was reported.
Assessment of the original project:
The ICR did not provide any direct evidence on the nature and depth of Borrower’s capacity improvements in
different areas except for Financial Management and WSS service delivery. Priority was given to the
modernization and implementation of the FM systems, WSS, and drainage service delivery capacity while
other services included in the PDO only focused on the policy level. It was not clear how effective the TA and
capacity building activities financed by the project were in terms of improving borrowers’ capacity with respect
to urban, environmental, and local economic development. The ICR did not provide any beneficiary
Page 9 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
assessment on this matter from the borrower side, nor provided robust evidence on the improvement of any
services. Given the weak evidence of outcomes, the efficacy of the first objective is rated Modest.
Rating
Modest
OBJECTIVE 1 REVISION 1
Revised Objective
Modernize and improve the management capacity of the Teresina Municipal Government in the financial,
urban, environmental, service delivery, and economic development fields.
Revised Rationale
During the 2013 restructuring, education, urban, and environmental management activities were significantly
scaled down. The related outcome indicators were dropped, and even though some of the activities were
partially reinstated as part of the 2016 AF, the ICR did not provide any evidence of the outcomes of these
interventions. Project interventions were downscaled to FM improvements, WSS, and drainage service
delivery capacity. For FM, the project met the following indicators/ targets introduced as part of the
restructuring.
Outcomes
Municipality aggregate PEFA score increased from C+ to B as against the original target of B+ and
revised target of B.
Municipal revenues (tax and primary revenues, excluding transfers and extraordinary revenues)
indicated a 437 percent real growth with the establishment of an electronic invoice and receipts
system under the project. This indicator was introduced as part of project restructuring in 2013 with an
original target of 35% and the target was revised to 50% during the 2016 project restructuring as part
of the AF. The ICR team clarified to IEG that even though the introduction of the electronic invoice
system under the project did contribute towards increased collections, there were other external
factors also that contributed toward the achievement of the significant increase in municipal revenues.
This includes the passing of the Fiscal responsibility law (Lei de Responsabilidade Fiscal) which made
tax collection procedures mandatory.
Municipal revenue spent on staff costs reduced from 50 percent to 47.3 percent (against the target of
48 percent).
Even though the project met the revised targets for improved financial management, given the weak evidence
of outcomes with respect to the urban, education, and economic development aspects, the above rating of
Modest holds.
Revised Rating
Modest
Page 10 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
OBJECTIVE 2
Objective
Improve the quality of life of the low-income population of the Lagoas do Norte region of the city.
Rationale
Theory of Change: The activities included a combination of the following interventions in the LDN region of
the city (i) undertaking of urban-environmental improvements, including water supply, wastewater,
environmental restoration of lagoons and river banks, urban upgrading, construction of parks and community
facilities, housing and road network improvements, and resettlement of beneficiary families from areas of risk;
and (ii) social and economic development interventions, including the provision of improved health, education
services, employment programs, and community mobilization for works interventions. It was expected that
these activities would result in (i) increased access to water supply, wastewater, and drainage services,
improved roads, housing, and restored lagoons and riverbanks; and (ii) increased coverage of households
under the health, education, and employment programs. These intermediate results were expected in turn
to lead to improved quality of life as reflected in an increase in population satisfied with (a) urban services,
and social services and the quality of the local environment. The investments were also expected to
contribute to enhanced property values, increased access to income and employment, and an increase in
children receiving pre-schooling. It was expected that raising the well-being of some of the country’s most
vulnerable populations would contribute to the higher-level outcome of a more equitable, sustainable, and
competitive region as outlined in the Government’s vision.
Although the original results framework appeared to reflect the TOC and was appropriate to measure the
outcomes being sought, there were some significant shortcomings. While the activities appeared appropriate
to achieving the desired outcomes, it should be noted that the TOC discussion in the ICR does not specifically
analyze whether they were of an adequate scale and if all the outcomes (e.g., increase in property value)
could be attributed to the interventions or achieved within the project duration. The lack of assumptions and
reflection of the PDO at a ‘higher-level outcome’ in the TOC as discussed in the previous objective also holds
here.
OUTPUTS
766 families (against the target of 668) benefited from upgrading/improvements of their current
household (114% achieved).
22,636 people (against the target of 60,000) were provided with access to ‘improved sanitation
facilities” under the project (38% achieved)
8,848 new household sewerage connections (against the target of 14,000) were constructed (63%
achieved)
16.28 Km (against the target of 58km) of non-rural roads rehabilitated (28% achieved)
4mg/l reduction in the pollution levels of the lakes (dissolved oxygen) against the original target of
7.58mg/l. (53% achieved) Target refers to measurements in Lagoons Cabrinha, Lourival, and
Pantanal.
3.31 m2 per inhabitant (against the target of 5m2/per inhabitant) areas were protected and recovered
for leisure and collective use (66% achieved)
Page 11 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Only the target of a number of families benefitting from upgrading/improvements was achieved
(exceeded). All the other targets were not achieved. No data was available for the following indicators since
they were subsequently dropped in the 2013 restructuring.
Reduction in the incidence of water- and excreta-related disease transmission (episodes/year)
Reduction in travel time between the city center and the Lagoas do Norte region
Annual reduction in the number of road accidents in the region.
Improvement in water supply services: (Continuity; Pressure; Water Quality)
Increase in eligible families registered in income transfer projects
Increase in population with personal employment documentation
Increase in access to job/income creating projects
Increase in working-age adults benefiting from work/skills training
Increase in new small business set up
Increase in children attending pre-school
Increase in handicraft sales
OUTCOMES
1056 households (out of 2699) living in flood-risk areas were resettled as part of the project. This
indicator was originally included as an intermediate indicator in the RF and moved to the PDO
indicator as part of the AF in 2016.
80% of clay workers (oleiros) working informally were reinserted into the formal work market.
No data were reported on the following outcome indicators which were subsequently dropped in the
2013 restructuring:
Increase in PMT’s standard urbanization indicator (IPMU – ‘Índice de Padrão Mínimo de
Urbanização’)
Increase in property values (hedonic pricing) in the Lagoas do Norte region
Increase in population reporting improvements in / satisfaction with (i) urban services (WSS, drainage,
solid waste management, public lighting, local transport, etc.); (ii) social services (health, education,
job, and training programs, etc.), and (iii) quality of the local environment
Increase coverage of the Family Health Program from the Lagoas do Norte region
Increase coverage of the Bolsa Familia Program for eligible families from the Lagoas do Norte region
Population with personal and work identification documents
Access to income and work generation programs (Banco Popular, micro-credit operations in the north
region of Teresina)
People benefitted from technical training
Number of new businesses started
Percentage of 0-6 old children receiving preschool education
The project did not demonstrate any progress in achieving the PDO indicators for the “quality of life” objective
as originally interpreted from the outcome indicators. The efficacy rating for this objective during the first
period is rated Negligible.
Page 12 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Rating
Negligible
OBJECTIVE 2 REVISION 1
Revised Objective
During the first restructuring, significant modifications were introduced. Activities related to planned
wastewater improvements were dropped and the scope of the drainage, restoration of the lagoons,
urbanization around the restored lagoons (including leisure areas), and resettlement interventions in certain
areas were reduced. Eleven outcome indicators measuring the quality of life in LDN, including the
urbanization index, satisfaction with urban services, access to housing and social services, and income
generation, were dropped. The ICR team clarified that some indicators were dropped since they were
deemed to be ambitious and affected by multiple factors outside the project scope (such as the increase in
property prices and the increase in coverage of Family Health programs). In some instances, the indicators
were dropped since the corresponding activities were dropped (e.g., education) or were difficult to measure.
Post-restructuring, improved quality of life was defined narrowly to be people benefiting from any single or
combined set of improvements in urban services (WSS, drainage, roads, housing, and environmental
restoration) as well as the families living in flood-risk areas that were resettled given the lack of a clear
definition (ICR, para 32).
Revised Rationale
OUTPUTS
In addition to the outputs mentioned under the original objective, the following outputs were achieved.
4mg/l reduction in the pollution levels of the lakes (dissolved oxygen) against the revised target of
5mg/l (80% achieved). Target refers to measurements in Lagoons Cabrinha, Lourival, and Pantanal
5,026 number of people attended the Upgraded Theater Teatro do Boi on average in the last five
years (against the target of 5000). This average considered zero attendance in 2020 due to the
COVID-19 restrictions. Without the 2020 null attendances, the average is at 6,283 (105% achieved).
225 socio-environmental activities implemented (against the target of 80). These include sports
events, fairs, summer camps, community cultural events, and music shows. (181% achieved)
Gender aspects were mainstreamed into the municipal violence prevention plan. The Plan
incorporates actions to prevent gender-based violence, including training and sensitization of public
officers (teachers, medical personnel, police, etc.) on gender-based violence, sensitization of the
youth, and services to assist victims of gender-based violence.
OUTCOMES
Post-restructuring, an additional PDO indicator on direct project beneficiaries was introduced. The progress
was still less than envisaged, and the direct project beneficiaries reached 78% of its target. 71,784 people
(out of 92,000 targeted) directly benefited from various urban upgrading, drainage, water and sanitation, and
environmental restoration improvements under the Project in the neighborhoods of São Joaquim, Poti Velho,
Olarias Mocambinho, Matadouro, Nova Brasilia, Parque Alvorada and Vila São Francisco. The percentage of
female beneficiaries was 53 percent (against the 52 percent planned). During the AF, the intermediate
Page 13 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
indicator of “households living in flood-risk areas resettled to new housing” was moved to to the PDO indicator
level. The families resettled was low at 39% with only 1056 households (out of 2699 targeted) resettled as
part of the project.
Rating: The project’s efficacy in achieving PDO 2 during this second period is rated Modest. Assessment of
quality-of-life improvements is based on a narrow definition of the relocation of flood-risk families and people
benefitting from access to urban services. However, in the narrower definition of the “improved quality of life”
after restructuring, the inclusion of “any single” improvement in urban services does not seem appropriate,
since it masks the failure to achieve the delivery of urban services in a comprehensive manner. In other
words, providing water access, for example, while maintaining inadequate drainage, poor roads, insufficient
or poor quality housing, and degraded environmental conditions would erroneously be accepted as a signal
that the PDO of improving quality of life has been achieved. Although no beneficiary satisfaction survey was
carried out, the ICR reported visibly improved living conditions in the new areas and many of the resettled
people reported satisfaction with their new housing (ICR, para 62). The validity of visual inspection and verbal
anecdotes as evidence of PDO achievement was not assessed in the ICR. In addition, only part of the
planned hydrological system restoration and environmental upgrading activities was implemented, thus
limiting the functionality of the system as a reliable flood risk management mechanism.
Revised Rating
Modest
OVERALL EFF TBL
OBJ_TBL
OVERALL EFFICACY
Rationale
Assessment of the revised project:
Based on the sub-ratings presented above, the ratings for each restructuring period assessed under the split
evaluation are as follows:
From effectiveness to first restructuring on July 15, 2013: Negligible Toward the first objective of
“modernizing and improving the management capacity of the Teresina Municipal Government in the financial,
urban, environmental, service delivery, and economic development fields”, results were achieved in the areas
of financial management and service delivery, and there was very limited progress for the urban,
environmental, and economic development fields. For the second objective related to “improving the quality of
life”, there was negligible progress on several key activities and outcome indicators that were eventually
dropped. Given the low achievement of results under the first objective and negligible achievement under the
second objective, the overall efficacy is rated as Negligible under the original design of the project.
Overall Efficacy Rating Primary Reason
Negligible Insufficient evidence
OBJR1_TBL
Page 14 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
OVERALL EFFICACY REVISION 1
Overall Efficacy Revision 1 Rationale
From July 2013 restructuring till April 15, 2022, when the project closed: Modest
Toward the objective of “modernizing and improving the management capacity of the Teresina Municipal
Government in the financial, urban, environmental, service delivery, and economic development fields”,
results were achieved in the areas of financial management and service delivery, and there was very limited
progress for the urban, environmental and economic development fields. The revised target for financial
management was met, though there were external factors also that contributed to achieving the results of the
significant increase in municipal revenues. For the second objective related to “improving the quality of life”,
the rating is modest due to limited achievement of results based on the new albeit narrow definition of ‘quality
of life’. Given the modest achievement of results under both objectives, the overall efficacy is rated as Modest
post-restructuring.
Overall Efficacy Revision 1 Rating Primary Reason
Modest Insufficient evidence
5. Efficiency
The ex-ante economic analysis at the time of the appraisal of both the original project and the AF was
conducted using a cost-benefit analysis. Economic benefits were quantified for the (i) urban improvements using
the hedonic price approach to estimate the properties’ appreciation; (ii) water and sanitation improvements using
the willingness to pay approach; (iii) improving the Teatro do Boi, and the municipal management modernization
by estimating the increase of revenues. The projection for costs and benefits for the assessments was
calculated based on a 30-year period, discounted at a rate of 10% per year. The present value of net benefits
was US$29 million, and the overall expected internal rate of return (EIRR) was 18 percent.
The ICR used the same methodology as in the PAD and the Additional Financing in updating the project’s
economic internal rate of return (EIRR) and NPV. The ex-post economic analysis included a cost-benefit
analysis using the same lifetime period and discount rates used at appraisal. Actual costs and benefits were
transformed into prices at the time of appraisal to eliminate currency fluctuation and make them comparable
given the significant depreciation of the Brazilian Real against the US$. The investment cost of interventions
whose benefits were quantified (urban upgrading, sanitation, modernization of Municipality’s financial system,
and Teatro do Boi) corresponded to 53 percent of all investments implemented under the project. The ex-post
analysis also included the cost of the other 47% of investments whose benefits were not quantified
(rehabilitation of facilities and parks, roads, and resettlement). Despite the reduced scope and even though the
investments were not fully implemented, the ex-post estimates yield a net benefit of US$2 million and an ERR of
16%, ranging from 13% (water and sanitation) to 52% (municipal management modernization). Actual results
were lower than expected (18 percent ERR) yet positive.
Administrative and Operational Efficiency:
There were significant operational and administrative inefficiencies. The project implementation period was 14
years. There were 3 restructurings prior to the AF to extend the closing date by 36 months due to the slow start
of implementation, procurement delays, low institutional capacity, and delays in the disbursement of counterpart
funds. The AF further extended the closing date by 70 months to allow time to complete sub-projects that faced
Page 15 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
delays as well as those under the enhanced scope of work. However, the extension did not deliver the expected
results and at the time of closing a substantial portion of infrastructure had yet to be completed and less than
half of the project funds has been disbursed. A final extension of four months was given in December 2021 to
enable the PMT to complete the priority outstanding safeguards activities and close ongoing works contracts.
Also, the planned project management costs (US$ 500,000) were exceeded by almost four times (>US$ 1.8
million) due to the extended project duration. With an average of US$153,000/per year in project implementation
support, the Bank invested about US$ 2.1 million in project management, twice as much as the average cost for
projects of similar nature within the APL (ICR, para 37). The project failed to strengthen PMT’s capacity
sufficiently despite the costs incurred over 14 years of project implementation. The value for money was not
achieved.
Overall, while the economic benefits of the project investments proved adequate despite the reduced scope, the
project efficiency is rated Modest, due to significant administrative and operational inefficiencies.
Efficiency Rating
Modest
a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal
and the re-estimated value at evaluation:
Rate Available? Point value (%) *Coverage/Scope (%)
0
Appraisal 18.00
Not Applicable
0
ICR Estimate 16.00
Not Applicable
* Refers to percent of total project cost for which ERR/FRR was calculated.
6. Outcome
Overall outcome rating pre-restructuring: The project had major shortcomings. The Relevance of Objectives
is rated Modest given the ambitious PDO relative to the institutional capacity of the municipality, the lack of
clarity of definition of outcomes, and not revising the objective despite the reduction in the ambition and scope of
the project. Given the low achievement of results under the first objective and negligible achievement under the
second objective, the overall efficacy is rated Negligible under the original design of the project. Efficiency is
rated Modest due to significant administrative and operational inefficiencies. The combined outcome rating is
Unsatisfactory.
Overall outcome rating post-restructuring: The relevance of Objectives is rated Modest as discussed above.
Achievement of the objective is rated Modest based on limited evidence of achievements and taking into
account the revised albeit narrow definition of the ‘improving the quality of life’ objective. Efficiency is rated
Page 16 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Modest due to significant administrative and operational inefficiencies. The combined outcome rating is
Unsatisfactory.
Overall outcome - Unsatisfactory
According to IEG/OPCS harmonized guidelines, a split assessment (for efficacy) was considered to derive the
overall project outcome rating by weighting the proportion of the IBRD funding that was disbursed before and
after the restructuring. Since the outcome ratings are Unsatisfactory both before and after restructuring, the
overall outcome rating is Unsatisfactory.
a. Outcome Rating
Unsatisfactory
7. Risk to Development Outcome
The ICR discussed (paras 70-71) three main risks that could potentially impact the development outcome.
Stakeholder risks: The non-completion of activities and lack of adequate stakeholder engagement could
lead to a lack of trust towards the PMT from the communities that did not benefit from the project. Any re-
engagement and developments in LDN in the future might be challenging, and costly from a social
perspective.
Institutional Capacity: While the project did build PMT’s capacity during project implementation, it is still not
sufficient to manage infrastructure projects and mobilize the communities effectively. There is a need for
continued commitment from PMT along with strong support from the government.
Environmental risk: There is the risk of renewed flooding since the lagoon system draining the LDN was
not restored as designed. The increased densification of informal settlements in the lagoons further
exacerbates this risk. The ICR team shared that the completion of the contingency plan in case of dikes
failure is still pending finalization by the Municipality of Teresina and is expected to be approved by the end
of the fiscal year.
8. Assessment of Bank Performance
a. Quality-at-Entry
The current project was one of eight similar projects in Brazil, under the Municipal Lending Program.
While the strategic relevance of the Project was well captured by the Bank team during the Project
appraisal, insufficient attention was given to the institutional context and capacity of the Teresina
Municipality to manage a complex project with limited resources. The Project’s physical design was too
demanding in practice for a single, five-year operation and existing capacity. The project activities were
not aligned sufficiently with the over-ambitious PDO; particularly the scale and type of activities could not
capture the high level of the capacity-building objective which spanned five sectors of government.
Page 17 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Moreover, there were no efforts to integrate early lessons based on other Municipal APLs under
implementation (ICR, para 46). The lessons reflected in the PAD were partially integrated into the design,
i.e., avoiding overly complex objectives and a ‘Christmas tree’ approach in design (PAD, page 11). The
ICR acknowledges that the project was not ready for implementation at Board approval (ICR, para 47).
The project cost estimates at appraisal were based on preliminary assessments and engineering designs
were not ready. There were significant weaknesses in the M&E design (see Section 10 below), which
made the evaluation of the achievement of the development objective difficult.
Given major shortcomings in the Bank’s performance at entry, the Quality at Entry is rated Unsatisfactory.
Quality-at-Entry Rating
Unsatisfactory
b. Quality of supervision
The Bank team was responsive to challenges that arose during the project implementation period,
including the rise in costs and the varying degrees of readiness in the sectors. Regular implementation
support missions were conducted, and technical experts were mobilized to assist the PMT on critical issues
pertaining to project activities. World Bank fiduciary and safeguards team provided advisory and technical
support to the PMT, particularly the PIU, on procurement, financial management, and safeguards
compliance with World Bank requirements. The Bank team tried to address capacity issues with training
and offering consultants to ensure compliance with procurement and safeguard requirements. The Project
restructurings were utilized to address design weaknesses and financing gaps.
However, despite these efforts and a lengthy implementation duration of 14 years, works were not
completed on time. The first restructuring with adjustments to the project design was made two weeks
before the original closing date.
There was significant TTL turnover towards the project closure (3 TTLs in the last four years), and the
Borrower’s comments note the change in TTLs during the project duration hindered the project
performance. (ICR, page 60). Even though the project was restructured eight times, the Bank team did not
adequately address the shortcomings in the M&E design. The ICR notes that in the early years of project
implementation, the ISRs and Aide Memoires were relatively lean and did not provide consistent progress
reporting on all project activities. ISRs ratings fluctuated from S to MS throughout most of the project
period, indicating a lack of candor in reporting. The ICR acknowledges that the Bank team could have been
more cautious in assessing the capacity and safeguards risks associated with tripling the financing
envelope and significantly scaling up the project scope, as well as risks associated with counterpart
financing, considering the lessons from the earlier phase of implementation. Despite a court case on a
land dispute and earlier resistance to the proposed resettlement by some groups of affected people, the
stakeholders' engagement was not adequately strengthened. Further, the ICR notes (page 27) that the
Bank's delayed decision (~10 months) to close the project led to the non-completion of some project
activities that could have been advanced if there had been clarity on the remaining implementation period
and could have potentially benefited more people.
Page 18 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Given significant shortcomings in the Bank’s performance at supervision, the Quality of Supervision is rated
as Unsatisfactory.
Quality of Supervision Rating
Unsatisfactory
Overall Bank Performance Rating
Unsatisfactory
9. M&E Design, Implementation, & Utilization
a. M&E Design
There were major weaknesses in M&E design. The PDO was complex with a focus on five sectors, in
addition to improving the quality of life of the targeted population. Moreover, there was a lack of clear
definitions of the PDO aspects. The Borrower’s ICR (ICR, page 61) states that the “definition of each
Indicator was at the interpretative criterion of the World Bank TTL...the meanings remained divergent for
the long term, representing an obstacle to concrete measurements”. The results framework at design was
extensive, with 44 indicators, many of which were not directly attributed to the project. Many of the PDO
and intermediate indicators lacked specificity in the definition and methodology used to assess progress.
The RF did not clearly distinguish between outputs and outcomes. For example, some outputs such as
people benefitting from technical training, and the formalization of WSS service delivery contracts were
proposed as outcome indicators concurrently with their outcome of increased access to income or
intermediate results of actual improvements in service delivery. For the first objective regarding capacity
building, most indicators focused largely on outputs at the policy level (e.g., education policy implemented,
urban legislation update) without linking this to the outcome of say, better municipal performance or service
delivery.
Further, some indicators (such as the percentage of people reporting improvements/satisfaction with
access to services and quality of life, etc.) would require specific periodic surveys that were not planned for
within the project. There were no baselines available for many indicators. The Borrower’s capacity to
undertake M&E was overestimated, and no structured support or budget for M&E was built into the design.
b. M&E Implementation
Some of the shortcomings in the M&E framework as designed were addressed during project
implementation, particularly during the 2013 restructuring under which PDO indicators and IR indicators
were significantly reduced, and some were revised to be more measurable and attributable to the project
activities. However, no revisions were made to clearly define the PDO and its scope. Even though some
of the activities such as local economic development strategy were dropped, the PDO was not revised.
Some of the indicators/ targets were further revised during the Additional Financing. However, these
revisions did not adequately address some of the shortcomings mentioned in the M&E design section.
The project’s implementation monitoring and updates to the RF were done through the information
provided by the PMT. The ICR notes (ICR, para 56) that data collection, analysis, and reporting were not
Page 19 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
systematized throughout project implementation, and reporting periods differed significantly between
indicators. At project closure, there was still a lack of clarity on the level of achievement of several
indicators.
c. M&E Utilization
The project-generated data could have been used more systematically to support implementation and
derive lessons to inform some implementation changes. For instance, during the early years of project
implementation, project performance was consistently rated satisfactory, despite the lack of evidence of
progress on the ground. The surveys that would have contributed to monitoring progress toward the
achievement of overall objectives were dropped. Even though the M&E data enabled the Bank and the
PMT to take necessary corrective actions through project restructurings, these were not enough. M&E
for this project remained one of its weakest aspects throughout the project’s lifetime.
Overall, the M&E quality is rated Modest. The M&E system as designed and implemented was not
adequate to assess the achievement of objectives and consequently restricted the utilization of M&E
findings to shift the project’s direction to achieve its objectives.
M&E Quality Rating
Modest
10. Other Issues
a. Safeguards
The project was classified as Category A. Following safeguard policies were triggered: OP 4.01
(Environmental Assessment); OP 4.04 (Natural Habitats); OP 4.11 (Physical Cultural Resources); OP 4.12
(Involuntary Resettlement); and Safety of Dams (OP 4.37).
Environmental Safeguards
At appraisal, in accordance with Environmental Assessment (OP 4.01), an Environmental Assessment
(EA), including an Environmental Management Plan was prepared. A public consultation process was
carried out and documented. The relevant safeguard policy documents related to the Project were sent to
InfoShop for the requisite disclosure on March 30, 2007, and simultaneously disclosed by PMT in the
country (PAD, page 21). Natural Habitats (OP4.04) policy was triggered because of the Project’s
proposed interventions in the lagoon and river ecosystems. Per ICR, the sub-projects generally complied
with all the triggered World Bank safeguards and national environmental policies (ICR, para 60).
Environmental issues were addressed in accordance with the EMP prepared for the project. The Project did
not affect any critical habitat, and project interventions improved the environmental health of some of the
lagoons and surrounding areas, with reduced environmental pollution from wastewater. By the Project
closure, the project had three pending environmental safeguards actions concerning uncompleted works,
such as restoration of the natural vegetation, embankment protection, and clean and safe disposal of
contraction waste. Per ICR (para 60), these pending actions were assessed as temporary and specific in
nature, and the post-closure action plan by the Municipality of Teresina was deemed adequate. The ICR
Page 20 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
team shared (via email dated April 18) that all actions were implemented except one related to the dikes
contingency plan that is close to being completed.
OP 4.11 on physical and cultural resources was triggered given project intervention on the restoration of
the Teatro do Boi theatre and enhancement of the Parque de Cabeça de Cuia municipal park located at the
confluence of the Poti and Parnaiba Rivers. The ICR team noted that the project was in compliance with OP
4.11. The project consulted with cultural leaders as of 2014 and held public consultations with different local
entities. The project assessed physical cultural resources related to the Afro-Brazilian groups present in
Lagoas do Norte, and commissioned a stand-alone anthropological study, completed in 2018. The
inspection panel carried out an investigation in 2020 and reviewed the implementation of OP4.11 as part of
this process. It considered the project in compliance with the implementation of OP4.11 but still suggested
to continue implementing the recommendations of the 2018 anthropological study as part of the
Management Action Plan. The specific action was to hire an anthropologist to provide expert advice where
RAP implementation involves the relocation of Afro-Brazilian cultural services. This action was completed in
2021.
The Safety of Dams Policy (OP4.37) was triggered given the planned works and resettlement activities
related to the river protection dikes along Parnaiba and Poti Rivers. A Dam Safety Panel of Experts was
engaged to support the Project, including in determining the risks associated with the current condition of
the structures and, in particular, construction activities on both slopes of the dikes and overtopping in case
of a severe flood. The engineering, social and environmental assessment of the risks were inconclusive,
given challenges with accessing the site for detailed structural and geotechnical assessments. The ICR
team shared (via email dated April 18) the Contingency Plan in case of dikes failure is at an advanced stage
and expected to be completed by the end of the fiscal year.
Environmental safeguards were assessed as Moderately Satisfactory at the project closure.
Social Safeguards
The Project triggered OP 4.12 on involuntary resettlement. An estimated 1288 households were expected
to be resettled from areas of permanent flood-risk areas around the lagoon and alongside the rivers, and
additional 302 households would be affected by engineering works pertaining to the road upgrade and
construction of wastewater pumping stations (PAD, page 21). Key required safeguards instruments such as
Resettlement Action Plans (RAPs) were prepared and approved by the Bank before the implementation of
works, with efforts to minimize the number of people adversely affected by the resettlement. However, the
land resettlement process was much more complex than estimated at appraisal, as the estimated benefits
by the project design were not perceived as benefits by all the affected people, particularly for the
households residing along the dikes. By the project closure, 1056 households had been resettled through
the project, with a combination of resettlement options, including relocation within and outside the
catchment area. About 80 households benefited from monitored resettlement, an approach that minimized
disruption and allowed project activities to progress while awaiting the completion of the housing complex.
Social safeguards implementation was generally challenging, with planned activities taking significantly
longer than estimated (particularly housing), thus delaying project implementation. The project was subject
to an investigation by the Inspection Panel. A request for inspection was received by the World Bank
Inspection panel on August 23, 2019, for the Project from 202 complainant households who opposed their
resettlement. According to the Inspection Panel report, by the time of the inspection, the Project lacked a
systematic and comprehensive application of the Bank Policy on Involuntary Resettlement, leading to
divided opinions between the project beneficiaries and some of the affected people. The ICR acknowledges
Page 21 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
that some of the issues raised by the complainants could have been avoided or resolved with a more
candid risk assessment at the original project and AF appraisal, a more proactive stakeholders’
engagement process, improved access to information, and adequate monitoring and record keeping (ICR
para 63.). In response to the complaint and prior to the investigation, the Bank took measures to assist PMT
in addressing the stakeholder engagement issues and ensure adequate monitoring on the ground. On
December 1, 2020, the World Bank’s Board of Directors accepted the proposed Management Action Plan
(MAP) included in the Management Report and Recommendation which included actions to address the
Inspection Panel investigation’s findings. All MAP actions were completed by project closure and the
completion report was sent to the Board in November 2022.
At project closure, the social safeguards compliance was assessed as Moderately Satisfactory.
b. Fiduciary Compliance
FINANCIAL MANAGEMENT
FM capacity was deemed inadequate regarding human resources, budget, financial reporting, and auditing
and significantly affected project implementation, financial monitoring, and timely data required for project
management. Key FM implementation issues included persistent delays in the submission of IFRs and
audits, inadequate management of project funds flow and designated account, qualified audit reports
related to noncompliance with the requirements set in the financial agreements, and delays in processing
financial information. (ICR, para 65). The FM project management system was not automated and IFRs
were processed manually. PMT staff were consistently trained to strengthen their capacity in financial
management. By project closure, the FM specialist assessed the FM arrangements to be mostly adhering
to the World Bank standards, and FM performance was assessed as Moderately Satisfactory. The ICR
team informed IEG that the final audit report was submitted by the PMU on Nov 30, 2022. The final report
issued an unmodified opinion and there were no relevant issues to be addressed.
PROCUREMENT
The project suffered significant procurement shortcomings. Overall, procurement took longer than
expected due to the lack of readiness and inexperience with managing World Bank-funded projects.
Procurement delays remained a critical bottleneck through project implementation. Although procurement
risk was rated high at appraisal and adequate mitigation measures were proposed, PIU capacity building
was significantly affected by changes in municipal government and PIU restructurings (ICR, para 64). High
demand for construction services in Brazil, particularly in Teresina, led to a lower-than-expected market
response and triggered the re-bid of major works. The changes in market conditions driven by external
macroeconomic factors, particularly the COVID-19 pandemic further increased the pressure on the
procurement team towards the later stage of the project. In response, the PMU increased the number of
skilled procurement staff and Banks procurement staff increased close supervision efforts by holding
biweekly meetings with the PIU. Procurement performance was rated Moderately Satisfactory at closing.
c. Unintended impacts (Positive or Negative)
Page 22 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Per ICR (para 45), ‘the project interventions have likely triggered some negative impacts related to
population influx in the LDN’. The land cover in the vicinity of the lagoons has significantly changed over
the project implementation period, with accelerated densification post-AF. The ICR notes that benefits
arising from the resettlement activities and the lack of a robust monitoring and enforcement system under
the project may have partially driven the migration of people into some of the protected areas planned for
intervention under the project, seeking housing opportunities.
d. Other
---
11. Ratings
Reason for
Ratings ICR IEG
Disagreements/Comment
There were shortcomings in the
Relevance of Objectives that are
Moderately
Outcome Unsatisfactory rated as Modest. Overall
Unsatisfactory
Efficacy and Efficiency are also
rated Modest.
Quality of Entry and Quality of
Moderately Supervision is rated as
Bank Performance Unsatisfactory
Unsatisfactory Unsatisfactory since there were
major shortcomings.
Quality of M&E Modest Modest
Quality of ICR --- Substantial
12. Lessons
This review has drawn the following lessons based on information in the ICR.
Flexibility and adaptability in the use of instruments is critical in urban and municipal
operations: The project was an integral part of Brazil’s Municipal Lending Program with most of the
operations following a similar design and implementation structure. Even though the project lending
at the municipal level was responsive to bottom-up demand, the project’s physical design spanning
five sectors was too demanding for a single operation. In the context of limited municipal capacity
such as in Teresina, there is a need to rethink how to engage in urban and municipal operations.
Multiple, interrelated areas of intervention critical for urban poverty reduction in the Teresina context
require a series of operations with focused objectives and tailored interventions based on an
understanding of the development constraints in a particular area. A simpler project design with
modest objectives, as well as flexibility to adapt to changed circumstances, should have been
considered.
Page 23 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
Project readiness is critical for smooth implementation: The preparation of feasibility studies
and procurement designs took relatively long, with works only starting two and half years after
project approval. The lack of detailed estimates at appraisal also led to an underestimation of project
costs and eventually lowered project ambition. Full establishment of the PIU with the critical fiduciary
functions took at least a year. The land resettlement process was much more complex than
estimated at appraisal, as the estimated benefits by the project design were not perceived as
benefits by all the affected people. During project preparation, more attention should be provided to
project readiness, with a specific focus on institutional requirements, detailed engineering designs,
and resettlement requirements to ensure these are within the limits of the financial and technical
capacity of the borrower. Ensuring a critical mass of design documents and related permits are
ready for procurement at effectiveness to avoid delays. Addressing land acquisition issues prior to
effectiveness and ensuring the timely engagement of people adversely affected helps reduce
implementation delays.
Candid project ratings are key to timely proactivity to address project shortfalls. Throughout
the first ten years of project implementation, project ratings were not adjusted to reflect emerging
issues and risks. The scaling up of project scope under the AF was also not fully backed by a candid
assessment of the results achieved prior to AF and the likelihood of achieving results within the
planned scope and implementation period. Candor in project ratings during implementation could
have been critical in raising the level of dialogue and attention at the management level early on in
the project.
Same-catchment relocation in case of involuntary settlement may garner stronger
stakeholder support. The initial approach to resettlement was to relocate all the households
affected by involuntary resettlement to a new complex, outside the LDN area, with significant
disruptions to the affected households’ livelihoods. The delays in the implementation of the housing
complex led the team to proactively identify some innovative approaches to involuntary resettlement,
including same-catchment relocation. Affected households, were given the opportunity to identify
available houses within LDN. This mechanism allowed households to remain within their
socioeconomic environment while accessing better living conditions and was strongly supported by
those affected.
13. Assessment Recommended?
No
14. Comments on Quality of ICR
The ICR is generally well-written, candid, and internally consistent. The ICR provides a comprehensive
overview of this complex operation given the multi-sector PDO and frequent changes in scope and
restructurings. The project description and the many changes introduced by various restructurings are nicely
and clearly laid out, also with the support of tables. The quality of evidence and analysis is generally aligned
with the messages outlined in the ICR. The ICR is candid in its assessment of the shortcomings in Bank
performance. The ICR provided clear lessons based othe n evidence outlined in the ICR.
Page 24 of 25
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
BR Municipal APL3: Teresina (P088966)
The analysis is generally evidence based but constrained to some extent by weaknesses in the project's M&E
system. For the assessment of the achievement of the objectives, the ICR focuses on compliance with outputs
and outcomes, rather than on the achievement of the objectives themselves. The report followed OPCS
guidelines except for exceeding the number of suggested pages (28 pages in the main text versus the
suggested 15). However, this is understandable given the complex operation. Also, there was a mistake in
using the split rating methodology. The achievement under the original outcome targets should be assessed at
the end of the project (across the entire project duration) and not what was achieved at restructuring. The data
on the actual costs in Annex 3 was not correct, and the team shared the correct data based on IEG’s request.
The ICR does not report on compliance with audit requirements.
Overall, the Quality of the ICR is rated Substantial, but with minor shortcomings.
a. Quality of ICR Rating
Substantial
Page 25 of 25