Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
Report Number : ICRR0020754
1. Project Data
Project ID Project Name
P072644 NG-Rural Access & Mobility - Ph. 1
Country Practice Area(Lead) Additional Financing
Nigeria Transport & ICT P159089
L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD)
IDA-44080 31-Dec-2014 72,000,000.00
Bank Approval Date Closing Date (Actual)
01-Apr-2008 30-Jun-2016
IBRD/IDA (USD) Grants (USD)
Original Commitment 60,000,000.00 0.00
Revised Commitment 59,478,437.42 0.00
Actual 56,524,534.95 0.00
Prepared by Reviewed by ICR Review Coordinator Group
Natsuko Toba Fernando Manibog Christopher David Nelson IEGSD (Unit 4)
2. Project Objectives and Components
a. Objectives
The objective of the project was to improve road access for rural communities in Kaduna State and to
improve management of Kaduna State’s road network in a sustainable manner (page 5, Financial
Agreement, 2008).
b. Were the project objectives/key associated outcome targets revised during implementation?
No
Page 1 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
PHEVALUNDERTAKENLBL
c. Will a split evaluation be undertaken?
No
d. Components
Component A: Upgrading, Rehabilitation, and Maintenance of Transport Infrastructure (appraisal total
US$63.81 million, of which IDA US$51.91 million and Borrower US$12 million; actual total US$60.77
million, of which IDA US$48.62 million and Borrower US$12.15 million)
(a) improvement and maintenance of about 427 kilometers (km) of rural roads within six intervention areas
of the state using an output- and performance-based road contract (OPRC) and (b) construction and
rehabilitation of about 132 river crossings spread across the Kaduna state.
Component B: Institutional Strengthening, Reforms, and Capacity Building (IDA financing only, appraisal
total US$8.09 million; actual total US$2.67 million)
(a) project implementation support by providing goods, materials and equipment and by ensuring the
availability of project management capacity and skills at the federal and state levels; (b) strengthening of
capacity to manage the road network, and rationalizing the establishment and enhancement of skills in
strategic planning, program and project scheduling, designing, implementation and maintenance of rural
road infrastructure; (c) development and implementation of institutional reforms to enhance efficiency in
resource allocation, procurement, and quality control responsibilities at the state government level; (d)
other cross cutting issues such as awareness creation and related work on road safety, gender and human
immuno-deficiency virus (HIV) and acquired immune-deficiency syndrome (AIDS); and (e) preparation of
state-funded follow-on this Rural Access and Mobility Project (RAMP).
e. Comments on Project Cost, Financing, Borrower Contribution, and Dates
Project cost
The total project cost at appraisal was US$72 million. The actual project cost was US$63.44 million, lower
than the appraisal, partly due to the exchange rate fluctuations of US$ and SDR.
Financing
The IDA credit at appraisal was US$60 million equivalent of SDR. Almost all (99 percent) of the original
IDA credit in SDR terms was disbursed. The actual US dollar disbursement was US$51.29 due to losses
in the exchange rate fluctuations.
Borrower Contribution
At appraisal, the Borrower contribution was estimated at US$12 million. The actual contribution was
slightly higher at US$12.15 million.
Dates
The World Bank board approval of the project was on April 1, 2008. After eight months, the project
became effective on December 16, 2008. The delay in project effectiveness was due to the processing
Page 2 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
time of the Credit Agreement and the on-lending agreement. The project had two level-two restructurings
on December 2, 2014 and December 29, 2015 only to extend the project closing date to complete
activities. The total of the two extensions was one and half year. The original project closing date was
January 1, 2015. The project actually closed 18 months later on June 30, 2016. IDA disbursement rates
were high at both restructurings, at 82 percent and 93 percent, respectively.
3. Relevance of Objectives & Design
a. Relevance of Objectives
At appraisal, the PDO was relevant to Nigeria and the World Bank. The medium-term (2003–2007) National
Economic Empowerment and Development Strategy included developing the transport infrastructure, in
particular, the state-level transport sector. About 70 million of the 130 million population were living on less
than a dollar a day. To reduce the rural mobility gap and poverty, the federal government adopted the National
Policy on Rural Travel and Transport (NPRTT). The World Bank-managed Sub-Saharan Africa Transport
Policy Program supported the preparation of the NPRTT. This project was directly linked to the NPRTT. The
government initiated a systematic investment in rural connectivity under the Rural Accessibility and Mobility
Program (RAMP). Kaduna state was identified to launch the program as it was the third most populous state
with a good agricultural economy. The state had only few rural roads, which were poorly maintained. The road
classification was inadequate. The state responsibilities for the road asset management and maintenance
were unclear.
The World Bank Country Partnership Strategy (CPS) for 2005 -2009 supported transport infrastructures and
included Kaduna state. The World Bank’s 2005 Africa Action Plan included three of the outputs: road network
rehabilitations, implementation of reforms, and establishment of sustainable financing and management
mechanisms. This project would contribute to all three outputs. The World Bank Nigeria Competitiveness and
Growth report in 2008 noted that the costs of poor roads to the economy at about 3 percent of GDP, causing
15 to 20 percent of the agricultural produce not to reach markets. The report recognized that Nigeria’s major
problem in the transport sector was not so much the lack of financing but rather the inefficient resource
utilization. Hence, the report recommended reforming the road sector to ensure efficient utilization of local
resources and unlock economic growth especially in the rural areas with a high development potential.
At the project’s completion, the PDO remained relevant to Nigeria and the World Bank. The CPS for 2014-
2017 was built on Nigeria’s national vision 20:2020 and the transformation agenda. The CPS identified the
road infrastructure among others as one of the constraints to Nigeria’s economic development. The CPS
recognized that the poor infrastructure increased transport costs for farmers and affected human capital, as
poor households lack access to schools and social and health services. The CPS supported improving
efficiency of the federal and state governments.
Rating
Page 3 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
Substantial
b. Relevance of Design
The project’s design was relevant to the PDO due to the following reasons. Upgrading, rehabilitation, and
maintenance of rural roads and river crossings (component A) would contribute to the first PDO to improve
road access for rural communities. Institutional strengthening, reform, and capacity buildings of the road sub-
sector at the state government level (component B) would contribute to the second PDO to improve
management of Kaduna state’s road network in a sustainable manner. The results framework (page 33, PAD)
was straightforward and presented clear linkages between the PDO, the final and intermediate outcomes, and
the project’s activities and inputs. It was also quantitatively specific in its annual, cumulative outcome targets
during the 5-year implementation period, and in the frequency as well as instruments for data collection. The
PAD (page 12) also assessed the implementation and data collection capacity of the State Project
Implementation Unit (SPIU) as adequate.
Rating
Substantial
4. Achievement of Objectives (Efficacy)
PHEFFICACYTBL
Objective 1
Objective
To improve road access for rural communities in Kaduna State
Rationale
Outputs
• Under OPRC, the total length of roads rehabilitated and maintained was 475 km. This achievement was
48 km (11.2 percent) more than 427 km of roads targeted due in part to lower unit costs than originally
estimated.
• The 146 rehabilitated river crossings that was achieved exceeded the target of 132 crossings by 10.6
percent.
Outcomes
• Road usage by the rural population exceeded the target by 52 percent. Based on the indicator of
number of road trips per day,76 trips were achieved, which was significantly higher than the target of 50
trips and the baseline of 37 trips. Data was based on average daily vehicle counts and reconfirmed
through household interviews.
• The targeted addition of 1.5 million of the rural population with improved access to an all-weather road
Page 4 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
within 2 km was achieved 100 percent, compared to the baseline of 1 million. Data was established by first
converting the baseline population to a percentage of the people with access to an all-weather road and
estimating changes in the percentage with local interviews. The data collection method used at the project
completion differed from that at the appraisal, which used absolute numbers. Sampling based estimates
could risk biased results if the sample did not represent the entire population. However, the sample of
local interviews added qualitative value rather than just measuring or estimating the absolute number of
people.
• The amount of agricultural produce transported across improved river crossings increased to 4.6 million
tons. This achievement exceeded the target of 3.8 million tons by 25 percent. The baseline was 3.5
million tons. The monitoring report presented the proportion of respondents indicating increased
transportation of agricultural produce across the new river crossings. As in the case of the improved
access above, the project completion’s data collection method differed from that at the appraisal, which
used absolute numbers. Despite the potential sampling bias, sample interviews provided more insights of
the attribution issues as discussed below.
• Other factors such as weather, agricultural productivity, market demand, inputs supply and demand,
etc., could also contribute to this achievement of agricultural products transported. Hence, whether this
achievement could be fully attributable to this project needed to be ascertained. The Bank team informed
IEG that the project’s contribution to this achievement was checked through stakeholder interviews.
Stakeholders along the project corridors informed the Bank’s team that farmers used to lose up to 30
percent of their agricultural product due to lack of access to markets. This loss was because their
agricultural products could not be sold, hence they became spoiled and were discarded. However, the
team was informed that sales of agriculture products have increased due to the improved road and river
crossings. Farmers could get to the markets and traders came to villages to buy the products.
Accordingly, with increased market sales, the local formers could also increase production.
• The local construction industry developed skills through their participation in the OPRC. With the
introduction of the OPRC a culture of long-term road asset management approach was to be
institutionalized. However, there was no evidence of the OPRC’s direct contribution to the efficacy in
achieving the outcome. As the OPRC was the first of its kind in Nigeria, it delayed the implementation due
to the OPRC's need to learn and make adjustments to the reality of local industry, such as removal of
prequalification requirement for bidding procedures.
Rating
Substantial
PHREVDELTBL
PHEFFICACYTBL
Objective 2
Objective
To improve management of Kaduna State’s road network in a sustainable manner
Page 5 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
Rationale
Outputs
The following were achieved:
• Training needs assessment and detailed capacity building framework was prepared for the Civil
Engineering Department of the State Ministry of Works and Transport (SMOWT) in June 2015. Its
recommendations were yet to be implemented.
• The three-year rolling plan, a new road reclassification, and digitization of the state’s road network were
completed, but only partially used.
• Studies were completed on (i) road information management system, (ii) road management institutional
development and capacity-building reforms, and (iii) “Evaluation of Community Involvement and Gender
Issues in the Rural Access and Mobility Project (RAMP-I) in Kaduna State”.
• Road management operational equipment was provided (e.g., operational vehicles, computers, printers,
copiers, etc.).
• Road sector reform studies and a policy document were completed.
• Road safety features were incorporated in road design. Road safety awareness building campaigns and
HIV/AIDS sensitization workshops were conducted.
However, the following were not achieved:
• The next phase of RAMP in Kaduna state was not prepared.
• A simplified road management and road inventory database was not prepared, even though the
database is a basic entry point towards professional road planning and is relatively easy to implement.
Outcomes
• The allocation of funds to Kaduna State Public Works Agency improved following the adoption of the
three-year rolling plan.
• The development and use of road management systems were only partially achieved, because some of
the above output activities and studies to develop the system were completed but not fully used.
• Implementation of the Kaduna state road sector institutional reform implementation was not initiated by
the project’s completion. However, the Bank team informed IEG on June 5, 2017 that the Kaduna state
legislative house (House of Assembly) approved the draft bill intended to become a law setting up an
independent agency for road management in the state (Kaduna State Road Authority, KADRA). The bill
was prepared under the project. This development indicates continuation of what the project left behind.
The new authority would separate the implementation function from the policy function that has been the
responsibility of the State Ministry of Works.
Rating
Modest
PHREVDELTBL
Page 6 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
PHREVISEDTBL
5. Efficiency
Ex-ante economic analysis
An ex-ante economic analysis estimated the options to provide communities with the rural transport
infrastructure. Two approaches have been employed: a cost effectiveness analysis method (CEA) and a cost
benefit analysis (CBA).
The CEA of very low-volume roads was carried out by estimating the investment cost per beneficiary. The CEA
covered nine local government areas where about 239,948 additional people would have improved
accessibility. An investment of US$189 per person over five years was determined as favorable cost-
effectiveness. However, the CEA’s coverage of the total project cost at appraisal was inconsistent in the PAD:
page 91 indicates about 57 percent while table 9-B on page 92 showed 63 percent.
The CBA for the roads with traffic between 50 and 200 annual average daily traffic (AADT) were assessed.
AADT is the annualized average 24-hour volume of vehicles at a given point or section of road, called a traffic
count. It is normally calculated by determining the volume of vehicles during a given period and dividing that
number by the number of days in that period. The benefits were road user costs savings. The analysis for 10
years was done only for the Fadan Kamantan-Yangal-Walijo Road. This road segment was chosen because it
was assumed to yield the lowest benefits so that the results of the analysis would be conservative. The gravel
option was estimated to be the best option with a net present value (NPV) of US$1.32 million at 12 percent
discount rate with an economic internal rate of return (EIRR) of 26 percent. The paved option with single
surface dressing was also economically justified with an NPV of US$0.47 million at 12 percent discount rate
and an EIRR of 15 percent. The paving option was justified because of the expected traffic diversion from an
alternative road with a distance savings of 11 km.
Economic benefits of the specific river crossing were not assessed.
Ex-post economic analysis
An ex-post analysis used the same methodologies used at the ex-ante analysis. The CEA covered 80 percent
of the actual total project cost and 309,641 people, more than those at the appraisal. The investment cost per
beneficiary was estimated to be US$165 per beneficiary, lower by 13 percent than the ex-ante analysis of
US$189. It was lower by 18 percent than the minimum threshold of US$200 per beneficiary for inclusion of a
road as part of the RAMP intervention network (paragraph 5, page 91, PAD, 2008). The CBA results showed
that the best alternative was the upgrading of the road to single surface dressing, with an NPV of US$3.34
million at 12 percent discount rate and an EIRR of 30 percent. However, an alternative of upgrading to double
surface dressing was also economically justified with an NPV of US$3.14 million and an EIRR of 27 percent.
However, the ICR noted inadequate evidence that the roads and river crossing would be sustainably
Page 7 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
maintained (page 17, ICR). The unsustainable road management could deteriorate the quality of roads, which
could decrease the normal traffic volume, the traffic growth rate, and the generated traffic. As a result, the
benefits would be reduced. A sensitivity analysis of the CBA addressed these parameters’ changes to the NPV
and EIRR by increasing or decreasing these parameters by 25 percent. The results of the sensitivity analysis
produced NPV of between US$2.73 million and US$4.85 million and EIRR values of between 20 percent and
34 percent. The analysis showed that the option would still be economically justified even when the normal
traffic reduces by 46 percent or the annual maintenance cost per kilometer (km) at US$35,100, which means a
total cost of US$765,180 for the 21.8km road link over Fadan Kamantan - Yangal -Walijo. This indicated the
robustness of economic impacts of the road upgrades even when the unsustainable road maintenance reduced
the traffic. Nevertheless, the project covered 457 km and CBA covered only 5 percent (21.8km) of the total
project road links. Therefore, this robust economic impact could not be fully assumed for the rest of the road
links. The rest of the road link was covered by CEA and had lower road traffic volumes (less than 50 AADT)
than the Fadan Kamantan - Yangal -Walijo road link (between 50 AADT and 200 AADT).
The economic benefits from the largest river crossing constructed under this project was evaluated. The largest
river crossing was a two-span reinforced concrete bridge, with each span measuring 12.5 m, located in Kajuru
Local Government Authority of Kaduna State. Benefits could be the impact of transport investments on local
agricultural productivity, increases in agricultural output, improved market prices for agricultural produce, and
reduced costs of agricultural inputs. Non-quantifiable benefits could include better access to schools, health
facilities, and markets. Data were not available to permit an objective evaluation of the benefits derived from
the investment made. However, information from a recent study on monitoring and evaluation (M&E) indicators
noted that 76.4 percent of farmers, who before the provision of improved river crossings experienced problems
in transporting agricultural produce during the wet season, were able to do so without problems. The study
estimates that the volume of agricultural produce transported across new bridges, culverts, and drifts has
increased by an average of 30.7 percent.
No financial analysis was available in the PAD or the ICR.
An analysis of the estimated costs and contract prices indicated that the prices of the road upgrading contracts
were, on an average, about 8 percent lower than the estimated costs. The significant depreciation of the
Nigerian naira (NGN) against the U.S. dollar happened during the project (from NGN 117.8 per U.S. dollar at
the time of the Credit approval to about NGN 279.7 per U.S. dollar at the project completion). Therefore, the
project is reported to have saved some costs as the project cost was mostly in local currency. An actual
expenditure of sub-component A1 (rehabilitation and maintenance of roads) indicated a savings of 10 percent
of the original estimates.
There was no evidence that the OPRC contributed to the cost efficiency. While the actual unit cost of the road
rehabilitation and maintenance was lower than the estimate at appraisal, there was no evidence this was the
result of the OPRC.
Administrative and Implementation Efficiency
The administrative efficiency was modest as the original project implementation period of 6 years and four
months was extended by 1.5 years. The key sources of delay included: civil works documents still being
Page 8 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
drafted when the project was approved; slow administrative processes; poor response and quality of bidders;
intermittent provision of counterpart funding; poor staff commitment due to inadequate civil service
remuneration levels; and precarious security situation in the project sites.
Due to the new concept of OPRC in Nigeria, the implementation efficiency deteriorated as it took time to learn
and make adjustments. The World Bank underestimated the heavy burden placed on the Borrower regarding
the preparations needed for successful implementation of the OPRC, especially in view of the Borrower’s
inadequate internal capacity. Requirements for the OPRC delayed the procurement of civil works. It took 18
months before the contracts could finally be awarded. This long process was due to the poor response and
quality of bidders at the prequalification stage.
Efficiency Rating
Substantial
a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal
and the re-estimated value at evaluation:
Rate Available? Point value (%) *Coverage/Scope (%)
0
Appraisal 26.00
Not Applicable
0
ICR Estimate 30.00
Not Applicable
* Refers to percent of total project cost for which ERR/FRR was calculated.
6. Outcome
The relevance of the PDO is substantial. The relevance of design is substantial. The efficacy of the first PDO is
substantial but the second PDO is modest. The efficiency is substantial. Overall project outcome is moderately
satisfactory.
a. Outcome Rating
Moderately Satisfactory
7. Rationale for Risk to Development Outcome Rating
The roads and river crossings financed by the project have a risk in terms of sustainable maintenance. This risk
mainly is stemmed from the uncertainties of (i) budget allocation and (ii) actual utilization of the technical and
institutional capacity (e.g., modern road management tool), which were strengthened by the project.
Page 9 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
Unsustainable road maintenance could lead to deterioration of quality of the road, resulting in the reduced road
traffic thereby reducing the benefits of the project. Sustainable maintenance is still a concern.
Despite the government intentions to finance the road maintenance and the project’s provision of capacity
building, the uncertainties remain. The contracts of the newly completed river crossings were terminated
without clear long term maintenance plans. The institutional foundations, backed by the reforms, were needed
to support the realization of the state government budget allocation for the project roads and the rest of the road
network and the utilization of the strengthened capacity. At the time the ICR mission, the state had decided to
extend some of the OPRCs. This decision was a good sign because with the introduction of the OPRC a
culture of long-term road asset management approach was to be institutionalized.
As noted in the section 4 above, the state has approved a law to set up an independent agency for road
management in the state. As the planned reforms at the project appraisal did not materialize, the political risk
to reverse this development remains. However, once approved, it will be a positive sign that the risk to
sustainable maintenance would be mitigated.
While the state is still considering an appropriate approach toward road sector reforms, the Department of
Roads could already start addressing two main challenges: adoption of a more robust road network
management approach and ensuring that road maintenance was well institutionalized and funded.
a. Risk to Development Outcome Rating
Substantial
8. Assessment of Bank Performance
a. Quality-at-Entry
The project was well prepared to address the need of Nigeria to improve the rural mobility. The project
incorporated lessons learned from the previous IDA-supported Second Multi-State Roads Project (Credit
2485-UNI), which closed on December 31, 1999. This previous project was rated unsatisfactory on
sustainability, and medium on institutional development impact. One of the lessons learned from the
operation was more focus on institutional capacity building.
The Bank team appropriately identified key risks. These risks included the use the long-term OPRCs as it
was new concept to Nigeria and the adoption of the proposed reforms by the state government.
Nevertheless, the mitigation measures appeared insufficient. It was premature to include the adoption of a
state road sector reform policy in an environment that lacked a comprehensive federal model. The World
Bank underestimated the heavy burden placed on the Borrower regarding the preparations needed for
successful implementation of the OPRC, especially given the Borrower’s inadequate internal capacity. The
Bank paid inadequate attention to the need for (i) designing a model suited for the capacity of the
construction industry and the government, (ii) raising high level of awareness and promoting private sector
Page 10 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
buy-in and (iii) all-inclusive training ahead of the starting the OPRC. The requirement of the Kaduna state’s
counterpart funding was too arduous as the state was largely dependent on federal allocations. This risk to
the counterpart funding was not identified at the appraisal.
Quality-at-Entry Rating
Moderately Unsatisfactory
b. Quality of supervision
The World Bank supervision included approximately two missions per year. The team made special
arrangements to avoid the risky circumstances occasioned by insecurity, especially in the last three years of
the project. The absence of monitoring data prevented the midterm review in November 2012 from
adequately assessing the project performance (page 19, ICR, 2017).
The Bank team actively supported to resolve implementation problems. For example, the Bank team
advised the State Project Implementation Unit (SPIU) to recruit support personnel on monitoring and
evaluation (M&E). This advice was taken and it resulted in remarkable improvement (page 19, ICR, 2017).
When the counterpart funding had stopped during the first half of the project implementation, the Bank team
explained the state administration the effects of the lack of counterpart funding. The State Government
thereafter reintroduced the counterpart funding in its state budget. Eventually, the counterpart funding was
adequately provided despite the initial delays.
Quality of Supervision Rating
Moderately Satisfactory
Overall Bank Performance Rating
Moderately Satisfactory
9. Assessment of Borrower Performance
a. Government Performance
The Federal Government demonstrated its commitment to the PDO as following examples. The
government had set up the policy and strategy during the project preparation. The government took initial
steps to coordinate its implementation. The government used the Rural Travel and Transport Program
(RTTP) to guide selection of participating states, define overall project outputs, and allocate internal
resources toward a national program. The government established a Federal Project Monitoring Unit
(FPMU).
Kaduna state eventually provided the counterpart funding after the initial delays. However, its commitment
toward reforms and the RAMP was lost due to the following reasons. Despite their agreement at appraisal,
Kaduna State did not prepare a follow-on state-funded RAMP due to their lack of funding for the project
implementation. The state did not complete the final critical steps of the road sector reform process to
institutionalize road maintenance by the project completion. However, as noted in the section 4 above, after
Page 11 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
the project’s completion, the Kaduna state legislative house approved a law to set up an independent
agency for road management (KADRA). This signaled the state’s sustained commitment to the PDO.
The Bank team informed IEG that all covenants were complied with.
Government Performance Rating
Moderately Unsatisfactory
b. Implementing Agency Performance
The project implementing entity was the Kaduna state (page 15, Financial Agreement, 2008). The
implementing agency was the Kaduna State Ministry of Works and Transport (SMOWT) (page i, ICR).
The State Project Implementation Unit (SPIU) in the SMOWT was responsible for managing and
coordinating project activities (page 3, Project Agreement, 2008). The SPIU contributed to achieving
higher than estimated outputs on the infrastructure aspect of the project. The Bank team informed the IEG
that all covenants were complied with.
Originally, the project staff was to be competitively and openly recruited within and outside the
government. However, the government decided to second the existing government staff directly. The
government only recruited outside the government for positions (e.g., with OPRC experience) that they
could not find the suitable candidates within the government.
The SPIU needed to report to the State Project Management Committee (SPMC). The SPMC included
representatives of various policy officials from stakeholder ministries and departments to enable
participatory decision making. However, the SPMC did not meet for a long period. Only the Permanent
Secretary was making decisions. In this regard, the project lost the opportunity of all-inclusive oversight.
This situation was reversed when a new government took over governance in the state in 2015. The
SPMC was reactivated. The SPMC was meeting regularly until end of the project.
Implementing Agency Performance Rating
Moderately Satisfactory
Overall Borrower Performance Rating
Moderately Satisfactory
10. M&E Design, Implementation, & Utilization
a. M&E Design
The PDO and intermediate indicators were generally adequate. Attributions of some of the indicators
were unclear as shown in the following examples. A PDO indicator of the reduction of transportation cost
per trip could be influenced by other factors such as cost of fuel, efficiency and size of the vehicle,
logistics, etc. The attribution of the PDO indicator of amount of agricultural produce transported across the
Page 12 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
new river crossings could be influenced by other factors than the project alone. However, the Bank team
informed IEG that they had checked with local people to verify the attribution. The ICR noted the crucial
and challenging role of the state road sector reform. Yet, the project had no indicator to monitor the
progress toward road sector reforms.
The SPIU was responsible for data collection. Most of data sources were from formal studies, one from a
report, and only two from the state statistical office. These arrangements were adequate as most of the
indicators were project specific. However, since the project documents did not provide guidance on how
the data was to be collected, the method used during the implementation diverged from the one used
during appraisal. Baselines were established at project appraisal. The Development Economics unit
(DEC) of the World Bank identified the project as a potential candidate to receive support for an impact
evaluation. The project’s M&E would partner with the Living Standards Measurement Study (LSMS) to
commence in April 2008 under the auspices of Nigeria Bureau of Statistics. The Bank team informed IEG
that DEC and/or LSMS did not actually support the project.
b. M&E Implementation
The inadequate capacity of SPIU delayed data collection. The quality of the data collection process was not
sufficiently checked. The midterm review (MTR) in November 2011 lacked sufficient M&E data to assess the
performance of the project. An M&E consultant contract was signed only on November 2013. Afterwards, the
SPIU arranged a regular data monitoring process in 2014 and produced quarterly reports. The reports were
general and used different methodologies from those at the appraisal. However, these different
methodologies did not significantly affect the evaluation at the project completion.
c. M&E Utilization
M&E was not utilized during the midterm review (MTR) in November 2011 when the M&E data could help
determine if any changes in directions or making other critical decisions. This was because of lack of M&E
data to assess performance at the MTR. The Bank project ICR team informed IEG that there was no explicit
reference to the utilization of the M&E in the material used to compile the ICR. However, the economic
benefits of river crossings were assessed using the information from a recent study on M&E indicators (page
32, ICR), which indicated the utilization of the M&E.
The PAD of Nigeria Second Rural Access and Mobility Project (2012, P095003) noted that the OPRC
experience of this project in Kaduna state would be closely monitored to address the road maintenance risk
(page 19). Also, the same PAD noted close monitoring and stock taking of OPRC pilot launched in Kaduna
state as part of as part of the implementation support plan (page 71). However, the same PAD did not
specifically indicate the use of the intermediate outcome indicator of number of kilometers of roads
rehabilitated under OPRC contracts of this project’s M&E. The Bank project ICR team informed IEG that the
team also could not find the evidence of the specific use of the intermediate indicator.
Page 13 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
M&E Quality Rating
Modest
11. Other Issues
a. Safeguards
Environmental safeguards
This project triggered the Environmental Assessment (OP/BP 4.01) policy and was assigned an
Environmental Screening Category of B. The safeguards instruments were prepared and disclosed in Nigeria
and the Bank’s InfoShop prior to project appraisal. The date of disclosure at the InfoShop was November 7,
2007 (page 94, PAD 2008). No major significant negative impacts during the project implementation were
anticipated. It was expected that the rehabilitation of the roads would result in positive environmental and
social impacts (page 22, PAD, 2008). The Bank team informed IEG that the environmental safeguards were
complied with. An independent audit did not reveal issues or concerns on compliance with safeguard
policies.
Social safeguards
The project triggered Involuntary Resettlement (OP/BP 4.12). The safeguards instruments were prepared
and disclosed in Nigeria and the Bank’s InfoShop prior to the project appraisal. The date of disclosure at the
InfoShop was November 7, 2007 (page 94, PAD). The Bank team informed the IEG that the social
safeguards were complied with. The project’s Abbreviated Resettlement Action Plan (ARAP) was disclosed
in Nigeria between November 9 and 12, 2011 and at the InfoShop on November 14, 2011. The supervision
missions of November 2012 and January 2013 observed that the ARAP document omitted some Project
Affected Persons (PAPs) on one of the roads. The ARAP document was revised after several months of
delay. The revised document was disclosed as an addendum to the ARAP. The process did not affect the
progress of the works. The PAPs were compensated. Following this incidence, the SPIU set up a
Resettlement Implementation Committee and Grievances Redress Committee to implement social safeguard
recommendations.
b. Fiduciary Compliance
Financial Management
The ICR reported no financial management (FM) concerns. The Bank team informed IEG that all audits were
unqualified. The Federal Project Financial Management Division located in the office of the Accountant
General of the Federation was responsible for FM for all World Bank-funded projects. Consequently, the
project maintained sound and robust internal control system. The project complied with the FM manual and
Page 14 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
the project implementation manual.
The accounting process was affected by the introduction of a new state government FM policy in May 2015.
The new procedures hindered the smooth operation of the project and delayed payments to service
providers. By November 2015, with the World Bank support, the state government resolved this situation.
Procurement
Requirements for the OPRC delayed the procurement of civil works. It took 18 months before the contracts
could finally be awarded. This long process was due to the poor response and quality of bidders at the
prequalification stage. Potential bidders lacked OPRC experiences and had a perceived risk of the OPRC
because Nigerian contractors were not used to pre-financing a project. Under many other government
contracts, contractors were well paid in advance of up to 50 percent. Furthermore, potential bidders feared a
perceived high risk of the long term payment period in long-term contracts. One of prequalification
requirements was the engineering capacity within the bidder’s establishment. Typically, medium- and small-
size contractors would not have significant engineering competencies. As a result, the contracts were re-
advertised. The pre-qualification was eliminated. Eventually, this revised approach was successful in
contracting. The works were completed on time. To fill the gaps of engineering capacity of the contractors,
the supervision consultants designed the roads.
Procurement of river crossing contracts did not have to follow a pre-qualification procedure. The
procurement took four months to complete. The procurement and construction of the Bailey bridge was
difficult because IDA financed only part of the works. A substantial part of the Bailey bridge had to be
salvaged from another existing bridge and was assembled by the Nigerian Army. The quality control of the
work was inadequate. In May 2014, the substructure failed. The Kaduna state government financed the
completion of the bridge. The contract was not completed by the project closing date.
The World Bank procurement team guided and trained the SPIU and the FPMU. The SPIU referred
procurement cases to the State Tender Committee for ratification before approaching the World Bank for
clearance. After the SPIU was disbanded at the end of the project, the procurement personnel were
reintegrated into the state procurement offices. Therefore, their skills continued to be utilized.
c. Unintended impacts (Positive or Negative)
Not applicable.
d. Other
Not applicable.
Page 15 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
12. Ratings
Reason for
Ratings ICR IEG
Disagreements/Comment
Moderately
Outcome Moderately Satisfactory ---
Satisfactory
Risk to Development
Substantial Substantial ---
Outcome
The ICR did not follow the
harmonized OPCS/IEG
guidelines. When two sub-
ratings are in the opposite
direction of satisfactory versus
unsatisfactory (i.e., MU for
Moderately
Bank Performance Moderately Satisfactory Quality at Entry and MS for
Unsatisfactory
Supervision in this case), the
overall rating (for Bank
Performance) follows the
direction of the overall
outcome rating (MS in this
case).
The ICR did not follow the
harmonized OPCS/IEG
guidelines. When two sub-
ratings are in the opposite
direction of satisfactory versus
unsatisfactory (i.e., MU for
Moderately Government Performance and
Borrower Performance Moderately Satisfactory
Unsatisfactory MS for Implementing Agency
Performance in this case), the
overall rating (for Borrower
Performance) follows the
direction of the overall
outcome rating (MS in this
case).
Quality of ICR Substantial ---
Note
When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the
relevant ratings as warranted beginning July 1, 2006.
The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as
appropriate.
13. Lessons
Followings are lessons learned largely drawn from the ICR’s lessons learned.
Page 16 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
1. For the OPRC to be successful, sufficient preparatory work is required. The project experienced
challenges as the OPRC was new to Nigeria. Prequalification requirements did not match the engineering
capability of small and medium sized contractors. An OPRC bid process could be carefully designed to reflect
the local reality and remove any responsibilities beyond the existing capacities. Supplementary measures can
fill the gap resulting from the removed responsibilities. If lump-sum prices are used, careful reviews are
required about the risks of lump-sum prices for works in the OPRC bid documents. The minimum costs for all
activities could be established as benchmarks for lump-sum priced items. In choosing the road links in an
OPRC package, it is important to minimize logistical challenges at implementation (e.g., each road segment
located far apart each other).
2. Investment in providing basic access could leverage government resources. The improvement of river
crossings had the advantage that it encouraged the state government to provide its own resources to improve
the ‘missing’ road links.
3. Absence of continuous M&E increases the risk of missed opportunities. At the midterm review,
insufficient data were available to assess the project’s progress toward achieving its objectives. As a result, the
project focused more on the rather obvious deliverables of the road infrastructure and less on the
transformational and institutional aspects. By the time the related activities were implemented, the project did
not have time for internalization of the outcomes and meaningful dialogue with stakeholders by the project
closing date.
4. Development and implementation of reforms in a single Investment Project Financing (IPF) operation
is a known challenge and could be complemented by Development Policy Financing (DPF). During the
total 8-year project implementation period, this project failed to deliver reforms although the steps leading to the
reforms were taken. Under the Nigeria Federal Roads Development Project (P090135), the federal-level road
sector reforms had also not progressed as planned. The proposed reforms would have included funding of road
maintenance using some form of road user-charging or taxation system. In the absence of a federal-level
model, the planned reforms in Kaduna state faced uncertainty. DPF could have complemented IPF to support
reforms. However, if the target is ambitious, such as tariff, user-charging system or taxation system, even the
combination of DPF and IPF may not always achieve the target (e.g., Côte d'Ivoire Urgent Electricity
Rehabilitation Project, P112573).
14. Assessment Recommended?
No
15. Comments on Quality of ICR
The ICR was candid and sufficiently analytical, and focused on identifying and presenting relevant evidence.
Safeguard policies trigged were not listed. The beneficial impacts of the project noted in the section of
Poverty Impacts, Gender Aspects, and Social Development (page 16, ICR) could have included sufficient
Page 17 of 18
Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review
NG-Rural Access & Mobility - Ph. 1 (P072644)
justifications of attributions. The ICR did not follow some of the ICR guidelines (updated version on July 22,
2014) and following are examples. Monitoring and Evaluation (M&E) Design, Implementation, and Utilization
section needed to include separate assessments of (a) M&E design, (b) M&E implementation and (c) M&E
utilization. When the rating for one dimension is in the satisfactory range while the rating for the other
dimension is in the unsatisfactory range, the rating for overall Bank or Borrower Performance normally would
depend on the Outcome rating.
a. Quality of ICR Rating
Substantial
Page 18 of 18