Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Report Number: ICRR0024189 1. Project Data Project ID Project Name P157054 MP Rural Connectivity Project Country Practice Area(Lead) India Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) COFN-C1520,IBRD-88330 15-Mar-2023 184,561,668.30 Bank Approval Date Closing Date (Actual) 14-Mar-2018 15-Sep-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 210,000,000.00 0.00 Revised Commitment 186,600,000.00 0.00 Actual 184,561,668.30 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Vibecke Dixon Avjeet Singh IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) as stated in the Loan Agreement (Schedule 1, page 5) and in the Project Appraisal Document (PAD, page 16) is: " To improve durability and enhance resilience to climate changes of the gravel surfaced rural roads in Madhya Pradesh (MP), while building the capacity of the state to manage its rural road network and road safety ". Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) For the purpose of this Implementation Completion and Results Report (ICR) Review, the PDOs are unpacked as follows. PDO 1. To improve durability and enhance resilience to climate changes of the gravel surfaced rural roads in MP. PDO 2. To build the capacity of the state to manage its rural road network. PDO 3. To build the capacity of the state to manage its road safety. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components There were four components (PAD, pages 18 - 23): 1. Road Upgrading, Construction and Maintenance. The estimated cost at appraisal was US$485.00 million. The actual cost was US$427.82 million. There were two sub-components: A. Surface sealing of gravel roads. Activities in this sub-component planned to seal the surface of eligible existing gravel surfaced roads developed under the Chief Minister's Graham Sadak Yojana (CMGSY) using conventional bitumen treatment or other alternative options, piloting post-construction road maintenance through women self-help groups (SHGs) in selected districts and providing SHGs with road maintenance training and hand tools. B. Provision of alternative connectivity. Activities in this sub-component planned to provide additional links to eligible villages already connected by a single road link but critically required additional connectivity to respond to the growing demand link to more social, economic and administrative centers. 2. Institutional Development. The estimated cost at appraisal was US$3.00 million. The actual cost was US$2.02 million. There were two sub-components: A. Rural road asset management system. Activities in sub-component included, defining system requirements and terms of reference (TOR) (for the system architecture, data collection and analytical modules) and procuring a road asset management system to upgrade the existing system by establishing an inventory of the road network with all its elements, current condition and network performance, an estimate of asset value, forecast of future demand of traffic and service needs, estimate of maintenance needs and cost, prioritization of the quality and performance objectives, funding scenarios for regular maintenance, upgrade the road asset and define a rural roads strategy and its implementation. B. Strengthening design, research and quality assurance capacity of the Madhya Pradesh Rural Road Development Agency (MPPRDA): Activities in this sub-component aimed to support MPPRDA's capacity by; (i) strengthening its design and research unit; (ii) training its design staff to enable them to do Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) in-house design to field units and review consultants designs; (ii) review and update current rural road design standards and technical specifications; (iv) reinforcing field laboratories in selected districts and training staff on laboratory and field tests; and (v) organizing study tours to share good examples of rural road design and research in India and abroad. 3. Road safety management capacity development. The estimated cost at appraisal was US$10.00 million. The actual cost was US$4.00 million. There were two sub-components: A. Road accident data management system. Activities in this sub-component planned to develop a comprehensive, multi-institutional and multi-sectoral road accident data management system including through: (i) a Geographic Information System (GIS)- based accident data collection software system for recording accidents, storage, analysis and dissemination; (ii) training at headquarters and district levels on database development, management and analysis of crash data. B. Pilot a comprehensive road safety program. Activities in this sub-component planned to pilot multi- sectoral road safety initiatives for a high-risk network, including technical assistance (TA), training and equipment for design and implementation of road safety engineering interventions further to an audit carried out with participation of the local community; road safety enforcement, post-crash emergency management; and road safety education and awareness. 4. Design, Implementation and Management Support. The estimated cost at appraisal was US$4.00 million. The actual cost was US$1.43 million. Activities in this sub-component entailed providing overall support to MPRRDA on project management, construction, supervision, quality control and technical and financial audits including by preparing cost-effective, climate resilient engineering designs and related surveys and investigations; construction supervision of civil works and quality control; implementing independent monitoring of quality of designs and works and contract compliance; and independent monitoring and assessment of safeguards and user satisfaction surveys. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal was US$502.00 million. The revised estimate after partial cancellation of the loan was US$462.60 million. The actual cost was US$445.13 million. Project financing. The project was financed by an International Bank for Reconstruction and Development (IBRD) loan of US$210.00 million. The amount disbursed was US$185.56 million. There was parallel financing of US$152.00 million from the Asian Infrastructure Investment Bank (AIIB). The amount disbursed was US$122.54 million. US$23.40 million was cancelled due to the savings arising on account of the depreciation of the Indian Currency (Rupee) relative to the US$. Borrower contribution. Borrower contribution of US$152.00 was planned at appraisal. Their actual contribution was US$138.05 million. Dates. The project was approved on March 14, 2018, became effective on July 16, 2018, and was scheduled to close on March 15, 2023. The Mid-Term Review was held on March 15, 2021. The project closed six months behind schedule on September 15, 2023. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Changes. The Bank supported the following changes through a Level 2 project restructuring on March 15, 2023.  Savings of US$39.00 million realized due to exchange rate changes were cancelled by the Bank at the government's request.  Extended the closing date by six months from March 15, 2023, to September 15, 2023, due to the delays caused by the restrictions in the wake of the COVID-19 pandemic.  Reallocated funds between disbursement categories. 3. Relevance of Objectives Rationale Country and State context. MP's economy, the second largest state in India in terms of its population, is agrarian. Over three quarters of its Gross Domestic Product (GDP) comes from the agricultural sector and only 7% of MP's population is engaged in non-farm activities. The heavy reliance on agriculture, coupled with an undiversified economy, resulted in a measured poverty rate of 31.6% in MP (as compared to the national average of 21.9%). Sector context. With a road network of 115,372 kilometer (km) in 2017, of which 80% was paved, MP was one of the least connected states in India. The lack of all weather roads in 6,636 villages, constituted a development barrier. With regard to rural roads, MP faced critical challenges such as: (i) local variation in implementation capabilities; (ii) low private sector participation; and (iii) poor service quality and sector governance. Regarding road safety, MP ranked third in India with 11% of the total number of road accidents and sixth at 6.4% of the number of road traffic fatalities in 2015. National and State Strategy. The PDO was consistent with the national strategy outlined in India's 12th Five-year plan (2012 - 2017) at appraisal. The plan underscored the need to improve transport infrastructure in rural areas. The PDO was relevant to the MP Government's Vision 2018 of connecting all villages in the state with all weather roads. The national government and the state government had launched initiatives aimed at connecting villages, prior to appraisal. The National Pradhan Mantra Gram Sadak Yojana (PMGSY) initiative aimed at connecting all villages with population over 500. The MP's - Chief Minister Gram Sadak Yojana (CMGSY) had a similar initiative to address villages not eligible under the PMGSY and to increase the service life of roads completed under the CMGSY program under sealed surface standards to address the adverse effects of climate change. MP adopted a Road Safety Policy in 2015 aimed at reducing the number of road traffic accidents and fatalities. This policy focused on nine strategic areas for improving the four E's of road safety that is: (1) Education; (2) Engineering (roads); (3) Enforcement; and (4) Emergency Care. World Bank (WB) Strategy. The PDO is well-aligned with the Bank strategy for India. At appraisal, the first and the third areas of the Country Partnership Framework for 2013 - 2017 highlighted the need for integration and inclusion. The PDO is well-aligned with the three focus areas of the WB's current CPF for 2018 - 2022: (1) promoting resource efficient, inclusive and diversified growth in the rural sector; (2) Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) improving disaster risk management and resilience to climate change; and (3) improving connectivity and logistics. When this project closed, the new CPF for India was under preparation. Bank experience. The Bank has a history of financing transport projects in India. This project build on the PMGSY and CMSGY initiatives to support the rural connectivity agenda. The project interventions, in addition to addressing connectivity challenges, aimed to address cross-cutting issues like climate change adaptation and road safety. Relevance of the PDO. The development objectives of improving the durability and enhancing the resilience of rural roads to climate change factors and improving road safety directly responded to the country requirements and hence highly relevant, given that less than 5% of MP's population were occupied in non-farm activities. All three objectives of the PDO formulation were pitched at the outcome level of the results chain and appropriately pitched to address the identified development problem. In the state context, the project targeted population in rural areas who were not eligible under the national program. The PDO continued to be relevant to the Bank's current CPF for India. The overall relevance is therefore rated as High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective PDO 1. To improve durability and enhance resilience to climate changes of the gravel surfaced rural roads in MP. Rationale Theory of change. The project activities aimed at improving the durability and enhancing resilience to climate changes of the gravel surfaced rural roads. The causal links between project activities (inputs), outputs and the intended outcomes were logical. The outcomes were monitorable. The outputs of activities such as sealing the surface of gravel surface roads to conventional bitumen standards or other alternative options, providing additional road connectivity options to eligible villages and piloting post- construction road maintenance through women self-help groups (SHGs), were likely to help in realizing the desired outcomes of improving the durability and resilience to climate changes of rural roads. The outputs were expected to aid in the long-term long-term development outcome of improving the welfare of the rural population in MP. The results framework is predicated on the assumption that the rural roads are regularly maintained. Outputs. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054)  10,979 kilometers (km) of roads were upgraded to bitumen surface, exceeding the target of 10,000 km. Of this, 2,398 km were constructed using technology with plastic waste, exceeding the target of 2,100 km. This technology enabled reduction of plastic in the environment and provided a lifespan of at least two times more than the bitumen surface. The site-specific climate resilience standards that were implemented included: (i) raising embankments in flood-prone areas; (ii) new water crossings; and (iii) water drainage systems; Road safety features were provided (road markings and traffic calming measures) through integrating Intelligent Transport Systems (ITS). Other enhancements included hygiene improvements around hand pumps, protection works along water bodies and an additional 158.06 km of last mile connectivity link from main roads to socio-economic facilities (schools, hospitals and cremation areas).  484 km of alternative roads were constructed, slightly short of the target of 510 km.  At closure, 38,570,552 people were connected by all weather paved roads, exceeding the target of 36,500,000.  Ten women self-help groups of over 100 members (including tribals from three districts) were employed for routine road maintenance, exceeding the target of five.  The percentage of high school girls shifting from walking to biking to schools increased from 10% at the baseline to 84.02% when the project closed, exceeding the target of 80%. This result was however not wholly attributable to the project activities. Outcomes. The outputs described above were expected to contribute to two outcomes: (i) reduce annual road maintenance cost; and (ii) improve the International Roughness Index (IRI).  The annual road maintenance cost per km decreased from US$4,696.62 per km at the baseline to US$3,244.19 per km, realizing savings of 30.9% against the estimated target of 25%.  The average Roughness Index (RI) of roads at the end of the project was 3.22 meter (m), as compared to 7m at the baseline and exceeding the target of 3.5 m.  MP carried out an assessment of the impact in the project area through a desk review and interviews and focus group discussions in ten divisions in the state, 20 districts and 100 villages. 800 households were interviewed. The key findings were: (i) The travel time to visit agricultural markets decreased from 90 to 120 minutes before the project to 30 to 60 minutes after the project: (ii) The travel time to visit an urban center decreased from 60 to 90 minutes to 30 to 60 minutes after the project; (iii) The use of fertilizers and improved seeds increased from 78.8% to 97.8% in the project-intervened areas; (iv) The prices of land around the villages increased from one to four lakh rupees before the project to 8 to 12 lakhs after the project; and (v) Off-farm employment opportunities increased from 100 to 200 days before the project to 200-300 days after the project. The achieved outputs made a significant contribution to realizing the intended outcomes of improving the durability and enhancing the resilience to climate changes of the gravel surfaced rural roads in MI. Therefore, efficacy of this PDO is rated High. Rating High Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) OBJECTIVE 2 Objective PDO 2. To build the capacity of the state to manage its rural road network. Rationale Theory of change. The project aimed at strengthening the capacity of the Madhya Pradesh Rural Road Development Authority's (MPRRDA) to manage rural roads. The outputs of project activities (inputs), such as designing and implementing a Rural Road Asset Management System (RRAMS) to upgrade the existing system, establishing an inventory of the network (with all its elements such as current condition and network performance, its asset value, forecast of future demand traffic and service needs, estimate of maintenance needs and cost, prioritization of the quality and performance objectives and funding scenarios for regular road maintenance), training MPRRDA's research units on using the system, updating rural road design standards, reinforcing field laboratories in selected districts and organizing study tours for MPRRA staff, were likely to help in realizing the intended outcome of strengthening MPRRDA's capacity to manage rural roads. The causal links between the project activities, outputs and outcomes were logical. The intended outcomes were monitorable. The theory of change explicitly assumes that the MPRRDA is adequately staffed. Outputs.  The Rural Roads Asset Management System (RRAMS) was rolled out as targeted. The system is now being used for monitoring performance of the entire MP road network, including state highways (well beyond the rural roads envisaged under the project). At closure, MP had prepared and adopted a five, ten and fifteen-years' maintenance policy. The system included a Rural Accessibility Index (RAI), an important global indicator used in the transport sector for monitoring progress of the rural connectivity agenda.  E-learning management system modules were completed and design packages were procured as targeted. 320 MPRRDA staff were trained on using the system. Technical trainings were provided to MPRRDA staff as targeted on topics such as Road safety audit, Pavement design, construction, evaluation and new technologies for rural roads.  A design and research unit was established in MPRRDA as targeted.  Field laboratories of MPRRDA at the district and block levels were provided with modern information technology and testing equipment as targeted. Outcomes. The outputs described above were expected to strengthen the capacity of MPRRDA to manage the rural road network for scientific based maintenance prioritization and investment plans.  At the baseline, there was no Geographic Information System (GIS) network road inventory data and no scientific based maintenance prioritization and investment plans. The RRAMS was operational to prioritize rural roads maintenance plans. When the project closed, the MPRRDA had prepared multiyear road maintenance plans, that were approved by the competent authorities. The RRAMS is now being used for monitoring performance of the entire MP road network, including state highways (well beyond the rural roads envisaged under the project). Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) The achieved outputs made a significant contribution to realizing the intended outcomes of building the capacity of the state to manage rural roads. Therefore, efficacy of this PDO is rated high. Rating High OBJECTIVE 3 Objective PDO 3. To build the capacity of the state to manage its road safety. Rationale Theory of change. The project activities (inputs) aimed to improve the capacity of the state to improve road safety. The outputs of activities such as developing a multi-institutional and multi-sectoral Road Accident Data Management System (RADMS) (including a Geographic Information System (GIS) based accident data collection software system for recording accidents, storage, analysis and dissemination), piloting community participatory road safety initiatives, implementing road safety engineering interventions and training relevant staff on road safety enforcement, post-crash emergency management and road safety education, were likely to aid in realizing the intended outcome of building the state's capacity to improve road safety. The casual links between project activities (inputs), outputs and outcomes were logical. The intended outcomes were monitorable. The results framework explicitly assumes that there is effective coordination between the respective agencies during implementation. Outputs.  An Integrated Road Accident Database (IRAD) was developed and implemented to manage road safety through timely reporting of road accidents as targeted. Key stakeholders, including the police, health and transport departments were given access to the system to analyze road accident data. When the project closed, the entire MP road network (including state highways, major and ordinary district roads and village roads) was covered by IRAD.  500 Gram Panchayats reported on road traffic crashes when the project closed as targeted. (Gram Panchayats are village councils. They are basic governing institutions in Indian villages and the members of the gram panchayats are elected directly by the people).  16 crash locations, blackspots, junctions and pedestrian facilities were improved, short of the target of 50.  A pilot community and participatory road safety program (PCPRSP) was launched for the first time in India in three districts of MP, focusing on education, engineering, enforcement and emergency management, as targeted. The content and final scope of the pilot program was informed by road user surveys and focus group discussions that included drivers, pedestrians, students, teachers, Non- Governmental Organizations (NGOs), rural women groups, engineering professionals and police groups. The program included safety interventions such as the use of roller crash barriers, solar powered speed display boards and speed-limit markings on road pavements. Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Outcomes.  The scope of the Road Accident Data Management System (RADMS) was extended beyond the state highway to cover the entire road network of MP (not just the rural roads). When the project closed, 100% of the state highway network were covered under RADMS. The achieved outputs made a significant contribution to realizing the intended outcomes of building the capacity of the state to manage road safety. However, there was moderate shortcoming in achieving the target of crash locations, blackspots, junctions and pedestrian facilities improved. Therefore, efficacy of this PDO is rated Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The results framework is clear and the project achieved outputs from the activities made a significant contribution to realizing the intended PDOs of improving the durability and enhancing the climate resilience of the gravel surfaced roads, building the capacity of the state to manage its rural road network and improving road safety. Therefore, overall efficacy is rated high. Overall Efficacy Rating High 5. Efficiency Economic analysis. The Bank conducted an economic analysis of project activities, which accounted for about 96% of the appraisal estimate and actual cost. Since the project interventions were on low traffic volume rural roads, the methodology used a consumer surplus approach. (Consumer surplus refers to the difference between the price a consumer would be willing to pay for a product and the price they actually pay. It is an economic measure of the extra benefit consumers receive when they pay less than they are willing to pay). Under this approach, the primary project benefits were assumed to come from: (i) savings in vehicle operating costs (VOC); (ii) travel time savings; and (iii) reduction in carbon savings. The Net Present Value (NPV) at 6% social discount rate at appraisal was Indian Rupees (IR) 50, 684 and the ex-ante Economic Internal Rate of Return (EIRR) was 25.5%. The NPV at closure was IR 33,324 and the ex- post EIRR was 19.4%. The ex-post EIRR was lower than the ex-ante EIRR as the observed traffic flows at closure were lower than the forecasted traffic flows at appraisal. Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Administrative and operational issues during implementation. Savings were realized during implementation due to the depreciation of the IR relative to the US$. At closure, 98.91% of the loan was utilized and the balance cancelled due to the savings realized during implementation; The project closed only six months behind schedule, despite the prolonged negative effects of the COVID-19 pandemic. Overall efficiency is rated Substantial in view of the minor shortcomings. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 96.00 Appraisal  25.50  Not Applicable 96.00 ICR Estimate  19.40  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the PDO to the current Bank strategy for India and the national and MP strategies is rated High. Overall efficacy is rated High and efficiency is rated Substantial. Taking these ratings into account, outcome is rated as highly satisfactory. a. Outcome Rating Highly Satisfactory 7. Risk to Development Outcome The ICR (para 64) noted that there is moderate risk to the sustainability of development outcome. Government commitment/ ownership. The ICR noted that both the national and the state government remain strongly committed to improving the rural road connectivity and providing the rural population with all- weather roads. Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Technical risk. The ICR (para 64) noted that the project roads had five-year maintenance post construction embedded in the contract. This would ensure that there is road maintenance in the short run (for a five year period). Institutional capacity risk. The Rural Roads Asset Management System (RRAMS) developed under the project is expected to support MP in preparing and implementing multi-year budget plans for the managing the rural road network in the medium and long run (for up to fifteen years). By the end of the project, MPRRDA had trained the required field staff for preparing the road maintenance plans under this project. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank prepared this project based on the experiences from previous Bank-financed transport projects in India. The lessons incorporated at design were: (i) a simple design, as experience with transport projects in India had shown that projects packed with complicated institutional development activities that did not consider the weak capacity of the implementing agencies encountered difficulties: (ii) customizing the Performance Assessment Tool (PAT) developed by the transport team in Europe and Central Asia for monitoring highway projects for rural roads; (iii) recognizing that rural roads are used more by pedestrians than cars, adapting the design to address the needs of pedestrians; and (iv) leveraging the project to address sector-wide issues (although project investments were focused on rural roads, the road crash database management system was designed to address road safety issues on the entire road network) (PAD, pages 18 - 19). The lessons incorporated at appraisal were appropriate and there was no modification of design during implementation. The World Bank prepared this project in close collaboration with the Asian Infrastructure Investment Bank (AIIB) who were co-financing 40% of the project cost. The implementation arrangements of this project were: (i) the Madhya Pradesh Road Development Company (MPRDC) was overall in charge of coordination; (ii) Project Implementation Units (PIUs) at districts were to be in charge of day-to-day implementation; (iii) project implementation arrangements included a Project Management Consultant (PMC) at headquarters and a Project Implementation and Supervision Consultant (PISC) at field level. The arrangements were appropriate following the implementation arrangements made at appraisal (PAD, para 39). The preparation team identified risks associated with the weak implementation capacity and fiduciary risks. Mitigation measures were incorporated at design. With these measures, overall project risk was rated as moderate at appraisal (PAD, para 54). The measures were adequate, as project performance was not compromised by the risks identified at appraisal. The arrangements made at appraisal for safeguards and fiduciary compliance were appropriate (discussed in sections 9 and 10). There were minor shortcomings. One, the absence of timely involvement of other key stakeholders besides the MPRRDA negatively affected the completion timelines of the road safety activities. And two, one of the intermediate indicator was not fully attributable to project activities (discussed in section nine). Bank performance at Quality-at-Entry is rated as satisfactory. Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Quality-at-Entry Rating Satisfactory b. Quality of supervision Twelve supervision missions were conducted over the project lifetime of approximately six years. The project had three Task-Team Leaders (TTLs) from preparation to closure and transition from one TTL to the next was smooth, as most task team members remained unchanged during implementation. The supervision team offered guidance on issues affecting project performance by exploring workable solutions with MPRDDA. Through regular interactions with the Government, the supervision team mobilized the relevant stakeholders to complete the delayed road safety activities. Following the recommendations of the Mid-Term Review (MTR) held on March 15, the Bank held a knowledge exchange workshop to discuss best practices at the country level and in the region and recommended changes in procurement for some activities that were behind schedule. This enabled the project to be completed with only a six-month extension of the closing date. The support provided by the supervision team aided in safeguards and fiduciary compliance (discussed in section 10). Bank performance during supervision is rated as satisfactory. Overall Bank performance is rated as satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was clear. The selected key outcome indicators were appropriate for measuring progress towards achievement of the PDO (discussed in section four). The MPRDC was in charge of monitoring and evaluation, with data from the project units, district offices, head Quarters and road crash data from the police department. The data sources were clearly specified at appraisal. There were some minor shortcomings. The basis for the baseline and end target for the PDO indicator "Annual maintenance cost per km" could have been clearly defined to be the maintenance cost of the project-supported roads. The intermediate indicator " the percentage of girls shifting from walking to cycling" was not entirely attributable to project activities. Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) b. M&E Implementation Data required on the results framework were collected regularly during implementation and the quality of the Quarterly Progress Reports (QPRs) was adequate to monitor progress (ICR, para 52). Implementation was also tracked through specific reports from the Project Management Consultant (PMC) to the Bank on actions agreed in project management, safeguards and fiduciary aspects. c. M&E Utilization The ICR (para 52) noted that M&E reports, in addition to tracking progress, were used to take appropriate actions during the MTR on changes to procurement, closing date extension and cancelling part of the loan due to the savings realized during implementation. In sum, overall M&E is rated as substantial, due to the minor shortcomings with the M&E framework at appraisal. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category B (partial assessment) project under WB safeguard policies: Three environmental policies were triggered at appraisal: Environmental Assessment (OP/BP 4.01); Physical Cultural Resources (OP/BP 4.11); and Indigenous Peoples (OP/BP 4.10). (PAD, page 4). Environmental Assessment and Physical Cultural Resources. The adverse impacts (such as erosion, debris disposal and work side safety), were expected to be site-specific and temporary. Safeguards on physical cultural resources was triggered in case of chance finds of cultural property in project areas. An Environment Management Framework (EMF) was prepared and publicly-disclosed at appraisal to address the safeguards (PAD, page 92). The ICR (para 54) noted that safeguard aspects were reflected in standard bidding documents, environmental and social and health and safety (ESHS) management strategies and implementation plans. The ICR (para 54) noted that environmental safeguard requirements specified in the environmental and social management plans were adhered to during implementation and there were no major environmental or physical cultural issues. Indigenous Peoples. The safeguards on indigenous peoples was triggered, as the project included activities in 21 tribal districts. A Social Management Framework (SMF) and a Vulnerability Framework (VF) was prepared and publicly disclosed to address issues pertaining to indigenous peoples (PAD, para 80). There were no issues with indigenous peoples during implementation and the project redressed all the 318 grievances when the project closed. Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) b. Fiduciary Compliance Financial Management (FM). The MPRRDA was in charge of FM. MPRRDA had implemented previous Bank-financed transport projects. An FM assessment conducted at appraisal, concluded that the FM arrangements were adequate. The FM risk was rated as moderate (PAD, para 73). The project's FM was rated as satisfactory throughout implementation, with unqualified external audit reports submitted in a timely fashion. Procurement. The Bank conducted a procurement assessment at appraisal. The assessment concluded that the procurement arrangements were adequate and procurement risk was rated as moderate at appraisal (PAD, para 74). There were no procurement issues when the project closed (ICR, para 58). However, there were two procurement challenges during implementation. One, 285 packages for road works had to be retendered due to the change in Goods and Service tax (GST) introduced in India in 2017. and (ii) There was low response to the consultancy packages for the Rural Roads Assessment System (RRAMS) in the initial years. These issues were resolved by project closure. c. Unintended impacts (Positive or Negative) There were no unintended impacts. d. Other Not applicable. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Highly Satisfactory Highly Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- High 12. Lessons The ICR draws the following main lessons from the experience of implementing this project, with some adaptation of language. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) 1. Good Quality-at-Entry, based on the lessons from previous projects, may raise the potential for success of projects. This project built on previous rural connectivity programs at the national level and at the state level. This was supported by knowledge exchange workshops during implementation. Based on the experience from the national level program, timely procurement for most project activities (with 30% of works contracts awarded prior negotiations) contributed to effective and timely arrangements. These factors, coupled with appropriate implementation arrangements, contributed to the project success. 2. Early involvement of relevant stakeholders can help in preventing delays during implementation. This project experienced delays in completion of road safety activities due to the lack of timely responsiveness of the many key stakeholders involved. The lesson from this project is that where implementation requires joint efforts, early involvement of the stakeholders can help in preventing delays during implementation. 3. A combination of measures aiming at responding to the challenges on the ground (as opposed to "one size fits all") may be required for designing climate mitigation measures. Given the state context, this project had various interventions to cater for both heavy rains and high temperature impacts (Bitumen treatment and plastic waste) that were selected to respond to the challenges of the ground. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well-written, concise and adheres to the recommended page length. The theory of change expounded in the text clearly shows the casual links between project activities, outputs and the intended outcomes. The results framework explicitly states the assumptions underlying the theory of change. The ICR provides adequate evidence for assessing project performance. The ICR draws good lessons from the experience of implementing this project. The pictorial illustration provided in Annex seven enables the reader to visualize the changes made to the rural roads in the project-intervened areas. Overall, the quality of the ICR is rated as High. a. Quality of ICR Rating High Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MP Rural Connectivity Project (P157054) Page 16 of 16