Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Report Number: ICRR0023436 1. Project Data Project ID Project Name P121731 IN: ICDS Syst Strength & Nut Imp Prog Country Practice Area(Lead) India Health, Nutrition & Population L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-88390,IDA-51500 31-Oct-2015 269,922,583.88 Bank Approval Date Closing Date (Actual) 06-Sep-2012 30-Aug-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 106,000,000.00 0.00 Revised Commitment 283,583,925.58 0.00 Actual 269,922,583.88 0.00 Prepared by Reviewed by ICR Review Coordinator Group Shiva Chakravarti Salim J. Habayeb Eduardo Fernandez IEGHC (Unit 2) Sharma Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The PDO was to support the Recipient and the Participating States to strengthen: (a) the ICDS policy framework, systems and capacities, and facilitate community engagement, to ensure greater focus on children under three years old; and (b) convergent actions for improved nutrition outcomes (Financing Agreement, p.5). The initial operation envisaged a two-phased Adaptable Lending Program (APL) to improve nutritional outcomes of children in India (PAD, p. ii), but the APL approach was discontinued in 2015 (ICR, p. 7). The Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) objectives were revised on March 30, 2018. The new PDO was to improve the coverage and quality of ICDS nutrition services to pregnant and lactating women and children under 3 years of age. Explanatory note on ICDS: According to ICR (p. 52), The Integrated Child Development Services (ICDS) program is the Government of India’s flagship program that caters to pregnant/lactating women and 0-6-year- old children. It offers a range of services including supplementary food, health education, and immunization to enhance nutrition. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 30-Mar-2018 c. Will a split evaluation be undertaken? Yes d. Components Component 1. ICDS Institutional and Systems Strengthening (Appraisal: US$ 62.80 million; Actual: 197.81 million): The component aimed to enhance the effectiveness of the Integrated Child Development Services (ICDS) program. For the initial project (2012-2015), the envisioned activities revolved around six areas. The first was reviewing, refining, and aligning policies, guidelines, and procedures across ICDS structures, encompassing activities like human resource reform. The second was to broaden the ICDS monitoring and evaluation system by activities such as revising existing management practices and tools. The third involved capacity building, entailing the review and adaptation of training models and testing decentralized capacity-building methods. The fourth area was to bolster convergence between the ICDS program and another Indian government program, 'National Rural Health Mission', amplifying coordinated multi-sectoral nutrition efforts. The fifth area aimed to foster innovation through piloting initiatives such as adapting the ICDS program for urban settings. Finally, the activities under Component 1 aimed to fortify program leadership by enhancing leadership capacity at decentralized levels. Under the restructured project (2015-2018) and Additional Financing phase (2018-2021), the project activities under Component 1 shifted to narrower, high-impact interventions. Firstly, the project intended to enhance the capacities and skills of ICDS field workers and supervisory staff, employing the Incremental Learning-by-doing approach (ILA). Secondly, the project aimed to employ Information and Communication Technology (ICT) tools for enhanced program management, monitoring, and beneficiary outreach. Component 2. Community Mobilization and Behavior Change Communication (BCC) Strengthening (Appraisal: US$ 27.30 million; Actual: 71.09 million): The component aimed to enhance community mobilization and strengthen behavior change communication. For the initial project (2012-2015), the project emphasized four sets of interventions. One, the project aimed to strengthen the supply and demand-side interface of the ICDS program. Two, the project intended to review past community participation experiences and piloting promising approaches, with plans to scale up successful pilots. Three, the project intended to promote community involvement through initiatives such as mothers' support groups and Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) change agents, such as tribal leaders. This focus area also encompassed empowering community-based organizations and piloting partnerships with NGOs. Finally, the project intended to stimulate demand by strengthening BCC, including a review and analysis of existing BCC practices, and developing and implementing community-driven and evidence-based BCC strategies. The restructured project (2015- 2018) and the Additional financing (2018-2021) focused on fewer high impact interventions. One, the project intended to implement an intensive mass media campaign on maternal, infant, and young child nutrition. Two, the project intended to strengthen community engagement through the organization and institutionalization of monthly community-based events. Component 3. Performance-Based Incentives (added during 2018-2021 Additional financing) Strengthening (Appraisal: US$ 0 million; Actual: 79.44 million): The component aimed to support performance-based incentives to ICDS field functionaries and state leadership. The project supported monthly incentives to ICDS field functionaries for specified positive outcomes such as home visits to prioritized households and organization of monthly village health and nutrition days, which were measured through the ICT monitoring tool. The project also included incentives to project states on the achievement of agreed upon outcome indicators. Component 4. Convergent Nutrition Action (Appraisal: US$ 0 million; Actual: 25.22 million): This component aimed to support the Government of India's efforts to initiate and expand a comprehensive convergent nutrition response to complement the ICDS to address the multiple determinants of malnutrition. Key activities under this component included development of frameworks and tools for facilitating multi- sectoral convergent actions, strengthening of structures at central and state levels for planning, coordination and monitoring nutrition actions across multiple sectors, development of action plans, and implementation/evaluation of pilots, focused evaluations, and operations research to systematically build the evidence base for convergent nutrition actions. The restructured project (2015-2018) and the Additional financing (2018-2021) focused on pilots and innovations to promote cross-sectoral convergence. Component 5. Project Management, Monitoring & Evaluation (Appraisal: US$ 0 million; Actual: 26.44 million): This component aimed to support effective project management through activities such as financing of specialists and office equipment; renting of vehicles; travel costs for key program staff; participation in planning, review, and dissemination workshops; participation in study tours and conferences financing administrative costs; and the monitoring and evaluation of project outcomes. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates At appraisal, the project was estimated at US$151.50 million, out of which US$106 million were financed by the World Bank through an IDA credit and US$45.50 million were borrower contributions. An additional financing in the amount of $400 million was approved on March 30, 2018, out of which US $200 million were financed by the World Bank through an IBRD credit and US$200 million were borrower contribution. US $20 million remained undisbursed by the end of the project. Actual project cost at closing was US $400 million. The project became effective on November 26, 2012, and closed on August 20, 2022. The project underwent five restructurings. Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) 1. The first restructuring of September 8, 2015, discontinued the APL instrument and introduced Disbursement-linked indicators (DLIs). The restructuring also made changes to the results framework, components, cost allocation, institutional and financial management arrangements, implementation and closing fates. 2. The second restructuring of March 30, 2017 reallocated funds between disbursement categories. 3. The third restructuring and additional financing occurred on March 30, 2018 changed project development objectives (PDOs), results framework, added a component on performance based incentives, and changes the closing dates. 4. The fourth restructuring of February 28,2019 changed disbursement arrangements. 5. The fifth restructuring of October 25, 2021 changed components, costs, and the results framework. 3. Relevance of Objectives Rationale At the time of approval, high levels of malnutrition were a significant human development constraint – one third of children born in India were “born with a low birth weight; by age five, 43 percent were underweight; and almost half of all children were chronically malnourished (stunted) with a very high risk of serious negative consequences for life” (ICR. P. 1). The project was aligned with the World Bank 2008 Country Assistance Strategy that identified “stubbornly high levels of malnutrition (CAS, p. 19)” as one of the key human development constraints. Enhancing service delivery through strengthened institutional arrangements that improve the enabling environment comprised one of the key CAS pillars. The PDO to support policy framework, systems and capacities on the supply side and mobilization of communities on the demand side, along with creating mechanisms to facilitate multi-sectoral actions were aligned with the CAS objective of enhancing the enabling environment. The objective was aligned with the 2018 SCD emphasis on the policy area of “early nutrition and stimulation” (SCD, p. 64). The project continued to be consistent with the country strategy at closing: Country Partnership Framework (FY 2018-2022) identifies the project as a key component of the country strategy under Focus Area 3, ‘Investing in Human Capital.’ The project was also aligned with the government strategy. According to the Government of India’s Eleventh Five Year Plan (2007-2012), one of the key goals was the reduction of malnutrition among children of age group 0-3. The PDO was revised in 2018 from the institutional focus to a service delivery approach. The Government of India wanted the project to be scaled up and aligned with the new National Nutrition Mission. The lowering of ambition and discontinuation of the APL instrumental presented some challenges to the relevance. At approval, the project directly aimed at impacting the enabling environment through a wide array of activities such as human resource reforms and decentralized district planning. However, in 2015, the ambition of the project design was scaled back, and resources were diverted to a more targeted Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) approach. The CAS also put forth the APL as an effective approach “to building capacities and strengthening systems.” However, the APL instrument was discontinued in 2015. The project was aligned with the World Bank sector experience in the country. The World Bank was engaged in India to combat malnutrition in the 1980s through a Nutrition project in the south Indian state of Tamil Nadu. The engagement continued through three projects focused on the ICDS program: (i) ICDS I in Andhra Pradesh and Orissa; (ii) ICDS II in Bihar and Madhya Pradesh; and (iii) ICDS III in Kerala, Maharashtra, Rajasthan, Uttar Pradesh and Tamil Nadu, along with a national training component (PAD, 11). While there were no lending initiatives after 2006, the World Bank was closely engaged with India through advisory and analytical assistance with the Ministry of Women and Child Development (MWCD) to “define the needed reforms and secure the required buy-in to reform ICDS and enable it to achieve its potential on nutritional outcomes” (ICR, p. 2). Overall, considering Bank's three-decade involvement in India’s Nutrition sector and the ICDS project's 5 years implementation at the time of objective revision, the revised objectives fell short of the necessary impact alignment for significant influence on child nutritional outcomes in India. The revised PDO, emphasizing access and service delivery quality, fall short to build upon the previously robust systems strengthening approach. Additionally, the service delivery approach provides limited insights into the demand side, despite the initial phase strongly emphasizing its importance. According to the original theory of change (ICR, p. 2), the project aimed at improving child nutrition as the ultimate outcome, a facet not fully addressed by the service delivery orientation. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Strengthen the ICDS policy framework, systems, capacities, and community engagement Rationale While the ICR presents the Theory of Change (ToC) diagram, it lacks further elaboration or clarification. The ICRR reconstructs the rationale behind the ToC by drawing from the ICR narrative found in the component section, the project appraisal document (PAD), and the project paper. The first objective of the project aimed to strengthen the ICDS system to improve child development services, which comprises nutrition counselling and outreach services for the pregnant and lactating women and children under the age of 3 and enhance community to ensure a greater focus on children under three years of age. The ICDS program is a Government of India flagship program initiated in 1975 aimed at improving the health and wellbeing of mothers and children under the age of six years through a combination of services including “health and nutrition education, health services (including immunization and growth monitoring), Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) supplementary food, and pre-school education, thus encompassing the troika of food, health and childcare and development interventions” (PAD, p. 2). These services are delivered by Anganwadi workers (AWWs: community workers) who are chosen by the community to serve the community in Anganwadi centers (rural childcare centers). Under the first objective, the project aimed to address both the demand and supply sides. To bolster the supply side, focusing on fortifying the ICDS policy framework, systems, and capacities, the project outlined a series of activities. These encompassed the review, refinement, and harmonization of ICDS policies, guidelines, and procedures, including human resource reform. Additionally, the project sought to revise existing management practices and tools to enhance the ICDS monitoring system, and it planned to review and adapt training models while testing decentralized capacity-building methods. Initiatives such as piloting the adaptation of the ICDS program for urban settings were also part of this supply-side strengthening. Concurrently, on the demand side, the project directed efforts toward amplifying the demand for ICDS services. Activities in this area involved refining and assessing existing behavior change communication (BCC) practices and implementation of community-driven and evidence-based BCC strategies, spanning media campaigns, community events, and mobilization drives. The project also planned outreach programs targeting groups like mothers' support groups and influential figures such as tribal leaders. Finally, the project intended to support the "interface between the supply and demand sides of the ICDS program, particularly during the delivery of nutrition services," with meticulous monitoring and evaluation of this interface (PAD, p. 9). After negligible performance, the project was heavily restructured in 2015. The key changes were the refocus of project activities, change in PDO indicators, and introduction of disbursement-linked-indicators. The APL instrument was discontinued and the refocus comprised “fewer high impact interventions” (ICR, p.7). Under the first objective, the project directed resources towards strengthening the capacities and skills of ICDS field functionaries and supervisory staff on the supply side. The capacity-building approach was to be executed through an "Incremental Learning-by-doing approach." Second, the project invested in technology tools, including mobile and tablet-based ICT tools, to enhance management, monitoring, and beneficiary outreach. These tools facilitated consistent real-time reporting and monitoring of service delivery at AWCs. On the demand side, the project focused on (i) an intensive mass media campaign on maternal, infant, and young child nutrition and (ii) the organization and institutionalization of monthly community-based events at the AWCs. While lower in ambition following the restructuring, the theory of change for the first objective was clear and convincing. An incremental learning approach to strengthen the capacity of the ICDS functionaries along with a fortified management and monitoring system on the supply side and an intensive media campaign and institutionalized community-based events on the demand side could reasonably be expected to strengthen the ICDS policy framework, systems, capacities, and community engagement. The key assumption of the first objective is that the listed project activities will strengthen policy frameworks, systems, capacities, and community engagement, which will thereby improve the quality, coverage, and utilization of the ICDS services. At the time of project approval, the ICDS was failing in meeting its goals and eradicating malnutrition in children. Evaluation work by the Government of India and the World Bank pointed out some reasons of the inadequacy of the ICDS efficacy. These reasons included (i) an overemphasis on food-based interventions at the cost of other important prerequisites such as utilization of health services, sanitation, and appropriate child feeding and caring practices (ii) low capacity of implementation agencies (iii) weak stewardship and funding. Moreover, in practice, ICDS largely focused on children between aged 3 and 6, which left out the critical first 1000 days of life. The capacity building activities, management and monitoring Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) system, and community-based events to focus on the early years can reasonably be expected to address the listed shortcomings. Intermediate Indicators 388,542 health personnel received training, exceeding the target of 280,000 62 percentage of Anganwadi Centers (AWCs) in project districts implemented monthly community-based processes focused on nutrition and health counselling (at least 5 events in the past 6 months), falling short of the target of 80 percent. Outcomes 100 percent project districts implemented the “incremental capacity building” program, exceeding the target of 80 percent. 60 percent of targeted AWCs reported key nutrition and service delivery indicators using ICT systems every month for the last 3 months, meeting the target of 60 percent. The PDO indicators were insufficient for measuring the broader outcome of strengthened “ICDS policy framework, systems, capacities, and community engagement " For instance, the first two indicators report on the existence of ICDS management information system and if the incremental capacity building is implemented. However, the indicators do not offer insights into the effectiveness of the training program or the MIS's contribution to reinforcing the policy framework, systems, and capacities. The Disbursement-linked indicators provide additional evidence to substantiate the PDOs. Additional evidence comes from Disbursement Linked Indicators. For the first DLI to be achieved, several indicators had to be accomplished: Standardized operating procedures /specifications for ICT monitoring and service management system issued by MWCD (DLI.1.1), Contracts for ICT hardware and supporting consulting services signed by each Participating State in accordance with guidelines set forth in the Operations Manual (DLI 1.2), At least 70 percent of AWWs in Project Districts in each Participating State trained in the ICT-based management, monitoring and communication system in accordance with the Training Plan (DLI. 1.3), National workshop held by MWCD on state nutrition and ICDS Program reporting using ICT (DLI. 1.4). All states achieved the DLIs, except Uttar Pradesh that did not achieve DLI 1.2. Similarly, for the second DLI to be achieved, the following actions were completed: At least 60 percent of District Resource Groups and 60 percent of Block Resource Groups in Project Districts in each Participating State are trained in 3 thematic modules for incremental learning (DLI 2. 1), At least 80 percent of DRG in Project Districts in each Participating State are trained in another 3 thematic modules for incremental learning over Year (DLI 2. 2), 10 thematic training modules for incremental learning developed in accordance with the Communications Plan (DLI 2.3). All states achieved DLI 2. For the third DLI to be achieved, the following measures were implemented: In all states, 80 percent of AWCs in Project Districts conducted at least 5 community processes in the previous 6 months (DLI 3.1), Content for TV and radio spots for mass media campaign was finalized and approved by MWCD in accordance with the Communications Plan (DLI 3.2), 80 percent of AWCs in Project Districts in each Participating State had conducted at least 10 community processes in the previous 12 months (DLI 3.3). Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Considering the achievement of DLIs and the two PDO indicators, this ICRR rates the objective as Substantial. Rating Substantial OBJECTIVE 2 Objective Strengthen convergent action for improved nutrition outcomes Rationale The second objective aimed to strengthen convergent actions to improve nutrition outcomes. While the ICR presents the Theory of Change (ToC) diagram, it lacks further elaboration or clarification. The ICRR reconstructs the rationale behind the ToC by drawing from the ICR narrative found in the component section, the project appraisal document (PAD), and the project paper. To bolster convergent actions for improved nutrition outcomes, a series of comprehensive interventions were planned. These interventions encompassed the development of conceptual frameworks, the reinforcement of national and regional planning and coordination structures, the pilot testing of multi-sectoral convergent actions, and the establishment of an evidence base for such actions. The project's aim was to strengthen both central and state-level structures for planning, coordinating, and monitoring nutrition-related initiatives across sectors, which included the creation of an inter-sectoral nutrition action coordination committee, designed to engage stakeholders in discussions about potential structural adjustments in policies and procedures to facilitate cohesive actions. Finally, the project aimed to evaluate and conduct research to substantiate the effectiveness of convergent nutrition actions in India. After a negligible initial performance, the project was heavily restructured in 2015. Under objective 2, the testing of pilots and nutritional action plans were prioritized. Additionally, the project provided flexibility to states to design and implement nutrition action plans as per their need and priority, while the MWCD (the central implementing agency) ensured that the pilots proposed by the state were within the purview of project’s development objectives. The key assumption underlying Objective 1 was that addressing nutrition outcomes requires collaborative efforts across multiple sectors. The PAD substantiates this by drawing on past project experiences and highlighting how various determinants influence nutrition outcomes (p. 34). These determinants encompass activities related to food and nutrient intake, healthcare, maternal and child feeding practices, hygiene, water, sanitation, and considerations of gender, equity, and governance. Another important assumption was that strengthening of institutional norms through activities such as developing of conceptual frameworks and reinforcing planning and coordination structures would facilitate convergent actions. The PAD draws on evidence from other World Bank projects, such as the World Bank Bachpan project in India and a nutrition project in Bangladesh, to demonstrate the effectiveness of these initiatives in promoting convergent actions. However, the assumption neglects to account for administrative and bureaucratic complexities, which indeed hindered the implementation of these reforms. The ICR acknowledges that during the initial phase, bureaucratic obstacles posed significant challenges (p. 22). Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Intermediate Outcome • National menu of options for convergent nutrition action pilots/studies for project states were prepared and discussed in a consultative workshop by the Central Project Management Unit (CPMU). • Annual Learning events organized were organized • 89.50 percent of project districts operationalized convergent nutrition action plans among those that developed them in the past year against a target of 60 percent Outcome District convergence committees in seven states developed convergent nutrition action plans, exceeding the target of six states. Additional evidence comes from the disbursement-linked indicators. For the fourth DLI to be achieved, the following activities were accomplished: (i) DLI 4.1. List of priority innovation areas and pilots eligible to be implemented under Part C of the Project agreed between MWCD and all Participating States and approved by the Association and (ii) DLI 4.2. At least one pilot designed by each Participating State to be implemented under Part C of the Project approved by MWCD. The expected outcome was the improved alignment of sector services, quantified by the PDO indicator "Number of project states in which convergent nutrition action plans have been developed by the district convergence committees in at least one district." The PDO indicator falls short in capturing the broader outcome of "strengthened convergent actions for improved nutrition outcomes." The indicator focuses on the development of action plans, without offering evidence of their effectiveness or their role in bolstering convergent actions for improved nutrition. Considering the partial achievement of DLIs and the PDO indicator, this ICRR rates the objective as Modest. Rating Modest OBJECTIVE 3 Objective Improve the coverage of ICDS nutrition services (New objective) Rationale For the additional financing (AF), there was a shift from “a process and capacity building focus to a service delivery focus” (ICR, p. 7). The AF expanded the intended coverage from 162 high burden districts across 8 states to 315 districts (out of the 766 total districts) across all states/UTs of India. The additional financing comprised the first phase of roll out of the National Nutrition Mission of the government of India. Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) The ICR does not present the ToC for the revised objectives. The ICRR reconstructs the ToC rationale using the ICR narrative in the component section and the project paper. The theory of change for the additional financing was clear and logical. The project continued to support the activities under the previous phases, while scaling them up to additional districts. These activities included the incremental learning approach for capacity building; community engagement and behavior change communication activities (specifically, CBEs, mass and mid-media campaigns); and the ICT-based management and monitoring tool. Additional initiatives under the additional financing included “the strengthening of existing citizen engagement and grievance redress systems to improve community engagement, the introduction of performance-based incentives to ICDS field functionaries and project states for improving service delivery, and the development and operationalization of convergent action plans (CAPs) for nutrition at the state, district, and block levels” (Project paper, p. 11). Intermediate Indicators 388,541 health personnel received training, exceeding the target of 280,000 99.8 percent of targeted AWCs reporting using POSHAN Tracker every month for the last 6 months, exceeding the target of 60 percent 903,027 ICDS personnel received training on POSHAN Tracker, exceeding the target of 560,000 37 percent project states had at least one mid-media campaign deployed in the past year, exceeding the target of 25 percent. 61 percent of AWCs in project districts implemented at least 4 CBEs focusing on agreed MIYCN themes in the last 6 months, almost meeting the target of 65 percent. 11 percent of CBEs implemented in project districts in the last 6 months that focus on community influencers, almost meeting the target of 15 percent. 85.68 percent of Block Resource Groups (BRGs) and sectors conducted at least four monthly learning and review meetings on nutrition knowledge and services in the last six months, exceeding the target of 60 percent. Outcomes 60,594 pregnant/lactating women (PLW) and children under 3 years in project districts were reached by the ICDS services, almost meeting the target of 64,000 12,099 children under 3 years in project district were reached by the ICDS services, exceeding the target of 8,700 The ICR also presents supplementary evidence in the efficiency section. Stunting rates showed more rapid improvement across the 11 priority states targeted by the project in comparison to the non-priority states. Within the priority states, the stunting rate decreased by 4 percentage points, contrasting with the 0.04 percentage point reduction in the non-priority states. Although not establishing causation, since multiple factors might contribute to these outcomes, especially considering that slower-performing states have the Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) potential for quicker growth, the ICR notes a higher utilization of ICDS services in priority states – 76.3 percent to 63.2 percent, as opposed to non-priority states – 72.3 percent to 66.5 percent. Additional evidence came from the DLIs. The project tracked the DLIs under Objective 1 and Objective 2. Considering the achievement of DLIs and the two PDO indicators, this ICRR rates the objective as Substantial. Rating Substantial OBJECTIVE 4 Objective Improve the quality of ICDS nutrition services (New objective) Rationale For Theory of Change, See Objective 3 Intermediate Objectives 388,541 health personnel received training, exceeding the target of 280,000. 100 percent of project districts with district resource groups trained to implement the incremental capacity building system, exceeding the target of 95 percent. 76.94 percent targeted AWWs trained on the ICT based management, monitoring and communication system, almost meeting the target of 80 percent. 56.9 percent of AWWs in project districts who have completed all online capacity building modules, almost meeting the target of 60 percent. 99.8 percent of targeted AWCs reporting using POSHAN Tracker every month for the last 6 months, exceeding the target of 60 percent. 903,027 ICDS personnel received training on POSHAN Tracker, exceeding the target of 560,000. 37 percent of states strengthened grievance redress system, exceeding the target of 25 percent. 61 percent of AWCs in project districts implemented at least 4 CBEs focusing on agreed MIYCN themes in the last 6 months, almost meeting the target of 65 percent. 11 percent of CBEs implemented in project districts in the last 6 months that focus on community influencers, almost meeting the target of 15 percent. Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) 37 percent of project states had at least one mid-media campaign deployed in the past year, exceeding the target of 25 percent. 95 percent of project districts operationalized convergent nutrition action plans among those that developed them in the past year, exceeding the target of 60 percent. 1.5 percent of AWWs provided with training on POSHAN Tracker, who – in the last six months – received their monthly performance incentive at least 4 times, falling short of the target of 60 percent. 85.68 percent of Block Resource Groups (BRGs) and sectors conducted at least four monthly learning and review meetings on nutrition knowledge and services in the last six months, exceeding the target of 60 percent. Outcomes 60.40 percent of children age 6-8 months received solid or semi-solid food and breastmilk, exceeding the target of 55 percent. 38 percent of children 6-23 months of age received foods from 4 or more food groups, exceeding the target of 30 percent. 43 percent of AWCs delivered a minimum set of services at AWCs in the last month, falling short of the target of 50 percent. The project tracked the DLIs under Objective 1 and Objective 2. Considering the achievement of DLIs, the three PDO indicators, this ICRR rates the objective as Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The original objectives to strengthen the ICDS policy framework, systems, capacities, and community engagement to ensure greater focus on children under three years of age was almost fully achieved; and strengthen convergent action for improved nutrition outcomes was partially achieved. The achievements are consistent with a marginally Substantial overall efficacy rating. Overall Efficacy Rating Substantial Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale The two new objectives to improve the coverage and quality of ICDS nutrition services to pregnant and lactating women and children under 3 years of age were almost fully achieved. The achievements are consistent with a Substantial rating for overall efficacy. Overall Efficacy Revision 1 Rating Substantial 5. Efficiency The PAD noted that a rate of return analysis for the project was deemed inappropriate by a leading team of Human Development economists. The recommendation was to focus on “why the institutional reforms and capacity development that are proposed in the project are critical to improving service delivery” (PAD, p. 15). Consequently, the PAD stated that the project was designed to improve allocative efficiency by promoting interventions focusing on children 0-3 years of age, the first 1000 years that are intended to produce the most important outcomes. Following the PAD, the ICR focuses on allocative and technical efficiency. The ICR mentions that the original and restructured project enabled the additional financing investments to be directed to the interventions that were evidence-based and demonstrated results. For instance, the project “incentivized the setup of convergence platforms where different ministries and departments that implement interventions impacting nutrition outcomes, came together to achieve a synergy of nutrition interventions to target undernutrition” (ICR, p.18). The ICR also reported that the project’s support to MWCD enabled the ministry to increase its resource envelop and sustain project activities – the national nutrition mission, as a share of the ICDS program increased from 8 percent to 15 percent between 2017-2018 to 2020-2021. The ICR reports a $5 per person per year cost of the project but does not provide any evidence on the efficiency aspect of this amount, i.e., if this amount is comparable to other nutrition projects. The original project 2012- 2015 faced several implementation challenges, with only 6 percent of amount being disbursed. There was a significant lack of activity (ICR, p. 23). The complexity of the project design, limited capacity of the implementation agency, and bureaucratic hurdles were major impediments to implementation. While the project assessed capacity risk as high, the mitigation measures were clearly inadequate. The restructuring helped improve the pace of the project by re-focusing the project on fewer but high impact interventions, introducing the disbursement-linked-indicators, and replacing the technical agency with a Bank executed Multi donor Trust Fund that provided “high quality technical assistance” (p. 6). Yet, 7.3 percent of the funds remained undisbursed due to delayed disbursement in the second half of AF due to CoVid and shift in the monitoring tool. Despite nudging by the World Bank, the undisbursed amount was not cancelled and returned. During the implementation period, there was large variability in implementation and performance among the states, and the transition from the project monitoring tool to a government tool of tracking resulted in a lack of data availability for substantial periods of time. Efficiency Rating Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of project objectives is rated Substantial throughout the project duration, as the objectives were mostly aligned at project closing with the Bank's Country Partnership Framework for India (FY 2018-2022) and the government's policy efforts and reforms. Efficiency is rated as Modest based on the allocative and technical efficiency of the project. The implementation delays and the undisbursed amount under additional financing impacted the project's efficiency. Original Project – Moderately Satisfactory Efficacy is rated Substantial, as the project almost fully achieved its original development objectives to Strengthen the ICDS policy framework, systems, capacities, and partially achieved community engagement to ensure greater focus on children under three years of age; and Strengthen convergent action for improved nutrition outcomes. The outcome rating is Moderately Satisfactory because there was modest achievement of one objective used in the assessment of efficacy. Project after Revision – Moderately Satisfactory Efficacy is rated Substantial, as the two new objectives to improve the coverage and quality of ICDS nutrition services to pregnant and lactating women and children under 3 years of age were almost fully achieved. Rating Dimension Original Objectives Objectives after first revision Relevance Substantial Objective 1: Strengthen ICDS policy framework, systems and capacities, and Substantial community engagement Objective 2: Strengthen convergent Modest actions for improved nutrition outcomes Objective 3: Improve the coverage of Substantial ICDS nutrition services Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Objective 4: Improve the quality of ICDS Substantial nutrition services Overall Efficacy Substantial Substantial Efficiency Modest Outcome Rating Moderately Satisfactory Moderately Satisfactory Numerical value of the outcome rating 4 4 Amount Disbursed (US$ million) US $80.91 million US $182.01 million Disbursement (%) 31 69 Weight Value 1.24 2.76 Total weights 4 Overall Outcome Rating Moderately Satisfactory a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The ICR assessment assigned a Low risk rating to development due to the following reasons: a) High Government Commitment: The ICR underscores the commitment exhibited by both central and state governments, with example of this commitment being institutionalization of project activities. These activities encompass a range of actions, such as prioritized home visits, integration of training modules for ICDS functionaries and supervisors into the ministry's training curriculum, and the establishment of a regular practice of performance-based incentives. Moreover, the notable adoption of the incremental learning approach by the Government of India's Capacity Building Commission across all administrative echelons further reinforces this high level of commitment. b) Continuation of Project Interventions: The project's impact extends beyond its duration through the preservation of the enhanced capacities and systems fostered under its purview. Noteworthy among these is the Poshan Tracker, an instrument dedicated to monitoring nutrition outcomes, and the effective grievance mechanisms. Both of these will endure beyond the project's timeline. Overall, there is sufficient evidence to suggest that the high government commitment and continuation of project interventions will sustain outside of the project. 8. Assessment of Bank Performance a. Quality-at-Entry Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) The project was aligned with the World Bank 2008 Country Assistance Strategy. One of the key pillars of CAS included improved service delivery India through enhanced enabling environment. The project development objective to support policy framework, systems and capacities on the supply side and mobilization of communities on the demand side, along with creating mechanisms to facilitate multi- sectoral actions were precisely aligned with the CAS objective of enhancing the enabling environment. The project included an equity plan to ensure that the project equally benefitted the tribal communities, low caste population, and religious minority. The project drew on existing evidence from the World Bank experience and other large scale nutrition programs to inform project design. The project used a technical assistance agency to provide technical support to country project management units and state project management units. The PAD highlighted that the procurement of the technical assistance agency was a critical factor for the success of the project. However, the capacity of the technical agency was not assessed adequately. The TA agency had to be replaced by qualified consultants at the state and central level under the Multi-donor trust fund. The procurement for the project was planned as per World Bank Group guidelines: Procurement of Goods, Works and Non-Consulting Services under IBRD Loans and IDA Credits & Grants by World Bank Borrowers (PAD, p. 40). The project relied on country systems for the audit process and the fiduciary implementation arrangement required states to make a two-fold submission that proved complex (ICR, p. 26). The project’s quality at entry suffered from underestimation of risks commensurate to the complexity of project design. The ICR stated that the “weakness was appropriately highlighted as a high risk; however, mitigation measures were insufficient according to the ICR (p. 22) and mitigation impacts were overestimated in terms of time” (p. 27). The underestimation was evident from the underperformance of the project before the restructuring. During the 2012-2015, only 6 percent of the project funds were disbursed. At entry, project implementation arrangements did not consider the time and resources required to reform human resources (ICR, p. 22). The ICR also noted the complexity of the project design, which was “trying to right every wrong documented (ICR, p. 22).” Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Project supervision remained consistent, adapting even during COVID lockdowns, utilizing virtual platforms for field visits. In the initial years of the Original Project, teams ensured regular supervision. Despite determined efforts and increased hands-on support, bureaucratic challenges persisted. Inadequate mitigation strategies, combined with disappointing Technical Agency performance, resulted in implementation coming to a halt. The 2015 restructuring marked a pivotal reinitiating. The restructuring identified "the main implementation hurdles, and proposed a highly effective new approach and a critical change to the TA provision to the MWCD and the States" (p. 27). The revised approach simplified the ambitious plan, concentrating on evidence-based interventions and employing existing government structures for delivery. High-quality Technical Assistance (TA) was ensured through the MDTF, remedying earlier shortcomings. Shifting from traditional input-based financing to results-based financing via new Disbursement-Linked Indicators (DLIs) at central and state levels promoted accountability. Joint implementation review missions with government, WB, and MDTF donors were successful, enhancing Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) quality and coordination. Coordination and harmonization with donor partners, facilitated by the MDTF, emerged as a valuable practice. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design In the original project phase, clearly defined objectives lacked appropriate indicators, hindering the measurement of broader outcomes. For example, while the PDO indicators for the first objective report on the existence of ICDS management information system and if the incremental capacity building is implemented, the indicators provide no information on if and how the ICDS management information system and the incremental capacity building program contributed to the strengthening of the policy framework, systems, and capacities. These shortcomings were partly addressed by the disbursement- linked indicators. In the revised phase, the PDOs were appropriate for measuring the service-delivery outcomes of the additional financing. Moreover, the original M&E framework of the project supported the ongoing reforms of the national ICDS program, which supported simplifying data collection, validation, monitoring, and evaluation. b. M&E Implementation Throughout the implementation, a significant focus was placed on the ICDS-CAS system and subsequently the Poshan tracker. These systems ensured consistent data collection and the timely application of this data for informed decision-making. The implementation faced several constraints during the initial phase. The development and testing of the IT system took place only under the Restructured Project, with its implementation expanding to cover all States. This continued under Additional Financing (AF) and was eventually replaced by the Poshan Tracker in early 2021, as per a government decision. However, this transition led to two significant disruptions: one due to the shift to the Poshan Tracker and another caused by the COVID-19 pandemic. The introduction of the Poshan Tracker notably boosted the sustainability of the monitoring systems. This was attributed to the fact that the government owned and oversaw its implementation (as mentioned in the TTL Interview, July 20, 2023). c. M&E Utilization The IT platform greatly facilitated prompt M&E, eased beneficiary data recording, and tracked child growth. It served as both a job tool and a time-saving aid for AWWs, enhancing service quality. ICDS supervisors benefited from improved supervision, course correction, and data analysis. The system empowered AWWs to stick to plans, prioritize needy families, and better allocate time between tasks. Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Clear and measurable indicators were employed for Communication for Social and Behavioral Change. The project set realistic targets after revising ambitious ones, aligning with CBEs and outreach visits by trained AWWs to encourage positive feeding practices. Verification methods validated AWW and CBE field conduct. Findings from the POSHAN Knowledge and Behavior survey informed recommendations, emphasizing improved nutrition behaviors through robust service delivery. Leveraging the MDTF, research and evaluation flourished. Insights from systematic assessments, interviews, and surveys shaped interventions and innovations, enhancing overall project effectiveness. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category C project because of the lack of risks related to infrastructure work. No environmental assessment was undertaken under the project. No other safeguards were triggered (PAD, p. 20). The project prepared an equity plan covering disadvantaged communities including tribal population, Scheduled castes, and religious minorities, to "guide the project in ensuring it responds to the special needs of these underserved populations" (PAD, p. 20). b. Fiduciary Compliance While the project’s financial management was rated as "Moderately Satisfactory" for most of the project, it dipped to "Moderately Unsatisfactory" in February 2022. This downgrade was due to delays in submitting and auditing financial reports. The project relied on the country's audit system instead of having its own. For the project's success, specific states needed to submit two reports: one detailing their spending (Eligible Expenditure Program) and another from the Auditor General. This became complex as not all states met the timeline. Additionally, COVID-19 led to extended submission times for reports in 2019-20 and 2020-21, alleviating some pressure. A state's delay caused a rating drop, but eventually, all required reports were submitted, restoring the financial management rating to "Moderately Satisfactory" upon project completion. c. Unintended impacts (Positive or Negative) No unintended impacts were applicable for inclusion in this section. d. Other None reported Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Efficiency was impacted by Outcome Satisfactory Moderately Satisfactory implementation delays and undisbursed funds Insufficient risk mitigation and Bank Performance Satisfactory Moderately Satisfactory complexity of project design at entry. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons Employing specific and clear incentives can serve as important motivators to achieving results. The project's transition from input-driven funding to disbursement-linked indicators effectively enhanced state involvement to achieve outcomes. Similarly, offering incentives to ICDS field functionaries for enhancing their performance, including timely home visits and organizing community events, emerged as appropriate motivational tools, driving tangible results. Harmonized donor coordination can enhance efficacy and efficiency. The Multi-donor Trust fund mechanism, established in partnership with the Bill and Melinda Gates Foundation (BMGF), Children’s Investment Fund Foundation (CIFF), and Tata Foundation, played a pivotal role in course correction by offering valuable technical assistance. The harmonized donor coordination through a single trust fund proved effective in aligning efforts under a unified thematic framework and optimizing resource utilization. Leveraging technology can bolster accountability and efficiency. Through project implementation, the project demonstrated use of technology solutions like the ICDS-CAS and subsequently, the Poshan Tracker. These mobile technologies demonstrated notable advantages such as substantial time savings for field workers, quality of service delivery, and the capacity to track outcomes in real time. By capitalizing on these innovations, the program demonstrated how technology integration can significantly enhance operational effectiveness and provide an accurate and up-to-date account of progress. An incremental and institutionalized approach to capacity building ensures effectiveness and sustainability. The incremental learning approach, as executed in the program, encompassed identifying issues, devising locally viable solutions, and gradually applying this knowledge in a phased manner. This involved addressing challenges faced by functionaries in prior training rounds and implementing solutions in the subsequent month's practices. The strategy focused on progressively building upon small increments of learning, gradually integrating all skills, Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) comprehension, and actions into regular practice, and ensuring full internalization by the functionaries. Incorporating project interventions into national systems fosters long-term sustainability. Project initiatives such as the incentives-based payments were adopted as routine practice in the ICDS program. Similarly, the grievance redressal system will continue beyond the project. For instance, the grievance redress mechanism established as a helpline in Gujarat has evolved into a two-way system, enhancing accountability between various tiers and empowering beneficiaries by providing them a platform for voicing their concerns. Lessons learned from the project were integrated into the Government approach to combat malnutrition. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a comprehensive overview of the project. It was candid and reasonably concise. Its outcome orientation was at an intermediate level, but related moderate gaps were rooted in the initial M&E design rather than the ICR itself. The project underwent substantial restructuring on two occasions, and these transitions were effectively discussed and the changes to the PDO indicators were distilled into tables with clarity. By drawing on the implementation experience, the ICR successfully gathered evidence and openly acknowledged the deficiencies and limitations encountered during the initial phase of the project. It offered several important lessons derived from project experience. The ICR exhibited some shortcomings. The split rating table ought to have been grounded in objective-specific ratings, evaluating the accomplishment of the original objectives throughout the project's entire lifespan, until their revision/drop in 2018. Additionally, the ICR failed to present a satisfactory explanation of the theory of change, which the ICRR reconstructs by drawing from the ICR narrative found in the component section, the project appraisal document (PAD), and the project paper. a. Quality of ICR Rating Substantial Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: ICDS Syst Strength & Nut Imp Prog (P121731) Page 21 of 21