Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Report Number: ICRR0024209 1. Project Data Project ID Project Name P162956 TD Climate Resilien Ag & Product Proj Country Practice Area(Lead) Chad Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D3020,IDA-D8060 31-Dec-2023 48,016,190.65 Bank Approval Date Closing Date (Actual) 30-Apr-2018 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 41,000,000.00 0.00 Revised Commitment 56,000,000.00 0.00 Actual 48,016,190.65 0.00 Prepared by Reviewed by ICR Review Coordinator Group Shashidhara Laxman Vibecke Dixon Avjeet Singh IEGSD (Unit 4) Kolavalli 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objective stated identically in the Financing agreement (p. 5) and the PAD (p. 5) is to “promote the adoption of improved technologies leading to increased productivity and to enhance the climate resilience of agricultural production systems in the areas targeted by the project,” For this review, the objective will be parsed into two: Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Objective one: to increase productivity by promoting the adoption of improved technologies in the areas targeted by the project. Objective two: to enhance the climate resilience of agricultural production systems in the areas targeted by the project. CERC (Component C) was activated. The efficacy in achieving its objective will be assessed under objective two. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components The project had four components: Component A: Institutional support for sustainable agriculture development and climate resilience (Appraised: US$12.6 million; Actual: US$8.25 million). This component included three subcomponents: 1. Strengthening the agricultural research and development (AR&D) system (a) Support for adaptive research systems including, inter alia: (i) the rehabilitation of the infrastructure (laboratories and facilities) and equipment of selected facilities for the Chadian Institute for Agriculture Research and Development (ITRAD); (ii) specialized short and long term training of researchers; (iii) support to the setting-up of an innovative and sustainable Agricultural Research and Development (AR&D) funding system; (iv) the production of high-quality breeder and foundation seeds for targeted crops; and (v) competitive research and development (R&D) funding. (b) Support for strengthening research-extension-farmer linkages including, inter alia, for consultation and cooperation among the stakeholders in agricultural R&D. (c) Support for the Recipient's participation and integration in regional networks and cooperation in agriculture, including, inter alia, through: (i) extension to the Recipient of West and Central African Council for Agricultural Research and Development (CORAF's) technical networks on strengthening regional cooperation and sharing of experience, knowledge and training; (ii) development and implementation of annual communication action plans based on the regional communication strategy prepared by CORAF, including the networking of knowledge management, information, and communications systems to accelerate the sharing of agricultural technology, tools, and best practices; (iii) support to the Recipient in preparing and implementing an action plan to mainstream climate change considerations in R&D programs; (iv) support for developing and implementing an action plan to mainstream gender considerations in R&D programs based on the gender strategy prepared by CORAF; (v) expansion of CORAF's regional Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) competitive agricultural research grant scheme to the Recipient; and (vi) technical assistance for the creation and implementation of a national competitive agriculture research grant mechanism. 2. Strengthening of national support services for sustainable agricultural development and climate resilience (a) Support for revitalizing agricultural advisory support services and mainstreaming ICT including, inter alia, support to the Recipient's National Agency for Rural Development (ANADER). (b) Support for strengthening the capacity of producer organizations and the federation of producer organizations, through inter alia: (i) training of trainers; (ii) short and specialized training courses; (iii) workshops, showcase events and study tours; (iv) linkage with development partners and the private sector; and (v) acquisition of equipment. (c) Support for developing farmer-oriented agro-climatic services including, inter alia, support for the operation of the agro-meteorological multidisciplinary working group (GTP) and the acquisition of weather forecasting digital solutions. 3. Strengthening the framework for sectoral strategies and reforms preparation and Monitoring Support for development and dissemination of sector policies and strategies, preparation of studies, gender impact evaluation and training in policy analysis and evaluation for the staff of the Recipient's Ministry of Agriculture. Component B: Supporting adoption of demand driven technologies and climate-smart agriculture Appraised: US$ 27.2 million [IDA: US$23.6 million; Beneficiaries: US$3.6 million]; Actual: US$19.34 million [IDA: US$17.31 million; Beneficiaries: US$2.03 million]) This component included four subcomponents: 1. Improving the efficiency of agricultural support services in the targeted areas (a) Support for fostering demand-driven agricultural advisory services, including, inter alia: (i) financing the implementation of innovation platforms; (ii) capacity building of producer organizations; (iii) capacity building of public and private local extension services; and (iv) supporting agro-climatic services delivery. (b) Support for effective technical support services for sustainable agriculture through the provision of Matching Grants to finance Sub-projects focusing on, inter alia: (i) planning and production of improved seeds and seedlings for the targeted crops; (ii) multiplication and dissemination of improved livestock breeds and fish fingerlings; and (iii) access to inputs and small mechanization services. 2. Accelerating adoption of improved technologies and innovations (a) Support for improved technology adoption and innovation to promote sustainable intensification of targeted local production systems including, inter alia, through provision of Matching Grants for Sub- projects. (b) Support for diversification and value addition through the provision of Matching Grants to selected private sector entities, for the use of technologies and innovations. Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) 3. Support to the Implementation of Climate-Smart Agriculture Plans (CSA-Plans) Support for the preparation of local community investment plans and implementation of climate-smart agriculture activities, including through the provision of Matching Grants for Sub-projects. 4. Resilience building of targeted refugees, returnees and selected host communities Building the resilience capacity of targeted refugees, returnees and vulnerable members of host communities through the provision of: (a) Cash for Work (CfW) Transfers; and (b) training, aimed at: (i) contributing to their purchasing power; (ii) increasing their financial management capacity; and (iii) contributing to productive asset building. Component C: Contingency emergency response (Appraised: US$0.0 million; Actual: US$13.08 million). Provision of rapid response to an Eligible Crisis or Emergency, as needed. Component D: Project management, coordination, monitoring and evaluation and knowledge management (Appraised: US$4.8 million; Actual US$7.5 million) Support for Project implementation, including, inter alia: (i) coordination and management, including human resources, financial management and procurement; (ii) monitoring and evaluation and technical studies; (iii) knowledge management and communication; (iv) safeguards and citizen engagement, including piloting an iterative beneficiary monitoring (IBM) system for obtaining feedback and informing on Project implementation; (v) establishing a grievance redress mechanism; (vi) financing of Incremental Operating Costs; (vii) rehabilitation of office space required for Project management; and (viii) equipment required for Project management and supervision. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost The project was appraised to cost US$59.6 million and the actual cost was US$50.05 million. Financing The project was financed by two IDA grants totaling US$48.02 million. Borrower Contribution At appraisal, local beneficiaries were expected to contribute US$3.6 million; they contributed US$2.03 million. Dates The project was approved in April 2018 and became effective in November 2018. A Mid-Term Review was held in March 2022. The project closed as planned in December 2023. Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Restructurings The project underwent three restructurings: May 2020: to integrate the residual activities of the Emergency Food and livestock Crisis Response project (PURACE) into Climate Resilient Agriculture and Productivity Enhancement Project (PROPAD) to consolidate the agriculture portfolio under the World Bank financing in Chad. It resulted in i) changes to results framework (RF)(Three intermediate level Indicators were added), ii) changes in components and costs, iii) reallocation between disbursement categories and iv) changes in institutional arrangements. August 2022: to bring additional finance into the project to fill the financing gap created by CERC which was activated in May 2020. It resulted in i) additional financing, ii) changes to RF (Two intermediate level indicators were added), and iii) reallocation between disbursement categories. June 2023: to implement the recommendations of the Mid-Term Review (MTR). It resulted in i) changes to RF (Three intermediate indicators were added; the targets for three PDO level indicators were revised upwards, while that of the sub-indicator of one of them was revised down; and targets for six output level indicators were revised down, ii) reallocation between disbursement categories, and iii) changes in institutional arrangements. Changes to RF  The first restructuring added three intermediate level indicators under activity B (total beneficiaries who participated in cash for work program (number), productive assets rehabilitated (number), and people trained (number)).  The second restructuring introduced two intermediate level indicators under component C (Beneficiaries that received emergency food assistance (number) and Smallholder farmers provided with improved seeds and small agricultural equipment (number)).  The third restructuring i) added three intermediate level indicators under component C for activities implemented by FAO (Quantity of cereals and oilseeds seeds distributed (Ton), Quantity of seeds of garden vegetables and fruits distributed (ton), Vegetable gardening tools distributed (Ton)), ii) revised upwards the PDO indicators, and iii) revised downwards some of the intermediate indicators on which the project was behind. The revisions to targets are summarized below. Level Indicator Original target Revised target Farmers reached with agricultural assets or services PDO 360,000 500,000 (No.) - Farmers reached with agricultural assets or services – PDO 108,000 150,000 female (no.) PDO Surface area under improved technology 21,400 40,000 - Surface area under Climate-smart Agriculture PDO 10,700 3,000 technologies/practices Output Young researchers trained 17 16 Output Young researchers trained – female 50 25 Output Agriculture sector reforms supported 5 4 Output Improved seeds produced by targeted beneficiaries 3,000 850 Output Diversification subprojects supported 360 250 Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Output Pilot local community CSA subprojects financed 100 50 Split Rating The PDO remained the same through the restructurings. The project became more ambitious. Several intermediate level indicators were introduced as the first and the second restructurings brought additional resources into the project. The targets for three PDO indicators were revised upwards, whereas the target for one sub-indicator was revised down. The last restructuring revised down the target for several intermediate level indicators. As a split rating would not have changed the ratings with equally ambitious PDO level indicators and downward-revised output indicators, this review will not apply a split rating. 3. Relevance of Objectives Rationale Country context When the project was appraised in early 2018, Chad’s oil and agriculture-dependent economy was under stress (ICR, para 1). Annual Gross Domestic Product (GDP) growth had plummeted from 6.3 percent in 2014 to 0.2 percent in 2017 due to a sharp drop in oil prices in 2014. Agricultural growth also exhibited excessive volatility, from 10.1 percent in 2015 to -6.0 percent in 2016 due to increasingly erratic weather patterns. These unfavorable developments were exacerbated by the growing insecurity spilling over from neighboring countries (Sudan, Nigeria, Central African Republic, and Libya). The confluence of these factors was undermining food security and triggering the urgency for improving the performance of the agriculture sector on which the country was highly dependent – the sector employed 80 percent of the Chadian workforce. Of the country’s 14 million inhabitants at the time, 47 percent of them lived below the poverty line. Average yields of the country’s staple cereals were among the lowest in Sub-Saharan Africa. An Agriculture Sector Review in 2017 showed that some of the factors behind the poor performance included inadequate institutional capacity, particularly in agricultural research and extension services, widespread use of low-yielding agricultural technologies (less than 5 percent of the farmers were using improved seeds), climate shocks, and lack of coherent agricultural policies (ICR, para 2). Country strategy The Government of Chad developed the National Development Plan (PND 2017-2021) and the National Rural Investment Plan (PNSIR 2016-2022) as frameworks to guide sector investments. The PDO remains relevant to Chad’s Nationally Determined Contribution (NDC), which includes the development of adaptation actions in the agricultural sector to ensure food and nutrition security through enhanced crop and livestock productivity. The PDO remains consistent with the government’s overall goals outlined in its “Vision 2030: The Chad We Want”, and the National Development Plan (PND 2017-2021), which seek to transform Chad through diversification, given the country’s over-reliance on the oil sector (ICR, para 17). Bank strategy Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) The PDO aligned with World Bank’s Country Partnership Framework (CPF) for Chad for FY2016 – 2020, which emphasized support to the country to improve returns to agriculture and reduce vulnerability, and the World Bank Group (WBG) Africa Climate Business Plan, especially its focus on contributing to farmers’ access to climate services and the adoption of climate-smart agricultural practices (ICR, para 3). The PDO remains consistent with i) WBG’s Country Engagement Note for the period FY23-FY24 especially with its objective of promoting “natural resource management and sustainable agriculture to prevent and manage conflicts based on natural resource scarcity and climate change”. (These themes have been continued in the new Bank-funded project, the Chad Agribusiness and Rural Transformation Project (P179238).) ii) World Bank’s Western and Central Africa Regional Strategy, especially Goal 4 pertaining to promoting “climate resilience.” Iii) the Systematic Country Diagnostic approved in April 2022, which underscored the importance of supporting climate-smart agricultural practices and ensuring sustained agricultural productivity growth through innovation. Furthermore, the PDO is also aligned with key elements of the WBG Strategy for Fragility, Conflict, and Violence (FCV), 2020-2025, and the Global Crisis Response Framework especially regarding “building the resilience and preparedness of communities, including the ability to manage climate change and environmental degradation.” The level at which PDO is pitched The PDO of increased crop yields and enhanced climate resilience of agricultural systems was pitched at the outcome level of the logical chain. Achieving the objective would contribute directly to the country strategy of achieving food and nutrition security, and it is pitched at an appropriate level to address the development problem of the country’s dramatic decline of agricultural growth and increasing vulnerability to climate change induced natural hazards. Summary: Chad’s oil and agriculture dependent economy slowed down considerably when oil prices fell sharply in 2014. The agriculture sector also performed erratically because of low adoption of technology and climate shocks. The PDO to increase productivity and make the agriculture systems climate resilient was highly relevant to Chad’s objective of achieving food and nutrition security and diversifying away from oil and the World Bank’s strategy to improve to returns to agriculture and reduce its vulnerability to climate shocks. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase productivity by promoting the adoption of improved technologies in the areas targeted by the project. Rationale Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Theory of Change (ToC) The retrospectively developed ToC articulated in ICR (para 4 and Figure 1) states that project inputs/activities, such as strengthening of research and extension capacity, strengthening demand driven value chain development through innovation platforms and supporting policy formulation, which produced outputs, such as researchers trained, quantity of foundation seeds produced, and the number of innovation platforms supported, would help farmers increase productivity of selected crops. The critical assumptions in the ToC were that the project recommended technologies would be superior to technologies farmers were already employing, they would adopt the technologies, and that the technologies would increase yields on the farms of adopters. The ToC was plausible if the technologies were adequately tested for suitability over the entire project area and farmers were supplied with inputs including information to adopt the technologies. Outputs  15 young researchers were trained, falling short of the revised target of 16  Of them, 25 percent were female researchers, meeting the target of 25  122.40 MT of foundation seeds of cereals were produced, exceeding the target of 97 MT  41.10 MT of foundation seeds of other crops, falling short of the target of 48.00 MT  21 improved technologies were disseminated by the project, exceeding the target of 20  4 agricultural sector reform/policies and strategies were supported, meeting the target of 4  9 operational and regional innovation platforms were supported, meeting the target of 9  95 benefited directly from using agricultural climate services, exceeding the target of 95  1,117 MT on improved seeds were produced by the target beneficiaries, exceeding the target of 850 MT  20,000 smallholder farmers were provided with improved seeds and small agricultural equipment, meeting the target of 20,000 Outcomes  685,286 farmers were reached with agricultural assets or services, exceeding the revised target of 500,000  189,844 farmers that were reached with agricultural assets or services were female, exceeding the revised target of 150,000  68,483 ha came under improved technology disseminated by the Project, exceeding the revised target of 40,000 ha  30.50 percent higher were the average yields obtained by direct beneficiaries compared to average yields of non-beneficiaries, exceeding the target of 25 percent.  78 percent of the targeted beneficiaries rated the project interventions ‘Satisfied’ or above, falling short of the target of 80 percent  79 percent of the targeted female beneficiaries rated project interventions ‘satisfied’ or above, falling short of the target of 90 percent. Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) The achievement of the objective was measured by one indicator: percentage increase in average agriculture yield of direct beneficiaries compared to average yields in the project area. The wording of the indicator suggests a 25 percent higher increase in yields among beneficiaries compared to average yields in the project area, which presumably includes beneficiaries as well. The ICR reports average yields obtained by beneficiaries and non-beneficiaries in different years (Table 5, page 14). The yields obtained by beneficiaries was 30.5 percent higher than that of non-beneficiaries during the last season of the project (2023-24). And the yields were also 39 percent higher than the baseline yields (ICR, para 18). Comparable information on any increase in yields among non-beneficiaries is not reported. The information on yields came from annual surveys of 2,216 households that included 997 beneficiaries and 1,219 non-beneficiaries. There were a few limitations in meeting the target set by the indicator. One, the 30.5 percent higher yields is a mathematical average for the eight crops. For four crops, which were on nearly 46 percent of the area, the yield increase was lower than the target: sorghum: 17.2%; cowpea: 10.1%; groundnuts: 19.5% and maize: 12.1%. Additional information provided by the team on August 19, 2023, suggested that the weighted average yields too were higher by 36.8 percent – the average remains high because of 64.9 percent higher yields of sesame grown on nearly 35 percent of the area. Two, the increase in yields was not consistent across the three participating regions (ICR, Table 6). Three, the yields of beneficiaries did not exceed that of the control group until the last year, 2023-24 (ICR, Table 5). The ICR attributes the delayed increase in yields to the improved seeds reaching farmers only at the end of the project and combined effect of adoption of complementary technologies over time (para 22). The project increased the yields by i) reaching more than the targeted number of farmers with agricultural assets or services and ii) bringing more area than planned under technologies disseminated by the project. Crop yield growth was driven by access to inputs and knowledge (ICR, para 25). The project nearly met or exceeded the target for producing foundation seeds – beneficiaries produced more seeds than targeted. The Food and Agricultural Organization of the United Nations (FAO), which was contracted to implement CERC activities, also supplied seeds. The project trained farmers in improved agricultural practices. Summary: The project aimed to enable beneficiaries to achieve yields that are 25 percent higher than the average yields, which the ICR has interpreted to be yield among non-beneficiaries. The average of yield differences of eight crops was higher than the target, but the yields of four of the eight crops, which were nearly 47 percent of the area, were not sufficiently higher. The yields were also not higher among the three regions. The yield increases can be attributed to project activities that included production and supply of seeds, training, and support to subprojects. With the minor shortcoming in not raising the yields of all the selected crops and across all regions, efficacy in the achievement of objective 1 is rated substantial. Rating Substantial OBJECTIVE 2 Objective To enhance the climate resilience of agricultural production systems in the areas targeted by the project. Rationale Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) ToC The retrospectively articulated ToC in the ICR (para 4 and Figure 1) states that project inputs or activities, such as i) promoting early maturing, drought tolerant, and disease resistance crop varieties, ii) addressing predisposition to climate vulnerability, iii) diversifying sources of income to avoid reliance on vulnerable crops and iv) piloting community wide natural resource management, would be expected to produce outputs such as an increased area under climate smart agricultural technologies, number of subprojects supported to improve resilience at community levels , and number of farmers supported with seeds and implements would make agricultural production systems more climate resilient. A critical assumption was that the climate smart technologies recommended to farmers were appropriate to manage the climate risks they faced and that they could be assisted to diversify into countercyclical activities that would make them more climate resilient. The evidence to prove that they have become more resilient, given the project activities, included information on improved yields from the technologies and increased diversity in income sources. Outputs  179 diversification subprojects were supported, falling short of the target of 250  45 CSA sub-projects for resilience at the community levels were supported, falling short of the target of 50  2,080 sub-projects to intensify sustainable production systems were supported, falling short of the target of 2,100 CERC  194,952 people received emergency food assistance, meeting the target of 190,000  20,000 smallholder farmers were provided with improved seeds and small agricultural equipment, meeting the target of 20,000  16MT of seeds of garden vegetables and fruits distributed, meeting the target of 16 MT  100,000 vegetable gardening tools distributed (FAO), meeting the target of 100,000 Activities transferred from the PURACE project  12,367 participated in Cash for Work rotations (IOM), falling short of the target 13,600  25 productive assets were rehabilitated, meeting the target for 25  5,201 were trained (IOM), falling short of the target 5,183 Outcomes  31,545.7 ha came under Climate-smart Agriculture technologies/practices, exceeding the original target of 10,700 ha and the revised target of 3,000 ha No single indicator in the RF captured resilience (ICR, para 18). Given the activities, the proxies for improved resilience were the performance of climate smart technologies, increased diversification into other activities, and better preparedness for climate events. The only indicator in the RF that met the requirement was the Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) “area under climate-smart technologies/practices”. No information was collected on how the climate smart technologies, a subset of the technologies promoted by the project, performed. The 21 technologies promoted by the project were improved varieties of the selected crops. The seeds distributed under CERC were also drought and disease resistant, in addition to possessing higher yield potential, and were expected to contribute to building resilience (ICR, para 27). The project succeeded in bringing the targeted area under climate smart technologies. To help beneficiaries diversify their income sources, the project financed sub-projects, which enabled households to acquire and rear small ruminants and CERC activities distributed vegetable seeds, built irrigation systems, and distributed agricultural equipment. The project also rehabilitated vaccination parks to enhance resilience to animal diseases (para 28) and supported community subprojects that were designed to build resilience through landscape management involving agro-forestry activities (para 30). The activities implemented to address predisposition to vulnerability included CERC-financed food distribution, cash for work activities, which addressed the precarious food situation the beneficiaries faced by helping them bridge the gap before the next harvest and discourage them from consuming seeds meant for sowing or disposing off their productive assets (ICR, para 28). Summary: The project expected to make the production systems more resilient by promoting the use of climate resilient technologies, diversifying their sources of income, and making households less predisposed to climate events. The project increased the area under climate smart technologies, and satisfactorily implemented activities designed to help beneficiaries diversify their incomes sources and be better prepared for climate events. A minor shortcoming was the absence of evidence on how the proxies for improved resilience may have performed, the efficacy of achieving this objective is Substantial with moderate shortcomings. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The ToC hypothesized that supplying improved technologies, primarily improved seeds, along with building capacity of research and extension systems and development of appropriate policies would bring larger areas under improved technologies and increase the yields obtained by beneficiaries. Additionally, the theory hypothesized that supplying farmers with climate smart/resilient technologies, helping them diversify their income sources, and supplying them with food and other supplies, which would make them less predisposed to climate events, could help make their production systems climate resilient. The average yields of beneficiaries were higher than that of the non-beneficiaries, the difference exceeding the target. But the yields of four of the eight crops were only marginally higher, and significant differences in yields were observed only during the fifth year. The efficacy in achievement of objective one was rated Substantial. The project satisfactorily implemented activities designed to contribute to improving resilience. With a minor shortcoming Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) in the absence of evidence on how the proxies for resilience may have performed, the efficacy in achieving objective two was rated Substantial with moderate shortcomings. The overall efficacy in the achievement of project objectives is rated Substantial with shortcomings. Overall Efficacy Rating Substantial 5. Efficiency Economic efficiency At appraisal, the economic analysis considered i) direct benefits to farmers, ii) community level benefits from small rural enterprises, iii) and environmental benefits, which could be attributed to activities of Component B (PAD, para 56). Eleven crop and activity models were prepared to compare ‘with’ and ‘without-project’ situations. Without the valuation of environmental benefits, the EIRR was estimated to be 14.2 percent (PAD, para 60). With environmental evaluation at market prices, the estimated EIRR increased to 22.6 percent. Including GHG mitigation valued at the low range (US$49/) and high range (US$98), the EIRR increased to 56.7 and 97.1 percent respectively. The analysis repeated at closure developed 13 models, ten on crop production and 3 on income generating activities. The EIRR was estimated at 13.9 percent. Including environmental benefits at low carbon prices, the estimated EIRR increased to 61.6 percent. Administrative efficiency Several factors influenced the project’s implementation. The startup was delayed because of slow onboarding of critical staff, high staff turnover, and their slow replacement – resulting in several changes in counterparts. Lengthy procurement processes, with an average of 9 to 12 months for procurement valued more than US$52,000 (para 46) slowed down implementation. Floods in 2022 and 2023 disrupted production of seeds (para 49). Disbursements were also suspended for three months following political developments in April 2021. Many factors helped in more efficient implementation of the project, despite the initial delays: i) the project could make use of technologies developed by a regional project (para 36), ii) coordinating with other projects in Chad helped in providing weather forecasts to farmers, iii) and CERC interventions protected the agriculture sector from COVID-19 related setbacks. Government made a timely request to reallocate resources to CERC to mitigate the disruptions caused by the covid outbreak. Summary: Estimated returns close to rates projected at appraisal and higher than opportunity cost of capital combined with implementation without delays earns this project an efficiency rating of Substantial. Efficiency Rating Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  14.20  Not Applicable 0 ICR Estimate  13.90  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Following an economic crisis precipitated by a decline in petroleum prices, oil and agricultural dependent Chad wished to diversify the economy and intensify agriculture in its most productive regions. The objective to increase productivity and make its production systems climate resilient was highly relevant to the country’s objective and the World Bank’ strategy to support the country. The average yields of eight crops increased to surpass the target, but the increase in yields of four out of eight crops fell short of the target. The project increased the area under climate smart technologies, and satisfactorily implemented activities designed to help beneficiaries diversify their incomes sources and be better prepared for climate events. The efficacy in achieving the objectives was, therefore, rated substantial with shortcomings. The project, however, was implemented without any delay and the estimated returns were close to the projections at appraisal. The efficiency was, therefore, rated Substantial. The outcome rating emerges as Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR identified a few risks to development outcomes, although the project was followed by other World Bank funded projects that would continue to support related activities: the Food System Resilience Project (P178132) and the Chad Agribusiness and Rural Transformation Project (P179238). Continued access to improved inputs Farmers’ continued access to seeds and reasonably priced fertilizers poses a risk to sustaining outcomes. Chad’s domestic system for improved seed multiplication and distribution is at a nascent stage. Farmers’ organizations took the lead in seed multiplication in collaboration with Chadian Institute of Agricultural Research for Development (ITRAD). This collaboration is expected to continue. However, the government needs to continue to support and monitor to maintain seed quality and supply, in addition encouraging Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) private enterprises to invest in the sector. Similarly, given the current high prices of fertilizer, which had its roots in the Ukraine war, access to remuneratively priced fertilizer is likely to remain limited, although the government is working with the private sector and farmers’ organizations such as National Consultation Council of rural producers (CNCPRT) on the issue (ICR, para 61). Institutional risk Continued access to advisory services also poses a risk. The project has put in place a framework to improve the advisory services, which demonstrated that outreach can be expanded at low cost, but it is also in a nascent state and needs support to mature. 8. Assessment of Bank Performance a. Quality-at-Entry a. Quality-at-Entry The project was designed to meet priorities of the government to increase productivity and make agriculture more resilient as embodied in PND and PSNIR. The activities, which broadly included strengthening of research and extension systems, the supply of improved technologies including climate- smart ones, and support to communities to make the necessary investments, could be expected to contribute to achieving the objectives the project. The design benefited from an agricultural sector review that had been just concluded, which gave the project a sound technical ground (ICR, para 44). The project also drew from the regional experience of the bank financed West Africa Agricultural Productivity program (WAAPP) and the regional research body West and Central African Council for Agricultural Research and Development (CORAF). The project made arrangements that ensured effective implementation. For example, the project struck partnership with the National Consultation of Council of Rural Producers of Chad (CNCPRT) to ensure the project’s relevance to the intended beneficiaries (ICR, para 44). The project also built into design coordination and synergy with other ongoing projects (ICR, para 58). A detailed M&E, which included annual surveys, was adequate to monitor implementation but less so for tracking the theory of change. Although no single indicator could capture resilience, the RF would have benefited from indicators to capture the performance of proxies for resilience, such as yields of climate smart technologies, level of diversification of income sources, and preparedness of climate events. Summary: The project was designed – with components that could be expected to achieve the objectives - to meet the strategies and objective of the country. The implementation plan entailed coordination and partnership with relevant organizations. Due to the minor shortcoming in an M&E that did not adequately capture the proxies for resilience, the Quality-at-Entry is rated Satisfactory. Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Quality-at-Entry Rating Satisfactory b. Quality of supervision The World Bank team conducted the required biannual missions, 11 in all, with appropriate expertise. They offered candid aide memoirs and action plans (ICR, para 59). The local office provided consistent support with a TTL, who was based in the country, and a team whose composition remained stable. The team proactively addressed the concerns that arose, including by restructuring the project, but missed the opportunity to identify relevant outcome indicators to measure proxies for improvements in climate resilience, such as performance of climate resilient technologies and changes in diversification of income sources. Providing the expected support and for proactively addressing the concerns of the project, quality of supervision is rated Satisfactory. Summary: The design, built on relevant experience, delivered a project that could be expected to achieve the objective, consistent with strategies of the country and the World Bank. Implementation arrangements that included collaborations with local organizations and coordination with related projects were sound. The design, however, had a minor shortcoming in the M&E system, which did not capture the outcomes of the proxies for resilience and this weakness was not addressed during implementation. The Bank team provided the support necessary for effective implementation and proactively addressed the issues as they emerged. The overall Bank Performance is also rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The extensive M&E design, with planned annual surveys, adequately met the requirements for monitoring implementation but fell short of generating adequate information to test the ToC. The PDO indicators were measurable but they were not enough to capture relevant outcomes, related particularly to improved resilience (ICR, para 20). Of the four PDO indicators, only two related to the outcomes associated with achievement of project objectives. PDO level indicators that measured changes Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) in resilience, such as changes in diversification of income sources, could have been included, given that the annual surveys included generated the required information. The design made provisions for baseline surveys and continuous data collection. b. M&E Implementation Changes were made to the RF during two restructurings, which brought additional activities and resources into the project. The sub-activities were expected to contribute to improving resilience – through diversification of incomes and addressing predisposition to climate vulnerability. The changes introduced tracked implementation of activities but did not measure the outcomes of those activities. The baseline study, midterm evaluation, and final evaluations were carried out as well as annual surveys. Daily information was also generated on the usage of e-extension and e-voucher activities (ICR, para 52). c. M&E Utilization The daily recording of the use of e-extension helped the project calibrate the language capabilities needed at the call centres and stock the types of inputs farmers were demanding (ICR, para 52). Summary: The M&E design was elaborate including baseline, endline, and annual surveys. The system was implemented as designed and used to monitor and finetune project activities. Even though the M&E design had minor shortcomings in monitoring of outcomes, the ICR provided additional evidence to justify the Satisfactory rating. Overall, the M&E quality is rated Substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental The project, classified as a category B project, triggered Environmental Assessment OP/BP 4.01 and Pest Management OP 4.09 safeguards. The mitigation measures specified in the Environmental and Social Management Plans (ESMPs) and the Pest Management Plans (PMPs) were implemented. Environmental screening and social assessments were carried out at the level of all subprojects financed by the project in line with the World Bank's environmental and social safeguard policies. The Project complied with environmental management safeguards. Social Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) The Project complied with social safeguards. The project triggered Physical Cultural Resources OP/BP 4.11 and Involuntary Resettlement OP/BP 4.12 social safeguards. There were no cases of physical cultural resources involved in the project. The project relied on a joint Grievance Redress Mechanism (GRM) developed for three Bank-funded projects that were being implemented in the same target areas, namely the ProPAD, the Sahel Irrigation Initiative Project (P154482), and the Rural Mobility and Connectivity Project (P164747). The GRM was accessible to project-affected people. Local resolutions committees were active in regions where project activities were being implemented. The project had set up a functional complaints management committee in each of the project cantons. The project recorded 18 complaints. All complaints were resolved in an inclusive manner involving the complainants, the committees, and the PCU, and were archived. COVID-19 control measures were adopted consistently throughout the pandemic period although, unfortunately, a regional director was lost to the pandemic. There were no reports of Occupational, Health and Safety (OHS) incidents during project implementation. The project had not recorded any cases of Gender Base Violence (GBV) (ICR, paras 55). b. Fiduciary Compliance Procurement Procurement documents for all contracts financed by IDA proceeds were uploaded into Systematic Tracking of Exchanges in Procurement (STEP) as well as all signed contracts, acceptance certificates and payment documents. There was no reported malfeasance in the procurement processes. Financial Management Interim Financial Reports were submitted on time and were of acceptable quality. Project audits were generally submitted on time and financial statements were certified without reservation. The disbursement rate under IDA financing was 90.07 percent because of slow project implementation as discussed earlier (ICR, para 57). The ICR does not offer any information on whether an independent/external audit was conducted or the outcome of one. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR presents five lessons, which are also presented here with some adjustments by IEG. E-extension, when curated to meet the audience requirements and preceded by user publicity, has the potential to expand the outreach of agricultural advisory services. The experience from the project suggests that extensive testing within a limited geographical area before the full roll out, widespread publicity to elicit demand, and meeting specific requirements of the audience (for example, after-hour services or provision for a particular language) are necessary to make e-extension effective. It also must be complemented with non-electronic services – through extension staff – in areas without good phone network. E-vouchers that are delivered to mobiles can be an effective tool for efficiently targeting input subsidies to poor farmers. The experience suggests that qualification criteria must be clearly defined, ample time must be provided for enrollment before the season, and input availability needs to be ensured through coordination with suppliers. Coordination is also necessary with other programs that can assist households without a mobile and incapable of paying the unsubsidized component of the price, to make the system inclusive. A multi-faceted approach to disseminating technologies may contribute to achieving results in a low institutional capacity environment. This project used a judicious combination of delivery mechanisms, including (i) strategic partnerships with farmer organizations, (ii) contracting private operators to provide advisory services, (iii) using “relay farmers” trained and equipped to assist fellow farmers, and (iv) establishing call-centers for on-demand advise. These combined efforts helped the project achieve significant results in a country with a poorly staffed, poorly equipped, and poorly funded agricultural extension service. Community seed production can complement a private sector-based seed production that is still in a nascent stage. It required effective collaboration between the primary stakeholders – the researchers as parent seed suppliers, the government as certifying agent, and farmers’ federations as producers, support to government certifying agencies to supervise the sites, and support producers to improve their production conditions through measures like small-scale irrigation. Focusing on strongly felt needs can enhance the chances of implementing reforms in a politically and institutionally fragile environment. During this project’s implementation, senior government officials were changed frequently (e.g., five ministers in charge of agriculture for a project that was never extended). Yet, the project was able to successfully support reforms in research and extension, and overall sector management. This project’s experience would suggest Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TD Climate Resilien Ag & Product Proj (P162956) that for successfully supporting reforms in a fragile environment, the reforms must address felt needs that transcend the interests of leaders of the day. 13. Assessment Recommended? No 14. Comments on Quality of ICR As required by the guidelines, the ICR report comprehensively covers various aspects of the project, beginning with the relevance of the objective to the outcomes. The evidence presented is drawn primarily from annual surveys; the report provides adequate details in appendices. The report makes full use of available information. With a results orientation, the report explains how project activities could have contributed to the outcomes. The analysis is of acceptable quality. The report draws out lessons rooted in the project experience that are likely to have wide relevance. The writing is crisp and internally consistent. The overall quality of this ICR is rated as Substantial. a. Quality of ICR Rating Substantial Page 19 of 19