ICRR 12514 Report Number : ICRR12514 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/31/2006 PROJ ID :P071445 Appraisal Actual Project Name :Demobilization And Project Costs 42.0 18.5 Reintegration Project US$M ) (US$M) Country :Cambodia Loan/ US$M ) Loan /Credit (US$M) 18.4 1.2 Sector (s):Board: ): SDV - Other social US$M ) Cofinancing (US$M) 2.9 2.6 services (82%), Central government administration (18%) L/C Number :C3564 FY ) Board Approval (FY) 02 Partners involved : Netherlands, CIDA, SIDA, Closing Date 12/31/2004 12/31/2004 Japan, World Food Programme Evaluator : Panel Reviewer : Division Manager : Division : Roy Gilbert Lily L. Chu Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives a) To assist the Government in its effort to demobilize 30,000 soldiers and to promote their successful adaptation to civilian life; and b) to contribute to a reallocation of budgetary resources to the social sectors in order to build human, social and economic assets. b. Components (or Key Conditions in the case of Adjustment Loans ): a) Preparation of personnel database and management information systems to establish a reliable military personnel database and computerized payroll system . It includes improvements to existing identification procedures and documentation of soldiers to determine program eligibility . Unregistered soldiers would not be . (appraisal cost - US$0.20m.; actual cost - US$0.01m.). b) Demobilization, incl: (i) assembly and disarmament of soldiers; (ii) discharge operations to mark the formal departure of a soldier from his/her military life; (iii) provision for a transitional safety net (TSN) for soldiers being demobilized; (iv) transportation of veterans to their communities . (appraisal cost - US$13.2m.; actual cost - US$3.5m.). c) Reintegration, incl: (i) health screening; (ii) provision of information and counselling for the veterans and sensitization of the host communities about the problems that veterans may encounter; (iii) the provision of package of assistance according to the choice of option made by the veterans; iv ) the design and implementation of skills/vocational training programs and job counselling . (appraisal cost - US$22.8m.; actual cost - US$14.3m.). d) Project implementation and technical assistance : i) to strengthen the planning and implementation capacities of the General Secretariat of Council for the Demobilization of Armed Forces (CDAF), the Provincial Veterans Committees (PVC) and their Executive Secretariat and the concerned line ministries such as the Ministry of Women's and Veteran Affairs (MoWVA); ii) to improve financial and data management including monitoring and evaluation; and iii) to facilitate the coordination and increase the linkage between the provision of reinsertion and immediate reintegration with the on-going community development program in communities where veterans are settled (appraisal cost - US$3.9m.; actual cost - US$0.8m.). c. Comments on Project Cost, Financing, Borrower Contribution, and Dates Less than half the intended project spending was incurred, with the Bank's financing closing at only 6% of the total, down from an appraisal expectation of 44%. Although the ICR does not provide standard annex tables for project financing, we can deduce that government counterpart rose to 79% of the total cost at completion, well up from the 49% expected at appraisal. Bank Controllers' data show that each cofinancier spent nearly its targeted amount of funds on the project. The ICR also doesn't provide actual project costs by type of procurement . Nevertheless, it gives the misprocurement of motor cycles (that eventually led to anti corruption investigation and disbarment of five individuals and four firms by the Bank ) as the cause for the suspension of the project implementation . Despite this interruption, the actual closing date was the same as expected at appraisal . 3. Relevance of Objectives & Design : IEG recognizes that this project set highly ambitious objectives in what were extraordinarily difficult circumstances in a country torn by conflict and violence for nearly thirty years until 2000. The first project objective is now only negligibly relevant to the current priorities of the Bank and the Borrower in Cambodia several years after the conflict . The 2005 CAS is focused upon the achievement of the Millennium Development Goals, in contrast with the 2000 CAS, where demobilization was cited as a top priority . Recently, the government proposed the introduction of formal drafting of personnel into the army . The second objective, however, remains highly relevant to current government and Bank priorities since its building of human capital can contributed the MDG goals of the 2005 CAS. Overall, project design was only modestly relevant to achieving the project objectives, being substantially so for the first, (especially for demobilization, but little for reintegration ), but negligibly so for the second (government budgetary reallocations to social sectors ) for which there was no operational component to achieve it . Project design was also weak in not providing specific instruments for reintegration, although the SAR did mention that health screening, counselling, housing provision, agricultural implements, and vocational training were "expected". The ICR reports that the soldiers identified land, cows and cash as their primary needs, items that were not eligible for funding by the project, according to the ICR. Whether the sub-component of 13,764 motor cycles for demobilized soldiers, the bidding for which led to misprocurement and closure of the project, was relevant to soldiers' reintegration and transition as farmers needing agricultural equipment and tools, is an important question . 4. Achievement of Objectives (Efficacy) : Objective a) demobilization and reintegration - Negligible : only 15,000 soldiers (half the intended 30,000 beneficiaries) were assisted by the project in demobilization and the ICR provides no convincing evidence that they were successfully reintegrated into their local communities and economies . The ICR provides no information about the other 15,000 soldiers who were targeted for assistance, but received none . That demobilization was not a government priority is evident from the decision to reintroduce the obligatory drafting into the army . On the reintegration side, the ICR provides no evidence that the project successfully adapted the 15,000 soldiers demobilized to civilian life as intended, and reports that no evaluation of this was carried out [because of problems with project, according to later comments by the Region ]. Yet, even so, the ICR concludes that this project achieved was partially achieved. Objective b) reallocating government budget to social sectors - Negligible : while the ICR claims that this occurred and that defense expenditure fell, there is no evidence that these result can be attributed to a project that had not means to achieve them. If the project expenditure itself is regarded as social sector expenditure, the operation itself would have incurred a significant shortfall, since it spent less than half the amount planned . In subsequent comments upon this ICR Review, the Region asserted that any saving of expenditures on 15,000 soldiers "by definition, must" inevitably lead to increased social spending . This review is not persuaded that there was an automatic military-to-social transfer of finance, nor that the project provided an assured mechanism that the transfers of savings on the military would not be spent on anything else . Significantly, the ICR provides no convincing evidence of actual results in this regard. 5. Efficiency : No ERRs were estimated for this project either at appraisal or completion . The SAR references the project's economic analysis as being in SAR Annex 4, but there is no such analysis in that annex . The ICR simply notes that, since no ERR was estimated at appraisal, none was estimated at completion . More effort could have gone into the economic analysis of the project, considering as future benefit streams former soldiers' earnings and added agricultural output, for instance . Cost effectiveness studies could also play a part in establishing unit cost targets for this kind of operation. Without actual solid evidence of reintegration results, however, project efficiency is difficult to assess. But, compared with other cases, it does not look good . In Bosnia and Herzegovina, for instance, a Bank project (Cr2906) succeeded in demobilizing and reintegrating 17,378 soldiers at a total cost of US$ 9.2m., less than half the amount spent by this project in Cambodia for fewer soldiers whose actual reintegration status remains uncertain. 6. M&E Design, Implementation, & Utilization: M&E was poorly designed for this project and not implemented, even before implementation was interrupted through misprocurement. For instance, there was no measurable indicator that would show that 30,000 veterans and their families had maintained their standard of living in terms of health, housing and economic activities, as intended at appraisal. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Unsatisfactory Unsatisfactory The project failed to achieve its objectives, and was disruptive to Bank-Borrower relations. Institutional Dev .: Negligible Negligible Sustainability : Unlikely Unlikely Bank Performance : Unsatisfactory Unsatisfactory Mainly at preparation stage through poor design, and poor assessment of borrower ownership. Also implementation ratings did not register problems with this project . The ICR reports that all four supervision missions continued to award 'satisfactory' performance ratings. Only with the final ICR mission itself (date not given in the ICR report) was there a downgrade to 'unsatisfactory' performance. [In later comments, the Region mentioned that problematic issues were addressed openly during supervision and that some lower ratings were given (but these are not reflected in the ICR)]. Borrower Perf .: Unsatisfactory Unsatisfactory Through failure of project management and the loss of borrower ownership of the project, evidenced by a documented unwillingness to accept responsibility for project failures. [The Region later commented, however, that the government's willingness to use its own resources for demobilization is a demonstration of borrower ownership early on in the project.] Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: It is essential to ensure that projects are underpinned with realistic assessments of the political issues through a good and frank political analysis of the context for project implementation . When citizens exert pressure upon donor agencies, such as the Bank, to exact accountability by government, then trust between the Bank and government can break down . Good knowledge and understanding of military issues, and objectives of military reform should be an essential element of designing and implementing demobilization . The Bank cannot provide this, and therefore needs to work with others such as military attaches of other countries . But this still may not be enough to ensure success . Registration of soldiers for demobilization is a one shot operation, since it cannot be easily corrected if done wrongly. Care (and expense), therefore, should be applied to a process that should also involve civilian authorities and civil society . Demobilization design should ensure that soldiers clearly see it as a once -off event for them that will not be repeated. Otherwise, they might see an incentive to stay on in the army in order to receive further hand outs . Bank procedures and guidelines can only begin to be adopted in a country where governance and rule -of-law is weak if there is more Bank supervision and closer monitoring by civil society . TORs and contracts should always make clear the limits of the responsibilities of consultants, who can never assume a government's responsibility for compliance with a project's legal covenants . The Bank can help borrowers be aware of their rights in case of negligence on the part of the consultants and encourage borrowers to exert these rights when there has been wrong -doing. When a project faces a crisis as this one did, strong leadership by high -level Bank management is essential . In this case, the Country Director and other senior regional staff continually guided and supported the task team . 10. Assessment Recommended? Yes No Why? To learn if reintegration, for which 77 percent of project resources were spent, was truly effective for the 15,000 demobilized and to learn what happened to the other 15,000 soldiers targeted for demobilization, who were not assisted by the project . 11. Comments on Quality of ICR: This is an unsatisfactory ICR in a number of respects . First, it does not adequately explain what happened to the 15,000 soldiers targeted for demobilization but not assisted by the project . Second, other than describing "reintegration" packages of food, medicines and mosquito nets handed to demobilized soldiers, the ICR provides no solid evidence of the successful adaptation to civilian life of the 15,000 soldiers demobilized by the project . Evidence on the location, health, housing and economic conditions of the ex -soldiers would have helped establish the extent of their adaptation to civillian life, but surveys to collect such evidence were not carried out . Third, although the ICR does candidly recognize several project problems, this is not reflected in the assessment of the performance of the project components and activities, nearly all of which are rated "satisfactory" [In later comments, the Region affirms that the ICR's component ratings refer only to the period of successful implementation and did not cover the failure to continue implementation. IEG considers that the overall performance of the intended activities should be taken into account. Failure to do so points to a self -evaluation that was only partial ] . Fourth, important data, required by the guidelines, is missing from the report, notably actual costs by procurement arrangement and project financing by component. The main ICR text (p. 15) does include a table under "costs and financing" but it indicates (presumably in US dollars, in a table without a title or labels ) total disbursements of US$24.0 million, far in excess of total expenses of only US$18.6 million. Yet the ICR does not discuss if this is evidence of misspending or corruption . Fifth, other ICR evidence presented very weakly, for example, juxtaposing items that cannot be compared (e.g. changes in social sector spending in absolute terms on the one hand and changes in defense spending expressed as a share of total spending on the other). Speculation is presented as evidence (e.g. claims that M&E "would have been satisfactory " had the project continued"; and "had the project gone ahead, the demobilization of the second 15,000 soldiers would have taken place in an equally efficient and satisfactory manner "). Sixth, annex data on Bank inputs is incomplete, particularly the composition of identification /preparation and appraisal missions, and the date of the ICR mission . Seventh, the ICR has no list of acronyms, that are nevertheless widely used in the report . Finally the self-evaluation of the ICR would have benefited from the perspectives and conclusions of co -financiers, as important project partners. It is not adequate for the ICR to simply state that co -financiers comments had been taken into account without attaching the comments received . In summary, the report does not read as a candid account of a significant failure from which very important lessons could and should have been drawn .