Reimbursable Advisory Services Agreement on Capacity Building to Accelerate the Transition from Institutional to Community-based Care for Children Deprived of Parental Care (P168507) Output 4: First monitoring report English version June 2021 www.poca.ro Disclaimer This report is a product of the International Bank for Reconstruction and Development/the World Bank. The findings, interpretations, and conclusions expressed in this document do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. This report does not necessarily represent the position of the European Union or the Romanian Government. Copyright Statement The material in this publication is copyrighted. Copying and/or transmitting portions of this work without permission may be a violation of applicable laws. For permission to photocopy or reprint any part of this work, please send a request with the complete information to either (i) National Authority for the Rights of People with Disabilities, Children and Adoptions (Bd Gheorghe Magheru no. 7, Sector 1, Bucharest, Romania) or (ii) the World Bank Group Romania (Vasile Lascăr Street, no. 31, 6th floor, 2nd sector, Bucharest, Romania). This report was delivered to the National Authority for the Rights of People with Disabilities, Children and Adoptions in June 2021 under the Reimbursable Advisory Services Agreement on “Capacity Building to Accelerate the Transition from Institutional to Community-based Care for Children Deprived of Parental Care” signed between the National Authority for the Protection of Children Rights and Adoption1 and the International Bank for Reconstruction and Development on August 26, 2019. It corresponds to Output 4 under the above-mentioned agreement: “First monitoring report”. 1 The project, initially implemented by the National Authority for the Protection of Children Rights and Adoption, has been taken over by the National Authority for the Rights of People with Disabilities, Children and Adoptions – institution established through the Government Emergency Ordinance no. 68 of November 6, 2019 through taking over of activities, attributions and structures of the National Authority for the Protection of Children Rights and Adoption (and of the National Authority for Persons with Disabilities), which was disbanded. 2 Acknowledgements Output 4 prepared under the Reimbursable Advisory Services Agreement on Capacity Building to Accelerate the Transition from Institutional to Community-Based Care for Children Deprived of Parental Care was developed under the supervision of Mr. Cem Mete (Practice Manager), with overall guidance from Ms. Tatiana Proskuryakova (Country Manager). The report preparation activities were coordinated by Vlad Grigoraș, Manuela Sofia Stănculescu, and Emil Teșliuc (team leaders). The team was composed of Georgiana Blaj, Bogdan Corad, Camelia Badea, Dan Croitoru, Alin David, Angelica Hîrju, and Cristina Iulia Vladu. The team also received support from Andrei Zambor and Ramona Lipara. The World Bank would like to acknowledge the excellent cooperation, guidance and timely feedback provided by the representatives of the National Authority for the Rights of People with Disabilities, Children and Adoption, in particular by Ms. Elena Tudor (project manager) and Ms. Cristina Cuculas (monitoring responsible). Also, the World Bank would like to express its gratitude to the representatives of General Directorates for Social Assistance and Child Protection who carried data collection activities and partnered the World Bank team in implementing monitoring round 1. 3 Acronyms AMP Foster care (to a professional maternal assistant) ANDPDCA National Authority for the Rights of People with Disabilities, Children and Adoption (former ANPDCA) CP Placement centres (institutions) for children CTF/AP Family-type homes and apartments DGASPC General Directorates for Social Assistance and Child Protection MC Case management OPA Accredited Private Organisations (NGOs service providers) PFAM Family placements (foster care) RAS Reimbursable Advisory Services REZALT Other types of protection services for children deprived of parental care, namely emergency placement centres - CPRU, maternal centres - CM, day or night shelters - AZ/N SPREV Community-based prevention services 4 Contents Introduction .......................................................................................................................................................................... 6 1. Setting up an M&E system .......................................................................................................................................... 7 1.1. General context ................................................................................................................................................................ 7 1.2. The need for an M&E system ..................................................................................................................................... 7 1.3. The M&E instrument ...................................................................................................................................................... 8 1.3.1. Data collection instruments ................................................................................................................................ 9 1.3.2. iProgres tool............................................................................................................................................................10 2. Monitoring round 1 ..................................................................................................................................................... 11 2.1. Data collection process ...............................................................................................................................................11 2.1.1. Preparation: Setting up a network .................................................................................................................11 2.1.2. Training the network in using the iProgres tool .......................................................................................12 2.1.3. Data collection period .........................................................................................................................................12 2.1.4. Support team ..........................................................................................................................................................12 2.2. Overview of the process .............................................................................................................................................13 Annexes ................................................................................................................................................................................ 14 Annex 1 – Templates of statistical reports ..................................................................................................................14 Annex 2 – County reports for monitoring round 1 ..................................................................................................14 Annex 3 – National level report for monitoring round 1 .......................................................................................14 5 Introduction The Reimbursable Advisory Services regarding the Capacity Building to Accelerate the Transition from Institutional to Community-Based Care for Children Deprived of Parental Care are provided by the World Bank within the project implemented by ANDPDCA: “PROGRESS in ensuring the Transition from Institutional to Community-Based Care”, co-financed from the European Social Fund under the Operational Program for Administrative Capacity 2014-2020. The advisory services are complementary to those provided by the World Bank between May 2016 and December 2018 within the RAS Development of Plans for the Deinstitutionalization of Children Deprived of Parental Care and Their Transfer to Community-Based Care, provided within the project Development of the Plan for the Deinstitutionalization of Children in Residential Care and Their Transition to Community-Based Care – SIPOCA 2, co-financed from the European Social Fund under the Operational Program for Administrative Capacity and implemented by the ANPDCA. The current RAS has two specific objectives, namely: (1) to support the development of evidence- based policies in the field of child protection, and (2) to develop the capacity of local public administration units to prevent the separation of the children from their biological families and increase the coverage, expand the availability and improve the quality and efficiency of child related services that they provide. The main expected outputs are: 1. Technical report including inputs into the design of an instrument for monitoring and evaluation of the progress of the transition from institutional to the community-based care (delivered and approved by ANDPDCA); 2. Data management and visualization tools to support the Recipient to enhance its website to collect and report data regarding the progress of the transition from institutional to community-based care(delivered and approved by ANDPDCA); 3. Handbook of alternative models for the development of the activity to prevent the separation of the child from the family (delivered and in the process of being approved by ANDPDCA); 4. First monitoring report; 5. Report including recommendations for the preparation of a policy note by the recipient regarding the improvement and the diversification of alternative services; 6. Advisory support for the preparation of 100 development plans for prevention services; 7. Study on the progress of the transition from institutional to community-based care; 8. Second monitoring report; 9. Report including recommendations for the preparation of a policy note by the recipient, regarding the improvement and diversification of services intended to prevent the separation of the child from the family. This report refers to the expected Output 4 – First monitoring report. 6 1. Setting up an M&E system 1.1. General context The general context for setting up the M&E system created in this project lays in the transition from institutional to community-based care for children deprived of parental care. For the child protection system, deinstitutionalization represents the process of planning the restructuring and closing of placements centers (institutions), while setting up, developing and consolidating modern and effective community-based services for children and families in order to prevent the separation of children from their families and to foster reintegration of children within families. Consequently, the closing of an institution is treated as an opportunity to step up efforts “to maintain or reintegrate children into the care of their families or, if this is not possible, to find other appropriate and permanent solutions such as adoption”2 for all children currently in institutions. In addition, the closing of an institution must be accompanied by the development and strengthening of community services that prevent the separation of children from their families, to ensure the reduction of the number of children separated from their families who need care in the special protection system. The whole process must be planned and carried out by considering the best interests of the child and the individual needs of the child and of his/her family. Two conditions are required for a proper deinstitutionalization process. On the one hand, no institution should be closed until better solutions are identified for each beneficiary in the respective center. Institutionalized children are a very diverse group, and (re)integration is not possible for all of them. Some of them must continue to be protected either in family-type foster care services (such as family placement or foster care) or in small-sized residential-type facilities (family-type homes or apartments). All these alternative services must be available and should be developed under right-based standards and they should be result oriented so as to provide better conditions and care for children. On the other hand, within the community, a whole range of education, health, habilitation/rehabilitation, housing and social services must be developed in order to avoid the separation of children from their families and to ensure that the children’s right to development is respected. 1.2. The need for an M&E system To support the last steps of deinstitutionalization, a monitoring and evaluation mechanism of this process should be in place. In the event that the placement centers are closed and that transition from institutional to community-based care is ensured, monitoring and evaluation are essential for each child and family, as well as for all newly established services. A transparent monitoring system allows the identification of centers/counties where the deinstitutionalization process is stopped/slowed down or is not being carried out properly and would allow for faster corrective measures to be taken. Moreover, the existence of such a system allows for a comparative analysis between the counties in terms of quality of the alternative services and, by updating the legislative framework and through specific professional training, it could standardize county practices. 2 European Expert Group on the Transition from Institutional to Community-based Care (2012). 7 The transition to community-based care should not replicate the institutional culture or recreate institutional care practices in the newly created services. This is why, on the one hand, it is necessary to monitor and evaluate the training of staff working within the new services; and, on the other hand, it is necessary to have new standards for the quality of the care that focus on the quality of life and on the rights of children and not on technical and organizational aspects related to the social service provider. Currently, there is insufficiently detailed systematic reporting on alternative services to care in placement centers that could allow for the correction of the problems existing at national level or of the regional discrepancies. The monitoring and evaluation system should not only consider the closure of the institutions and the development of alternative services, but also the development of prevention services at community level. Children are the main beneficiaries of the deinstitutionalization process, and, as such, monitoring and evaluation should have them and their families at their center. The closure of the placement centers would not be sustainable without the implementation of measures aimed at preventing the separation of children from their families and at reducing the number of entries in the protection system. 1.3. The M&E instrument The M&E instrument created within the current RAS aims to the development and enforcement of a system of evidence-based policies in the field of child protection. Consequently, it is designed so that to produce systematic data constantly generated and presented in a transparent manner, at the county and national level. Also, the M&E instrument covers all four strategic directions of action for the deinstitutionalization of children deprived of parental care that has been established by the ANPDCA3 and are transposed into the “National Strategy for the Protection and Promotion of the Children’s Rights 2014-2020”,4 namely: (i) closure of the placement centers, because every child must grow up in a family (ii) development of alternative services to placement center care, especially family-type ones, both numerically and in terms of quality (iii) strengthening case management to ensure the quality and adequacy of the protection services (iv) development of community-based prevention and support services to reduce the number of entries in the system and stimulate family reintegration. The M&E instrument5 is organized into two broad components. The first component of the instrument is dedicated to data collection. Specific tools were created to this aim structured in nine registries both for the protection services and the children benefiting from these services (regarding children's welfare and voice of children). These tools include the logical conditions to ensure data validation (both within and between registries) and the technical specifications to enable the calculation of the relevant monitoring indicators (as shown in Output 1, volume 1). The second component refers to automated data analysis and data visualization. Templates for statistical reports for all registries (regarding services and children) were developed, based on the data collected at the national, county, and local level (as presented in Output 1, volume 2). 3 Currently, the ANDPDCA. 4 GD no. 1113/2014. 5 The M&E instrument is detailed in Output 1, Volumes 1 and 2. 8 The system of indicators used by the M&E instrument is aimed at measuring changes at the child level and for all the child protection system components. Two main types of indicators are used: process and outcome indicators. The process indicators are meant to monitor the evolution regarding the implementation of plans for closing placement centers and for developing new services, and the outcome indicators are meant to measure the success that the activities implemented within the deinstitutionalization process have had in terms of achieving the proposed objectives. In line with UNICEF recommendations6 related to the monitoring of child protection programs, the rounds of collection of new data should take place every six months for process indicators and once a year for outcome indicators. The two rounds of monitoring provided for in this project will be carried out annually, but the proposed M&E instrument can be continuously updated, and a six-month interim report is recommended to be used by the DGASPCs, to make any real-time adjustments. 1.3.1. Data collection instruments In order to track the changes occurred at the level of the four strategic directions of action for deinstitutionalization, the monitoring instrument covers both the special child protection services and the community-based services for preventing the separation of children from their families. Accordingly, data collection instruments developed in Output 1, volume 1 and used starting with the first monitoring round are organized as follows: CHILDREN IN THE SPECIAL PROTECTION SYSTEM (i) Residential • monitoring the stage of the process of deinstitutionalization institutions of children in placement centers (CP) • monitoring the development of small-sized residential services: family-type homes (CTF), apartments (AP) • monitoring other residential institutions: emergency placement centers (CPRU), maternal centers (CM), day or night shelters (AZ/N) (ii) Family-type • monitoring the development of family-type institutions: institutions family placements (PFam) and foster care (AMP) (iii) Case management • case management (MC) consolidation monitoring (iv) The special • monitoring the impact on the SPSC, with a focus on protection system children’s wellbeing and voice CHILDREN FROM THE FAMILY AND COMMUNITY (v) Prevention services • monitoring the development of services aimed at preventing the separation of children from their families at local level 6 UNICEF (2015) CHILD PROTECTION RESOURCE PACK. How to Plan, Monitor and Evaluate Child Protection Programmes, p. 168-170. Available at: https://www.unicef.org/protection/files/CPR-WEB.pdf 9 The data collection instruments collect a large volume of information in a disaggregated manner and are interconnected and interdependent. Multiple links are set up between the data collection tools; for example, each child under a protection measure is linked to the service where he/she is being placed as well as to his/her case manager. Moreover, the system covers all children and young people living in the special protection system (with a special protection measure), regardless of the type of service in which they are placed and regardless of whether the service is provided by an accredited public or private social service providers. Also, the social services registries and the case managers’ registry must include data on both the social services and case managers of accredited public social service providers as well as private providers who offer direct services or ensure case management for children with special protection measures. 1.3.2. iProgres tool The M&E instrument, both the data collection and visualization component, was incorporated in iProgres tool7 that has the following main features: 1. allows data collection in an efficient (by means of a user-friendly and easy-to-use interface) and accurate (based on the included logical and validation conditions) manner; 2. provides automated aggregation of data on several levels - county and national; 3. generates statistical reports on a predefined format based on the collected data. Please refer to Annex 1 for the template of the statistical reports on all registries that are automatically generated at the county, respectively at the national level. The iProgres tool was developed to work offline. Thus, it can be opened and used on any DGASPC computer. The entered data is saved securely (encrypted) on the computer that it was entered on. The absence of a server to collect/save and aggregate data in real time requires a manual process of data validation and aggregation at county level. All users (accounts in iProgres) have access to all possible actions provided by the instrument, namely (i) data collection – data entering/editing/deleting; (ii) data consolidation and validation - aggregating in one database and validating all information collected and entered by the specialists at the county level; (iii) export the consolidated and validated database for their county; and (iv) visualization of statistical reports - data aggregated at the level of types of services/children’s welfare/voice of children, at the county or national level, based on the predefined templates. Only the system administrator at the national level is allowed: (i) to export the entire data set collected at the national level in order to perform in-depth analysis based on which new policies or changes of the existing regulations could be made and to (ii) to export the indicator set calculated against locality, county or national level, as applicable, which are to be viewed on the interactive platform.8 7 Described in detail in Output 2, part 1. 8 Described in detail in Output 2, part 2. 10 2. Monitoring round 1 2.1. Data collection process 2.1.1. Preparation: Setting up a network In preparation of the data collection for the first monitoring round, a series of steps have been taken to ensure maximum engagement in the process of the counterparts at the county level and to prevent as much as possible administrative or technical faults. In the first monitoring round the DGASPC from all counties in the country took part. In the first step, the DGASPCs were informed about the project and the preparation activities for the first monitoring round. An introductory workshop was held with representatives from all DGASPCs to present the project - the rationale, objectives, scope, and timeline of the project and to set a work frame for monitoring round 1. The methodology used for data collection and the iProgres tool were also shortly presented. In the second step, each DGASPC designated a contact person responsible for the relationship and communication with the project. This person worked closely with the World Bank team on all aspects related to the smooth running of the monitoring round 1. In the third step, teams of specialists were appointed in each county. The specialists were split into two groups with two different roles: the first group were assigned to do data entry, while the other group were supposed to do data consolidation (to merge the datasets prepared by the data entry specialists and check the possible duplicates). Overall, at the national level, 1740 individual accounts9 were created in iProgres to be assigned to the specialists (including the personal information associated and the pre-registered information regarding the specific register they will work on) In the fourth step, summary data at the county level were collected regarding the total number of each type of services, as well as of the total number of children with a protection measure within each type of service, to serve as a benchmark for the information collected during monitoring round 1. In the next step, a dry run iProgres installation was performed. The project team together with the contact person at DGASPC level set up a test session for the installation process of the iProgres on DGASPCs’ computers. Where problems were identified, the WB team worked with IT specialists at the DGASPC level to address the issues preventing the proper installation of iProgres. As a secondary result of this action, some of the counties changed some of the equipment available – bought new computers and/or upgraded the software used. In the last step, the iProgres user manual was made available to the county level specialists. The manual includes the data collection questionnaires with explanations of the logical conditions that are implemented in the iProgres tool (information about skipping questions, answer parameters) and provides additional explanations for the various error messages that may occur when data is entered inadequately. 9 In total, 58 persons played consolidation roles, while 1685 were involved in the data entry process. 11 By the end of December 2020, a national network of specialists who would be assigned with the roles of data entry and data consolidation for the first monitoring round was established. The network consisted of about 2,000 specialists from the public (DGASPC) and private (OPA) child protection services, from all counties. 2.1.2. Training the network in using the iProgres tool The entire national network of DGASPC specialists was trained in using iProgres in January 2020. The WB team conducted 6 online training sessions10 on data entry and one on consolidation with all DGASPC and OPA specialists involved in the project. All counties and the six Bucharest sectors participated in the online training sessions. The data entry training sessions included: • Overall presentation of the project and the objectives regarding deinstitutionalization; • Objectives of the monitoring exercise round 1; • Existing roles at the county level and responsibilities associated with the project (data entry, consolidation, coordination, and reporting); • General presentation of iProgres structure – modules and actions (data entry, data consolidation, data visualization – statistical reports) and functionalities (login, accounts, data saving, etc.); • Data entry of dummy cases on each register and discussion on the overall content of the registry and the use of it; the logic of the questionnaire and how different answers affect the questions that need to be completed; the logical conditions implemented in the tool that allow skipping some questions, force only some possible responses or alert the data entry specialist about possible mistakes in the data introduced; the interconnections between the registries, etc.; • Discussions on particular situations at the county level; • Setting time frame for data collection for monitoring round 1; • Setting the workflow between the specialists at the county level and the WB team. The training on data consolidation focused on the way in which the data is saved and transmitted to the person responsible for the consolidation and the way in which the information is consolidated at the county level. Some practical examples of data consolidation were provided and specific situations were discussed and solved. 2.1.3. Data collection period The data collection started in the first 8 counties on 15 January 2021, after the first training session. The last 8 counties started data collection on January 20, 2021. 2.1.4. Support team Given the offline nature of the iProgres tool and the lack of real-time information about the data collection process, combined with the large amount of data that needed to be completed in a short 10 Between 15 and 20 January 2021, with more than 1000 zoom connections from DGASPC offices around all counties in the country and the 6 Bucharest sectors. For each training session about 8 counties were grouped in alphabetical order. The average duration of a training session was 120 minutes. 12 period of time, a team of WB experts was set with the role of providing 24/7 support services to the counties. Each expert was responsible for addressing the problems raised by specialists at the local level and offering support and guidance through the data collection and consolidation processes for about 8 counties. Communication with specialists at the local level was done daily by all channels – phone, email, online one-on-one meetings. Every week, the team collected information on the progress of data collection and data entry at the county level. A WhatsApp group was created among WB team members to facilitate communication and to share particular situations identified at the local level. This way, the entire team was rapidly informed on decisions and capable of communicating similar responses to different counties. 2.2. Overview of the process The first monitoring round registered large participation of specialists from all counties in the country, both DGASPC and OPA. On the one hand, this fact has translated into a very good coverage of the data collected with M&E purposes and increasing capacity in working with a new (iProgres) tool that will be further used in daily activities. On the other hand, setting up a national M&E system remains a challenge while working with so many people who only partially take responsibility for the process and in the absence of an articulated mandatory working structure. The process was successful as a large quantity of data were collected from all over the country by using the same tools (iProgres). Nevertheless, the data collection process unfolded at a slower pace than expected due to two main reasons. Firstly, the national network involves „essential workers” in the COVID-19 pandemic context, working directly with vulnerable persons. Many of them were active actors in the process of organizing COVID-19 vaccinations actions at the county level and some were directly affected by COVID-19 and, during part of the process, being isolated or unavailable. Secondly, being in the first line in the work with vulnerable people can disrupt the daily/ weekly planning of activities due to unexpected events such as a fire that left an entire street homeless and in need of social protection/intervention. The statistical reports based on the data collected by county and registry are available in Annex 2. 13 Annexes Annex 1 – Templates of statistical reports Please see attached Annex 1 archive that comprises the template reports for the 9 registries: 1.CP (placement centres), 2.CTF/AP (family-type homes and apartments), 3.REZALT (emergency placement centres - CPRU, maternal centres - CM, day or night shelters - AZ/N), 4.PFAM (family placements), 5.AMP (foster care), 6.MC (case management), 7.KIDS (children’s wellbeing), 8.VOICE (voice of the children), 9.SPREV (prevention services). Annex 2 – County reports for monitoring round 1 Please see attached Annex 2 archive that comprises the county reports for monitoring round 1. Annex 3 – National level report for monitoring round 1 Please see attached Annex 3. 14 Project co-funded from the European Social Fund through the Administrative Capacity Operational Program 2014-2020! Project title: "PROGRESS in ensuring the transition from institutional to community-based care” Project code: SIPOCA 577/MySMIS 127380 Name of beneficiary: National Authority for the Rights of Persons with Disabilities, Children and Adoptions Date of publishing: June 2021 The content of this material does not necessarily represent the official position of the European Union or of the Romanian Government. Material distributed for free