World Vision International (WVI) Nepal has implemented “Community Resilience/Disaster Support: COVID-19 Recovery in Nepal” (ASTELLAS Project) in three different districts (Mahottari, Sarlahi and Rautahat) of Madhesh Province of Nepal, in partnership with three different local organizations from November 2021 to November 2022. The project aimed to support the local Government to strengthen healthcare systems to build community resilience against COVID-19. This report is the summary of the endline study conducted to examine the extent to which the ASTELLAS project intervention has been successful in achieving its goal and outcomes.
The study employed a descriptive cross-sectional study design using both qualitative and quantitative methods. The study method primarily consisted of the household questionnaire survey, the health facility survey, interaction with project/partner staff, key informant interviews, and focus group discussions. To collect the quantitative information, a total of 305 household heads and 18 health facility in-charges interviewed. The qualitative study included five focus group discussions and 20 key informant interviews. The field work for the evaluation was conducted in October 2022 in all six project municipalities in three districts. Quantitative data were analyzed using statistical tools (mean, SD, proportion, etc.) by SPSS version 23, and the findings were compared with the baseline study conducted in February 2021. Qualitative findings were analyzed by using thematic analysis, and conclusions were drawn about the outcomes of the project. The project was evaluated based on different evaluation criteria, which included relevance, coherence, effectiveness, impact, and sustainability.
Major findings of the study
Limit the spread of COVID-19
The project has contributed to limiting the spread of COVID-19 through community awareness and providing educational activities. Project activities such as health education to communities through risk communication and community engagement strategy, involving community leaders, volunteers, schoolteachers, and child clubs were highly effective. The project provided hygiene kits to the vulnerable population, including pregnant and recently delivered women, and people with disabilities. The project also supported the government in conducting an immunization camp for the COVID-19 vaccine by mobilizing volunteers, providing furniture and tents, drinking water and handwashing facilities in the camp, distributing sanitizer and masks, and supporting record keeping. Information on vaccine camps was disseminated through community level miking, local radio stations, and mobilizing volunteers. The support provided by the project in “Search and Immunize Campaign (Khojau ra Khopau Abhiyan)” was effective in identifying and immunizing vulnerable populations such as pregnant women, lactating mothers, elderly people, and people with disabilities. Ultimately, these activities contributed to increasing COVID-19 vaccine coverage in the project municipalities.
Strengthened health system for COVID-19 preparedness
The project contributed to strengthening the public health system by providing essential supplies including PPEs, pulse oximeters, infrared thermometer, ice lined refrigerator, RDT kits, sanitizers, masks, etc. These supplies are instrumental in providing quality health services during the time of COVID-19. Project provided training on COVID-19 to health workers on clinical management including providing services applying precautionary measures, use of PPEs, use of RDT kits testing, and interpretation of results. Those who were infected with COVID-19 were provided with home isolation kits. Transportation support was provided to critically ill patients from poor and marginalized families to reach the nearest hospital. The project provided ILR fridge to health facilities which was highly appreciated by health workers as it helped health facilities to store the vaccines and conduct mobile camps at community level.
The table below shows that the project achieved most of the targets as planned in the log-frame. The comparison with the baseline value showed a significant improvement in most of the project indicators, indicating the effectiveness of the project.
|Objectives||Indicators||Definition||Baseline value||Target for end line||End line value|
|Objective 1. Limit spread of COVID-19 in the community through education on COVID-19||1.1 Proportion of community members who have adequate knowledge on means of COVID-19
|% of respondents who can state at least three different means for the transmission of COVID-19||41.1%||66.1%||50.8%|
|1.2 Proportion of community members who have adequate knowledge on prevention from COVID-19||% of respondents who can state at least the SMS technique (Soap
(handwashing), mask and social distancing) to prevent from COVID-19
|Objective 2. Strengthened health system for COVID-19 preparedness and recovery||2.1 Proportion of health facilities with required minimum equipment and logistics for delivering services during COVID-19||Minimum equipment and logistics are available as per MoHP’s standard for
delivering services during COVID-19
|2.2 Proportion of health workers having adequate knowledge while treating people infected with COVID-19||Adequate knowledge refers to knowledge score higher than mean value.||89.6%||95.0%||94.8%|
The summary and overall conclusion of project evaluation has been summarized in the table below.
|Evaluation||Results||Main findings/Major facts identified|
|Project design, management, and coordination||Very Good (4)||The Involvement of relevant stakeholders was sought from the initial phase of the project. Project activities were planned together with the government stakeholders and were revised during the implementation to meet the actual demand and needs of the health facilities and communities. As an emergency response project, there was some delay in the implementation of the project, as the project was implemented towards the end of the second wave of COVID-19.|
|Relevance and coherence||Highly Relevant (5)||The project was highly relevant to meet the humanitarian needs of the affected people in the recovery phase. Project activities were targeted towards meeting the ‘real needs of community people. The projects and its intended outcomes have taken into consideration the national and local priorities during the time of emergency and the needs of intended beneficiaries in terms of improving community resilience against COVID-19 and increasing vaccine coverage. The project was in line with the Government of Nepal’s Health Sector Emergency Response Plan (HSERP) 2020, SDG Target 3.3 (End the epidemics of communicable disease) and SDG Target 13.1 (Strengthen resilience and adaptive capacity of communities), and WVI’s principle of serving the vulnerable population. The activities were in line with the overall objectives of the project, and there was coherence in the hierarchy of project outcomes, outputs, and activities.|
|Effectiveness||Effective (4)||The project was successful in improving knowledge of community people on COVID-19 transmission and prevention methods; improving availability of with required minimum equipment and logistics for delivering services during COVID-19; and improving health workers knowledge on COVID-19. The project has contributed towards improved health service. The comparison with baseline also showed that there was a significant improvement in all indicators. Targets for three (out of four) outcome indicators have been met.|
|Impact||High (4)||Improving community awareness on COVID-19, capacity building of health workers to provide health services during the time of COVID-19 pandemic and strengthening health system to deal with pandemic were the significant impacts of the project. The project also contributed to increase the vaccine coverage. COVID-19 vaccine coverage in project municipalities in Sarlahi district raised from less than 50% before the project implementation to above 80% after the project. Although the level of knowledge during end line showed satisfactory on COVID-19 prevention, but the practice related to the prevention was poor, even deteriorated compared to the baseline.|
|Sustainability||Sustainable (4)||The project activities were planned jointly with municipalities and health facilities, increases ownership in the project from the beginning. The material and equipment support provided such as ILR and deep refrigerator in health facilities, medical tent, and handwashing facilities can be used for long duration. The project however, missed the opportunity to build capacity of ward and municipality level disaster response and relief community which might have contributed better sustainability of the project.|
|Overall conclusion||Very Good (4)||Overall, the project is found in “very good” performance position.|
The project contributed to limit the spread of COVID-19 in the community by providing health education, providing hygiene and sanitation supplies, and improving access of the vulnerable and left behind population to COVID-19 vaccination. The project also strengthened the public health system for COVID-19 preparedness and recovery by providing equipment and supplies and improving the capacity of health workers. The endline findings showed that the project outcome indicators have been significantly improved compared to the baseline. The endline study has identified some of the important learnings of the project, and made recommendations for future programming.
 The average score of all the five evaluation criteria is 4.2 = 4 (Very good).
Project Area: Mahottari, Salahi and Rautahat, Madhesh Provence
Duration: September to November 2022