Consultant for BEST IV Final Evaluation/ PREPS 2020-2022 Progress Review

Back

Organization : Save the Children

Consultant for BEST IV Final Evaluation/ PREPS 2020-2022 Progress Review

Background

Save the Children has been working on Thailand for over 35 years, supporting the most vulnerable children with Education, Child Protection, Child Safety and Health & Nutrition programs. Whenever a disaster strikes in Thailand we are ready to deploy an emergency response. In 2020 we have deployed a multi-site, multi-sectoral response to the COVID19 pandemic with a focus on the Southernmost Provinces (Yala, Pattani, Narathiwat) and the border with Myanmar where more than 91,000 refugees still reside in 9 refugee camps and a large migrant population resides.

Since 2013, Save the Children is the lead of the EU consortium ensuring access to basic education services to school-age children residing in the 9 temporary shelters located in Mae Hong Son, Tak, Kanchanaburi and Ratchaburi Provinces. SC is also the co-lead – alongside the Jesuit Refugee Support (JRS) – of the Committee for Coordination of Services to Displaced Persons in Thailand (CCSDPT)’ education sub-committee. SC’s efforts align with its’ 2019-2021 CSP goal ‘The most deprived children receive a high quality multi-dimensional education from infancy to adulthood’.

Refugee Education Program

Over the past decades, more than 200,000 Myanmar nationals sought refuge in Thailand, fleeing conflict and ongoing violence in South East Myanmar. Presently, 91,777 (46,347f, 45,430m) verified Myanmar refugees reside in 9 camps alongside the Thai/Myanmar border (UNHCR, September 2020). 40% of them are children (17,805g; 18,539b). For these children, access to a quality basic education and the recognition of their academic achievements is a fundamental right, an essential component of their wellbeing and development as well as a steppingstone to live a peaceful and prosperous life.

Brief program outline

The BEST/PREPS project contributes to the achievement of durable education solutions for the refugee camp population in Thailand by supporting the provision of quality basic education for students in nine refugee camps along the Thailand-Burma/Myanmar border, supporting teachers to build education capacity, and facilitating progress towards sustainable education solutions. Specifically, BEST/PREPS aims to achieve the following outcomes:

1. Basic education services are maintained and the quality of basic education is improved in nine (9) refugee camps.

2. Children, teachers, and parents have increased knowledge and skills to support successful transition and reintegration into education services in Myanmar.

3. Increased linkages, synergies, and cooperation between relevant local actors on both sides of the border leads to positive experiences of return, particularly related to education transition.

Scope of Evaluation
1. Purpose and Key Questions

This evaluation and assessment is being conducted at the end of the BEST IV project and the middle of Year 1 for the PREPS project. It will build upon the external final evaluation of the BEST III and mid-term review of PREPS previously conducted in May 2019.

The purpose of the evaluation is twofold:

1) to understand the extent to which the BEST IV and PREPS 2020-2022 achieved/ are achieving its intended results/outcomes according to indicators on the logical frameworks and

2) to establish baseline indicator values for select indicators as shown on an updated logical framework that will be used for Phase V of BEST project starting in March 2021.

The evaluation of BEST IV will answer the following key questions:

To what extent has the BEST/PREPS project had an impact in the lives of beneficiaries especially refugee children, teachers and education staff (KRCEE and KnED) in basic education system within the target areas?
How relevant, effective, efficient, sustainable, participatory, gender-sensitive and inclusive are the programme interventions?
To what extent was the BEST/PREPS project adaptive and responsive to COVID-19?
What lessons learned can be drawn from the project to further plan, design and develop future programmes?
What are the most critical areas of focus for future programs in the basic education for refugee children, e.g. how can we have the greatest impact on children in the coming years?

The baseline assessment data collection will collect additional data on a few indicators relevant to select basic education activities in the temporary shelters. Agreement of indicators will be determined during the inception period in consultation with the evaluation team.

The Evaluation team will be required to undertake consultation with the SC MEAL Coordinator and the Education Technical Advisor at the commencement of the assignment in order to further refine the evaluation questions.

1. Scope

Due to the changing context and often restricted access within the camps due to COVID-19, the evaluation team will be requested to conduct the evaluation remotely with support from Save the Children. For this, the evaluation will be limited to three camps known to have access to strong internet and/or phone signal. These include Mae La, Umphiem and Nu Poe camps in Tak Province.

1. Stakeholders/Audiences

The main stakeholders/targeted audiences for this evaluation are:

Stakeholder	Further information
Project donor	European Union and Australian DFAT
Primary implementing organisation	Save the Children Thailand, BEST/PREPS Education Team
Implementing partners	Adventist Development Relief Agency Thailand (ADRA); Center for Rural Education Development Foundation (CRED), Jesuit Refugee Service (JRS) with Karen Refugee Committee Education Entity (KRCEE) and Karenni Education Department (KnED); Karen Women’s Organization (KWO)
Government counterparts	Ministry of Interior
Community groups	Parent Teacher Associations
Beneficiaries	Children and adults involved in the program/project/s and the evaluation
International development/humanitarian research community	CCSDPT members and other refugee education stakeholder groups such as KKEST; Migrant education stakeholders

In addition to assessing the impact of the BEST IV project, findings from the evaluation will be used to evaluate progress of PREPS VII, determine baseline data for BEST V, and inform lessons learnt and good practice for the on-going PREPS project and BEST V extension (2021-2023).

The Evaluation team will be required to propose how the evaluation findings will be shared with each of the different stakeholders in the table above, particularly outlining how reporting back to communities, beneficiaries and children will be conducted in an accessible and child friendly manner.

1. Secondary Questions

a) Relevance:

Considering the current context and situation for refugees residing in camps along the Thai-Myanmar border, to what extent were project objectives and activities relevant to the needs, interests and priorities of target communities?
What are the socio-cultural barriers to the approach adopted by Save the Children (SCI) in delivering project outcomes, and how has SCI worked to identify and address these barriers?
To what extent were beneficiaries/stakeholders (partners, children & their families, teachers, community members, local authorities) involved in the design, planning, implementation, feedback mechanism establishment and process of the project?
To what extent to the project adjust to the needs of refugees during the COVID-19 pandemic?

b) Effectiveness:

To what extent were the objectives of the project achieved or likely to be attained? What are the barriers to the achievement of project objectives?
Are there any differences in outcomes achieved by different groups?
How effective are the monitoring systems of the project? How has the project used the information from monitoring activities to improve the quality of the project interventions and inform decision making process of the project?

c) Impact:

To what extent is the program producing intended and unintended results or changes (positive and negative)? What are the driving factors for intended and unintended results?
What is the impact or effect of the project in proportion to the overall situation of the target group or those effected?

d) Efficiency:

Is the relation between input of resources (materials, financial, services, staff etc.) and results achieved appropriate and justifiable?
In what ways has the consortium model contributed to greater efficiencies, cost-savings and fiduciary controls? How efficient were the partnership mechanisms and collaboration between SCI and partners?
How efficient were the complaint and response mechanism that had been established in the targeted communities?

f) Sustainability:

To what extent can activities, models, initiatives, results and effects be expected to continue or replicate at the conclusion of the project?
How the project planning and implementing sustainability strategy?

f) Cross cutting issues Participation, Gender, Safe programming and inclusion

What kind of gender gaps were identified in refugee education context and how did the project manage to tackle those gaps?
What were the protection measure undertaken in project implementation i.e. child safeguarding policy implementation, safe school facilities? How has the project considered gender sensitivity in the implementation of these policy?

To what extent have SCI integrated gender and social inclusion (i.e. disability) concerns and child safeguarding issues into the design and implementation of its interventions?

Evaluation Methodology
1. Research design and sampling

The suggested research design for this evaluation is a mixed methods design including participatory methods if and where possible. SCI anticipates the methodology to include a review of project documents, semi-structured interviews and focus group discussions with relevant stakeholders including beneficiaries, observations, and review of project monitoring data. The consultant is encouraged to propose additional methodology that they consider appropriate to the project, which is subject to Save the Children’s approval and must meet ethical guidelines (Annex I).

The evaluation will focus on three camps in Tak province. It will involve children, teachers, relevant local education authorities, project partner organizations, and SCI project team. Participants in the study will be selected through purposive sampling.

1. Data

All primary data collected during the course of the evaluation must be disaggregated by sex, age, people with disabilities, status and camp location and inclusive of children’s representation.

Save the Children has existing data collection instruments and tools that can be drawn on in the evaluation. These will be shared with the evaluation team.

Save the Children will not provide enumerators to assist with primary data collection. Data triangulation is expected for this evaluation. It will be a requirement of the Evaluation team to source additional external data sources to add value to the evaluation, such as consultations with partners and review of project documents.

A range of project documentation will be made available to the Evaluation team that provides information about the design, implementation and operation of the Program. Example of documents include previous evaluations, mid-term reviews, proposals, and logframes.

1. Ethical considerations

It is expected that this evaluation will be:

Child-friendly and participatory. Children should be meaningfully involved in the evaluation as a holistic process and not only as informants. Refer to the Practice Standards in Children’s Participation (International Save the Children Alliance 2005); and Global Indicator technical guidance (SCI M&E handouts Package, Volume 2).
Inclusive. Ensure that children from different ethnic, social and religious backgrounds have the chance to participate, as well as children with disabilities and children who may be excluded or discriminated against in their community.
Ethical: The evaluation must be guided by the following ethical considerations:
- Child safeguarding – demonstrating the highest standards of behaviour towards children
- Sensitive – to child rights, gender, inclusion and cultural contexts
- Openness - of information given, to the highest possible degree to all involved parties
- Confidentiality and data protection - measures will be put in place to protect the identity of all participants and any other information that may put them or others at risk.
- Public access - to the results when there are not special considerations against this
- Broad participation - the relevant parties should be involved where possible
- Reliability and independence - the evaluation should be conducted so that findings and conclusions are correct and trustworthy

It is expected that:

Data collection methods will be age and gender appropriate.
Evaluation activities will provide a safe, creative space where children feel that their thoughts and ideas are important.
The Evaluation team will adhere to the Save the Children Child Safeguarding, Data protection and Privacy policies that are required throughout all project activities.

Expected Deliverables

The evaluation deliverables and due dates (subject to the commencement date of the evaluation) are outlined below. The lead evaluation consultant will advise [the SC BEST/PREPS Project Manager] immediately of any risks or issues that may impact on their ability to provide the deliverables by these due dates.

Deliverables and Due Dates

Deliverable	Due Date
The Evaluation Team is contracted and commences work	January, 2021
Phase 1: INCEPTION PHASE
The Evaluation Team will conduct initial desk research and tool review and develop and submit the Inception report that will include: evaluation objectives and key evaluation questions description of the methodology, data sources, draft data collection tools (preferably against the key evaluation questions and selected indicators for baseline assessment) and sampling considerations caveats and limitations of evaluation key deliverables, milestones and timelines risk and issue management plan a stakeholder communication and engagement plan consultation protocols for consulting with children and other vulnerable groups (if applicable) Logistical or other support required from Save the Children Once the report is finalised and accepted, the evaluator/ evaluation team must submit a request for any change in strategy or approach to the Education Technical Advisor and BEST/PREPS Coordinator	To be submitted within 15 days after signed contract
Ongoing Desk Research of project documents and secondary sources	On-going January-February
Feedback by SCI	Within 5 working days after receiving the inception report
SCI Approval of Inception Report and data collection tools	Within 1 week after review completed –End of January
Phase 2: Data Collection and Analysis
Field work plan and logistical arrangements Translation of tools Data collection Data analysis	Early-February (15 days)
Phase 3: Reporting
Preliminary findings presentation and verification workshop with partners Summary of interim findings Any emerging program issues or risks (if applicable) Key tasks for the next stage of the evaluation and any proposed refinements or changes to methodology (if applicable)	Mid- February (1 day)
Draft Evaluation Report* including the following elements: Executive summary Background description of the Program and context relevant to the evaluation Scope and focus of the evaluation Overview of the evaluation methodology and data collection methods, including an evaluation matrix Findings aligned to each of the key evaluation questions Specific caveats or methodological limitations of the evaluation Conclusions outlining implications of the findings or learnings Recommendations Annexes (Project logframe, Evaluation TOR, Inception Report, Study schedule, List of people involved)	3^rd week in February
Feedback and approval by SCI	Within 5 working days after receiving the inception report
Final Evaluation Report* with submission of data and analysis incorporating feedback from consultation on the Draft Evaluation Report	End of February (3 days)
Knowledge translation materials: PowerPoint presentation of evaluation findings Evidence to Action Brief**	End of February (1 day)

*All reports are to use the Save the Children Evaluation report template. Please also refer to Save the Children technical writing guide.

** The Evidence to Action Brief is a 2-4-page summary of the full report and will be created using the Save the Children Evidence to Action Brief template.

Dates and times for deliverables will be changed upon consultation with the candidates.

All documents are to be produced in MS Word format and provided electronically by email to the SC BEST/PREPS Coordinator and Education Technical Advisor. Copies of all PowerPoint presentations used to facilitate briefings for the project should also be provided to Save the Children in editable digital format.

Reporting and Governance

The consultant will report to the Senior Field Coordinator. Additional technical advice will be provided by the Education Technical Advisor, MEAL Coordinator and technical team from Save the Children members. Save the Children should approve all plans and documents developed by the consultant.

The lead consultant is to provide reporting against the project plan. The following regular reporting and quality review processes will also be used:

Regular email at least once a week to the Save the Children BEST/PREPS Project Manager documenting progress, any emerging issues to be resolved and planned activities for the following week.

At the end of the field visit, the evaluator will hold a meeting with the project team and project partners to discuss the preliminary findings of the evaluation exercise.

A draft report should be submitted for feedback and comments. The report should be written in English and approximately 30 pages with executive summary (appendices not included). The final evaluation report will comprise the following contents:

Table of Contents
List of Acronyms
List of Tables
Executive Summary
Background and context
Introduction
Literature Review
Scope of Evaluation
Research Design and Methodology
Data analysis
Key Findings
Conclusions and Recommendations
Annexes
- Evaluation TOR
- Inception report including workplan and budget
- Project Logframe
- Evaluation matrix and tools
- List of people involved – including list of key informants interviewed
- Raw data

The consultant will revise the report according to the agreed feedback and comments.

The final report will be assessed against Save the Children’s Evaluation Report Scoring checklist (Annex II). The MEAL Coordinator and Technical Advisor will review and sign-off for final submission to the Program Director, who will be accountable for approving the Final Evaluation.

Evaluation Management

What	Who is responsible	By when	Who else is involved
Evaluation tender submissions	BEST/PREPS Project Coordinator	December	Senior Field Coordinator
Tender review and selection of evaluation team	BEST/PREPS Project Coordinator	Mid-January	Procurement teams; Finance: Education Technical Advisor, MEAL Coordinator
Inception report with data collection tools informed by desk review	Evaluation team	End of January	BEST/PREPS Coordinator Education Technical Advisor, MEAL Coordinator, SCA MEAL Advisor.
Review of inception report	BEST/PREPS Coordinator	End of January	MEAL Coordinator Technical advisor, Regional MEAL Managers, Head of Research and Evaluation - Asia
Logistical arrangements	Evaluation team with support from BEST/PREPS Coordinator	Early Feb	SC Senior Field Co./Logistics team
Data collection	Evaluation team with their enumerators	Early Feb	N/A
Data management and analysis (coding, transcriptions, data cleaning, integration and analysis)	Evaluation team	Mid-February	N/A
First draft of the Final evaluation report	Evaluation team	Mid-Feb
Review of first draft report	BEST/PREPS Coordinator	TBD	MEAL Coordinator Technical advisor, SCA MEAL Advisor, Regional MEAL Managers, Head of Research and Evaluation Asia]
Meeting with evaluators and evaluation team to finalize the report	BEST/PREPS Coordinator	TBD	MEAL Coordinator Technical advisor
Validation of evaluation findings and recommendations	Evaluation Team	Mid-Feb	BEST/PREPS Coordinator, SC MEAL staff, Technical advisor, BEST/PREPS Consortium Partners
Final evaluation report and submission of data and analyses	Evaluation team	End of February	N/A
Knowledge translation materials	Evaluation team	End of February	N/A
Project team meeting to develop Evaluation Response Plan	SC BEST/PREPS Field Coordinator	March	Senior Field Coordinator, Technical advisor, MEAL Coordinator
Evaluation final report (together with response plan) posted on OneNet and reviewed	SC BEST/PREPS Field Coordinator	March	Technical Advisor, SC Peer reviewers

Consultation

Key Save the Children stakeholders to be involved in the evaluation are the BEST/PREPS Coordinator; Senior Field Coordinator; MEAL Coordinator; Education Technical Advisor

External stakeholders to be consulted include representatives from the following partners and local education authorities: ADRA, JRS, KRCEE, OCEE, CRED, KnED, and KWO.

Evaluation Team

To be considered, the Evaluation team members together must have demonstrated skills, expertise and experience in:

Masters’ degree in Social sciences, education, international relation or relevant area to the project.
Proven experience designing and conducting high quality project evaluations based on OEDC-DAC evaluation criteria
Proven ability and track record conducting ethical and inclusive research with vulnerable populations while ensuring rigorous ethics, integrity, and safeguarding especially in refugee setting;
Thai or international consultant who has experience to work in Thailand
Ability to work independently and meet tight deadlines
Language proficiencies required inclu

Contact : Yonradee.Wangcharoenpaisan@savethechildren.org

ที่ปรึกษา / Consultant