Back

Consultant for BEST IV Final Evaluation/ PREPS 2020-2022 Progress Review

Consultant for BEST IV Final Evaluation/ PREPS 2020-2022 Progress Review

: Save the Children

: Nonprofits / องค์กรไม่แสวงหาผลกำไร

: 1322

: 19 January 2021

25 January 2021

Consultant for BEST IV Final Evaluation/ PREPS 2020-2022 Progress Review

  1. Background

Save the Children has been working on Thailand for over 35 years, supporting the most vulnerable children with Education, Child Protection, Child Safety and Health & Nutrition programs. Whenever a disaster strikes in Thailand we are ready to deploy an emergency response. In 2020 we have deployed a multi-site, multi-sectoral response to the COVID19 pandemic with a focus on the Southernmost Provinces (Yala, Pattani, Narathiwat) and the border with Myanmar where more than 91,000 refugees still reside in 9 refugee camps and a large migrant population resides.

Since 2013, Save the Children is the lead of the EU consortium ensuring access to basic education services to school-age children residing in the 9 temporary shelters located in Mae Hong Son, Tak, Kanchanaburi and Ratchaburi Provinces. SC is also the co-lead – alongside the Jesuit Refugee Support (JRS) – of the Committee for Coordination of Services to Displaced Persons in Thailand (CCSDPT)’ education sub-committee. SC’s efforts align with its’ 2019-2021 CSP goal ‘The most deprived children receive a high quality multi-dimensional education from infancy to adulthood’.

Refugee Education Program

Over the past decades, more than 200,000 Myanmar nationals sought refuge in Thailand, fleeing conflict and ongoing violence in South East Myanmar. Presently, 91,777 (46,347f, 45,430m) verified Myanmar refugees reside in 9 camps alongside the Thai/Myanmar border (UNHCR, September 2020). 40% of them are children (17,805g; 18,539b). For these children, access to a quality basic education and the recognition of their academic achievements is a fundamental right, an essential component of their wellbeing and development as well as a steppingstone to live a peaceful and prosperous life.

Brief program outline

The BEST/PREPS project contributes to the achievement of durable education solutions for the refugee camp population in Thailand by supporting the provision of quality basic education for students in nine refugee camps along the Thailand-Burma/Myanmar border, supporting teachers to build education capacity, and facilitating progress towards sustainable education solutions.  Specifically, BEST/PREPS aims to achieve the following outcomes:

1. Basic education services are maintained and the quality of basic education is improved in nine (9) refugee camps.

2. Children, teachers, and parents have increased knowledge and skills to support successful transition and reintegration into education services in Myanmar.

3. Increased linkages, synergies, and cooperation between relevant local actors on both sides of the border leads to positive experiences of return, particularly related to education transition.

  1. Scope of Evaluation
    1. Purpose and Key Questions

This evaluation and assessment is being conducted at the end of the BEST IV project and the middle of Year 1 for the PREPS project. It will build upon the external final evaluation of the BEST III and mid-term review of PREPS previously conducted in May 2019.

The purpose of the evaluation is twofold:

1) to understand the extent to which the BEST IV and PREPS 2020-2022 achieved/ are achieving its intended results/outcomes according to indicators on the logical frameworks and

2) to establish baseline indicator values for select indicators as shown on an updated logical framework that will be used for Phase V of BEST project starting in March 2021.

The evaluation of BEST IV will answer the following key questions:

  • To what extent has the BEST/PREPS project had an impact in the lives of beneficiaries especially refugee children, teachers and education staff (KRCEE and KnED) in basic education system within the target areas?
  • How relevant, effective, efficient, sustainable, participatory, gender-sensitive and inclusive are the programme interventions?
  • To what extent was the BEST/PREPS project adaptive and responsive to COVID-19?
  • What lessons learned can be drawn from the project to further plan, design and develop future programmes?
  • What are the most critical areas of focus for future programs in the basic education for refugee children, e.g. how can we have the greatest impact on children in the coming years?

The baseline assessment data collection will collect additional data on a few indicators relevant to select basic education activities in the temporary shelters. Agreement of indicators will be determined during the inception period in consultation with the evaluation team. 

The Evaluation team will be required to undertake consultation with the SC MEAL Coordinator and the Education Technical Advisor at the commencement of the assignment in order to further refine the evaluation questions.

    1. Scope

Due to the changing context and often restricted access within the camps due to COVID-19, the evaluation team will be requested to conduct the evaluation remotely with support from Save the Children. For this, the evaluation will be limited to three camps known to have access to strong internet and/or phone signal. These include Mae La, Umphiem and Nu Poe camps in Tak Province.

    1. Stakeholders/Audiences

The main stakeholders/targeted audiences for this evaluation are:

Stakeholder

Further information

Project donor

European Union and Australian DFAT

Primary implementing organisation

Save the Children Thailand, BEST/PREPS Education Team

Implementing partners

Adventist Development Relief Agency Thailand (ADRA); Center for Rural Education Development Foundation (CRED), Jesuit Refugee Service (JRS) with Karen Refugee Committee Education Entity (KRCEE) and Karenni Education Department (KnED); Karen Women’s Organization (KWO)

Government counterparts

Ministry of Interior

Community groups

Parent Teacher Associations

Beneficiaries

Children and adults involved in the program/project/s and the evaluation

International development/humanitarian research community

CCSDPT members and other refugee education stakeholder groups such as KKEST; Migrant education stakeholders

In addition to assessing the impact of the BEST IV project, findings from the evaluation will be used to evaluate progress of PREPS VII, determine baseline data for BEST V, and inform lessons learnt and good practice for the on-going PREPS project and BEST V extension (2021-2023).

The Evaluation team will be required to propose how the evaluation findings will be shared with each of the different stakeholders in the table above, particularly outlining how reporting back to communities, beneficiaries and children will be conducted in an accessible and child friendly manner.

    1. Secondary Questions

a) Relevance:

  • Considering the current context and situation for refugees residing in camps along the Thai-Myanmar border, to what extent were project objectives and activities relevant to the needs, interests and priorities of target communities?
  • What are the socio-cultural barriers to the approach adopted by Save the Children (SCI) in delivering project outcomes, and how has SCI worked to identify and address these barriers?
  • To what extent were beneficiaries/stakeholders (partners, children & their families, teachers, community members, local authorities) involved in the design, planning, implementation, feedback mechanism establishment and process of the project?
  • To what extent to the project adjust to the needs of refugees during the COVID-19 pandemic?

b) Effectiveness:

  • To what extent were the objectives of the project achieved or likely to be attained? What are the barriers to the achievement of project objectives?
  • Are there any differences in outcomes achieved by different groups?
  • How effective are the monitoring systems of the project? How has the project used the information from monitoring activities to improve the quality of the project interventions and inform decision making process of the project?

c) Impact:

  • To what extent is the program producing intended and unintended results or changes (positive and negative)? What are the driving factors for intended and unintended results?
  • What is the impact or effect of the project in proportion to the overall situation of the target group or those effected?

d) Efficiency:

  • Is the relation between input of resources (materials, financial, services, staff etc.) and results achieved appropriate and justifiable?
  • In what ways has the consortium model contributed to greater efficiencies, cost-savings and fiduciary controls? How efficient were the partnership mechanisms and collaboration between SCI and partners?
  • How efficient were the complaint and response mechanism that had been established in the targeted communities?

f) Sustainability:

  • To what extent can activities, models, initiatives, results and effects be expected to continue or replicate at the conclusion of the project?
  • How the project planning and implementing sustainability strategy?

f) Cross cutting issues Participation, Gender, Safe programming and inclusion

  • What kind of gender gaps were identified in refugee education context and how did the project manage to tackle those gaps?
  • What were the protection measure undertaken in project implementation i.e. child safeguarding policy implementation, safe school facilities? How has the project considered gender sensitivity in the implementation of these policy?
  • To what extent have SCI integrated gender and social inclusion (i.e. disability) concerns and child safeguarding issues into the design and implementation of its interventions?
  1. Evaluation Methodology
    1. Research design and sampling

The suggested research design for this evaluation is a mixed methods design including participatory methods if and where possible. SCI anticipates the methodology to include a review of project documents, semi-structured interviews and focus group discussions with relevant stakeholders including beneficiaries, observations, and review of project monitoring data. The consultant is encouraged to propose additional methodology that they consider appropriate to the project, which is subject to Save the Children’s approval and must meet ethical guidelines (Annex I).

The evaluation will focus on three camps in Tak province. It will involve children, teachers, relevant local education authorities, project partner organizations, and SCI project team. Participants in the study will be selected through purposive sampling.

    1. Data

All primary data collected during the course of the evaluation must be disaggregated by sex, age, people with disabilities, status and camp location and inclusive of children’s representation.

Save the Children has existing data collection instruments and tools that can be drawn on in the evaluation. These will be shared with the evaluation team.

Save the Children will not provide enumerators to assist with primary data collection. Data triangulation is expected for this evaluation. It will be a requirement of the Evaluation team to source additional external data sources to add value to the evaluation, such as consultations with partners and review of project documents.  

A range of project documentation will be made available to the Evaluation team that provides information about the design, implementation and operation of the Program. Example of documents include previous evaluations, mid-term reviews, proposals, and logframes.

    1. Ethical considerations

It is expected that this evaluation will be:

  • Child-friendly and participatory. Children should be meaningfully involved in the evaluation as a holistic process and not only as informants. Refer to the Practice Standards in Children’s Participation (International Save the Children Alliance 2005); and Global Indicator technical guidance (SCI M&E handouts Package, Volume 2).
  • Inclusive. Ensure that children from different ethnic, social and religious backgrounds have the chance to participate, as well as children with disabilities and children who may be excluded or discriminated against in their community.
  • Ethical: The evaluation must be guided by the following ethical considerations:
    • Child safeguarding – demonstrating the highest standards of behaviour towards children
    • Sensitive – to child rights, gender, inclusion and cultural contexts
    • Openness - of information given, to the highest possible degree to all involved parties
    • Confidentiality and data protection - measures will be put in place to protect the identity of all participants and any other information that may put them or others at risk.
    • Public access - to the results when there are not special considerations against this
    • Broad participation - the relevant parties should be involved where possible
    • Reliability and independence - the evaluation should be conducted so that findings and conclusions are correct and trustworthy

It is expected that:

  • Data collection methods will be age and gender appropriate.
  • Evaluation activities will provide a safe, creative space where children feel that their thoughts and ideas are important.
  • The Evaluation team will adhere to the Save the Children Child Safeguarding, Data protection and Privacy policies that are required throughout all project activities.
  1. Expected Deliverables

The evaluation deliverables and due dates (subject to the commencement date of the evaluation) are outlined below. The lead evaluation consultant will advise [the SC BEST/PREPS Project Manager] immediately of any risks or issues that may impact on their ability to provide the deliverables by these due dates.

Deliverables and Due Dates

Deliverable

Due Date

The Evaluation Team is contracted and commences work

January, 2021

Phase 1: INCEPTION PHASE

The Evaluation Team will conduct initial desk research and tool review and develop and submit the Inception report that will include:

  • evaluation objectives and key evaluation questions
  • description of the methodology, data sources, draft data collection tools (preferably against the key evaluation questions and selected indicators for baseline assessment) and sampling considerations
  • caveats and limitations of evaluation
  • key deliverables, milestones and timelines
  • risk and issue management plan
  • a stakeholder communication and engagement plan
  • consultation protocols for consulting with children and other vulnerable groups (if applicable)
  • Logistical or other support required from Save the Children

Once the report is finalised and accepted, the evaluator/ evaluation team must submit a request for any change in strategy or approach to the Education Technical Advisor and BEST/PREPS Coordinator

To be submitted within 15 days after signed contract

 

Ongoing Desk Research of project documents and secondary sources

On-going January-February

Feedback by SCI

Within 5 working days after receiving the inception report

SCI Approval of Inception Report and data collection tools

Within 1 week after review completed –End of January

Phase 2: Data Collection and Analysis

  • Field work plan and logistical arrangements
  • Translation of tools
  • Data collection
  • Data analysis

Early-February (15 days)

Phase 3: Reporting

Preliminary findings presentation and verification workshop with partners Summary of interim findings

  • Any emerging program issues or risks (if applicable)
  • Key tasks for the next stage of the evaluation and any proposed refinements or changes to methodology (if applicable)

Mid- February (1 day)

Draft Evaluation Report* including the following elements:

  • Executive summary
  • Background description of the Program and context relevant to the evaluation
  • Scope and focus of the evaluation
  • Overview of the evaluation methodology and data collection methods, including an evaluation matrix
  • Findings aligned to each of the key evaluation questions
  • Specific caveats or methodological limitations of the evaluation
  • Conclusions outlining implications of the findings or learnings
  • Recommendations
  • Annexes (Project logframe, Evaluation TOR, Inception Report, Study schedule, List of people involved)

3rd week in February

Feedback and approval by SCI

Within 5 working days after receiving the inception report

Final Evaluation Report* with submission of data and analysis incorporating feedback from consultation on the Draft Evaluation Report

End of February (3 days)

Knowledge translation materials:

  • PowerPoint presentation of evaluation findings
  • Evidence to Action Brief**

End of February (1 day)

*All reports are to use the Save the Children Evaluation report template. Please also refer to Save the Children technical writing guide.

** The Evidence to Action Brief is a 2-4-page summary of the full report and will be created using the Save the Children Evidence to Action Brief template.

Dates and times for deliverables will be changed upon consultation with the candidates.

All documents are to be produced in MS Word format and provided electronically by email to the SC BEST/PREPS Coordinator and Education Technical Advisor.  Copies of all PowerPoint presentations used to facilitate briefings for the project should also be provided to Save the Children in editable digital format.

  1. Reporting and Governance

The consultant will report to the Senior Field Coordinator. Additional technical advice will be provided by the Education Technical Advisor, MEAL Coordinator and technical team from Save the Children members. Save the Children should approve all plans and documents developed by the consultant.

The lead consultant is to provide reporting against the project plan. The following regular reporting and quality review processes will also be used:

  • Regular email at least once a week to the Save the Children BEST/PREPS Project Manager documenting progress, any emerging issues to be resolved and planned activities for the following week.

At the end of the field visit, the evaluator will hold a meeting with the project team and project partners to discuss the preliminary findings of the evaluation exercise.

A draft report should be submitted for feedback and comments. The report should be written in English and approximately 30 pages with executive summary (appendices not included). The final evaluation report will comprise the following contents:

  • Table of Contents
  • List of Acronyms
  • List of Tables
  • Executive Summary
  • Background and context
  • Introduction
  • Literature Review
  • Scope of Evaluation
  • Research Design and Methodology
  • Data analysis
  • Key Findings
  • Conclusions and Recommendations
  • Annexes
    • Evaluation TOR
    • Inception report including workplan and budget
    • Project Logframe
    • Evaluation matrix and tools
    • List of people involved – including list of key informants interviewed  
    • Raw data  

The consultant will revise the report according to the agreed feedback and comments.

The final report will be assessed against Save the Children’s Evaluation Report Scoring checklist (Annex II). The MEAL Coordinator and Technical Advisor will review and sign-off for final submission to the Program Director, who will be accountable for approving the Final Evaluation.

  1. Evaluation Management

What

Who is responsible

By when

Who else is involved

Evaluation tender submissions

BEST/PREPS Project Coordinator

December

Senior Field Coordinator

Tender review and selection of evaluation team

BEST/PREPS Project Coordinator

Mid-January

Procurement teams; Finance: Education Technical Advisor, MEAL Coordinator

Inception report with data collection tools informed by desk review

Evaluation team

End of January

BEST/PREPS Coordinator Education Technical Advisor, MEAL Coordinator, SCA MEAL Advisor.

Review of inception report

BEST/PREPS Coordinator

End of January

 

MEAL Coordinator
Technical advisor, Regional MEAL Managers, Head of Research and Evaluation - Asia

Logistical arrangements

Evaluation team with support from BEST/PREPS Coordinator

Early Feb

 

SC Senior Field Co./Logistics team

Data collection

Evaluation team with their enumerators

Early Feb

 

N/A

Data management and analysis (coding, transcriptions, data cleaning, integration and analysis)

Evaluation team

Mid-February

 

N/A

First draft of the Final evaluation report

Evaluation team

Mid-Feb

 

 

Review of first draft report

BEST/PREPS Coordinator

TBD

 

MEAL Coordinator
Technical advisor, SCA MEAL  Advisor, Regional MEAL Managers, Head of Research and Evaluation  Asia]

Meeting with evaluators and evaluation team to finalize the report

BEST/PREPS Coordinator

TBD

 

MEAL Coordinator
Technical advisor

Validation of evaluation findings and recommendations

Evaluation Team

Mid-Feb

 

BEST/PREPS Coordinator, SC MEAL staff, Technical advisor, BEST/PREPS Consortium Partners

Final evaluation report and submission of data and analyses

Evaluation team

End of February

 

N/A

Knowledge translation materials

Evaluation team

End of February

 

N/A

Project team meeting to develop Evaluation Response Plan

SC BEST/PREPS Field Coordinator

March

 

Senior Field Coordinator,
Technical advisor, MEAL Coordinator

Evaluation final report (together with response plan) posted on OneNet and reviewed

SC BEST/PREPS Field Coordinator

March

 

Technical Advisor, SC Peer reviewers

  1. Consultation

Key Save the Children stakeholders to be involved in the evaluation are the BEST/PREPS Coordinator; Senior Field Coordinator; MEAL Coordinator; Education Technical Advisor

External stakeholders to be consulted include representatives from the following partners and local education authorities: ADRA, JRS, KRCEE, OCEE, CRED, KnED, and KWO.

  1. Evaluation Team

To be considered, the Evaluation team members together must have demonstrated skills, expertise and experience in:

  • Masters’ degree in Social sciences, education, international relation or relevant area to the project.
  • Proven experience designing and conducting high quality project evaluations based on OEDC-DAC evaluation criteria
  • Proven ability and track record conducting ethical and inclusive research with vulnerable populations while ensuring rigorous ethics, integrity, and safeguarding especially in refugee setting;
  • Thai or international consultant who has experience to work in Thailand
  • Ability to work independently and meet tight deadlines
  • Language proficiencies required inclu

Contact : Yonradee.Wangcharoenpaisan@savethechildren.org


ที่ปรึกษา / Consultant