ThaiNGO

Term of Reference for Middle Term Evaluation

Title:Terms of Reference for Final Evaluation

1.            Executive Summary

Background to the Project

In January 2016, ADRA Thailand started a 36-month project entitled ‘Enhanced Capacities for Migrant Advocacy’ (ECMA) funded by the European Union, which targets 3,750 migrant factory workers in Mae Sot, Thailand. The project aims to strengthen and empower marginalized and vulnerable migrant workers in Mae Sot, Thailand. 

Thailand’s rapid economic growth in the past few decades has created a high demand for low-skilled and low-cost labour, attracting a large number of migrant workers from Thailand’s three neighbouring countries, namely Myanmar, Lao People’s Democratic Republic, and Cambodia, to fill the labour shortage gap. An estimated 3 million migrant workers from these three countries hold either regular or irregular status in Thailand[1]. As located in a bordering province, Tak, Mae Sot, has eventually become a transit and destination for Myanmar migrants because of its high concentration of factories. According to statistics from the Tak Provincial Office, there were 27,059 Myanmar migrant workers registered at the Mae Sot District’s One Stop Service Centre during the migrant registration process in 2015.  Migrant workers are employed in factories in textiles, garments, food processing and ceramics.  The majority of migrant workers are from Myanmar, with Burmese representing the largest ethnic group, along with other ethnic groups such as Karen, Mon and Arakanese.

The major problems facing Burmese migrant workers include: i) a lack of awareness about their rights due to language barriers and accessibility to information. An additional problem is the limited human resources of the local Labour Protection and Welfare Office compared to the large migrant population within the area; ii) factory owners do not follow Thai Labour Law by registering their factories, leading migrant workers to become vulnerable in terms of working in unsafe, unhygienic conditions; iii) major health concerns of migrant workers included skeletal or muscular illnesses due to heavy workloads and poor occupational health and safety standards; iv) lack of capacity of CSOs to engage in the dialogue process, at national level - little effective coordination, limited resources and travel restrictions, which are among major factors that limit the ability of CSOs to engage directly and actively in policy advocacy dialogue; and v) poor living and unhealthy conditions, as migrants are often housed in overcrowded accommodation with inadequate facilities; some factories restrict their workers to the factory premises giving them no opportunity to find other accommodation.

Project Activities (according to the “Description of the Action”)

1.1 Conduct a project launch for joint initiative between LPO and ECMA project to increase systematic awareness raising on labour and human rights to cover a greater number of factories

1.2 Organize four (4) meetings on labour rights and migrant legislations for employers from 70 factories in Mae Sot, and organize seven (7) meetings on labour law awareness and safety employment with 20 factories in collaboration with the Labour Protection and Welfare Office (LPO), and the Federation of Thai Industries (FTI).

1.3 Conduct five (5) paralegal trainings for 150 members of community-based organization (CBO) and non-governmental organizations (NGO) led by Thai and Myanmar migrant community leaders in Mae Sot, in order to enhance their capacity on handling and presenting migrant legal case to the Labour Law Clinic (LLC) and LPO.  

1.4 Sub-grants and technical support provided to 2 CSOs (focused on labour/legal rights of migrants), to enhance their capacity for providing legal assistance/advice to migrant workers. This will provide a provision of direct assistance to facilitate 1,500 migrant legal caseworks of vulnerable migrants, in order to assist them to seek legal assistance and remedy.

1.5 Provide financial support to LPO to organize 3 workshops to present findings of monitoring results on labour rights legislation and standard monitoring systems.

1.6 Develop and produce promotional materials to distribute to all relevant stakeholders throughout the project implementation.

2.1 Facilitate eight (8) CSOs to participate in local and national networks meeting i.e. Migrant Rights Promotion Working Group (MRPWG), Community-Based Organizations (CBO) monthly meetings, and Migrant Working Group (MWG) through the provision of technical and financial support

2.2 Conduct six (6) workshops on labour and human rights advocacy, networking skills, and knowledge on migrant legislation and policy to eight (8) CSOs to strengthen their advocacy capacity, in coordination with LLC, LPO, and INGO.

2.3 Provide financial support to organize Migrant Rights Promotion Working Group (MRPWG) and Community-Based Organization network (CBO) monthly meetings, which serve as platforms for CSOs, INGOs, and government officials to strategise on the best ways to advocate for migrant rights.

2.4 CSO management/organization capacity building

3.1 Conduct three (3) ToTs for 120 members from different CSOs and factory Safety Officers on occupational health, work safety, and hygiene.

3.2 Support CSOs to organize 20 outreach activities to promote health and hygiene to 2,250 migrant workers from 20 factories.

3.3 Develop and produce 5,000 multilingual (Thai and Burmese) brochures on occupational health, work safety, and hygiene in order to increase awareness in target groups and their relatives.

3.4 Support CSOs to conduct five (5) trainings on occupational health and work safety to 250 migrant workers and provide 500 sets of basic safety equipment and manuals to migrants.

3.5 Hold an annual ‘lessons learned’ workshop upon the completion of each year’s project implementation. There will be a total of three (3) workshops.

Note: Please refer to the amended “Description of the Action” for further details on project activities.

Key Project Stakeholders

Migrant factory workers in Mae Sot have participated in project activities, such as safety and hygiene training, legal aid consultation, community outreach, and so on.

Labour Protection and Welfare Provincial Office (LPO) is an associate partner for providing guidelines on the implementation of this proposed action.  As a government agency working to promote and protect labour rights, LPO are expected to play a key role in providing legal assistance for migrant workers and monitoring the implementation of labour law in the target factories.

The Federation of Thailand Industries (FTI) is an associate partner for coordinating with factories and providing guidance on the project implementation.  FTI is expected to be part of a consultative team to identify target factories that ADRA will work with to promote labour law and improve working conditions.

Civil Society Organizations (CSOs) are local organizations led by Thai and/or Myanmar workers based in Mae Sot.  Eight CSOs have been identified to provide legal assistance, disseminate and advocate labour law and migrant rights as well as delivering training on occupational health and hygiene promotion to migrant workers in factories. Eight CSOs who received sub-grant under this project are Human Right Development Foundation (HRDF)/Labor Law Clinic (LLC), MAP Foundation (MAP), Yaung Chi Oo Workers’ Association (YCOWA), Arakan Labor Campaign (ALC), Mae Sot Lawyer Association for Human Rights Along the Border (LHB), Foundation of Education and Development (FED), Migrant Working Group (MWG), and Migrant Development Center (MDC). 

Factory owners in Mae Sot have participated in project activities, particularly those who signed up for the “Safety and Happy Factory Model”.  Under this model, LPO will certify the factories as “Model Factories” when they comply with the 4 criteria that include: 1) management and administration, 2) Health, hygiene and the environment, 3) Physical and social safety, 4) Knowledge and awareness of employees and employers.

Note: Please refer to the amended “Description of the Action” for further details on project stakeholders.

Key Components of the Final Evaluation

The main objective of this evaluation is to provide ADRA and the European Union (EU) with a review of the project performance, relevance, and to capture outcomes and possible impact of the broader project, and make suggestions of potential adjustments to the future project. 

The evaluation results are envisaged to identify and describe the lessons learned, through measurements of the changes in the set indicators, summarize the experiences gained, technically and managerially, and recommend the approaches and methodologies for future sustainability.

To achieve the above objective the Final Evaluation will address the following:

1. Assessment of the project progress towards attaining its objectives and results using logical framework matrix and indicators, particularly outcome indicators, (Note: A review of output indicators will be provided by the project team); Assessment of the Monitoring, Evaluation, and Learning (MEL) system in capturing and analysing data;

The following questions should be answered:

• To what extend the project has taken recommendations from the mid-term evaluation and incorporated into the remaining period of the project?
• What the achievement or underachievement of the project results really mean based on the qualitative and quantitative analysis?

2. Assess the relevance of project in addressing the problems stated in the proposal;

The following questions should be answered:

• Are the project interventions (methodologies and activities) relevant to achieve the project objectives?
• Are the project interventions (methodologies and activities) relevant to the context and target group?
• Is the project adequately adjusting to changing situations or contexts?

3. Assessment of the performance of the project in terms of efficiency and effectiveness of the activities undertaken, as well as in terms of the quality of the relationships established with key stakeholders;

The following questions should be answered:

• Is the project managed efficiently, in terms of human capital, staff, financial and other resources versus the results?
• Are beneficiaries sufficiently involved in the project implementation? Is there feedback from beneficiaries to project implementers?
• How effective was the capacity building support provided to CSO partners?
• What are the external constraints to achieving better efficiency and how well are they mitigated?
• Were synergies capitalised on with other actors (local and international) involved in similar projects?
• How has the relationship been between the partners and local government authorities?
• To what extent is the project achieving its specific objective “to strengthen capacity of local CSOs (Thai and Migrant Worker led) to effectively advocate for improvements in labor/human rights and working/living conditions for vulnerable migrant workers in Mae Sot.”?
• To what extend the project has demonstrated the value against the amount of funding utilized to deliver the project?

4. Assess the prospects of the sustainability of the project outcomes and benefits and make concrete recommendations for its further improvement for future projects based on the evidence collected;

The following questions should be answered:

• To what extent are the risk mitigation and sustainability plans in the project proposal, in

addition to unforeseen risks, addressed during project implementation? How realistic were these?

• What evidence is there to suggest the project’s interventions and/or results will be sustained after the project end?
• Can the project be scaled up or replicated in the future?
• To what extent the project has developed its exit strategy?

5. Assess the impact of the project in achieving the overall project goal;

The effect of the project on its wider environment, and its contribution to the wider policy or the project’s overall objective. In particular, the evaluation of impact should address the following key elements:

-              Policy level impact;

-              Social level impact;

-              Economic level impact;

-              Technical level impact.

The following questions should be answered:

• Is the project ultimately contributing to the overall objective stated in the proposal? i.e. “To strengthen and empower marginalized and vulnerable migrant workers in Mae Sot, Thailand”.
• What, if any, are the unintended impacts of the project intervention, both positive and negative?
• What are the external constraints to achieving better effectiveness and how well are they mitigated?
• If the project had not occurred, would there have been any change?

6. Identify and describe the main lessons learned from the project performance to date, notably taking into account the lessons learned workshops conducted by the project team.

The following questions should be answered:

• How was learning addressed within the scope of the project?
• What are the lessons learned on different levels – organization, implementation, beneficiaries, CSOs, and risks (both internal and external)?

A special attention should be given to the discussion and formulation of concrete recommendations for future replication of the project.

Methodology of the Final Evaluation

The evaluation will be based on the findings and factual statements identified from review of relevant documents including the project contractual documents and in particular the amended “Description of the Action”, the annual interim reports to the donor, the baseline survey report, mid-term evaluation, findings from primary data collected from the field visit, as well as any technical reports, Monitoring, Evaluation and Learning Plan, training materials and different promotional materials produced by the project.  A list of the above documents will be shared with the evaluator before the beginning of the mission. The mission will also undertake field visits and interview the stakeholders including target beneficiaries, government officials, CSOs, etc.

Participation of stakeholders in the evaluation should be maintained at all the times, reflecting opinions, expectations and vision about the contribution of the project towards the achievement of its objectives. In particular, the following stakeholders should be visited and interviewed:

• ADRA’s Project Team including Program Director, Program Coordinator, Project Manager, Monitoring and Evaluation Coordinator, and Field Coordinators for a detailed overview about the project performance, successes and challenges encountered;
• Labour Protection and Welfare Provincial Office (LPO) for an overview about the joint initiative (Safety and Happy Factory Model), relevance of the project and collaboration with ADRA’s project team;
• All partner CSOs who received sub-grants, namely HRDF/LLC, MAP, YCOWA, FED, MWG, and MDC for a detailed overview about the partnership with ADRA, effectiveness of the activities and challenges encountered;

• Minimum of 3-5 Factory owners who have participated in the joint initiative between LPO and ECMA for feedback on the “Safety and Happy Factory Model” and benefits for the factories;
• Minimum of 20 Migrant factory workers who have been participating in the project activities for feedback on the effectiveness of the activities that target them and any suggestions for improvement.
• Survey at least 200 factory migrant workers who have been participating in the project activities for feedback on the effectiveness of the activities that target them and any suggestions for future improvement.

Methodologies such as focus group discussions, semi-structured interviews, surveys, and direct observations, etc. will be applied as appropriate.

Products expected from the evaluation

The evaluator will provide ADRA with a comprehensive draft report for review and comments. Please note that the suggestions, which are made for future improvement, are expected to be incorporated in the final report to the EU.  The report should be preceded by an executive summary. The report will include:                         

• Purpose of the evaluation and the methodology.
• The main findings: project relevance, efficiency, effectiveness, impact of project activities, sustainability with recommendations for improvement. Key questions asked in the “key components of the final evaluation” of this Term of Reference need to be answered and incorporated into the sub-sections under the main findings.  
• Lessons learned: Assessment of attainment of indicators, operational and developmental lessons.
• Conclusions and recommendations.
• Annexes: TOR, Itinerary, List of people met, List of documents reviewed.                  

Note: We will encourage regular contact between ADRA and the evaluators to ensure a good understanding of the project activities and the evaluation recommendations. In this context, we envisage at least two meetings:

• Briefing (first day): the evaluators will explain the approach and methodology for the evaluation and ADRA Thailand/ECMA team will share expectations/areas of interest.
• De-briefing (last day of the field phase): the evaluators will share the raw main findings and recommendations, which will be developed in the draft report and ADRA Thailand/ECMA team will provide a first reaction to them, as appropriate.

Timeframe and duration of the evaluation

A total of 18 payable working days are planned for this evaluation:

• 2 days desk review (clarify matters arising from the documents read previously)
• 2 days designing the interview and survey questionnaires
• 6 days field assessment (visiting Mae Sot and interviewing key stakeholders) including briefing and debriefing with ADRA Thailand/ ECMA team, training of local enumerators;
• 2 days of data entry and analysis (including data entry and analysis for quantitative survey in appropriate software)
• 5 days writing of the draft report (give two weeks for ADRA UK/ECMA team to provide comments to the draft report)
• 1 days to address comments from ADRA ADRA-UK/ECMA team and finalize the report

The ECMA project team in Mae Sot will facilitate the planning of the field itinerary.  Accommodation, transportation including domestic flights, and food during the field mission in Mae Sot will be covered by the ECMA project.  ECMA project will provide translators/interpreters during the field mission.  ADRA can also assist in finding local enumerators to conduct quantitative survey and enter data in SPSS software.

Background of the consultant undertaking the evaluation

• Hold at least a bachelor degree in the development field, preferable master degree in related development field
• At least 10 years’ experience in the development field- particularly in migration issues is preferred
• Experience in working with migrant communities would be beneficial
• Experience in conducting similar scale evaluations
• Concrete experience in providing written reports in English

Submission of application

Applications should be submitted no later than 18 November 2018

Applications in the English language, including i) a resume of not more than 3 pages; ii) a cover letter; iii) detailed budget to carry out this evaluation, costs including consultant fee, international flight, accommodation and transportation in the Bangkok if necessary (costs in Mae Sot will be covered by the project; iv) evaluation proposal detailing the methodology, approach, proposed timetable, as appropriate; v) contact details of 2 references should be submitted by email to linda@adrathailand.org, hr@adrathailand.org and cc’d to emily@adrathailand.org

Evaluation of application

Applications will be evaluated by a selection committee within two weeks of the application deadline.  Only shortlisted candidates will be contacted.

[1] Thailand Migration Report 2014, International Organization for Migration (IOM).