ThaiNGO

Title : Consultancy for Final Evaluation of Road Safety Project

1. Background

According to the WHO, Thailand has the 2nd most dangerous roads in the world, where over 2,600 children die and more than 72,000 are injured on Thailand's roads every year. Research indicates that helmet use is the single most effective means of preventing injury among motorcyclists. It reduces risk of death by 40 percent and severe head injury by 70 percent. However, only 7% of children wear helmets when riding motorcycles, even when their parents do. The main challenges limiting helmet use among children in Thailand are found to be: weak enforcement, short journey, inconvenience to the users, perception of helmets as “unfashionable”, and cultural beliefs that fate is predetermined.

Save the Children (SC) launched the “7% Project” project in Thailand, since 2013, to decrease motorcycle deaths and injuries among Thai children by increasing helmet use. It started as a small pilot initiative in 2015, and subsequently received funding to scale up activities with a 3 Year project from June 2016 to May 2019. The project aims to improve the teaching of road safety in schools and encourage behaviour change to increase helmet usage, working with teachers, students, local authorities, police, media, corporate partners and communities across Bangkok. The key project components are:

• Education: Teaching children why and how to wear helmets, with a focus on utilising social and peer influence to make helmet wearing a habit.
• Enforcement: Working with schools, parents and police to ensure children wear helmets on every trip.
• Advocacy and Media Campaign: Launching a campaign using nationwide messaging, events, advocacy and social mobilisation.
• Increasing demand and acceptance: Designing helmets that youth want to wear and providing opportunities for children to customise their own helmets, including the seeking innovative methods to inspire change to people

300 schools were targeted in the 2016-2017 school year, 200 in 2017-2018 and 200 in 2018-2019. During Year 1, the project trained teachers and provided a toolkit for schools. Following a review of the approach, in Year 2 the project continued to provide “light touch” training and support to teachers and schools, but also worked with a smaller group of schools to provide “intensive” support with more direct engagement in the delivery and coordination of activities. This was continued in Year 3.

1. Purpose, Objectives, and Research Questions

After three years of implementation, Save the Children wishes to assess the impact, effectiveness, relevance and sustainability of the 7% Project. Save the Children is seeking for a consultant or team to conduct a Final Evaluation of the road safety project. The final evaluation should:

• Assess the extent to which the project has achieved its objectives and results
• Assess the project in terms of its relevance, effectiveness and sustainability.
• Provide a comprehensive analysis of the project’s design and implementation to highlight strengths and weaknesses, generate findings and document lessons learned with local stakeholders, partners and beneficiaries

The evaluation will require a mixed methods approaching, including an analysis of existing quantitative data and collection of new qualitative data.  The consultant will utilize all program data available to prepare a report that determine whether the project has achieved its objectives and results.

Key questions to be answered include:

Impact

• What are the results of the project intervention and the change seen on helmet use among children in Bangkok?
• Did the project activities and outputs achieve the desired outcomes?
• How did the outcomes of the project contribute to the impact?
• Were there any unintended positive or negative outcomes resulting from the project?
• What were the key successes and challenges in the project implementation from the perspectives of stakeholders and Save the Children’ staff and partners at all levels?
• What other external factors or influences that may have affected the achievement of intended results?
• What were the key driving factors for these success/challenges?

Relevance

• To what extent was the project rationale and design relevant to the needs of the beneficiaries (children and adults: teachers, parents, polices, etc)
• How was feedback from stakeholders applied in the project implementation?

Effectiveness

• How effective was the project design and activities (strengths and weaknesses)?
• How did revisions to the project design affect the project?
• What were the most effective elements of the project? What’s worked what did not work?
• What are the key learnings from project implementation which will inform the design of the future projects in other country?
• To what degree are various stakeholders satisfied with types programming as well as effectiveness of these programs?
• What is the nature and strengths of current strategic partners?

Sustainability

• Which elements of the project design and implementation do stakeholders considered most sustainable? And why?
• How likely are schools and to continue implementing road safety after the project ends?

III. Evaluation Method and Data Collection Plan

While the consultant will propose a detailed methodology to suit the evaluation, a consultative and participatory approach will be a basic minimum requirement. The main activities are expected to comprise of project document review, analysis of existing primary data, interviews or focus group discussions with project staff, beneficiaries, implementing partners and key stakeholders, and secondary data research or collection.

The consultant(s) will be expected to lay out their detailed methodology in an Inception Report following initial consultations with Save the Children. This should cover preparatory work (literature review, sampling, survey and interview tools), field work (consultations, data collection), and analysis plan (data processing, findings, report, presentation). The deliverables and timescale for the research are outlined below. The assessment should be undertaken in line with standard ethical considerations and adhere to Save the Children’s ethical guidelines.

In order to assess the multifaceted project, particularly given the complex interplay of factors which limit the extent to which impact can be determined, a purposive sampling approach is recommended.

The evaluation is also expected to analyse and utilise existing primary quantitative data during the project’s implementation by external and independent sources, such as helmet observation studies, KAP studies, and social media metrics.

Other documents to be provided include proposal, MEAL framework, baseline assessment, annual and semi-annual reports, research reports, monitoring reports, interview reports, campaign and IEC materials. A draft report detailing the program history and a set of good practise case studies will also be available for reference.

IV. Scope and Tasks

The main tasks to be undertaken by the consultant include, but are not limited to;

• Review of literature on:
  • the crashes situation and statistic in Thailand during 2011-2020, referred to The Decade of Actions for Road Safety
  • Governmental policies: laws, cities policies, educations, etc
  • Road safety Networking: Donors, Thai Health Promotion Foundation (Sor Sor Sor), Department of Disaster Prevention and Mitigation (DDPM), Road Accident Victims Protection Co., Ltd. (RVP), A.P. Honda, Child Safety Promotion and Injury Prevention Research Center (CSIP), AIP Foundation, SideKick, MCOT, etc
  • Save the Children’s Road Safety projects’ designs, partners, reports and material
• Preparation of Inception Report containing evaluation plan, methodology, evaluation tools, target group, budget estimate and work plan to be discussed and approved by Save the Children
• Analysis of quantitative project data.
• Data collection from primary sources in Bangkok
• Debrief of field data collection result (preliminary findings)
• Data analysis, presentation and draft report preparation
• Incorporate feedback from Save the Children
• Complete the final report

V.Timeline

The consultant will be required to dedicate an estimated 45 working days to this consultancy work during April – June 2019 subject to the requirements and agreements with Save the Children.

Key Deliverables

Deliverable 1: Review of project literature

Timeframe :    Upon contract signing

Deliverable 2 : Initial briefing and planning discussion on methodology and approach

Timeframe :    2 days after confirmation of contract

Remarks :       At SC office (central Bangkok)

Deliverable 3 : Draft Inception report and work plan

• Mapping of available data and plan for data collection
• Draft of evaluation framework
• Detailed research methodology including selection criteria for inclusion in research, no of KIIs and FGDs,
• Draft of evaluation tools

Timeframe :    10 days after confirmation of contract

Remarks :       SC will review the Inception report within 3 working days

Deliverable 4 : Submission of finalized Inception Report and work plan

Timeframe :    5 days after draft report submission

Deliverable 5 : Data collection/interviews/visits with project staff, partners, stakeholders and beneficiaries

Timeframe :    April 2019

Deliverable 6 : Presentations of preliminary findings to Save the Children

Timeframe :    May 2019

Deliverable 7 : Submission of First Draft Report

Timeframe :    End of May 2019

Remarks :       SC will provide comments within 5 working days

Deliverable 8 : Final Evaluation report completion

• Evaluation summary report (3-5 pages)
• PowerPoint slide deck of key findings, for sharing to stakeholders
• Full report (30 pages maximum excluding appendices)

Timeframe :    June 2019

Remarks :       Report structure to be agreed during inception stage

All documents are to be submitted in English.

Management of Consultancy

Save the Children should approve all plans, tools and documents developed by the consultant. The final report will be assessed against Save the Children’s Evaluation Checklist. The consultant will report to the Child Health & Safety Program Coordinator.

The report will be considered final only after the approval of the Thailand Country Office.

Consultancy Budget

The consultant is requested to submit the proposal and all-inclusive budget with their application letter. The financial competitiveness of the fee will be considered in the selection process. Costs involved in the assessment such as materials, travel and taxes, should be included in the budget proposal.

Consultant Qualifications and Profile

• Relevant academic background and extensive experience in research and evaluation
• Familiarity with development, education and behaviour change projects.
• Experience in Road Safety and Helmet Safety promotion is an advantage
• Contacts within Bangkok education and police agencies is desirable
• Thai and English language proficiency
• Excellent written English skills
• Ability to work within strict deadlines

Submissions for the Consultancy

Submissions for the consultancy must address the Terms of Reference and include:

1. Name and contact details of applicant
2. Motivational letter, comprehensive CV of the and details of at least 3 referees
3. A cover letter outlining the relevant experience and expertise of the consultant and initial ideas on the approach to be taken the achieve the consultancy objectives
4. A summary proposal of the research scope, design, methods, and field implementation approach (maximum 5 pages)
5. An example of a previous evaluation conducted
6. A detailed quote in THB (including travel and other costs)

The role holder is required to carry out the duties in accordance with SCI Health and Safety policies and procedures.

SCI offer competitive National package; Transportation & Communication allowance, Wellness Top up, 13th month salary & Annual leave 20 days, etc.

Please indicate in the subject as “Consultancy for Final Evaluation of Road Safety Project _(Name of candidate)”

Only shortlisted candidates will be notified.

“We need to keep children safe so our selection process, which includes rigorous background checks, reflects our commitment to the protection of children from abuse”

Submissions should be addressed to: hr.thailand@savethechildren.org

Annex 1: Project’s Logical Framework:

https://www.dropbox.com/s/om2codyf0c7c0m9/Annex%202_7%25%20Project%202016%20Log%20Frame-6%20Feb16_KK_nai%20revised_revNBP-TC_rev21March2016.xlsx?dl=0

Annex II: Evaluation report quality score tool:

https://www.dropbox.com/s/pommi7ua8m9wk5y/Evaluation%20Report%20Quality%20Score%20tool_final%20%281%29.xlsx?dl=0