Terms of Reference (ToR)
Independent Evaluation
of the Help – Hilfe zur Selbsthilfe e.V.
South-East Europe Project
- Introduction
Help- Hilfe zur Selbsthilfe (Help) is a non-governmental, non-profit Organization active in humanitarian aid and development cooperation.
Help has been operating in the Balkan region for 25 years and recently shifted from a country project to a regional project approach financed by the German Federal Foreign Office. Help is commissioning an independent evaluation of the regional project: “Support to socio-economic stability in the western Balkans region 2019-2020”.
This evaluation shall
- assess the introduction and implementation of the OECD DAC criteria on impact and sustainability of the project approach as well as
- draw recommendations to better support the planning of future programes and projects.
The evaluator is invited to support and cooperate with the project team. This may entail participating in an in-depth interview, providing requested information or documents and making use of the reporting templates developed by the team.
- Background
The project “Support to socio-economic stability in the western Balkans region 2019-2020” is implemented in the Western Balkans including Bosnia and Herzegovina, Serbia, Kosovo and Montenegro. The overall objective of the program is to empower economically and socially vulnerable groups and enable social inclusion by reaching the following specific objectives:
Specific Objective 1: Decreased levels of poverty among economically vulnerable groups
Specific Objective 2: Improvement of the treatment programs in correctional institutes and the process of re-integration into society
Specific Objective 3: Better inclusion of Roma and vulnerable social cases in the society and migration prevention
The main activities to achieve those objectives are:
- Direct job creation through educational and economic support to 550 micro businesses
- Improvement of the economic situation and treatment programs in 13 selected correctional institutions in Serbia, Kosovo and Montenegro through equipping workshops for the increased working engagement and different vocational and specialized trainings
- Construction of prefab-houses and buildings for Roma and other socially vulnerable families in Niksic, Danilovgrad and Bijelo Polje / Montenegro Construction of 4 prefab-houses in Niksic/ Montenegro
- Facilitating access to health, employment and social services for Roma Egyptian population
The program is implemented from 1 January 2019 until 31 December 2020.
Note: for Montenegro evaluation only yellow marked activities are relevant and subject of evaluation
- Purpose, key evaluation questions and general aims of the evaluation
Help aims to ensure quality in its work and thus builds on a strong culture of lessons learnt. As such, Help is committed to review its program through external evaluations.
3.1. Purpose of the evaluation
The evaluation will make a comprehensive assessment of the program describing the project’s relevance, its level of success in achieving its objectives, program efficiency and effectiveness, methodology, sustainability and impact of its interventions, identifying key weaknesses and assessing its contribution to socio-economic stability in the region.
This means:
- Assessment of the achievement of goals of the project
- Identification of the impact of the program and ways this may be sustained beyond the project
- Improve future program design and management
- Verify that funds were used effectively and efficiently to deliver results
- Assessment of the program structure and its functionality in regards to the current environment
Evaluation Principles: Respect for People; Independence & Impartiality; Utility; Credibility; Validity; Transparency; See Annex 1 for detailed interpretation of these principals
3.2. Main questions / relevant aspects to cover
The evaluation will address two Key Evaluation Questions (KEQs). All KEQs should be queried independently from each other. Following sub-questions are conceivable but not exclusive:
OECD DAC criteria: Impact
Quality Criterion: The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.
KEQ 1: Had the project activities the planned impact?
- Was the project design needs based?
- Did the project design consider risks and vulnerabilities of the target population?
- How did the project change the lives of target groups?
- How many people were influenced by the project and how?
- How did the beneficiaries involved directly in the project influence the other members of the target group?
- What are the potential unintended positive and negative effects of the implementation of the program?
- How did the program contribute to improving?
- Were the activities as reflected in the program proposal and work plans consistent with the overall goal and objectives?
OECD DAC criteria: Sustainability
Quality Criterion: The extent to which the net benefits of the intervention continue, or are likely to continue.
KEQ 2: Were local capacities strengthened and negative side effects avoided?
- Was the program build on local capacities?
- How is a longer-positive effect of the outputs ensured?
- Did the program create negative side effects and if so, how were they responded too?
- How were negative side effects avoided?
- Did the program strengthen the local capacities on the long-term?
- g. What were the project measures for sustainability and how were they implemented / achieved?
- How will the project continue after phase-out of project partners?
- What was the level of cooperation with local authorities?
- What are the main factors influencing the sustainability of the project in terms of employment RE persons?
The KEQs should be addressed with special, but not exclusive, consideration of the OECD/DAC criteria of appropriateness and relevance.
The KEQs listed above are to be considered as guiding questions only and the evaluation team is not limited to them. The refining and further elaboration of the questions should be done by the evaluator, which will propose a matrix of detailed evaluation questions.
3.3. General aims of the evaluation
The aims of this evaluation are to:
- identify good practices
- identify gaps and areas of unmet needs both in activities and from a cross-cutting perspective
- provide “lessons learnt” for future projects in order to improve the work of Help and its organisations
- formulate recommendations for Help.
Evaluation approach and methodology
The evaluation process is foreseen to be carried out in 3 phases: a Desk Phase, a Field Phase, and finally a Synthesis Phase.
The process will start by kick-off meeting participated by Help staff where objectives of the evaluation are discussed again and will follow with the review of the provided relevant project documentation by the evaluator and briefing session / sessions through both, personally and Skype with the project staff. As preparing the Evaluation Design, the evaluator should:
- Propose a set of evaluation questions justifying their relevance.
- Present an indicative methodology for the overall assessment of the project
- Describe the approach for answering evaluation questions
- Analyze the relevant available documents
- Propose adjusted work plan of the evaluation.
- Confirm the final schedule for the evaluation exercise
The Field Phase should start after approval of the Evaluation Design. During the field phase, the evaluator shall ensure adequate contact and consultation with the project partners and different stakeholders; using the most reliable and appropriate sources of information. At the end of the field phase, the evaluator shall summarize its work, consider the reliability and coverage of information collection, and present preliminary findings.
This Synthesis Phase is mainly devoted to the preparation of the final report. The evaluator will present the findings, conclusions and recommendations in a single document in accordance with the agreed structure. The evaluator will make sure that:
- The assessments are objective and balanced, statements accurate and verifiable, and recommendations realistic.
- When drafting the report, they will acknowledge clearly where changes in the desired direction are known to be already taking place and gather feedback on the findings and on recommendations
The evaluation should combine evaluation tools based on international standards and guidelines like the Code of Conduct of the Red Cross/Red Crescent societies, the adapted ALNAP and OECD/DAC criteria, the Core Humanitarian Standard on Quality and Accountability and the Sphere Minimum Standards in Humanitarian Response and Grand Bargain commitments.
5.Deliverers and deadlines
5.1. Draft evaluation report
Deadline: 20.09.2020
5.2. Final evaluation report including a summary
(max. 10 pages excluding annexes)
The report should include (but is not limited to) the following:
- Executive summary (max. 1-2 pages)
- Evaluation purpose, objectives, and scope
- Methodology (reflection and linking to the TOR and possible constraints leading to deviations from the TOR)
- Findings (related to the objectives of the TOR)
- Conclusions
- Recommendations
- Lessons learnt
- Appendices (including TOR, maps, questionnaires, list of interviewees, and bibliography)
Deadline: 15.10.2020
Language
All documents should be written in English.
The evaluation team will directly report to Help office and will be bound by Help rules of confidentiality. All material collected during the evaluation process will be handed over to Help prior to termination of the contract. The evaluation report and all background documentation will become property of Help and will be published according to Help rules and regulations.
The evaluation team will not be allowed to present any of the analytical results as its own work or to make use of the evaluation results for private publication purposes.
- Expected time-frame
| Activities | Deadlines |
| Kickoff- meeting | 10.08.2020 |
| Evaluation travel |
3 days Montenegro/
|
| Deadline for draft report | 20.09.2020 |
| Final evaluation report | 15.10.2020 |
The evaluator is requested to immediately inform Help if serious problems or delays are encountered. Any significant changes to the evaluation timetable must be approved by Help.
- Budget
Offers should include a proposed budget for the evaluation of the component in Montenegro only , covering all consultancy fees, visa, transport, accommodation and subsistence costs. The budget should present consultancy fees according to the number of expected working days over the entire period.
- Qualification of evaluator
The evaluator should have the following skills and experiences:
- Very good understanding of the OECD/DAC criteria, Core Humanitarian Standard on Quality and Accountability, the Sphere Minimum Standards in Humanitarian Response, and the adapted ALNAP, as well as an appreciation of key challenges and constraints to their application in the relevant context.
- Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and prepare well-written reports in a timely manner.
- Experience in evaluating development program targeting income generation projects, enhancement of economic stability.
- Knowledge of multi-methodological approaches (qualitative and quantitative methods) in evaluation (as evidenced by recent publications).
- A sound knowledge of the context in South-East- Europe.
- Experience in collecting data from vulnerable groups.
- Excellent oral and written communication skills.
- Demonstrated cross-cultural skills.
- Knowledge and prior experience of working in South East Europe.
- Excellent writing and presentation skills in English.
- Considerable knowledge of German, Serbian, Croatian or Bosnian is a plus.
