End Term Evaluation Terms of Reference

2

Table of Contents Background .........................................................................................................................................3

Introduction to PITCH ......................................................................................................................3 Background to the End of Term Evaluation ..................................................................................3

Evaluation Audience……………………………………………………………………………………….4

End term evaluation ............................................................................................................................4 Objectives, themes, and questions ................................................................................................4

Methodology….……………………………………………………………………………………………..6 Roles and Responsibilities……………………………………………………………………………….8

Deliverables………………………………………………………………………………………………….8 Timeframe……………………………………………………………………………………………………9

Request for Expressions of Interest................................................................................................10 Person Specification……………………………………………………….……………………………….10 Application Process and Deadline……….………………………………………………………………12 Contact Person and Details…………………………………….………………………………………….13

Annexes .............................................................................................................................................13

3

BACKGROUND Introduction to PITCH

PITCH is a five-year global advocacy programme designed to tackle the global AIDS epidemic. The programme is implemented in nine countries (Indonesia, Kenya, Mozambique, Myanmar, Nigeria, Uganda, Ukraine, Vietnam and Zimbabwe), two regions (Southern Africa and Eastern Europe & Central Asia) as well as at the global policy level.

PITCH is a strategic partnership between Aidsfonds, Frontline AIDS and the Dutch Ministry of Foreign Affairs (MoFA). As programme partners, civil society organisations (CSO) are contracted to advocate on behalf of and with the LGBT community, Sex Workers, People Who Use Drugs, and Adolescent Girls and Young Women, communities that are most affected by HIV, in order to transform the international HIV response. We have sought to achieve this by building and coordinating our partnership networks; enhancing the flow and use of evidence and intelligence; engaging advocacy targets; and strengthening civil society’s HIV advocacy capacity. For five years (2016-20), these partners have jointly worked towards ending the AIDS epidemic in the regions and countries most affected by HIV through an inclusive, in-country, in-depth approach to international lobbying and advocacy. The goals of PITCH are:

Equal access to HIV-related services Sexual and reproductive health and rights for those most affected Equal and full rights for key populations Strong civil society organisations are successful HIV advocates.

Further information regarding PITCH is provided within the annexes to this Terms of Reference.

Background to the End of Term Evaluation Implementation of PITCH will be complete by the end of 2020. As the programme’s funder, MoFA requires an independent, external End Term Evaluation of all programmes funded through the Dialogue and Dissent Strategic Framework to be carried out. The final End Term Evaluation report will be due for submission to MoFA by 1st December 2020. Accordingly, the PITCH Strategic Partnership is now seeking an independent consultant or consultancy to conduct this independent, external evaluation of the programme. The PITCH Mid Term Review was carried out during 2018, with the final report produced in November 2018, while a management response was prepared in February 2019. The reflections and lessons from the MTR process were well documented, as were the programme’s identification of next steps and formal response to the MTR recommendations. This experience has informed the planning of the End Term Evaluation.

4

Evaluation Audience The primary audience of the final report of the End Term Evaluation will be MoFA. The final report will need to provide clear analysis and recommendations for the benefit of Aidsfonds and Frontline AIDS as strategic partners in PITCH, as well as for the programme’s implementing partners at the country, regional, and global policy levels. It is anticipated that other organisations implementing advocacy programmes in the HIV/AIDS sector will also benefit from the learning opportunities presented by the evaluation, helping to inform the design and implementation of future advocacy programmes internationally.

END TERM EVALUATION

Objectives, themes, and questions The purpose of this End Term Evaluation is to facilitate a critical analysis of the programme’s contribution to evidence-based changes, in relation to the PITCH programmatic goals and advocacy strategies. The lead evaluation consultant will need to embrace the concept of gender as a cross-cutting consideration when developing interview questions and data collection tools, shortlisting participants, assessing the achievements of PITCH, and developing the evaluation’s recommendations. Moreover, the PITCH Strategic Partnership seeks to identify the strengths as well as opportunities for improvement in the programme’s approach to collaboration both internally and externally, as well as the sustainability of the programme’s endeavours more generally. The scope of the End Term Evaluation will be to review the programme period from its inception in January 2016, through to July 2020, at which point the data analysis and interpretation phase of the evaluation will begin, followed by the End Term Evaluation report write up. The PITCH Strategic Partnership has identified two objectives for the end term evaluation, as well as a series of evaluation questions designed to meet these objectives:

Objective Evaluation question 1. To assess the outcomes and

impact of PITCH, as well as their sustainability, in the context of the programme’s Theory of Change

1.1 Which significant advocacy outcomes has PITCH made a measurable contribution to? To what extent does evidence exist to support these claims of contribution?

1.2 How has PITCH contributed to any positive

or negative unexpected outcomes? What lessons have been learned, and how have these unexpected outcomes influenced partners’ advocacy planning?

5

1.3 Which PITCH advocacy strategies have been most effective in allowing PITCH partners to achieve their advocacy asks1? What lessons can be learned from this?

1.4 Reflecting on structural and legislative

changes, how sustainable are the achievements of PITCH beyond the programme’s lifetime, and in the absence of significant external funding?

1.5 To what extent has PITCH measurably and

sustainably strengthened the advocacy capacity of PITCH partners, including the capacity to capture evidence to support their advocacy?

1.6 To what extent has the capacity of PITCH

partners to apply a gender transformative approach to their work been strengthened? How has this been achieved? What impact has this had on the outcome of PITCH country level advocacy activities?

1.7 What lessons can be learned about how

gender informs advocacy carried out on behalf of Key Populations and Adolescent Girls and Young Women?

1.8 From the perspective of different PITCH stakeholders, including implementing partners, which programmatic strategies and approaches have partly or entirely failed? What lessons can we learn from this?

2. To understand the extent to which internal and external PITCH stakeholders have benefited from collaborating with each other

2.1. To what extent have country partners and global policy partners benefited from/connected with each other’s advocacy activities?

2.2 To what extent have country partners and

regional programme partners benefited from/connected with each other’s advocacy activities?

2.3 To what extent have regional programme and global policy partners benefited

1 An Advocacy Ask is a statement developed by a PITCH partner that identifies the specific change they seek to contribute to through their planned advocacy work

6

from/connected with each other’s advocacy activities?

2.4 To what extent have PITCH partners formed or joined coalitions with other civil society organisations that have helped to advance their advocacy, raise the profile of the experiences of Key Populations and Adolescent Girls and Young Women, and to set the advocacy agenda?

2.5 To what extent can examples of effective collaboration at the country level be demonstrated between PITCH and other MoFA funded programmes working to address HIV/AIDS?

2.6 To what extent have working relationships

between PITCH, and the Dutch embassies and Permanent Missions in the PITCH countries, contributed to advocacy outcomes through strategic collaboration?

As part of the preparation and review of the evaluation inception report and following discussions with the PITCH M&E Lead, it is anticipated that these evaluation questions will be adjusted and finalized by the lead consultant.

Methodology

Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) will be carried out by the evaluation consultants with relevant staff at MoFA, Aidsfonds, and Frontline AIDS as Strategic Partners, as well as with Country Focal Points and a sample of partner staff. External stakeholders will also be invited to participate in these evaluation exercises. These will include representatives from communities, and networks of communities, most affected by HIV (the LGBT community, Sex Workers, People Who Use Drugs, and Adolescent Girls and Young Women), relevant staff from Dutch permanent representations and embassies in relevant PITCH countries, international organisations, academia, and country level CSOs. It is anticipated that KIIs and FGDs will be used when answering the evaluation questions that correspond with objective 2, in addition to questions related to gender, sustainability, and perceived failures;

A Desk-based literature review and in-depth comparative analysis of documentation and data: the lead consultant will be asked to carry out an initial assessment of existing case studies, PITCH partners’ indicator reports, advocacy logs,

7

and country-level mid-year change reports. It is anticipated that this process will enable the lead consultant to answer questions 1.3, 1.5, 1.6, 1.7, and 1.8. Based on this desk review, the lead evaluator will choose which examples of outcomes and case studies will be selected for a more in-depth analysis, which will involve the application of the Stories of Change approach;

Two-day Reflection meetings will be held in each country, as well as at the regional and global policy levels. The Outcome Harvesting methodology will be applied during these reflection meetings, to better understand and document evidence of PITCH partners’ contribution to the most significant, observable changes identified. PITCH partner representatives will participate in these meetings. It is anticipated that each of these twelve meetings (nine country meetings, two regional meetings, and one global policy meeting) will be co-facilitated by a combination of the lead and local evaluation consultants, with support from PITCH country, regional, and global focal points. The evaluation consultants will then be expected to validate a sample of the harvested outcomes, and to take these forwards into the evaluation’s substantiation and data analysis and interpretation phase. Responsibility for developing relevant guidance and for organizing these meetings will sit with a combination of the consultants, the PITCH M&E Working Group, and the relevant country, regional, and global focal points. It is anticipated that these reflection meetings, and the broader Outcome Harvesting process, will enable the evaluation consultants to respond to evaluation questions 1.1, 1.2, and 1.4.

When designing the evaluation’s data collection tools and analyzing and interpreting the data collected during the evaluation, the consultants will be encouraged to reflect on the BOND principles for assessing the quality of evidence2. From this perspective, good quality evidence means: 1) Voice and Inclusion: the perspectives of Key Populations and Adolescent Girls and Young

Women are included in the evidence, and a clear picture is provided of who is affected and how, with attention paid in particular to those groups most marginalized due to their vulnerability, HIV status, and criminalization;

2) Appropriateness: the evidence is generated through methods that are justifiable given the nature of the purpose of the enquiry;

3) Triangulation: the evidence has been generated using a mix of methods, data sources,

and perspectives; 4) Contribution: the evidence explores how change happens, the contribution of the

intervention and factors outside the intervention in explaining change; and 5) Transparency: the evidence discloses the details of the data sources and methods used,

the results achieved, and any limitations in the data or conclusions.

2 https://www.bond.org.uk/ngo-support/evidence-principles-download

8

Roles and Responsibilities This evaluation will be directly managed by a lead consultant who will assume overall responsibility for the deliverables. It is anticipated that the lead consultant will recruit a small number of local consultants based in some of the PITCH countries as part of the evaluation process. The lead consultant will be expected to liaise with, delegate and manage the workload of these supporting consultants. Casey Davison O’Brien, M&E Lead for PITCH, will act as the first point of contact for the lead consultant and will be responsible for managing the evaluation. The PITCH M&E Lead will support all stages of the evaluation process, and will work with the PITCH Programme Team to: Review and refine the evaluation’s objectives, tools and methodologies in consultation with

the lead consultant Provide feedback on draft inputs presented by the lead consultant Sign off final deliverables Ensure a management response to the evaluation is written and recommended actions are

assigned to named individuals to implement. The lead consultant will be expected to carry out the desk-based literature review as part of the first phase of the evaluation. With support from the team of local consultants, the lead consultant will also be expected to carry out the Key Informant Interviews and Focus Group Discussions. The lead consultant will then lead on the design of the End Term Evaluation reflection meetings, outlining in their inception report how they envisage the Outcome Harvesting methodology will be applied in preparation for, during, and after these meetings. The lead consultant will be expected to identify and work with a team of local consultants to support the facilitation of these meetings. Relevant PITCH staff will take responsibility for supporting partners and for scheduling and coordinating the country, regional and global level reflection meetings. The M&E Working Group will provide all secondary data and literature required by the lead consultant to fulfil their Terms of Reference.

Deliverables The lead consultant will be expected to produce the following deliverables:

Deliverable Notes A draft and final inception report at the beginning of the consultancy

This inception report should represent an expanded, detailed version of the three-page proposal submitted as part of the expression of interest, including the following:

1. The evaluation design, which demonstrates how the evaluation methods will be applied in order to meet the evaluation objectives, including the data collection and sampling approach

9

2. A detailed work plan and schedule, as well as an updated, detailed budget. For consultancies able to provide logistical support when arranging the evaluation reflection meetings, the time and cost considerations should be factored in here

3. A list of participants that the lead consultant proposes to invite to engage in the evaluation process

4. A set of interview questions and guiding notes

that the lead consultant proposes to use

5. A detailed process for analyzing, interpreting, and substantiating the evaluation data

Two drafts and a final evaluation report at the end of the consultancy

The final evaluation report should be a maximum number of 70 pages, excluding the Table of Contents and annexes, including data visualisation tools such as infographics.

Two drafts and a final version of an accessible Executive Summary of the final version of the evaluation report

This should provide a succinct summary of the findings of the evaluation and its recommendations. This should be readable as a standalone document to be shared with partners and key stakeholders.

Two drafts and a final version of a PowerPoint visual presentation of the final evaluation findings (no more than 20 slides)

This should be presented by the lead consultant to the PITCH Strategic Team either in Brighton or Amsterdam in September 2020

The responsibility for the contents of these deliverables lies with the lead evaluation consultant, who will need to prepare all documents in English.

Timeframe This is a provisional timetable to be agreed with the selected lead consultant. The assignment is estimated to take between 50 - 70 days in total. The deadline for the submission of Expressions of Interest is Thursday 5th December, 12pm GMT.

Activity Time period 2019 Applicants shortlisted First week of December Applicants interviewed 12th and 13th December Lead consultant appointed 17th December 2020 Lead consultant carries out first part of the evaluation’s desk-based literature review

January

Lead consultant to develop and submit first draft of their inception report

January

10

PITCH M&E Working Group, End Term Evaluation Reference Group, and PITCH Programme Team to carry out a review of the draft inception report

February

Final version of inception report approved by PITCH Programme Team

February

Key Informant Interviews and Focus Group Discussions with internal and external stakeholders (combination of in-person and Skype)

April/May

Country, Regional, and Global reflection meetings

June/July

Lead consultant will use the Stories of Change approach to carry out an in-depth analysis of a sample of case studies and outcomes

July/August

First draft of End Evaluation Report to be produced for review by PITCH M&E Working Group, End Term Evaluation Reference Group, and PITCH Programme Team

September

Presentation and discussion of findings by lead consultant to the PITCH Strategic Team

September

Incorporation of feedback from this presentation into the second draft of the evaluation report

October

Final version of evaluation report presented to PITCH M&E Working Group by lead consultant

October

Evaluation report design and editing, in preparation for publishing and submission

November

Submission of PITCH End Term Evaluation report to MoFA by PITCH Programme Manager

1st December

REQUEST FOR EXPRESSIONS OF INTEREST Person Specification Aidsfonds and Frontline AIDS are seeking Expressions of Interest from experienced evaluation consultants. We are open to hearing from either independent consultants or from consultants affiliated with organizational consultancies. We would be particularly interested in receiving

11

applications from those with established working relationships with local consultants in the same countries and regions where PITCH is operational. The successful applicants will meet the following criteria: Essential:

Substantial experience in conducting reviews and evaluations of human rights and HIV/AIDS programmes

Demonstrated knowledge and understanding of advocacy for human rights programming

in the context of HIV/AIDS, with particular reference to the experiences of LGBTI people, Sex Workers, People who use drugs, and/or Adolescent Girls and Young Women

Knowledge of programming with a Gender Transformative Approach

Experience in undertaking assessments using quantitative and qualitative

methodologies in the context of advocacy programmes

Experience of using the Outcome Harvesting and Stories of Change approaches in an advocacy context

Ability to systematically analyse and present complex data and information, for example

using infographics

Proven ability to write high quality technical reports in English

Fluency in written and spoken English

Strong written and spoken communication skills

Ability to facilitate Focus Group Discussions and meetings, and to carry out Key Informant Interviews

Ability and commitment to deliver the expected results within an agreed period of time

Excellent organisational and administrative skills

Ability to work in complex environments

Desirable:

Experience of living as an LGBTI community member, Sex Worker, Person Who Uses Drugs, or as an Adolescent Girl or Young Woman living with/vulnerable to HIV

Experience of working with key populations, especially LGBTI, Sex Worker, People who use drugs, and Adolescent Girls and Young Women, as part of evaluation and learning exercises

Knowledge of the legal, policy, political, and social environments for these same key

populations in the countries where PITCH is present, with particular reference to these communities’ experiences of stigma, discrimination, criminalization, and violence

12

Application Process and Deadline If you would like to submit an Expression of Interest, please submit the following:

Item Notes A short letter outlining your qualifications against the essential and desirable criteria

Please explain your motivation to apply for this consultancy, as well as your availability in 2020, and how you feel you are qualified to take on the role of lead consultant in line with the essential and desirable person specification criteria.

A maximum three-page proposed methodology and approach for the consultancy

Applicants should present a clear explanation as to how they would deliver this evaluation, and how they would work with, recruit, and organize a team of local consultants (i.e. the size, membership, and experience of these local consultants). Applicants should explain here in which PITCH countries and regions they have professional contacts among local evaluation consultants, and what type of support they envision these local consultants providing as part of this evaluation. Applicants should also describe how they propose to use the preferred methodology highlighted in this Terms of Reference, and what role sampling would play in their approach, when working to respond to and answer the evaluation objectives and questions.

A detailed budget, and an approximate timeline, clearly highlighting the number of days and daily fee, as well as itemizing other costs (including travel) necessary in order to fulfil this consultancy

When developing the timeline please consider the time needed for the PITCH M&E Working Group, End Term Evaluation Reference Group, PITCH Programme Team, PITCH Board and, separately, the PITCH Strategic Team to review the drafts of the evaluation report in September and October 2020 (see annex for further details). In the case of expressions of interest from organizational consultancies, applicants are encouraged to consider including support costs for managing logistical arrangements related to coordinating and administering the country, regional and global policy level reflection meetings. Independent consultants will not be penalized on this basis as part of the application process. If an independent consultant is offered this role of lead consultant, then the PITCH Strategic Partnership will manage and coordinate these logistical arrangements.

13

Applicants are also requested to include an indication of the costs that will be incurred through paying the fees of a small number of local consultants during the course of the evaluation. Applicants should note that the PITCH programme has a separate budget of €49,000 to cover the costs of the reflection meetings in nine countries, two regions, and at the global level.

A copy of the applicant’s CV

Two examples of previous end term evaluation reports that the lead consultant has led on (or links to report available online)

The names and contact details of two references per consultant

Contact person and details Applicants are asked to submit their Expression of Interest to the following people, with “PITCH End Term Evaluation Consultancy” as the subject title. Casey Davison O’Brien Senior Advisor: M&E (PITCH) cdavisonobrien@frontlineaids.org Maartje van der Meulen M&E Policy Officer maartjevandermeulen@aidsfonds.nl The deadline for the submission of Expressions of Interest is Thursday 5th December, 12pm GMT. Shortlisted candidates will then be invited to participate in an interview on 12th and 13th December.

ANNEXES Annex I: PITCH Global Theory of Change Annex II: PITCH Global Results Framework Annex III: 2018 PITCH Annual Report Annex IV: PITCH Programme Structure and Description Annex V: Description of Evaluation Methodologies