INCEPT Evaluation Guide March 2016 -...

35
INCEPT: Inner North West Collaborative Evaluation Project INCEPT Inner North West Collaborative Evaluation Project INCEPT Evaluation Guide March 2016

Transcript of INCEPT Evaluation Guide March 2016 -...

Page 1: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

INCEPT: Inner North West Collaborative Evaluation Project

INCEPT Inner North West Collaborative Evaluation Project

INCEPT Evaluation Guide

March 2016

Page 2: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

2

The following Inner North West Primary Care Partnership partners have collaborated on INCEPT

and contributed to this evaluation guide.

City of Melbourne

City of Yarra

cohealth

Inner North West Primary Care Partnership

Merri Community Health Service

Moonee Valley City Council

Moreland City Council

North Richmond Community Health Service

North Western Melbourne Primary Care Network

University of Melbourne

Womens Health in the North

Womens Health West

For enquiries contact: Susan Conaghan IHP Project Coordinator Inner North West Primary Care Partnership [email protected] 03 9389 2249

Inner North West Primary Care Partnership acknowledges the support of the Victorian Government

Project Partners

Page 3: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

3

TABLE OF CONTENTS

Introduction…………………………………………………………………………………………… 4

Logic model…………………………………………………………………………………………… 6

Using this guide……………………………………………………………………………………….. 8

SECTION 1. Project characteristics…………………………………………………………………… 10

SECTION 2. Project impact…………………………………………………………………………..... 14

SECTION 3. Lessons learned………………………………………………………………………....... 20

SECTION 4. Project quality…………………………………………………………………………… 24

List of abbreviations…………………………………………………………………………………. .28

Glossary of terms……………………………………………………………………………………....29

APPENDIX 1. …………………………………………………………………………………………...31

APPENDIX 2. …………………………………………………………………………………………. .32

APPENDIX 3. …………………………………………………………………………………………. .35

Page 4: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

4

Background

INCEPT is a collaborative evaluation initiative of the Inner North West Primary Care Partnership

(INWPCP) Integrated Health Promotion (IHP) Alliance. The aim of the initiative is to strengthen

evaluation practice across the Inner North West catchment through a consistent approach to

evaluation, and to better understand the collective impact of local preventing violence against

women (PVAW) health promotion initiatives.

The University of Melbourne were commissioned to provide expert assistance with the

development of an evaluation framework for INCEPT using a participatory approach. After a series

of workshops and meetings with INCEPT partners and with a designated working group, it was

agreed that the initiative would involve the development of this evaluation guide on PVAW, which

includes shared evaluation indicators and questions.

Purpose

The purpose of this evaluation guide is to assist the collection of data about PVAW projects

occurring across the INW catchment. It is intended to be used in addition to current evaluation

measures that are in place for individual projects. It is recognised that it will not be possible to apply

all evaluation questions included in this evaluation framework to all projects. This guide and the

evaluation process will be piloted in 2016 with a number of partner projects.

The evaluation results will inevitably have limitations due to the diversity in target population

groups, project objectives, timing, numbers and study design. Therefore, rather than evaluating

direct project impacts, it will aim to capture indicators of change arising from a suite of locally

delivered projects all contributing to prevention of violence against women. Specifically, this

evaluation is designed to capture the collective impact of the various initiatives recorded across

Inner North West Melbourne. In addition this partnership will lead the way in developing and

piloting a set of indicators in the field of PVAW.

Introduction

Page 5: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

5

Context

The following documents and websites provide information about PVAW work on a national, state

and local level that give context to INCEPT.

1. Change the Story: A shared framework for the primary prevention of violence against

women and their children in Australia

2. Royal Commission into Family Violence

3. Victorian Public Health and Wellbeing Plan 2015-2019

4. Evaluating Victorian Project for the Primary Prevention of Violence Against Women: A

concise guide

5. Building a Respectful Community – Preventing Violence Against Women – A Strategy for

the Northern Metropolitan Region of Melbourne 2011-16

6. Preventing Violence Together: The Western Region Action Plan to Prevent Violence Against

Women

Page 6: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

6

The program logic model is a theoretical depiction of how the activities to prevent violence against

women in Inner North West Melbourne are anticipated to lead to the desired outcomes. These

outcomes are classified into individual, community, organisational and societal levels. It should be

noted that the logic aims to capture the range of activities occurring across the catchment but may

not include all.

This logic was developed by the INCEPT partners and was informed by the VicHealth Prevention of

Violence Against Women: A Framework for Action1 and other existing logics including those

developed by other PCPs in the Metropolitan Region2. The logic is a working document that will be

tested through evaluation and revised as evidence is produced to show otherwise.

The evaluation indicators and questions aim to collect data on a number of the anticipated

outcomes listed below in the logic diagram. These are depicted in the bolded boxes.

1 Vichealth Prevention Of Violence Against Women: A Framework For Action (2009), VicHealth Http://Assets.Justice.Vic.Gov.Au/Ccp/Resources/84bff852-E0d3-4731-867b-5d3ba166399a/Vichealthframeworkforrvawc.Pdf 2 Women’s Health in the North (2013) Program Logic for Building a Respectful Community - Preventing Violence Against Women. WHIN, Melbourne Women’s Health West (2010). Prevention of Violence Together, Western Region Action Plan to Prevent Violence Against Women (2010) WHW, Melbourne Women’s Health East (2014). Together for Equality and Respect Action Plan 2013-2017. WHE Melbourne.

Logic Model

Page 7: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,
Page 8: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

Guide components

Within this guide there are four sections. These sections are to be completed by the project staff

involved in the design and delivery of the local project.

Section 1 collects information on the project characteristics, reach, settings, action areas and target

groups. Section 2 is focused on the project impact and involves two indicators that require a set of

questions to be asked of participants. Section 3 is mainly qualitative and includes elements such as

project reflections and sustainability. The final section (Section 4) collects information on the

quality of the project.

Pilot timeline

Piloting of the evaluation initiative will occur over 2016 into 2017. Key dates are:

February to December 2016 - Collection of project data

January to March 2017 - Analysis of project data

April to May 2017 – Review and refinement of the evaluation manual

Pilot project selection

The pilot phase intends to evaluate a wide variety of projects. Consequently, the only inclusion

criterion is that the focus of the project be on PVAW. For the pilot it is hoped that each partner will

complete the evaluation template for at least one PVAW project. Any questions about projects for

inclusion can be directed to the INWPCP IHP Coordinator (see page 2).

Evaluation completion

This evaluation guide provides an electronic/paper copy of the four sections but the information is

to be entered through a survey. The link to the data entry site will be supplied to each partner.

This evaluation is not to replace your existing evaluation processes but to be done in addition.

Using this guide

Page 9: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

9

Support for INCEPT Data Collection

The INWPCP IHP Coordinator will be available to provide individualised support to agencies around

INCEPT data collection, with potential for additional support from University of Melbourne, when

required. Contact details for the INWPCP IHP Project Coordinator are included on page 2 of this

guide.

Page 10: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

10

For the project being evaluated, please complete the relevant project details below. (One project

per template)

Project details

Q1 Project name

2 Lead organisation

3 Partner organisation/s

4 Number of partners/organisations

5 Government area/region

6 Project start date

7 Project end date

8 Project manager

Name and contact details

SECTION 1: Project characteristics

Page 11: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

11

9 Please provide a brief description of the

project and expected outcomes (up to

100 words)

Page 12: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

12

Project reach, settings and priority areas can be combined across the Partnership to build a

comprehensive picture of all the projects. Appendix 1 is an example of how this can be succinctly

presented at the PCP level.

Project Reach

Reach is measured to create a picture of the number of individuals and settings the projects have

been able to ‘reach’ – or in other words, the number of individuals exposed to the project, and the

number of settings. As an example, a school project with family involvement could reach 10 service

providers (teachers/staff), 450 individuals (students) and 420 families (inclusive of the students),

within 10 educational settings (e.g. 10 schools), across 2 LGA’s.

It is acknowledged the method of delivery used in some projects can make it difficult to accurately

measure reach. For example, social marketing campaigns or wide distribution of health education

resources. In these cases, alternative methods to establish an estimate of reach may need to be

used, for example, number of downloads of a resource. If you would like support on how best to

measure reach, please contact the INWPCP IHP Coordinator (see page 2)

Instructions:

Count all people that have been directly reached by your project within the last year.

This is the actual number of participants that attended or were exposed, not the intended

reach.

If a participant attended a training program that was made up of 3 sessions only count that

participant once.

Please complete the reach and settings of the project with the actual number.

Reach (Number of)

Individuals

Families

Service providers/professionals

Neighborhoods/Local Government Areas

Page 13: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

13

Settings3 (Number of)

Education and care settings for children and young people

Universities, TAFEs and other tertiary education institutions

Workplaces, corporations and employee organisations

Sports, recreation, social and leisure spaces

Arts

Health, family and community services

Faith-based contexts

Media

Popular culture, advertising and entertainment

Public spaces, transport, infrastructure and facilities

Legal, justice and corrections contexts

Please select the priority action areas of the project.

Priority action area ** tick all that apply

Build healthy public policy □

Create supportive environments □

Strengthen community actions □

Develop personal skills □

Reorient health services □

**Appendix 2: The Ottawa Charter for Health Promotion 1986

http://www.who.int/healthpromotion/conferences/previous/ottawa/en/index1.html

Target group (Tick all that apply)

Un

iver

sal

Wo

men

Men

Ch

ildre

n/y

ou

th

Sta

ff

So

cio

-eco

no

mic

dis

adva

nta

ge

Eld

erly

(65+

)

GL

BIT

Q

CA

LD

Ab

ori

gin

al a

nd

To

rres

Str

ait

Isla

nd

er o

rig

in

Ho

mel

essn

ess

Peo

ple

wit

h M

enta

l

illn

ess

Peo

ple

wit

h D

isab

ility

Ref

ug

ees

and

new

ly

arri

ved

mig

ran

ts

□ □ □ □ □ □ □ □ □ □ □ □ □ □

3Our Watch, ANROWS, VicHealth. (2015). Change the story: A shared framework for the primary prevention of violence against women

and their children in Australia. Retrieved from http://www.ourwatch.org.au/getmedia/0aa0109b-6b03-43f2-85fe-a9f5ec92ae4e/Change-

the-story-framework-prevent-violence-women-children-AA-new.pdf.aspx

Page 14: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

14

There are two evaluation indicators that have been developed, each with associated evaluation

questions, as shown in the tables below. The questions for Indicators 1 and 2 are to be asked by the

maximum number of participants possible. These questions should not replace your current

evaluation process but should be used in addition to the evaluation already in place for your project.

Step 1 – Select the indicator questions you will ask your participants

Select all the relevant indicator questions that could be applied in the evaluation of your project. It

is recognised that not all indicator questions will be relevant to each project. It is, however,

important that the number of indicator questions used to evaluate your project is maximised to

provide a robust evidence base for future PVAW work.

It is important that the wording of the indicator questions remains the same. These are not to be

changed. If you would like to simplify the language please contact the INWPCP IHP Project

Coordinator.

If none of the question options is suitable for your project please contact the INWPCP IHP Project

Coordinator.

Step 2 - Asking participants questions (post exposure)

These questions are to be asked after the participant has completed the project. The timeframe for

asking the questions is 0 to 3 months after the participant completed the project. (Aim for over 60%

response rate).

Questions can be incorporated into surveys, interviews or focus groups. An example of a survey is

provided in Appendix 3.

INCEPT data will be sex disaggregated and participants therefore need to be asked their gender

identity. To ask this in a GLBTIQ inclusive manner it is suggested that you pose gender as an open

ended question (e.g. Gender? ______________) rather than having check boxes. Alternatively

follow your own organisational guidelines relating to this matter.

SECTION 2: Project impact

Page 15: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

15

When submitting results into INCEPT we have included only three categories (woman, man, other),

however you will be provided with an opportunity to state more specifically how people who

identified as other than woman or man, identified themselves when entering data into Survey

Monkey.

Step 3 – Informed consent

It is important to provide information to participants on how the data will be used so that informed

consent can be obtained.

To ensure that the data can be used for research and evaluation into the future the following points

are to be included:

I agree that the information I provide can be grouped with other participant information to:

1. Inform project improvements 2. Support the Primary Care Partnerships program 3. Increase promotion of respectful relationships 4. Contribute to research and reports

Step 4 – Complete the following section with the information you collected in Steps 1-2

What was the total number of participants for this project? _______________

What number of participants responded to the indicator questions you selected?

Indicator Set 1 _______________

Indicator Set 2 _______________

Who was asked? For example, ‘primary school children’

Indicator Set 1 _______________

Indicator Set 2 _______________

Page 16: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

16

Indicator set 1

Improved awareness, knowledge and skills towards gender equity

As a result of the [insert name of training/initiative]:

1. I am better able to identify gender equity in my home/workplace/community/school

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

2. I am better able to identify respectful relationships in my home/workplace/community/school

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

3. I have improved skills to maintain and build respectful relationships with my family/colleagues/friends

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

4. I feel more confident in standing up for, asserting or promoting respectful relationships and gender equity

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

5. I have an increased understanding of gender inequity as a driver of violence against women

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

Page 17: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

17

Indicator set 2

Improved organisation, partnership and leadership commitment to addressing VAW through modeling, promotion and facilitation

As a result of the [insert name of training/initiative]:

1. I have a greater awareness of the policies and processes within my organisation / area of work that ensure gender equity and respectful relationships

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

2. I have a greater understanding of the consequences of rigid gender roles and stereotypes in my organisation / area of work

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

3. I have more confidence applying a gender lens to all activities within my organisation/ area of work

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

4. I intend to apply a gender lens to all activities within my organisation / area of work

Woman Man Other

Strongly Agree

Agree

Unsure

Disagree

Strongly Disagree

Page 18: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

18

Additional impact related questions:

Has this project influenced any:

Policies within the organisation have been created or updated to promote gender equity?

Number in last year:

Names of policies:

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

Procedures within the organisation have been created or updated to promote gender

equity?

Number in last year:

Names of procedures:

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

Stories of change

This section is an opportunity to provide project outcomes beyond the evaluation indicators and

questions above. This could be existing qualitative data collected through the project evaluation (as

shown in list below) or by answering the following question with a brief story of change:

‘During the last 12 months, in your opinion, what was the most significant change4 that took place for

participants in the project?’

Please note that this question is just a prompt for you reflect on what you think is the most

significant change has been. You do not need to go through the full Most Significant Change

process.

4Informed by from Davies, R. & Dart J. The Most Significant Change technique MSC: A guide to its use. 2005

http://www.mande.co.uk/docs/MSCGuide.pdf

Page 19: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

19

You can provide materials to support/illustrate your story of change. This can be submitted to

INWPCP IHP Project Coordinator (see page 2). Examples of materials that could be provided:

Qualitative report from the project evaluation

A case study based on experiences of individual participant

Photos

Poetry

Other creative documenting of outcomes

Please remove any information that may lead to the identification of any participants.

This stories of change section is optional.

Page 20: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

20

Please consider the barriers and facilitators that may inform planning of similar projects in the

future. Reflecting on the logic model may assist in identifying both barriers and facilitators.

What barriers occurred and how were they/could they be addressed?

Word limit – 300

What factors, if any, facilitated the success of the project?

SECTION 3: Lessons learnt

Page 21: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

21

Logic Model

Do you have any reflections on the logic model (page 6) and how it aligns with INCEPT and your

project? Do you propose any changes to the logic model?

In what way could you run this project differently next time?

Page 22: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

22

Were there unanticipated outcomes (positive or negative) arising from the project?

Would you repeat this project if funds were available? Why?

Page 23: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

23

Sustainability

What are the expectations for the continuation, replication or expansion of the project and

strategies including timelines? Are there elements of the project that are being continued?

Are you happy to share details on project plans and evaluations?

Yes/No

Page 24: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

24

Please take some time to read through and complete this section.

These questions are intended as a reflective process as well as a method of collecting information to

measure the quality of the INCEPT partnership projects. Answers to this section will be de-

identified and collated before being shared in any way.

Results will inform where there are opportunities for further training for INCEPT partners, where

there are gaps in the projects best practice and where limited resources may be impacting on

project quality.

Please complete this section as accurately as possible so the Partnership can form an understanding

of where further support may be needed.

Principles for best practice

Theory of change or principles Description: The planning process considers/considered and applies/applied an appropriate theoretical understanding of effective approaches for achieving change in organisations and individuals.

To help determine this, the following question is to be answered:

Is there a theory for change that underpins your project, for example the ecological model or feminism theory?

Yes/No/Don’t know

If yes, please record the theory or principle(s) that your project prescribes to:

_____________________________________________________________________________

How is the change expected to occur: through education, leadership, facilitation, a mix or other?

________________________________________________________________________________

SECTION 4: Project quality

Page 25: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

25

Evidence / innovation Description: The planning process considers and applies best available evidence on the effectiveness of interventions; or explicitly seeks to test strategic, innovative approaches. To help determine this, the following questions are to be answered: Are you explicitly testing an innovative approach based upon best available or promising evidence, or theory, in order to contribute to evidence building?

Yes/No/Don’t know Are you implementing a previously tested intervention that is supported by available evidence on its effectiveness?

Yes/No/Don’t know

Project plan Description: There is a clear program logic for your project, depicting coherent connections between goals, objectives and strategies. The goals are appropriate and objectives are achievable, taking into account the likely size of effect and project reach. To help determine this, the following questions are to be answered: Are the objectives SMART and focused on changes logically connected to violence against women?

Yes/No/Don’t know Is there a mix of strategies all addressing the key objectives?

Yes/No/Don’t know

Evaluation planning Description: There is an evaluation plan, which identifies a chain of outcomes (process, outcome and impact evaluation) and assesses the extent to which all or some of the objectives are achieved. To help determine this, the following questions are to be answered: Is there a documented evaluation plan?

Yes/No/Don’t know If you currently use an evaluation tool that you would recommend to other partners please name and briefly describe it. _____________________________________________________________________________

_____________________________________________________________________________

Page 26: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

26

Structures for project management, governance and organisational relationships Description: There are clear governance structures and lines of accountability. To help determine this, the following questions are to be answered: Are there structures for efficient management decision-making with clear lines of accountability?

Yes/No/Don’t know

Are there structures to provide higher level strategic advice and links to other important opportunities?

Yes/No/Don’t know

Are all parties clear about their roles, responsibilities and relationships e.g. through memorandum of understanding (MOU), contracts, participation in joint groups etc.?

Yes/No/Don’t know

Are there systems for accountability regarding quality and timeframes?

Yes/No/Don’t know

Implementation partnerships Description: Implementation builds synergy between partner groups and contributes to community/agency/target group capacity. The partnership arrangements include clear structures and communication strategies. To help determine this, the following questions are to be answered: Are there processes e.g. protocols, reporting systems, MOU, contracts etc. to coordinate actions across partner groups so that the roles are clear to all partners?

Yes/No/Don’t know What are the arrangements for regular communication with partner groups and other community stakeholders? ______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

Page 27: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

27

Sustainability and community capacity Description: The project identifies existing structures and processes that can enable it to be maintained at agency and/or community levels. To help determine this, the following questions are to be answered: Have you given priority to implementing policy and environmental change strategies that can be sustainable?

Yes/No/Don’t know Are there specific strategies to build skills and knowledge and mobilise future resources for the project?

Yes/No/Don’t know Are there ways of securing additional resources to support sustained changes?

Yes/No/Don’t know

Adaptation and responsiveness Description: Implementation adapts to pursue additional opportunities as they arise, such as broadening the reach or linking with other partners. Responsiveness supports efforts to build capacity and promote sustainability. To help determine this, the following questions are to be answered: Does the project identify new opportunities and emerging evidence, and adapt and respond to them?

Yes/No/Don’t know Does the project actively support the development of knowledge, skills and sustainable systems in partner agencies and groups?

Yes/No/Don’t know

(Modified - King L, Gill T. Best Practice Principles for Community-based Obesity Prevention. 2009;

CO-OPS Secretariat, Deakin University, Geelong.)

Page 28: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

28

CALD: Culturally and Linguistically Diverse

GLBTIQ: Gay Lesbian Bi Transgender Intersex Queer

IHP: Integrated Health Promotion

INWPCP: Inner North West Primary Care Partnership

INCEPT: Inner North West Collective Evaluation Project

LGA: Local Government Area

PVAW: Preventing Violence Against Women

List of abbreviations

Page 29: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

29

Evaluation indicator5: Indicators are measure of progress or lack of progress used to assess

progress towards meeting stated objectives.

Gender6 – the socially learnt roles, behaviours, activities and attributes that any given society

considers appropriate for men and women; gender defines masculinity and femininity.

Gender Equity7: Fairness in women’s and men’s access to socio-economic resources.

Gender Lens6: Using a gender lens means approaching or examining an issue, paying particular

attention to the potentially different ways that men and women are or might be impacted.

Gender Roles6: These are the roles assigned to men and women respectively according to cultural

norms and traditions.

Gender Stereotype6: arise from presumptions about the roles, abilities and attributes of men and

women.

Impact8 (Australian definition): the long-term development impact that the activity contributes to

at a national or sectoral level

INWPCP Integrated Health Promotion Alliance: An alliance of agencies working to

collaboratively promote health and wellbeing in the Inner North West Melbourne catchment,

facilitated by Inner North West Primary Care Partnership.

Inner North West catchment: the local government areas of Melbourne, Moonee Valley, Moreland

and Yarra

5 AusAid. (2005). 3.3 The Logical Framework Approach. AusGuideline. Activity design: Commonwealth of Australia. Retrieved from http://www.sswm.info/sites/default/files/reference_attachments/AUSAID%20 6 Our Watch, ANROWS, VicHealth. (2015). Change the story: A shared framework for the primary prevention of violence against women and their children in Australia. Retrieved from http://www.ourwatch.org.au/getmedia/0aa0109b-6b03-43f2-85fe-a9f5ec92ae4e/Change-the-story-framework-prevent-violence-women-children-AA-new.pdf.aspx

7 http://www.rodicovstvo.sk/buletin/gender_def_Equal_TCA_June.htm

8 Funnell, S.C. and Rogers, P.J. (2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco. Jossey-Bass.

Glossary of terms

Page 30: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

30

Program Logic Model7: a representation of the theory of how an intervention is understood to

contribute to its intended or observed outcomes, usually in the form of a diagram.

Theory of change7: The central processes or drivers by which change comes about for individuals,

groups, or communities. It can be derived from a research-based theory of change or drawn from

other sources.

Outcome7 (Australian definition): the medium-term results in terms of benefits to the target group

that the activity aims to achieve.

Violence against women5 – any act of gender based violence that causes or could cause physical,

sexual or psychological harm or suffering to women, including threats of harm or coercion, in public

or in private life. This definition encompasses all forms of violence that women experience

(including physical, sexual, emotional, cultural/spiritual, financial, and others) that are gender

based.

Page 31: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

31

APPENDIX 1.

Page 32: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

32

APPENDIX 2.

The Ottawa Charter for Health promotion

http://www.who.int/healthpromotion/conferences/previous/ottawa/en/index1.html

First International Conference on Health Promotion, Ottawa, 21 November 1986

Health Promotion Action Means:

Build Healthy Public Policy

Health promotion goes beyond health care. It puts health on the agenda of policy makers in all

sectors and at all levels, directing them to be aware of the health consequences of their decisions

and to accept their responsibilities for health.

Health promotion policy combines diverse but complementary approaches including legislation,

fiscal measures, taxation and organizational change. It is coordinated action that leads to health,

income and social policies that foster greater equity. Joint action contributes to ensuring safer and

healthier goods and services, healthier public services, and cleaner, more enjoyable environments.

Health promotion policy requires the identification of obstacles to the adoption of healthy public

policies in non-health sectors, and ways of removing them. The aim must be to make the healthier

choice the easier choice for policy makers as well.

Create Supportive Environments

Our societies are complex and interrelated. Health cannot be separated from other goals. The

inextricable links between people and their environment constitutes the basis for a socioecological

approach to health. The overall guiding principle for the world, nations, regions and communities

alike, is the need to encourage reciprocal maintenance - to take care of each other, our

communities and our natural environment. The conservation of natural resources throughout the

world should be emphasized as a global responsibility.

Changing patterns of life, work and leisure have a significant impact on health. Work and leisure

should be a source of health for people. The way society organizes work should help create a

Page 33: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

33

healthy society. Health promotion generates living and working conditions that are safe,

stimulating, satisfying and enjoyable.

Systematic assessment of the health impact of a rapidly changing environment - particularly in

areas of technology, work, energy production and urbanization - is essential and must be followed

by action to ensure positive benefit to the health of the public. The protection of the natural and

built environments and the conservation of natural resources must be addressed in any health

promotion strategy.

Strengthen Community Actions

Health promotion works through concrete and effective community action in setting priorities,

making decisions, planning strategies and implementing them to achieve better health. At the

heart of this process is the empowerment of communities - their ownership and control of their own

endeavours and destinies.

Community development draws on existing human and material resources in the community to

enhance self-help and social support, and to develop flexible systems for strengthening public

participation in and direction of health matters. This requires full and continuous access to

information, learning opportunities for health, as well as funding support.

Develop Personal Skills

Health promotion supports personal and social development through providing information,

education for health, and enhancing life skills. By so doing, it increases the options available to

people to exercise more control over their own health and over their environments, and to make

choices conducive to health.

Enabling people to learn, throughout life, to prepare themselves for all of its stages and to cope

with chronic illness and injuries is essential. This has to be facilitated in school, home, work and

community settings. Action is required through educational, professional, commercial and

voluntary bodies, and within the institutions themselves.

Reorient Health Services

The responsibility for health promotion in health services is shared among individuals, community

groups, health professionals, health service institutions and governments.

Page 34: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

34

They must work together towards a health care system which contributes to the pursuit of health.

The role of the health sector must move increasingly in a health promotion direction, beyond its

responsibility for providing clinical and curative services. Health services need to embrace an

expanded mandate which is sensitive and respects cultural needs. This mandate should support the

needs of individuals and communities for a healthier life, and open channels between the health

sector and broader social, political, economic and physical environmental components.

Reorienting health services also requires stronger attention to health research as well as changes in

professional education and training. This must lead to a change of attitude and organization of

health services which refocuses on the total needs of the individual as a whole person.

Page 35: INCEPT Evaluation Guide March 2016 - Inwpcpinwpcp.org.au/.../INCEPT-Evaluation-Guide-Mar-2016.pdf · Section 1 collects information on the project characteristics, reach, settings,

35

APPENDIX 3.

Example for Data Collection

Post Participant Survey (paper/survey monkey/telephone etc)

Your feedback is important to us as it will help to improve the programs we deliver.

All feedback will remain confidential.

Gender? _______________________

As a result of completing the training with XX Please circle

I am better able to identify gender equity in

my workplace

Strongly

Agree

Agree Unsure Disagree Strongly

Disagree

I have improved skills to maintain and build

respectful relationships with my colleagues

Strongly

Agree

Agree Unsure Disagree Strongly

Disagree

I feel more confident in standing up for,

asserting or promoting respectful

relationships and gender equity

Strongly

Agree

Agree Unsure Disagree Strongly

Disagree

Any comments or suggestions

Thank you for your time