Evaluation Workshop Self evaluation – some workable ideas and approaches.
-
Upload
brianna-maxwell -
Category
Documents
-
view
220 -
download
0
Transcript of Evaluation Workshop Self evaluation – some workable ideas and approaches.
![Page 1: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/1.jpg)
Evaluation Workshop
Self evaluation – some workable ideas and approaches
![Page 2: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/2.jpg)
Why self-evaluation?
• Reflexive practice – knowing what has been successful or unsuccessful and why – Evidence for beneficiaries – Evidence for commissioners or partners
• Practitioner-led research - encouraged by the Children’s Workforce Development Council
• Strengthening the wider evaluation arrangements for the 0-7 Pilot Partnership
![Page 3: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/3.jpg)
Self evaluation: the stages involved
Getting Started
Doing your evaluation
Interpreting your findings
Reporting and sharing your findings
What do you want to evaluate or find out about?
What do you want to measure?Who will do the evaluation?
Who needs to be consulted?What methods will you use?What support is available?
How will you analyse your findings (surveys and
interviews)
How to write your reportSharing your findings with
others
![Page 4: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/4.jpg)
Getting started
• The Action Learning Sets: – Aims – Objectives – Milestones – Outcomes
• Leap of faith from testing the activities to proving that certain outcomes have been achieved
• Theory of Change model breaks this down…
![Page 5: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/5.jpg)
1. Start here…
The problem or issue to be addressed
7. Impacts
The ‘Big’ / wider effects that are brought about
2. Theory of Change
The idea or approach for tackling the problem
3. Inputs
Resources;time or money to be used
4. Activities
What will actually be done, and how
5. Outputs
The end result of the activities
6. Outcomes
What will be different for the people or service that you are trying to improve
Theory of change…
![Page 6: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/6.jpg)
What to measure?
Hard outcome: a clearly defined (quantifiable) change• ‘The family learning course improved my literacy and
numeracy skills’
Soft outcome: a harder to measure (qualitative) change • ‘The family learning course made me more confident in
speaking about my experiences with other parents’
Indicator: a sign that change has occurred over time • Numbers of new enrolments• Percentage (%) of parents completing OCN Level 1
![Page 7: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/7.jpg)
Showing that change has happened
Baseline
Time
Chosen Indicator
e.g. % of parents reading to their child every week
End of project
Start of project
![Page 8: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/8.jpg)
Gathering the information
![Page 9: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/9.jpg)
1. Secondary evidence
• Existing sources of information to show what has been done / achieved by the project
• Think - what data is already available? • Core service data * School visitor records• Child or parent-held records * Staff observations / CPD • CAF records * Partner evidence
• What else do you need, on top of this?• Adding to your records • Project-specific monitoring
![Page 10: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/10.jpg)
Practical suggestions
• Have something in place from the start – Much harder to collect evidence after the event
– Need a ‘baseline’ to measure what has changed
• Who will be responsible for gathering the information and how often? – Realistic approach – Match it to the available time and resources
• Don’t change the format half way through…
![Page 11: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/11.jpg)
2. Primary evidence
• New or additional information to show what has been achieved, e.g. insights, personal accounts, experiences and case studies
• Who needs to be consulted?– Stakeholders
– Beneficiaries of your project • Staff, parents, children
• Who will gather the information and how? – Self-reporting / gathered by someone else
![Page 12: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/12.jpg)
Examples…
Self-reporting • Feedback sheets • Comments boxes • Research diaries • Pictorial or video
evidence • SMS / Text or email
feedback • Blogs
Gathered directly • Observations • Structured interviews
or focus groups • Action research /
practitioner research • Piggy-backing:
– Involve existing parent or community panels
– Work into annual resident’s surveys
![Page 13: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/13.jpg)
Some key points
• Structure is needed - what are you asking people to feed back? – Structured doesn’t have to mean formal– Satisfaction is different to outcomes
• Impartiality and avoiding bias– Are you too close to ask the questions? – Who else could do this instead?
• Volunteers / parent advocates / partner staff • CRB checks and Health and Safety
• Confidentiality and informed consent – Do participants know how you intend to use the information?
![Page 14: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/14.jpg)
Questionnaires – some tips
• Make them confidential
• Language used – support for completion
• Appearance / length
• Style of questions – avoid bias Was the session:– A) Excellent – B) Very good – C) Good
x
![Page 15: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/15.jpg)
Questionnaires…
Overall, how would you rate your personal confidence, now? Tick the box that describes you.
a) I am confident in myself
(4)
b) I struggle a bit with my confidence
(3)
c) I need a lot of help with my confidence
(2)
d) I have very low confidence
(1)
How far would you agree with the following statements?
Strongly agree
(4)
Agree(3)
Disagree(2)
Strongly disagree
(1)
Don't know
a. "I feel nervous about using childcare services"
b. "I am confident about my child's development"
c. "I wish I knew more about services for families in my area"
![Page 16: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/16.jpg)
Analysing and reporting on evidence
![Page 17: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/17.jpg)
Analysing and reporting
1) Qualitative evidence
Case studies• Individual or service level• Telling a story:
– what was the issue? – how was it tackled? – what was successful? – what is different now?
Using quotes• Powerful form of evidence • Confidentiality
2) Quantitative evidence
Reporting survey findings • Less can be more• Non-numeric is often best::
– ‘approaching half’
– ‘nearly a third’
– ‘a sizeable majority’
Charts and tables • Presenting a clear message • Outcomes for different groups
![Page 18: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/18.jpg)
Support from ECOTEC…
![Page 19: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/19.jpg)
Support from ECOTEC
• Evaluation Toolkit – Key principles of doing evaluation – A source of ideas and examples – Benchmarking what you are doing – good practice examples
• One-to-one advice – Commenting on your individual evaluation plans – Q&A and troubleshooting via the support email / telephone
• Sharing findings from the independent evaluation – Interim and final reports – Case studies
![Page 20: Evaluation Workshop Self evaluation – some workable ideas and approaches.](https://reader035.fdocuments.us/reader035/viewer/2022062408/56649f175503460f94c2e11a/html5/thumbnails/20.jpg)
Thank you for listening