Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R....
Transcript of Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R....
![Page 1: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/1.jpg)
Kickstarting Your Evaluation
Stacie R. Powers, PhDPhilliber Research & Evaluation
September 13, 2017contact: [email protected]
1
![Page 2: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/2.jpg)
AgendaDay One
1. Defining evaluation2. Getting ready3. Logic models4. Customizing questions5. Summary
2
![Page 3: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/3.jpg)
3
AgendaDay Two
1. Writing an evaluation plan
2. Reporting and utilizing results
3. Summary
![Page 4: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/4.jpg)
Handouts
Slides
Example executive summaries
Data set
4
![Page 5: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/5.jpg)
Defining monitoring and evaluation
5
![Page 6: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/6.jpg)
6
![Page 7: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/7.jpg)
7
Monitoring is used to learn whether a program is being delivered as planned.
![Page 8: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/8.jpg)
Evaluation is used to learn whether the shorter-term and longer-term outcomes, results, or goals of a program are being achieved.
8
![Page 9: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/9.jpg)
9
![Page 10: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/10.jpg)
What kinds of evaluations are there?
10
Needs assessment
What should we do?
Process evaluation
Did we do it?
Cost benefit evaluation
Was it worth it?
Outcome evaluation
What happened as a result?
![Page 11: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/11.jpg)
11
![Page 12: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/12.jpg)
12
Why evaluate?
To guide practiceTo create a “science of helping”To determine relative costs and benefits of different strategiesTo ensure fidelity to a modelTo raise money with evidence
![Page 13: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/13.jpg)
Go further
13
The Arts in Education website from the US DOE has helpful links to resources for making the case for arts evaluation in k-12 schools:
https://arts.ed.gov/#program
The Animating Democracy website has several links on evaluating social impact and the arts:
http://animatingdemocracy.org/home-impact
The National Endowment for the Arts has several links specific to program evaluation:
https://www.arts.gov/artistic-fields/research-analysis/program-evaluation-resources-and-performance-measurement
![Page 14: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/14.jpg)
Getting ready
14
![Page 15: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/15.jpg)
What are your program’s objectives?
15
1. What does your program do?2. Whom does it reach?3. How do participants change as a result?4. What are the barriers to doing this?
![Page 16: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/16.jpg)
If you are stuck...
Try describing a “peak experience” in your program. When did it work particularly well?
16
![Page 17: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/17.jpg)
Which one of these program statements is not like the others?The goal of our theatre education program is to:
a. Enhance theatre arts awareness among youth.b. Ensure quality arts exposure for area students.c. In 15 one-hour sessions, teaching artists will utilize a variety
of theatre integrated techniques to improve social emotional learning skills and increase literacy learning for 1,600 underserved African-American students.
d. Enhance the community climate for theatre arts appreciation.
17
![Page 18: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/18.jpg)
Writing clear process objectives
Who will deliver
what
to whom
by when
18
![Page 19: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/19.jpg)
SMART and WISE
Specific
Measurable
Achievable
Relevant
Time bound
Whole system
Inspiring
Story and Synergy
Engaging
19
http://aea365.org/blog/bh-tig-week-developing-smart-and-wise-indicators-by-evangeline-danseco/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+aea365+%28AEA365%29
![Page 20: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/20.jpg)
Writing clear outcome objectives
Shorter term outcomes: Things that will happen almost immediately to those being served.
● Often seen as an increase in knowledge● Change in feelings and attitudes● Could also be an increase in resources (such as food,
shelter, etc.)
20
![Page 21: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/21.jpg)
Writing clear outcome objectives
Longer term outcomes: Things that will happen in the longer term to those being served.
● Increase in skill● Change in behavior● Attainment of life successes
21
![Page 22: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/22.jpg)
Less helpful outcomes objectives
● Participant satisfaction or testimony about how much they like you or your program
● Staff testimony about how hard they work● Poignant vignettes of high profile “wins”● Counts of successes and failure
22
![Page 23: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/23.jpg)
Examples in arts education
● Literacy and language development
● Math achievement● Underserved students● Creative thinking● Critical thinking● Problem solving and
reasoning● Engagement and
persistence● Positive behavior● Self-efficacy 23
● Arts participation● Collaboration and
Communication● Community building● Civic engagement● Cross-cultural
understanding● Social development
![Page 24: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/24.jpg)
Can you spot the process and outcome objectives? What do they DO? What CHANGES as a result?
In 15 one-hour sessions, teaching artists will utilize a variety of theatre integrated techniques to improve social emotional learning skills and increase literacy learning for 1,600 underserved African-American students.
24
![Page 25: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/25.jpg)
Activity
Take a moment to try to summarize the work of your program using process and outcome language.
25
![Page 26: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/26.jpg)
Go further
The Better Evaluation website has a great series on organizing your evaluation, including several ideas on how to describe your program:
http://www.betterevaluation.org/en/plan/define/develop_initial_description
26
![Page 27: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/27.jpg)
Putting ideas together
27
![Page 28: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/28.jpg)
28
What you do...and what changes as a result.
![Page 29: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/29.jpg)
29
Program Activities Shorter Term Outcomes
Longer Term Outcomes
What was delivered to whom, by when?
Changes in perception, awareness, knowledge, feelings, attitudes
Changes in behavioral intention
Changes in behavior
Possible categories in a logic model
![Page 30: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/30.jpg)
30
Program Activities Shorter Term Outcomes Longer Term Outcomes
The symphony will offer 100 free performances per year, including:● at venues such as
schools, senior centers, and corporations;
● African American churches;
● noon concerts downtown; and
● an informal concert series where audiences sit on stage with performers.
There will be:
● increased awareness of the activities of the Symphony and
● increased feelings/perceptions of connection with the Symphony.
There will be increased involvement with the symphony as evidenced by increased:
● ticket buying,● enrollment in the
Symphony Music School, and
● charitable contributions.
Example from a symphony outreach evaluation
![Page 31: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/31.jpg)
31
Program Activities Shorter Term Outcomes Longer Term Outcomes
Now you try...
![Page 32: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/32.jpg)
Check your model:
32
a. Are all the objectives in the correct columns? b. Is the target population the same from column to column? c. Is all the language clear?d. Are the outcomes either too trivial or too unrealistic?e. Is the model really logical?f. Will the program really happen?
g. Does the program use best practices?
![Page 33: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/33.jpg)
33
Program Activities Shorter Term Outcomes
Longer Term Outcomes
What was delivered to whom, by when?
Changes in perception, awareness, knowledge, feelings, attitudes
Changes in behavioral intention
Changes in behavior
More possible categories in a logic model...
![Page 34: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/34.jpg)
34
Inputs Program Activities Outputs Shorter Term
OutcomesLonger Term
Outcomes
What resources are required?
What is supposed to be delivered to whom, by when?
What/how much was actually delivered to whom, by when?
Changes in perception, awareness, knowledge, feelings, attitudes
Changes in behavioral intention
Changes in behavior
More possible categories in a logic model...NEW
![Page 35: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/35.jpg)
35
![Page 37: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/37.jpg)
In general, the success of any program will depend on:
a. Targeting. Does the program reach the right people?b. Coverage. Does it get to enough of those people?c. Intensity. Is the intervention strong enough to produce the
desired outcome?d. Effectiveness. Is the intervention able to change the
outcome in question?
37
![Page 38: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/38.jpg)
Go further
A good standard guide for getting started with your logic model is from the Kellogg Foundation:
W.K. Kellogg Foundation Guide to Logic Model Development
https://ag.purdue.edu/extension/pdehs/Documents/Pub3669.pdf
38
![Page 39: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/39.jpg)
Framing your evaluation
39
![Page 40: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/40.jpg)
40
![Page 41: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/41.jpg)
41
Four tasks in framing an evaluation
1. Identify primary intended users.2. Decide the purpose or intended use.3. Specify key evaluation questions based on your type of
evaluation.4. Specify what success looks like based on your type of
evaluation.
http://www.betterevaluation.org/en/plan/engage_frame
![Page 42: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/42.jpg)
42
Who might be your intended users?
Broadly?
Specifically?
![Page 43: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/43.jpg)
Decide the purpose
Improve the program
Continue or stop it
Build a broader evidence base
Advocate
Provide voice
Build trust and legitimacy
Increase accountability
What else?43
![Page 44: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/44.jpg)
Key evaluation questions
44
Needs assessment
What should we do?
Process evaluation
Did we do it?
Cost benefit evaluation
Was it worth it?
Outcome evaluation
What happened as a result?
![Page 45: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/45.jpg)
What kinds of information would you need to answer these evaluation questions?
1. Has learning improved?2. Has the program contributed to improved learning?3. Has the program been a success?4. Should the program continue?
45
![Page 46: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/46.jpg)
Additional considerations for barriers
Context
Unintended outcomes
Try this exercise:
The Five Whys. Come up with a barrier your program or community faces. Then ask Why? in response to each reason. This usually helps establish a cause and effect sequence.
46
![Page 47: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/47.jpg)
47
Determine what success looks like
1. Benchmarks2. Performance measures3. Improvement--participant level? neighborhood level?4. Consensus5. Change
![Page 48: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/48.jpg)
Go further
The Robert Wood Johnson Foundation has a nice guide and set of worksheets:
A Practical Guide to Engaging Stakeholders in Developing Evaluation Questions
http://www.rwjf.org/en/library/research/2009/12/a-practical-guide-for-engaging-stakeholders-in-developing-evalua.html
48
![Page 49: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/49.jpg)
Summary
49
![Page 50: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/50.jpg)
AgendaDay One
1. Defining evaluation2. Getting ready3. Logic models4. Customizing questions5. Summary
50
![Page 51: Kickstarting Your Evaluation - CSU Class · 2020-01-01 · Kickstarting Your Evaluation Stacie R. Powers, PhD Philliber Research & Evaluation September 13, 2017 contact: spowers@philliberresearch.com](https://reader033.fdocuments.us/reader033/viewer/2022050109/5f4718f21cb50a1e014ff523/html5/thumbnails/51.jpg)
Activity using Executive Summaries
1. Reading through these summaries, what is valued in these evaluations?
2. Can you identify the key processes and outcomes that would be captured in the logic models (or theories of change) for these programs?
3. Who do you think are the primary users of these evaluations? Why?
4. Were these programs successes? What challenges did the evaluation surface that were/were not addressed?
51