Program evaluation Sebastian Uijtdehaage [email protected] 310.794.9009.
26
-
Upload
patience-munsell -
Category
Documents
-
view
213 -
download
1
Transcript of Program evaluation Sebastian Uijtdehaage [email protected] 310.794.9009.
Why do you evaluate?
Five reasons of evaluating
1. FORMATIVEinforms program development
2. SUMMATIVEdemonstrate program’s success and impact
3. UNDERSTANDINGhow and why did program work?
4. DISSEMINATIONsharing best practices
5. RECOGNITIONscholarship, academic credit, leadership
Stufflebeam 2003
What type of evaluation should I conduct?
Outcome & Impact
Did it work?
Dan Stufflebeam:CIPP Model of Program Evaluation
Needs Assessment
What should we do?
Program Planning
How shall we do it?
Implementation
Are we doing as planned?
Outcome & Impact
Does it work?
INPUT
CONTEXT
PROCESS
PRODUCT
What type of evaluation should I conduct?
Formative or Summative?
formative & summative evaluationFormative Evaluation
Context What needs to be done?
Input How should it be done?
Process Is it being done?
Product Is it working?
Stufflebeam 2003
formative & summative evaluationFormative Evaluation Summative Evaluation
Context What needs to be done?
Were important needs addressed?
Input How should it be done?
Was program plan defensible?
Process Is it being done? Was the plan well executed?
Product Is it working? Did the effort make a difference?
Stufflebeam 2003
What type of evaluation should I conduct?
Quantitative or Qualitative?
Quantitative or qualitative measures?
Quantitative measures HOW MUCH…?
• How much did the participant change?
• How much did the community change?
Qualitative measuresWHY and HOW?
• Why did the program work?
• How did participants change as result?
What makes good evaluation?
Evaluation must be planned ahead
• Evaluation should be part of program planning
• Planning and resource allocation makes evaluation more successful– Example: long-term tracking of participants
Confidence being admitted in a medical school
Category 13.4
3.6
3.8
4
4.2
4.4
4.6
4.8
Start of programEnd of program
What else makes good evaluation?
• Scientific rigor• Evaluation produces useful information that
helps make decisions– Assessment should be done with a particular
purpose in mind• Evaluation is tailored to your program
objectives• Evaluation is done with conceptual framework
in mind
LOGIC MODEL
Resources Activities Output Outcomes Impact
The Planned Work The Intended Results
CONDITIONS and DRIVERS
LOGIC MODEL
CONDITIONS and DRIVERS
Current situation that precipitates the need for the program
LOGIC MODEL
Resources Activities Output Outcomes Impact
CONDITIONS and DRIVERS
Resources and inputs needed to operate the program:personnel, facilities, materials, equipment
LOGIC MODEL
Resources Activities Output Outcomes Impact
CONDITIONS and DRIVERS
Everything the program does if it had access to the resources
LOGIC MODEL
Resources Activities Output Outcomes Impact
CONDITIONS and DRIVERS
The “deliverables”—the events, products and services
Number of participants, classes, brochures, advisors, etc
LOGIC MODEL
Resources Activities Output Outcomes Impact
CONDITIONS and DRIVERSIf the program delivers, the participants
will benefit in certain ways:
Knowledge, skills,
behavior, attitudes
Immediate, mid-term and
long-term outcomes
LOGIC MODEL
Resources Activities Output Outcomes Impact
CONDITIONS and DRIVERS
If these benefits to participants are achieved, then certain changes in organizations and communities might be expected to occur
LOGIC MODEL
Resources Activities Output Outcomes Impact
The Planned Work The Intended Results
CONDITIONS and DRIVERS
LOGIC MODEL
• Links processes and outcomes:– Funders love this!
• Great for program planning– Work backwards
• Evaluation is built in!• Ain’t perfect
Your turn…
