CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?
-
Upload
candace-fletcher -
Category
Documents
-
view
219 -
download
0
Transcript of CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?
![Page 1: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/1.jpg)
CRJS 4466PROGRAM & POLICY EVALUATION
LECTURE #5
•Evaluation projects
• Questions?
![Page 2: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/2.jpg)
Qualitative Evaluation Methods
![Page 3: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/3.jpg)
Qualitative Methods – strengths
• Carol Weiss (1998) – the coming of age of qualitative methods
• note the Campbell versus Cronbach controversy here
• qualitative methods are an important technique for under- standing the ‘context’ of success or failure of programs
• qualitative methods most commonly used in process evaluation (formative) or the ‘how’ and ‘why’ of a program and its workings
• ability to represent subtleties and complexities of program functioning and dynamics
![Page 4: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/4.jpg)
Qualitative Methods - strengths
• many evaluators like the interpersonal nature of qualitative methodologies
• qualitative approaches are often ‘action oriented’
• a choice of ‘depth’ over ‘breadth’
![Page 5: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/5.jpg)
Qualitative Methods - limitations
• reliability and validity of measures
• interviewer bias
• anecdotal
• labour-intensive
• costly
• less useful for reporting on program outcomes
![Page 6: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/6.jpg)
Qualitative Methods – Single System Evaluations
• evaluating an intervention with a single client system, usually quantitatively during the course of treatment
• limited or questionable usefulness of qualitative methodology
![Page 7: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/7.jpg)
4. Qualitative Methods – Focused Qualitative Evaluation (FQE)
• reflects the immediacy of much evaluation – not time for extended research, ethnomethodological development
• use of FG and rapid ethnographic assessment
![Page 8: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/8.jpg)
Qualitative Methods – Mixed Methods
• ‘triangulation’, combining both quantitative and qualitative methods
• ‘temporal sequencing’ and qualitative subsamples
• benefit of both breadth and depth
![Page 9: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/9.jpg)
Qualitative Methods – Action (Participatory) Research
• overtly ‘action-oriented’ approach primarily among those conducting qualitative evaluations
• participatory, collaborative, empowering – now an expectation in some evaluation projects
• ‘praxis’ orientation
![Page 10: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/10.jpg)
Qualitative Methods – Research Design
• units of analysis and comparative designs
• gaining access, key informants, and sponsorship
• reciprocity, payback and feedback – and the methodological, ethical concerns here (‘false hope’)
• sampling considerations • deviant case sampling/typical case sampling• maximum variation sampling• snowball sampling• purposive sampling• convenience sampling• random sampling
![Page 11: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/11.jpg)
Qualitative Methods – Research Design
• data collection strategies• on-site observation• participant observation• reflexivity• use of interview guide• use of focus groups• cross-validation among interviewers• use of documents
• when to stop data collection – the issue of saturation
![Page 12: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/12.jpg)
Qualitative Methods – Data Management and Analysis
• use of qualitative data analysis programs: Ethnograph, AskSam QSR (Nudist)
• coding data
• emic (indigenous) and etic (researcher-created) coding
• quality control mechanisms
• reporting – issues related to qualitative methodologies
![Page 13: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/13.jpg)
Table 5-1: Key Differences Between Qualitative and Quantitative Evaluation Approaches
![Page 14: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/14.jpg)
Table 5.2: Summary of Key Questions in Conducting Qualitative Evaluation Assessments and Evaluation Studies
![Page 15: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/15.jpg)
Table 5-3: Sampling Strategies for Qualitative Evaluations
![Page 16: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/16.jpg)
Table 5-4: Open Ended Questions for the Economic Development Agreement Stakeholders Project
![Page 17: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/17.jpg)
Table 5-4: Open Ended Questions for the Economic Development Agreement Stakeholders Project
![Page 18: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/18.jpg)
Table 5-5: Some Basics of Face-to-Face Interviewing
![Page 19: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/19.jpg)
Table 5-6: Helpful Hints as You Analyze Qualitative Data
![Page 20: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/20.jpg)
Table 5-6: Helpful Hints as You Analyze Qualitative Data
![Page 21: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/21.jpg)
Table 5-7: Comparing Qualitative Validity with Experimental and Quasi-Experimental Validity
![Page 22: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/22.jpg)
Table 5-8: Ways of Testing and Confirming Qualitative Findings
![Page 23: CRJS 4466 PROGRAM & POLICY EVALUATION LECTURE #5 Evaluation projects Questions?](https://reader036.fdocuments.us/reader036/viewer/2022062423/56649eac5503460f94bb3250/html5/thumbnails/23.jpg)
Focus Group Question:
“Based on your own experiences of the Criminal Justice Program here at Nipissing University, what do you like about the program? What don’t you like about the program?