Definition of Evaluation
-
Upload
penn-state -
Category
Education
-
view
371 -
download
0
description
Transcript of Definition of Evaluation
![Page 1: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/1.jpg)
Evaluation of Investment InWorkforce Education & Development
![Page 2: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/2.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 3: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/3.jpg)
Decision types
ActualIntended
Ends
Means
PLANNING RECYCLING
STRUCTURING IMPLEMENTING
To determineobjectives
To judge and react to attainments
To design proceduresTo utilize, control, and refine procedures
![Page 4: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/4.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 5: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/5.jpg)
Describe use of criteria in evaluation
![Page 6: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/6.jpg)
Criteria
A decision requires judging among two or more alternatives.
Criteria are rules or tests for judging. Criteria are yardsticks for values.
Ö Values = estimated or assessed worth.Ö Values result from systems of beliefs, perceptions,
interpretations of facts, judgments of needs. Ö Values are shared and/or idiosyncratic.
Decision-makers apply criteria.Ö They must “own” criteria.Ö Often need assistance in identifying and finding criteria.
![Page 7: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/7.jpg)
Four kinds of values that impinge on establishment of criteria
Instiutional valuesÖ E.g., “What does the Board of Trustees expect our College to
do?”
Values external to the institutionÖ E.g., “What does the State Legislature believe the College
should produce?”
Subsystem maintenance valuesÖ E.g., “What do my College colleagues believe we should do?”
Private valuesÖ E.g, “What do I think our College should produce?”
![Page 8: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/8.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 9: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/9.jpg)
Delineate criteria for evaluation
![Page 10: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/10.jpg)
Criteria for evaluation
ScientificÖ Internal and external validity, reliability, objectivity
PracticalÖ Relevance, importance, scope, credibility, timeliness,
pervasiveness
PrudentialÖ Efficiency
![Page 11: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/11.jpg)
Criteria for evaluation:Scientific
Internal validityÖ No alternative explanations are possible for results obtained.
External validityÖ Findings are generalizable across people, places, and time.
ReliabilityÖ Accuracy of findings; reproducibility.
ObjectivityÖ Intersubjectivity; lack of bias.
![Page 12: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/12.jpg)
Criteria for evaluation:Practical
RelevanceÖ Information serves decisions.
ImportanceÖ “Best” information culled to serve decisions.
ScopeÖ All necessary information is provided.
CredibilityÖ Clients/users believe information.
TimelinessÖ Information provided when needed.
PervasivenessÖ Information provided to all clients/users.
![Page 13: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/13.jpg)
Criteria for evaluation:Efficiency
Conducted with the “best” use of resources.
![Page 14: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/14.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 15: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/15.jpg)
Suggest a meta-methodology for evaluation
![Page 16: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/16.jpg)
Work breakdown of evaluation design
![Page 17: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/17.jpg)
Delineation of information needs
![Page 18: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/18.jpg)
Delineation of information needs
![Page 19: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/19.jpg)
Delineation ofinformation needs
![Page 20: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/20.jpg)
Delineation ofinformation needs
![Page 21: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/21.jpg)
Work breakdown of evaluation design
![Page 22: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/22.jpg)
Plan forobtaining information
![Page 23: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/23.jpg)
Plan forobtaining information
![Page 24: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/24.jpg)
Plan forobtaining information
![Page 25: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/25.jpg)
Work breakdown of evaluation design
![Page 26: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/26.jpg)
Plan forproviding information
![Page 27: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/27.jpg)
Plan forproviding information
![Page 28: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/28.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 29: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/29.jpg)
Specify four types of evaluation
![Page 30: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/30.jpg)
Evaluation &decision-making
![Page 31: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/31.jpg)
Evaluation types
ActualIntended
Ends
Means
CONTEXT PRODUCT
INPUT PROCESS
For planningdecisions
For recyclingdecisions
For structuringdecisions
For implementingdecisions
![Page 32: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/32.jpg)
Evaluation types:Context
Defines relevant environment. Identifies unmet needs and unused opportunities. Diagnoses problems that prevent needs from being
met and opportunities from being used. Is macro-analytic and systematic. Involves contingency and congruence studies to set
direction.Ö Contingency - searches for opportunities and pressures outside
system to promote improvement within.Ö Congruence - compares actual and intended system
performance to affirm objectives or consider new objectives.
![Page 33: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/33.jpg)
Evaluation types:Input
Determines how to use resources to achieve aims.Ö Identifies relevant capabilities.Ö Specifies strategies for achieving goals.Ö Fabricates specific designs to accomplish aims.
End product of input evaluation is one or more procedural designs, with a lists of costs and benefits.Ö Assesses alternative designs in light of resource
requirements/limits.Ö Identifies procedural barriers, and risk/consequences of failure
to overcome.Ö Relevance of various designs to aims.
Build or make? Project proposals embody the results of input
evaluation.
![Page 34: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/34.jpg)
Evaluation types:Input (cont.)
Although context evaluation is systematic and macro-analytic, input evaluation is ad hoc and micro-analytic.
Some questions asked:Ö Is need documented?Ö What strategies already exist?Ö What are costs and benefits of competing strategies?Ö What is track record for competing strategies?Ö Are designs logical? Based on sound theory? Assumptions met?
Legal? Moral? Ö Side effects? Ö Likely to be adopted?
![Page 35: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/35.jpg)
Evaluation types:Process
Aims:Ö Detects or predicts defects in procedural design or in
implementation in early stages.Ö Provides information for programmed decisions.Ö Records implementation as it occurs.
Depends on context and input evaluation.Ö More adequate context and input evaluation, the less critical the
need for process evaluation.Ö Less adequate context and input evaluation, the more critical
process evaluation becomes.
More important than product evaluation in early development stages, but less important as more structure and knowledge is developed.
![Page 36: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/36.jpg)
Evaluation types:Product
Interprets attainments at end of project cycle or as often as necessary during cycle for accountability or improvement.
Involves: Ö Devising operational definitions of objectives.Ö Obtaining criterion measures.Ö Comparing measures with some absolute or relative standards.Ö Making rational interpretations of outcomes using recorded
context, input, and process information.
Criteria are instrumental or consequential:Ö Instrumental -- related to intermediate accomplishments
prerequisite to some end.Ö Consequential -- pertain to fundamental, penultimate ends
sought.
![Page 37: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/37.jpg)
What we will do…
Define evaluation as a process. Make decision-making a focus of evaluation:
Ö Decision-making processesÖ Decision settingsÖ Decision types
Describe use of criteria in evaluation. Delineate criteria for evaluation. Suggest a meta-methodology for evaluation. Specify four types of evaluation. Present a general evaluation model.
![Page 38: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/38.jpg)
Present a general evaluation model
![Page 39: Definition of Evaluation](https://reader034.fdocuments.us/reader034/viewer/2022051609/547b6dc2b379596a2b8b4d9c/html5/thumbnails/39.jpg)
Evaluation of Investment InWorkforce Education & Development