Team PresentationTeam PresentationEvaluationEvaluation

Team MemberBetty RogersSeptember 20, 2010

OverviewOverview

How evaluation techniques can be applied to libraries program and services.

Define evaluation

Purpose of evaluation

Evaluation Method

Evaluation Strategies libraries can use

Evaluation Types

Evaluation planning process

3

What being measured?

What does it indicate?

What is not being measured?

For What Purpose is the Evaluation?

Who are the audiences for the information from the evaluation?

From what sources should the information be collected?

When is the information needed?

“If you don’t know where you are going, how are you gonna’ know when you get there?”

Yogi Berra

DefinitionDefinition

First, evaluation is viewed as a systematic process

Second, evaluation involves collecting data

Third, evaluation is a process for enhancing knowledge anddecision making

Fourth, evaluation is a process of determining the success, impact, results, costs, outcome

SOURCE: Adapted from Russ-Eft and Preskill (2001)

Purpose of EvaluationPurpose of Evaluation

Provide “Useful Feedback

Planning/efficiency

Accountability

Implementation

Institutional strengthening

Evaluation MethodEvaluation Method

Input measurement

Output/Performance Measurement

Impact/Outcomes Assessment

Service Quality

Evaluation MethodsEvaluation Methods

Input MeasurementMoney,

Facilities,

Customers

Clients,

Program staff

Volunteers

External partners

Time

Equipment

Technology

Evaluation MethodsEvaluation Methods

Output measurement

ActivityEvents

Products

Workshops

Trainings

exhibits

Numbered of customer served

Evaluation MethodsImpact/Outcomes MeasuresIndicative of impacts or outcomes from the service, program, or library activity on those who are receiving the service

Outcome / Impactknowledge

Attitudes

Awareness

Opinions

Skills

Behavior

Educational

Environmental quality

Evaluation MethodsEvaluation Methods

Service QualityDifference between what customers expect and their perceptions of the service performance.

-Encompasses the interactive relationship between the library and the people whom it is supposed to serve

Does the service meet organizational or user expectation

Evaluation StrategiesEvaluation Strategies

Major Model Groups

Scientific-experimental models

Management-oriented systems modelsPERT(Program Evaluation and Review Technique

CPM(Critical Path Method)

Qualitative Models

Participant-oriented Models

ROI ModelROI Model

Solution Matrix • Cost-Benefit-Analysis http://www.solutionmatix.com/return-on-investment.html

Type of EvaluationType of Evaluation

Formative

Summative

Robert Stake likened the two stages of evaluation to making soup: when the chef tastes the soup,

its formative; when the diners (or a food critic) taste the soup it is summative

Types of EvaluationTypes of Evaluation

Formative Evaluation

– Needs Assessment

– Evaluability assessment

– Structured conceptualization

– Implementation evaluation

– Process evaluation

Research Methods Knowledge Base

Types of EvaluationTypes of Evaluation

Summative Evaluation

– Outcome evaluations

– Impact evaluation

– Cost-effectiveness and cost-benefit analysis

– Secondary analysis

– Meta-analysis

Research Methods Knowledge Base

Types of Evaluation

Planning Evaluation Cycle

Evaluation PhaseFormulation

Conceptualization

Detailing

Evaluation

implementation

Planning Evaluation Cycle

Planning PhaseFormulation

Conceptualization

Design

Detailing

Analysis

utilization

Planning-Evaluation CyclePlanning-Evaluation Cycle

Research Methods Knowledge Base

20

University of Wisconsin-Extension, Program Development and Evaluation

Typical activity indicators to trackTypical activity indicators to track

Amount of products, services delivered#/type of customers/clients servedTimeliness of service provisionAccessibility and convenience of service

Location; hours of operation; staff availability

Accuracy, adequacy, relevance of assistanceCourteousnessCustomer satisfaction

E.g.:# of clients served# of consultations# of workshops held# of attendees# of referralsQuality of service

Data Collection Techniques

TestsAssessmentsQuestionnairesInterviewsObservationEvaluationUse recordAnalysisContent analysisSurveys

Technique used to Evaluation my Library’s Technique used to Evaluation my Library’s Service/ FVSU H.A. Hunt Memorial LibraryService/ FVSU H.A. Hunt Memorial Library

Formative

Pretest

-Students

Summative

Survey

-faculty

Information Literacy Defined…

The ability to recognize a need for

information and to access, evaluate, and use that information

Conclusion/SummaryConclusion/Summary

In essence, Evaluations are important to library programs and services. The evaluation experience is likely to be more positive and its results are likely to be more useful if you build evaluation in form the start and make it an on-going activity.

ReferenceReference

University of Wisconsin-Extension, Program Development and Evaluation Logic Model. Retrieved. September 1, 2010; http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

These forms of evaluation are derived from the work of Owen published in Program Evaluation: Forms and Approaches (2006) . Retrieved September 1, 2010; http://www.murdoch.edu.au/teach/carrick_evaluation/purpose_scope.html

Haycock, K. & Sheldon ,B.E (2008) .The Portable MLIS; Insights from the Experts. Westport, CT: Libraries Unlimited.

Research Methods Knowledge Base. Retrieved. September 12, 2010; http://www.socialresearchmethods.net