Determining &
Demonstrating Value -
with the Logic Model
Moe Hosseini-Ara, City of Markham
Rebecca Jones, Brampton Public Library
TO WHOM
?
Ivanna U.R. Mony
• Socially connected
• Lobbying to have library
budget reallocated to other
projects
• Has never used the library
• Buys all her books from
Amazon
• Why does the library need
so many staff ? And why are
they paid so much?
Dr. Ural Doomed
• Libraries days are numbered
•
– Everyone has computers and internet
–
– Research happens in faculties not
libraries
• With eBooks and research online,
believes facilities can be made
smaller no need for print material
CFO Roi (Roy) Bottomline, MBA
• All about the bottomline
• Revenue and investments
• How do your services support revenue generation?
•revenue are a drain on the system
•it!
Kee Paiper
• Influential in many circles
• Has deep seated beliefs in
traditional library services and
importance of paper for legal
profession
• Libraries are institutions for
archives & research
Understand the Context
Know the Context & the Contexters
How do they
measure their
progress?
Know your
Stakeholders who
are they?
How is value
defined,
measured &
communicated?
Symptoms
Underlying causes
the water
causing the
symptoms?
•
•
performance
• Difficult and complex
•
decision-makers
• Identifying & illustrating value depends on conversations with your
stakeholders
•
& mental
models
How should
we frame
this issue?
Know the
language of
your context
MetricsValue
Customer Constituent
Student
Start with stakeholders
Name Position How do they
measure their
progress?
Goals &
objectives
What do we know about them?
What type of stakeholder are
they?
What do they
know about
our services?
Critical that you understand the context in which the measures are considered.
1. In your organization, how is value defined, measured and conveyed?
2. Who are the stakeholders who must understand the value of your services?
EXERCISE
The Logic Model
The Logic Model
goes beyond documenting what you did and
measures what difference you made in the life of
changed
Measuring Impact/Value
Change in the human condition?
• Behavior
• Skill
• Knowledge
• Attitude
• Circumstances
• Awareness
• Motivation
• Condition
• Status
INPUT
resource perspective
OUTPUT
operational perspective
OUTCOME
user perspective
IMPACT
stakeholder perspective
IMPACT
Equipment
INPUTResource
Perspective
Equipment
Space
Supplies
Funding
Staff
IMPACT
Equipment
OUTPUTOperational Perspective
Training module
Program
Report
# of Program Attendees
# of Programs Held
IMPACT
OUTCOMEUser
Perspective
Increased skill
Know-how or know-that
Change in status
Change in behavior or
attitude
IMPACT
IMPACTStakeholder Perspective
Long term results
Lower dropout rates
Increased employment
levels
Increased recovery rates
Job Skills Training
Input(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation
Job Skills Training
Input(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation
Job Skills Training
Input(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation
Job Skills Training
Input(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation
Job Skills Training
Input(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation
BEGINWITH
THEIN MIND
ENDCovey 1989
Public Library Example
Input(resources)
• Staffing; Program Design; Marketing Material, Facilities; Equipment
• Speakers = 6 female professionals working in STEM careers scientists, astronauts,
engineers, doctors, programmers
• $1,500 to pay for honorariums and supplies
Output(operations)
• 6 week program offered twice a year
• 25 girls ages 10-12 complete program each session (50 participants per year)
• Pre and Post-session questionnaire; (permission to follow-up in 3 years)
Outcome(user)
• Girls ages 10-12 have a better understanding of STEM careers
• Increased interest in sciences & maths as potential course options
• Positive response to questionnaire that the girls want to be scientists, astronauts,
engineers, programmers, etc.
Impact(stakeholder)
• Girls ages 10-12 are interested in pursing STEM careers (based on questionnaire
response (short-term)
• These girls take maths, sciences, computers in high school and pursue degrees in
STEM in university (mid-term)
• Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long-
term)
Public Library Example
Input(resources)
• Staffing; Program Design; Marketing Material, Facilities; Equipment
• Speakers = 6 female professionals working in STEM careers scientists, astronauts,
engineers, doctors, programmers
• $1,500 to pay for honorariums and supplies
Output(operations)
• 6 week program offered twice a year
• 25 girls ages 10-12 complete program each session (50 participants per year)
• Pre and Post-session questionnaire; (permission to follow-up in 3 years)
Outcome(user)
• Girls ages 10-12 have a better understanding of STEM careers
• Increased interest in sciences & maths as potential course options
• Positive response to questionnaire about what the girls want to be - scientists,
astronauts, engineers, programmers, etc.
Impact(stakeholder)
• Girls ages 10-12 are interested in pursing STEM careers (based on questionnaire
response (short-term)
• These girls take maths, sciences, computers in high school and pursue degrees in
STEM in university (mid-term)
• Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long-
term)
Public Library Example
Input(resources)
• Staffing; Program Design; Marketing Material, Facilities; Equipment
• Speakers = 6 female professionals working in STEM careers scientists, astronauts,
engineers, doctors, programmers
• $1,500 to pay for honorariums and supplies
Output(operations)
• 6 week program offered twice a year
• 25 girls ages 10-12 complete program each session (50 participants per year)
• Pre and Post-session questionnaire; (permission to follow-up in 3 years)
Outcome(user)
• Girls ages 10-12 have a better understanding of STEM careers
• Increased interest in sciences & maths as potential course options
• Positive response to questionnaire that the girls want to be scientists, astronauts,
engineers, programmers, etc.
Impact(stakeholder)
• Girls ages 10-12 are interested in pursing STEM careers (based on questionnaire
response (short-term)
• These girls take maths, sciences, computers in high school and pursue degrees in
STEM in university (mid-term)
• Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long-
term)
Public Library Example
Input(resources)
• Staffing; Program Design; Marketing Material, Facilities; Equipment
• Speakers = 6 female professionals working in STEM careers scientists, astronauts,
engineers, doctors, programmers
• $1,500 to pay for honorariums and supplies
Output(operations)
• 6 week program offered twice a year
• 25 girls ages 10-12 complete program each session (50 participants per year)
• Pre and Post-session questionnaire; (permission to follow-up in 3 years)
Outcome(user)
• Girls ages 10-12 have a better understanding of STEM careers
• Increased interest in sciences & maths as potential course options
• Positive response to questionnaire that the girls want to be scientists, astronauts,
engineers, programmers, etc.
Impact(stakeholder)
• Girls ages 10-12 are interested in pursing STEM careers (based on questionnaire
response (short-term)
• These girls take maths, sciences, computers in high school and pursue degrees in
STEM in university (mid-term)
• Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long-
term)
Academic Library Example
Input(resources)
• One liaison librarian
Output(operations)
• Faculty research support for a Grant Application
• Completed application submitted on time
Outcome(user)
• Grant dollars received by faculty
• Faculty member is able to hire 2 grad students to conduct
research
Impact(stakeholder)
• Increased university reputation
• A number of articles are submitted and accepted by academic
journals, increased citations for faculty member
• University increases visibility.
Government Library Example
Input(resources)
• $100K resource budget
• Staff to advise on development of policies
Output(operations)
• 2500 resources utilized by 75% of department employees
• Library has contributed to the development of 6 policies in past
12 months
Outcome(user)
• 6 policies approved and in place
• Eliminate department-specific resource purchases saving the
organization $35K
Impact(stakeholder)
• Credible policies
• -
by at least 25% (as reported during policy debriefings)
• Reduced costs to run department
Corporate Library Example
Input(resources)
• $80K electronic content contract
•
departments applications
• Skilled info pro to develop research portfolio for business development
department
Output(operations)
• Customized research portfolio in response to request for background on
deal between XYZ company & ABC government agency, including specific
data on terms how ABC likes deal structured
• Library resources are integrated into business development applications
Outcome(user)
• Client extrapolated how ABC agency likes to see deals structured and
structured deal appropriately
Impact(stakeholder)
• Organization successfully completed a deal with ABC government agency
for $10M, the 1st in 5 years
Measurement
Framework
Understand the Context
Align Strategies &
Objectives
Identify Services & Programs
Define Measures
Manage Measurement
Data
Translate Data into
Outcomes & Impacts
Communicate Results
Align Strategies &
Objectives
Align Strategies & Objectives
Your Goals & Objectives: How they
Contribute
Accelerate newcomers contribution to
the community & economy
Offer programs and services that
connect newcomers with relevant
agencies and services
Increased research grants Support the research process and
assist with submission and completion
of grants
Newcomers contributing to community
economically
Welcome newcomers with links to
language, literacy, employment &
social supports
Align objectives
• Articulate & document your goals &
objectives
What are you trying to accomplish?
• Clarify how these contribute towards
desired outcomes
EXERCISE
Identify Services & Programs
Identify Services
What services, programs or products contribute to your realizing
an objective?
Stakeholder Goals & Objectives
Your Goals & Objectives
Your Services
EXERCISE
Define Measures
What measures do
you keep now?
• No one magic measure
• Successful organizations:
• clarity of purpose
• understand their culture
• performance measurement system that fits that
culture
•
alternatives
• Identifying outcomes & especially impact takes a long time
& heavy duty analysis
Value
SatisfactionOperational
Three Types of Measures
Operational/Usage Measures
•
• We also need to look at differences:
• Peaks? Dips? Switches?
• What are these statistics really telling us? Or, not?
• Who needs to see these statistics?
• Are they telling a story that you want to tell?
Operational
Customer Satisfaction Measures
• Tends to focus on existing
products and/or services
• Usually measured through
surveys
• Are we also measuring
importance?
• Survey burnout, plus the
expectation is to score all
Satisfaction
• Are we adding value?
• What difference are we making?
• What is the impact of our
services and/or programs?
• How are we contributing to the
Value
What are meaningful measures?
Matter to you AND your
stakeholders
language
Focus attention on what is most
important for the organization
Are critical for managing,
planning & decision-making
Are organization-dependent
Measure: info about what is
Define success
• What will success look like?
• How will you track progress towards meeting goals?
• What indicators or measures will you use?
• Qualitative? Quantitative?
• What data needs to be collected?
• Keep it key
• Keep it simple
• Be practical
• or
overwork frontline staff
Service Impact
InputsPerson power, money, content, etc.
ActivityWhat you do
OutputsWhat the activity produces
OutcomesHow the client uses the output
ImpactWhat changes for the client
Outcome Measures Impact Measures
EXERCISE
Service Impact
InputsPerson power, money, content, etc.
ActivityWhat you do
OutputsWhat the activity produces
OutcomesHow the client uses the output
ImpactWhat changes for the client
Outcome Measures Impact Measures
EXERCISE
Manage Measurement
Data
Manage collection
• Determine a collection schedule• Work out agreements with other departments that
have needed data• Are you currently collecting data or indicators that
are no longer relevant?• How long do you need to keep data?• Who is responsible?
EXERCISE
Translate Data into
Outcomes & Impacts
Define outcomes & impact
• What will success look like?
•
OUTCOME IMPACT
INPUT OUTPUT
Define outcomes & impact
OUTCOME
User Perspective
Increased skill
Know-how or know-that
Change in status
Change in behaviour or
attitude
IMPACT
Stakeholder Perspective
Longer term results
Lower dropout rates
Increased employment
levels
Increased recovery rates
Measuring Impact/Value
Change in the human condition?
• Behavior
• Skill
• Knowledge
• Attitude
• Circumstances
• Awareness
• Motivation
• Condition
• Status
Interpret data
•
•
• Examine it from various angles
• What is the progress towards the goals &
objectives?
EXERCISE
Communicate the Results
Remember us
?
Communicate results
• Actually begins when you are understanding the context
• Your message: e have
contributed towards your goals
• If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable
EXERCISE
c
Value
SatisfactionOperational
Sweet Spot
What do you need to stop doing?
Resources
IMLS Webography
http://www.imls.gov/applicants/webography.aspx
Shaping Outcomes
http://www.shapingoutcomes.org/index.htm
Overcoming Our Habits and Learning to Measure Impact by Moe Hosseini-Ara and Rebecca Jones.
Information Today, Inc. June 2013.
http://www.infotoday.com/cilmag/jun13/Hosseini-Ara_Jones--Overcoming-Our-Habits-and-Learning-to-
Measure-Impact.shtml
Seven Easy Steps to Help You Measure Up by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc.
July/August 2013.
http://www.infotoday.com/cilmag/jul13/Hosseini-Ara_Jones--Seven-Easy-Steps-to-Help-You-Measure-Up.shtml
Top Related