Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects
description
Transcript of Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects
![Page 1: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/1.jpg)
Service Quality Assessment in Research Libraries
The Guelph/Waterloo ARL LibQUAL Projects
Mark HaslettRon MacKinnonSusan Routliffe
February 1, 2002
![Page 2: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/2.jpg)
Structure of today’s session
Part 1: History & context of LibQUAL
Part 2: Local administration Part 3: Local results Questions
![Page 3: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/3.jpg)
Part 1
History & context of LibQUAL
![Page 4: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/4.jpg)
ARL New Measures Initiative October 1999 Membership Meeting, the ARL
New Measures Initiative was established in response to the following two needs: Increasing demand for libraries to
demonstrate outcomes/impacts in areas important to the institution.
Increasing pressure to maximize use of resources - benchmark best practices to save or reallocate resources.
![Page 5: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/5.jpg)
ARL New Measures Initiative Assessing outcomes important to
students and faculty Maximizing access to information
resources Bench marking best practices Improving services Reallocating resources
![Page 6: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/6.jpg)
ARL New Measures Initiative
1. Higher education outcomes research review
2. LibQUAL (measures for library service quality)
3. Investigation of cost drivers (e.g. technical services cost study)
4. Development of a self-assisted guide for measuring performance of ILL/DD
5. E-Metrics (measures for electronic resources)
![Page 7: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/7.jpg)
Some more context
Traditional focus on inputs “Research libraries have always placed
value in describing their… resources & services.”
Strong history of statistical data collection
![Page 8: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/8.jpg)
Research libraries searching for improved measures
Past practice equated use with value and quantity with quality
Resulted in focus on tonnage But what about the outcome value
for faculty & students?
![Page 9: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/9.jpg)
Research libraries searching for improved measures
New strategic objective for ARL:
“the need for alternatives to expenditure metrics as measures of library performance…”
![Page 10: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/10.jpg)
We need to listen to our users
In order to help “describe and measure the performance of research libraries …”
Such listening should provide opportunities to: Develop & revise our services Use our information resources
effectively Provide for continuous assessment &
accountability
![Page 11: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/11.jpg)
Multiple methods of listening
Active listening Complaints and suggestions Focus groups Students & faculty on committees Web usability studies Surveys
![Page 12: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/12.jpg)
ARL’s LibQUAL proposal
A web survey instrument Identify user expectations -- and user
perceptions of how they're met Not a forecasting or predictive tool Not a ranking tool
![Page 13: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/13.jpg)
LibQUAL goals Establish a library service quality
assessment program at ARL Develop web-based tools for assessing
library service quality Develop mechanisms and protocols for
evaluating libraries Identify best practices in providing
library service
![Page 14: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/14.jpg)
LibQUAL
A research and development project Aim is to have a mature web survey
instrument within 4 to 5 years Focus on client expectations and
perceptions
![Page 15: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/15.jpg)
LibQUAL’s origins Based on SERVQUAL survey instrument
Developed in the 1980s for use in the for-profit sector
Utilizes gap theory to measure service quality
“Only the perceptions of the customers matter.”
![Page 16: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/16.jpg)
What does SERVQUAL measure?
FIVE “Dimensions” of service as perceived by customers: Tangibles Reliability Responsiveness Assurance Empathy
![Page 17: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/17.jpg)
LibQUAL (2000/2001)Nine dimensions of service
Tangibles Reliability Responsiveness Assurance Empathy Access to Collections Library as Place Self-Reliance Instruction
![Page 18: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/18.jpg)
LibQUAL start-up
Spring 1999 ARL meeting – ARL decision to engage in “LibQUAL” pilot project with Texas A&M
October 2000 – ARL Symposium on “Measuring Service Quality.”
![Page 19: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/19.jpg)
LibQUAL phases Phase 1: 1999/2000
12 participating libraries One Canadian – York
Phase 2: 2000/2001 43 participating libraries Three Canadian - Guelph, McGill, Waterloo
Phase 3: 2001/2002 171 participating libraries Four Canadian – Alberta, Calgary, McGill,
York
![Page 20: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/20.jpg)
Purpose of LibQUAL phase 2
Test what was learned in Phase 1 Increase sample size and diversity More Canadian universities Additional questions on, for example,
user self-reliance
![Page 21: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/21.jpg)
Benefits of participation in phase 2
ARL’s collective work Information about user expectations Opportunity to identify service areas for
further review Sharing best practices Experience with this type of survey Experience analyzing the data
![Page 22: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/22.jpg)
Part 2
Local administration of the survey
![Page 23: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/23.jpg)
Research ethics approval
UW
UG
![Page 24: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/24.jpg)
Survey population
How Many
How Selected
![Page 25: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/25.jpg)
Email addresses
How obtained Substitutions Accuracy
![Page 26: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/26.jpg)
Incentive to participate
Project wide incentive Palm pilot
Local incentives Gift certificates
![Page 27: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/27.jpg)
Demographic Detail Population total Students by discipline and year Faculty by status
![Page 28: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/28.jpg)
Start and finish dates
March 15 – March 30 PRAGMATIC FACTORS:
March Break End of classes Beginning of Exams
![Page 29: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/29.jpg)
Testing the questionnaire
Why What we found
![Page 30: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/30.jpg)
Messages to Survey Sample
Four messages Invitation URL Reminders
![Page 31: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/31.jpg)
Responding to questions/comments/complai
nts
Who How much time
![Page 32: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/32.jpg)
Nature of questions/comments/complai
nts Technical problems Can’t/won’t respond Respond later Already responded Spam Survey….
![Page 33: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/33.jpg)
Comments about the survey
Too long Redundant questions Rating scale is too broad and not
meaningful Questions are confusing
![Page 34: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/34.jpg)
Comments about the Survey More questions about collections No opportunity to provide comments Endless questions… Minimal/desired/perceived format not is
desirable; prefer strongly disagree/agree format
![Page 35: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/35.jpg)
Comments about the Survey Poor visual layout, small dots on beige
page Uninviting layout, too dense Questions didn’t all fit on a screen,
needed to constantly scroll back and forth
![Page 36: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/36.jpg)
Comments about the Survey No way to save a partially completed
survey Total number of questions should have
been indicated at the beginning
![Page 37: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/37.jpg)
Comments about the Survey Should have let respondents indicate
which library they were commenting on Age and sex are never relevant on a
survey Irrelevant questions
![Page 38: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/38.jpg)
Survey administration wrap-up
Summary report to Texas A&M / ARL project team
![Page 39: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/39.jpg)
Part 3
Local results: What we learned
![Page 40: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/40.jpg)
Three areas Demographic data Satisfaction data:
Expectations & perceptions Data models
![Page 41: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/41.jpg)
Area 1: Demographic Data Good match with known discipline
populations More in common than we thought May foster collaboration rather than
competition
![Page 42: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/42.jpg)
Respondents by DisciplineUW UG
Agr/Envl Studies 72 137 Architecture 5 3 Business 47 38 Education 1 2 Engineering 222 43 General Studies 1 5 Health Sciences 43 84 Humanities 78 50 Other 69 49 Performing & Fine Arts 11 10 Science 289 228 Social Sciences 124 145 Undecided 5 0 Total 967 794
![Page 43: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/43.jpg)
Library Use on Premises
![Page 44: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/44.jpg)
Electronic Resource Use
![Page 45: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/45.jpg)
Respondents by Sex
![Page 46: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/46.jpg)
Area 2: Satisfaction Data
Caution Do not over-interpret the data Mature methodology Not a mature instrument (R&D) Even when “mature”, gaps only
indicate probable concern further investigation
![Page 47: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/47.jpg)
Example: When it comes to complete runs of journal
titles…
My desired service level is 1..2..3..4..5..6..7..8..9
My minimum service level is 1..2..3..4..5..6..7..8..9
My perception of the library’s service performance is
1..2..3..4..5..6..7..8..9
![Page 48: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/48.jpg)
Zone of tolerance
Desired level---------------------------------
Zone of Tolerance
Minimum level-------------------------------
![Page 49: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/49.jpg)
Zone of Tolerance / sample
![Page 50: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/50.jpg)
Above and within the zone of tolerance
We do not exceed the zone of tolerance in any dimension
We are within the zone for most areas: Assurance Empathy Responsiveness Tangibles Self-Reliance Instruction
![Page 51: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/51.jpg)
Below the zone of tolerance There were questions for which we fell
below the zone of tolerance; these are in 3 dimensions: Access to Collections Library as Place Reliability
![Page 52: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/52.jpg)
Below the zone for…
Access to Collections
Full text delivered electronically to users
Comprehensive print collections
Complete runs of journal titles
![Page 53: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/53.jpg)
Below the zone for…
Library as Place
A haven for quiet and solitude Space that facilitates quiet study Space for individual/group study…
Of more concern to students than faculty
![Page 54: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/54.jpg)
Below the zone for…
Reliability
Accuracy in the catalogue, borrowing, and overdue records
![Page 55: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/55.jpg)
What matters the least?
Visually appealing facilities Employees who appear to enjoy
what they do
![Page 56: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/56.jpg)
What matters the most?
Accuracy in the catalogue, borrowing and overdue records
![Page 57: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/57.jpg)
What services are perceived as among the worst?
A meditative place Haven for quiet & solitude Complete runs of journal titles
![Page 58: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/58.jpg)
What service is perceived as the best?
“A library website enabling me to locate information on my own”
UW UG Perception 7.59 7.47 Minimum 7.04 7.17 Desired 8.46 8.47
![Page 59: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/59.jpg)
Area 3: Data models “Same old” and “not so same old” tables vs. zone and radar charts Radar Charts
Green: GreatYellow: Very GoodBlue: I can accept thatRed: Danger
![Page 60: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/60.jpg)
Radar chart / sample
![Page 61: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/61.jpg)
![Page 62: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/62.jpg)
“in-the-red” University
![Page 63: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/63.jpg)
“Lotsa Quality” University
![Page 64: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/64.jpg)
Undergraduates
![Page 65: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/65.jpg)
Faculty
![Page 66: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/66.jpg)
What next?
Updates and reports to various groups More in-depth look at the data (SPSS) Compare & contrast best practices Investigate possible problem areas:
follow-up focus groups, etc.)
![Page 67: Service Quality Assessment in Research Libraries The Guelph/Waterloo ARL LibQUAL Projects](https://reader036.fdocuments.us/reader036/viewer/2022062519/56815323550346895dc1499d/html5/thumbnails/67.jpg)
Mark Haslett [email protected]
519-888-4567 x 3568
Ron MacKinnon [email protected]
519-824-4120 x 3422
Susan Routliffe [email protected]
519-888-4567 x 3312
ARL LibQUALwww.arl.org/libqual/