LibQUAL+ ® in South Africa Sherrie Schmidt, Arizona State and ARL President Martha Kyrillidou, ARL...
Transcript of LibQUAL+ ® in South Africa Sherrie Schmidt, Arizona State and ARL President Martha Kyrillidou, ARL...
LibQUAL+LibQUAL+®® in South Africa in South Africa
Sherrie Schmidt, Arizona State and ARL PresidentSherrie Schmidt, Arizona State and ARL President Martha Kyrillidou, ARL Martha Kyrillidou, ARL
Stellenbosch, South AfricaAugust 13, 2007
old.libqual.org
old.libqual.org
OverviewOverview
• Introduction and Overview
• LibQUAL+® in South Africa
• The importance of LibQUAL® for ARL Statistics and Measurement Program
• LibQUAL+® in the UK
• LibQUAL+® at Arizona State
• LibQUAL+® at the U. of Virginia
• Share Fair
• What have we learned?
old.libqual.org
old.libqual.org
old.libqual.org
old.libqual.org
World World LibQUALLibQUAL++®® Survey Survey
old.libqual.org
Mission: Shaping the future of research libraries in the changing environment of public policy and scholarly communication.
Members: 123 major research libraries in North America.
Ratios: 4 percent of the higher education institutions providing 40 percent of the information
resources.
Users: Three million students and faculty served.
Expenditures: 37 percent is invested in access to electronic resources.
old.libqual.org
ARL Mission
• Non-profit organization of the libraries of research institutions in North America
• Forum for exchange of ideas
• Agent for collective action
old.libqual.org
. . . chart a course for every endeavor that we take the people's money for, see how well we are progressing, tell the public how we are doing, stop the things that don't work, and never stop improving the things that we think are worth investing in.
– President William J. Clinton, on signing the Government Performance and Results Act of 1993
Assessment – the Imperative
old.libqual.org
Assessment – the ChallengeAssessment – the Challenge
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Library Assessment at ARL
What are some of the developments with library assessments efforts?
ARL StatsQUAL™
E-Metrics
LibQUAL+®
DigiQUAL™
MINES for Libraries™
Where are the most critical needs and opportunities?
What are the lessons learned?
old.libqual.org
LibQUAL+ LibQUAL+ TM Brief History Brief History
• Experience with SERVQUAL in many libraries in the 1990s• Texas A&M SERVQUAL assessment in the 1990s• New Measures Initiative called a meeting of interested ARL libraries
(ALA Midwinter 2000)• Pilot with 12 ARL libraries (spring 2000)• External funding through FIPSE, U.S. Department of Education
(September 2000)• Participation and endorsement by consortia (Greater Midwestern Big
12 Plus Consortium, OhioLINK, etc.)• Consortia and related associations interest• 2003 UK participation• 2005 South Africa• More than 200 institutions every year
old.libqual.org
Rapid GrowthRapid Growth
• Languages– American English– British English– Chinese (2007)– French– Dutch– Swedish– Norwegian– Finnish– Danish
• Consortia– Each may create 5 local questions
to add to their survey
• Countries– Australia, Canada, Denmark,
Egypt, Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.
• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– Electronic– European Business– Family History– FFRDC – High School (2007)– Hospital– National Health Service England– Natural Resources– New York Public– Public– Smithsonian– State– University/TAFE
old.libqual.org
250
218
307308
204
164
13
43
152,111
176,360
151,460
113,480
78,863
4,407
20,416
128,958
0
50
100
150
200
250
300
350
2000 2001 2002 2003 2004 2005 2006 2007
Survey Year
Number ofInstitutions
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
180,000
200,000
Number ofResponses
Number of Institutions
Number of Responses
LibQUALLibQUAL++®® Participants Participants
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Languages Languages
American English
Dutch EnglishFrench Canadian DutchSwedish
Swedish(British English)
Afrikaans
DanishFinnishGerman Norwegian
British English
Continental French
old.libqual.org
Why LibQUAL+?
• To base decisions on data– Internally– Externally– Comparative– Longitudinally
• Cost effective• Timely data• Support available• Community building• Sharing of experiences• Learning and improvement
old.libqual.org
13 LibrariesEnglish LibQUAL+™ Version
4000 Respondents
QUAL
QUAN
QUAL
QUAL
QUAN
QUAL
PURPOSE DATA ANALYSIS PRODUCT/RESULTDescribe library environment;build theory of library service quality from user perspective
Test LibQUAL+™ instrument
Refine theoryof service quality
Refine LibQUAL+™ instrument
Test LibQUAL+™ instrument
Refine theory
Unstructured interviewsat 8 ARL institutions
Web-delivered survey
Unstructured interviews at Health Sciences and the Smithsonian libraries
E-mail to surveyadministrators
Web-delivered survey
Focus groups
Content analysis:(cards & Atlas TI)
Reliability/validityanalyses: CronbachsAlpha, factor analysis,SEM, descriptive statistics
Content analysis
Content analysis
Reliability/validity analyses including Cronbachs Alpha,factor analysis, SEM, descriptive statistics
Content analysis
VignetteRe-tooling
Iterative
Emergent2000
2004315 Libraries English, Dutch, Swedish,
German LibQUAL+™ Versions160,000 anticipated respondents
LibQUAL+LibQUAL+®® Project Project
Case studies1
Valid LibQUAL+™ protocol
Scalable process
Enhanced understanding of user-centered views of service quality in the library environment2
Cultural perspective3
Refined survey delivery process and theory of service quality4
Refined LibQUAL+™ instrument5
Local contextual understanding of LibQUAL+™ survey responses6
old.libqual.org
Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
old.libqual.org
Premises
Three Seminal Quotations
old.libqual.org
PERCEPTIONS SERVICE “….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.
LibQUAL+™ Premise #1
old.libqual.org
LibQUAL+™ Premise #2
“Il est plus nécessaire d'étudier
les hommes que les livres”
—FRANÇOIS DE LA ROCHEFOUCAULD
old.libqual.org
“We only care about the things we measure.”
--Bruce Thompson, CASLIN, 2006
LibQUAL+™ Premise #3
old.libqual.org
Extended GAPS ModelOrganizational Barriers to SQ Customers’ Assessment of SQ
Poor UpwardCommunication
Poor HorizontalCommunication
Poor Tech - JobFit
Perception ofInfeasibility
GAP 1
GAP 2
GAP 3
GAP 4
GAP 5
Reliability
Responsiveness
Assurance
Empathy
Tangibles
old.libqual.org
DimensionsDimensions
2000 2001 2002 2003-200741 items 56 items 25 items 22 items
Affect of Service Affect of Service Service Affect Service Affect
Library as Place Library as Place Library as Place Library as Place
Reliability Reliability Personal ControlInformation Control
Provision of Physical Collections
Self-RelianceInformation Access
Access to Information
Access to Information
old.libqual.org
Survey Structure Survey Structure (Detail View)(Detail View)
old.libqual.org
Interpreting Service Quality Data
Three Interpretation Frameworks
old.libqual.org
Benchmarking Against Peer Institutions
--1,000,000 Users; 1,000 Institutions!
NORMS! NORMS! NORMS!
Interpretation Framework #1
old.libqual.org
Score Norms
• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.
• LibQUAL+™ norms have been created at both the individual and institutional level
old.libqual.org
Benchmarking Against Self, Longitudinally
“Nobody is more like me than me!”--Anonymous
Interpretation Framework #2
old.libqual.org
Interpreting Perceived Scores Against Minimally-Acceptable and Desired
Service Levels (i.e., “Zones of Tolerance”)
Interpretation Framework #3
old.libqual.org
old.libqual.org
old.libqual.org
“22 Items and The Box….”
Why the Box is so Important– About 40% of participants provide open-
ended comments, and these are linked to demographics and quantitative data.
– Users elaborate the details of their concerns.
– Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.
old.libqual.org
“…and Five Ancillary Items”
Either Zero or Five Ancillary items are selected to address local or consortial concerns
– Items from the initial LibQUAL+TM item pool.
– Items written by previous consortial groups.
old.libqual.org
LibQUAL+LibQUAL+®® Resources Resources
• LibQUAL+® Web site:http://old.libqual.org
• Publications:http://old.libqual.org/publications
• Events and Training:http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm