ESP in the UK: from assessment to action
-
Upload
stephen-town -
Category
Education
-
view
203 -
download
0
description
Transcript of ESP in the UK: from assessment to action
![Page 1: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/1.jpg)
ESP in the UK:from assessment to action
J. Stephen Town, Steve Hiller, Jim Self and Martha Kyrillidou
![Page 2: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/2.jpg)
Summary
• Background to the program• The University of York• The ESP process in York• Findings• Actions • Observations and conclusions
![Page 3: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/3.jpg)
Background to the program
![Page 4: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/4.jpg)
ARL Supports the Assessment Community
• Tools– ARL Statistics– LibQUAL+®– MINES for Libraries®– ClimateQUAL™– ARL Library Scorecard Initiative (2009-)
• Building a Community of Practice– Library Assessment Conferences– Service Quality Evaluation Academy– Library Assessment blog– Workshops
• Individual Library Consultation (Jim and Steve) – Effective, Sustainable, Practical Library
Assessment
![Page 5: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/5.jpg)
Key Reasons for Developing an Assessment Consulting Service
(2005-)• LibQUAL+® results – what to do with them• E-Metrics data – how to understand them• New emphasis on outcomes-based assessment
from accreditation agencies and associations• Data driven university administrations• Establishing sustainable assessment programs• Article by Jim and Steve, “From Measurement to
Management” . . . Library Trends , Summer 2004, highlighted issues involved with data collection, analysis and use (or non-use) in libraries.
![Page 6: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/6.jpg)
ARL Assessment Consulting Service • Began in 2005 as “Making Library Assessment Work”
(MLAW) Assess the state of assessment efforts in individual research libraries, identify barriers and facilitators of assessment, and devise pragmatic approaches to assessment that can flourish in different local environments”
• Funded by participating libraries; limited to ARL members• Conducted by Steve Hiller and Jim Self under the aegis of
Martha Kyrillidou of ARL• In 2007 changed to “Effective, Sustainable and Practical
Library Assessment” (ESP) and opened up to all libraries• 41 libraries visited since February 2005
– 38 in North America (32 ARL libraries)– 3 others in Israel, South Africa and UK
![Page 7: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/7.jpg)
ESP Insights• Strong interest in using assessment to improve • Uncertainty on how to establish and sustain
assessment• Lack of assessment knowledge among staff• Underutilization of campus assessment resources• More data collection than data utilization• Effectiveness not dependent on library size or
budget• Each library has a unique culture and mission• Organizational issues play a significant role in
sustainable assessment
![Page 8: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/8.jpg)
Rationale
• http://www.arl.org/stats/initiatives/esp/• Those starting, prioritising or
embedding assessment activities• Evaluating current activities• Accreditation requirements• Developing a quality culture• LibQUAL+ active
![Page 9: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/9.jpg)
The University of York
and its Library & Archives
![Page 10: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/10.jpg)
The University
• Founded 1963• UK top ten; RAE 8th;
World 81st
• 11,000 students• >30 departments in
humanities, social sciences, science
• Campus growth• Collegiate and inclusive
![Page 11: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/11.jpg)
The Library & Archives
• > 1m items• >100 staff• Traditional divisions• Archives extensive &
unique• Developing digital
library expertise• New Director 2007
![Page 12: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/12.jpg)
The ESP process in York
![Page 13: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/13.jpg)
Sustainable assessment week 21-29 Jun 2008
• LibQUAL+ Results meeting• International benchmarking seminar• ESP Dinner with senior University staff• Senior Management Team session
– UK, University and quality contexts, including 2008 survey results
• Team sessions with Library divisions (4)• Future initiatives
– SPEC Kit; WUN collaboration
![Page 14: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/14.jpg)
Process observations
• No cultural issues, except …• Compact and efficient process• Commitment and insight• Similar to traditional UK consultancy or
peer review, but …
![Page 15: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/15.jpg)
Post visit
• Report delivered and finalised promptly• Incorporation into Library planning cycle• Feedback to broader University• Follow up value for more remote
participants?
![Page 16: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/16.jpg)
Findings
![Page 17: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/17.jpg)
Current position
• Budget issues• National Student Survey results
disappointing• Inadequate reporting of survey results• Assessment not embedded• Culture values precedent and regulation• LibQUAL+ scores low
![Page 18: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/18.jpg)
Suggestions and options 1
1. Practices and procedures for assessment1. Assessment group and leadership2. Training3. Culture
2. An Assessment plan3. Culture promotion
1. From budget to real costs2. From regulation to user perspective3. From risk aversion to ‘why not?’
![Page 19: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/19.jpg)
Suggestions and options 2
4. Processes review1. Classification system2. Acquisition process review
5. LibQUAL+ results follow-up6. Review collection development and
liaison7. Seek areas of collaboration between
Library and the Archives
![Page 20: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/20.jpg)
Actions and achievements
![Page 21: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/21.jpg)
Progress 1
• Culture change and promotion– Middle management training & quality focus– University Staff survey 2009– All staff Awayday 2009 (User perspectives on library value)– Project styles
• Process review– Redesign of acquisition process to remove repeat touches
and increase trust across organisational boundaries– Reclassification proposal under consideration; supported by
University senior managers
![Page 22: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/22.jpg)
Progress 2
• LibQUAL+ follow up and embedding– 2009 survey, results and breakdowns– Devolvement to proto-evaluation group– Improved results across almost all items and disciplines
• NSS results improvement– Moved from 78% overall to 81%– Met first improvement target
• Academic liaison transformation– Additional substantial investment– Role development and career paths
![Page 23: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/23.jpg)
University of YorkLibQUAL+ 2008 to 2009 trends
• Superiority mean scores improved across all but one of the 22 core and 5 local questions
• The other item remained constant
• Discipline adequacy scores improved in all but six cases
• Information control overall adequacy score moved from negative to positive
• Substantial improvements in convenient service hours and online course support items (the latter also out of the red)
![Page 24: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/24.jpg)
Confounding factors
• Departure of performance measurement lead and hiatus during replacement– Leadership and assessment group– In house capability and training
• Major library refurbishment program• Creation of Information directorate and
closer working with IT Services
![Page 25: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/25.jpg)
Observations and conclusions
![Page 26: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/26.jpg)
Benefits
• External view from professional experts• Succinct and compelling report• Broader scope than assessment alone• Specific recommendations outside the
comfort zone• Institutional and organisational
acceptance• Same language
![Page 27: ESP in the UK: from assessment to action](https://reader035.fdocuments.us/reader035/viewer/2022081505/5537e22a4a795977028b4629/html5/thumbnails/27.jpg)
Conclusions
• ESP is transferable to institutions outside North America
• The US origin can be a positive factor within aspirant world class institutions
• Practical and strategic insight very strong (and unexpected?)
• Successful in this case in contributing to culture, community and value