Value, Impact and Performance: the SCONUL VAMP Programme
description
Transcript of Value, Impact and Performance: the SCONUL VAMP Programme
![Page 1: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/1.jpg)
Value, Impact and Performance:the SCONUL VAMP Programme
Stephen TownCranfield University
Open University
Wednesday 27th June, 2007
![Page 2: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/2.jpg)
Summary
• An introduction to the issues
• The Value & Impact Measurement Program (“VAMP”)
• The VAMP Deliverables– A Community of Practice in Library Measurement– The “Performance Portal”
![Page 3: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/3.jpg)
The University Context (from the Library Assessment Conference,
Charlottesville, Va, September 2006)
Universities have two “bottom lines”
1. Financial (as in business)
2. Academic, largely through reputation in• Research (the priority in “leading” Universities)• Teaching (& maybe Learning)
![Page 4: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/4.jpg)
Library Pressures for Accountability
The need is therefore to demonstrate the Library contribution in these two dimensions:
1. Financial, through “value for money” or related measures
2. Impact on research, teaching and learning
This also implies that “competitive” data will be highly valued
![Page 5: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/5.jpg)
Cautions for measurement
The Aim & Role of Universities & their Libraries
• Research– ‘Mode 1’ Research & impact ‘transcendental’– ‘Mode 2’ Research & impact ‘instrumental’
• Reductionism– Value, Price & ‘Mandarinisation’ of research and its support– Libraries as symbols and ‘transcendent’ services
![Page 6: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/6.jpg)
The SCONUL Experience
The SCONUL Working Group on Performance Improvement
• Ten years of “toolkit” development to assist in performance measurement and improvement
• SCONUL ‘Top concern survey’ 2005, suggested inability to prove value, impact or worth, leading to VAMP
![Page 7: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/7.jpg)
Examples of tools developed 1
• Integration
• Efficiency & Comparability
Quality assurance Guidelines
SCONUL Statistics & interactive service
HELMS national performance indicators
E-measures projectBenchmarking Manual
![Page 8: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/8.jpg)
Examples of tools developed 2
• Satisfaction
• Impact
SCONUL Satisfaction Survey
SCONUL LibQUAL+ Consortium
LIRG/SCONUL Impact Initiative
Information Literacy Success Factors
![Page 9: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/9.jpg)
VAMP Objectives
• New missing measurement instruments & frameworks
• A full coherent framework for performance, improvement and innovation
• Persuasive data for University Senior Managers, to prove value, impact, comparability, and worth
![Page 10: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/10.jpg)
Missing methods?
• An impact tool or tools, for both teaching & learning and research (from the LIRG/SCONUL initiative?)
• A robust Value for Money/Economic Impact tool
• Staff measures• Process & operational costing tools
![Page 11: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/11.jpg)
Benefits?
1. Attainment & retention of Library institutional income
2. Proof of value and impact on education and research
3. Evidence of comparability with peer institutions4. Justification of a continuing role for libraries
and their staff5. Meeting national costing requirements for
separating spend on teaching and research
![Page 12: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/12.jpg)
VAMP Project Structure
• Phase 1 (March-June 2006)– Critical review– SCONUL Members Survey– Gap analysis & synthesis– SCONUL Conference Workshops
• Phases 2 & 3 (July 2006 - June 2007)– Development of new measures & techniques– Review and re-branding of existing tools– Web site development– Dissemination & maintenance strategy
![Page 13: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/13.jpg)
Critical Review Method
Review of:
• SCONUL initiated or promoted services• Other UK & European initiatives• Initiatives from other UK library sectors• International initiatives
starting from the perspective of ‘The Effective Academic Library’, 1995
![Page 14: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/14.jpg)
Review Findings
• The Impact Initiative work will be key, but needs to be solidified and embedded
• Eight JISC/EU projects in the last eight years relevant to the assessment of value and impact, many relating to e-resources
• Significant work in the USA, Australia and South Africa
![Page 15: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/15.jpg)
Review Conclusions
• Vast amount of relevant work, but without wide take up
• More critical analysis required of most products and tools
• Further development and simplification required to create credible and applicable instruments for SCONUL members
![Page 16: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/16.jpg)
Member Survey Findings
• 38 respondents; 27% of population• 70% undertaken value or impact measurement• Main rationales are advocacy, service
improvement, comparison• Half used in-house methodologies; half used
standard techniques• Main barrier is lack of tools, making time an issue• Buy-in of stakeholders is an issue
![Page 17: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/17.jpg)
Member Survey Conclusions
• There is a need to demonstrate value and that libraries make a difference
• Measurement needs to show ‘real’ value• Need to link to University mission• Libraries are, and intend to be, ahead of the game• Impact may be difficult or impossible to measure• All respondents welcomed the programme, and
the prospect of an available toolkit
![Page 18: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/18.jpg)
Synthesis
• Terminological confusion?• Is ‘impact’ = ‘effect’ or to ‘outcome’• ‘Higher order effects’ and level
– Individual, course, institutional, vocational, societal, national, international
• ‘Value and impact’ are not an item?• ‘Value’, ‘adding value’, ‘value for
money’, ‘cost-effectiveness’
![Page 19: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/19.jpg)
Expert Comment
• Return to why?– Advocacy or management?
• Critical gap in measurement– Effect of the Library on educational attainment– Effect of the Library on research attainment
• Robust and simple tools• Only a few existing tools effective, so
simplify and focus the range of offerings
![Page 20: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/20.jpg)
SCONUL Conference Workshops
• Accountability to a variety of structures and individuals… therefore a range of approaches required
• SCONUL Statistics heavily used• Directors want help with all VAMP lines• Pedagogic “Big project” needed?• Re-engineer processes rather than measure!
![Page 21: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/21.jpg)
Overall conclusions
• Wider sectoral involvement?– Health Evidence-based methods– National MLA Measures– British Library Contingent Valuation
• Not only new measures… but also supporting & directing processes
![Page 22: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/22.jpg)
Deliverable Plan 1
“Content” Products
2.1 Value & Impact Guidelines
2.1.1 Institutional Value (eg VFM & Economic Impact)
2.1.2 Impact on Teaching & Learning2.1.3 Impact on Research
![Page 23: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/23.jpg)
Deliverable Plan 2
“Content” Products
2.2 Staffing & Operational Measures Guidelines
2.2.1 Staff Costing2.2.2 Staff Added Value measures2.2.3 Other operational costing methods
2.3 Re-branding & packaging of existing tools
![Page 24: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/24.jpg)
Deliverable Plan 3
“Process” Products
3.1 Web Site
3.2 Community of practice establishment
3.3 Maintenance & sustainability strategy
![Page 25: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/25.jpg)
Progress on Content 1
2.1.1 Institutional Value (eg VFM & Economic Impact)
VFM tool in negotiation now with 2.2Contingent Valuation methods available
2.1.2 Impact on Teaching & Learning2.1.3 Impact on Research
Tool for both areas delivered (Information Management Associates)
![Page 26: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/26.jpg)
Progress on Content 2
2.2.1 Staff Costing2.2.2 Staff Added Value measures2.2.3 Other operational costing methods
Costing & value method in negotiation‘Transparency’ Meeting May 2007
2.3 Re-branding & packaging of existing tools
Mainly included within the Web Site development
![Page 27: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/27.jpg)
Progress on Process
3.1 Web Site
Portal to be launched in June 2007
3.2 Community of practice establishment
Invitations for beta testing of site May 2007
![Page 28: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/28.jpg)
Communities of Practice
“groups of people who share a passion for something that they know how to do,and who interact regularly to learn how to do it better”
“coherence through mutual engagement”
Etienne Wenger, 1998 & 2002
![Page 29: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/29.jpg)
Member’s Forum(Blog?Chat?)
Techniques in Use(Wiki?)
VAMP Home Page
SimpleIntroductio
ns
DetailedTechniques
Community of Practice
Techniques
![Page 30: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/30.jpg)
The ‘Performance Portal’
• A Wiki of library performance measurement containing a number of ‘approaches’, each (hopefully) with:– A definition– A method or methods– Some experience of their use in libraries (or links to
this)– The opportunity to discuss use
![Page 31: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/31.jpg)
![Page 32: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/32.jpg)
![Page 33: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/33.jpg)
![Page 34: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/34.jpg)
![Page 35: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/35.jpg)
The Ontology of Performance
• ‘Frameworks’• ‘Impact’• ‘Quality’• ‘Statistics’• ‘Value’
![Page 36: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/36.jpg)
![Page 37: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/37.jpg)
Frameworks
Mounted
• European Framework for Quality Management (EFQM)
Desired
• Key Performance Indicators
• The Balanced Scorecard
• Critical Success Factors• The Effective Academic
Library
![Page 38: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/38.jpg)
Impact
Mounted
• Impact tools
Desired
• Detailed UK experience from LIRG/SCONUL Initiatives
• Outcome based evaluation
• Information Literacy measurement
• More on research impact
![Page 39: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/39.jpg)
![Page 40: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/40.jpg)
Quality
Mounted
• Charter Mark• Customer Surveys
– LibQUAL+– SCONUl Survey– Priority Research
• Investors in People
Desired
• Benchmarking• Quality Assurance• ISO 9000s• ‘Investors in People’
experience• Opinion meters• Quality Maturity Model
![Page 41: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/41.jpg)
![Page 42: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/42.jpg)
Statistics
Mounted
• SCONUL Statistics & interactive service
• HELMS statistics
Desired
• Institutional experience of using SCONUL statistics for local advocacy
• COUNTER• E-resource tools
![Page 43: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/43.jpg)
Value
Mounted Desired
• Contingent valuation• ‘Transparency’ costing• Staff & process
costing, value & contribution
• E-resource value tools
![Page 44: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/44.jpg)
Discussion Tools
• An experiment in social networking & Web 2.0 technologies
![Page 45: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/45.jpg)
![Page 46: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/46.jpg)
Acknowledgments
• Angela Conyers, Evidence Base, UCE
• Claire Creaser & Suzanne Lockyer, LISU, Loughborough University
• Professor Peter Brophy, Manchester Metropolitan University
• The VAMP Subgroup of SCONUL WGPIMaxine Melling, Philip Payne, Rupert Wood
• The Cranfield VAMP Team, Darien Rossiter, Michael Davis, Selena Lock, Heather Regan
![Page 47: Value, Impact and Performance: the SCONUL VAMP Programme](https://reader035.fdocuments.us/reader035/viewer/2022062305/56814cee550346895db9ee81/html5/thumbnails/47.jpg)