Peer evaluation in further education colleges : shaping the quality agenda from within….
description
Transcript of Peer evaluation in further education colleges : shaping the quality agenda from within….
Peer evaluation in further education colleges : shaping the quality agenda from within….
Steve Cropper, Senior Librarian, Wirral Metropolitan College
What this session is about: I am not here to sell you anything! This is about an initiative which started
in a very small way and has now grown beyond our expectations
I want to show you how it works at a practical level
The learning objective is that you go away from here sufficiently interested in peer evaluation to want to try it for yourselves
The Circle of Merseyside College Librarians: Founded 27 years ago Members from 6 major FE providers on
Merseyside and two 6th form colleges Strong links with North Wales and
Cheshire Lively, well-attended meetings each term Annual benchmarking on key
performance indicators
The Peer Evaluation Scheme Has carried out 24 evaluations to date In Lancashire: Blackpool and the Fylde
College and Runshaw College In Cheshire: Widnes 6th Form College
and West Cheshire College In North Wales: Denbigh and Rhyl
Community Colleges and Coleg Llandrillo
Lancashire and North Wales have provided Evaluators for the scheme
The benefits of reciprocity: a broader perspective on quality
A knowledge of the Scottish FEFC standards and how they are applied.
Knowledge of the SLIC toolkit as used in Scotland.
Knowledge of the ESTYN standards as applied in Wales, and the fforwm toolkit.
How we began:
A brief introduction to how peer evaluation began on Merseyside and why
The climate that gave rise to peer evaluation: Many libraries struggling with inadequate
resources and management indifference An inspection regime for which we had
high hopes delivered very little Unqualified inspectors passing
judgement on libraries Inspection process too easy to subvert The arrival of the dreaded SAR!
That was then, this is now: The summer of 1999, “meltdown”
arrives… More businesslike, but how much has the
underlying culture really changed? Inspections in England and Scotland now
consider libraries only in the context of the curriculum area inspections.
We are still being inspected by the unqualified.
Inspection process is still being stage-managed to death.
How do we respond and what we want to achieve? To put libraries back on the inspection
agenda, as central to the learning process To counterbalance ill-informed judgements
with qualified, professional assessment To present inspectors and managers with
thorough, detailed reports from well-qualified independent inspectors that are hard to dispute
Central to our thinking is: That the professional knowledge and
experience required to evaluate college libraries effectively, rests with those who are managing those services every day.
That keeping the process transparent gives it added legitimacy.
That keeping it free places it outside the realm of “consultancy” and available to all.
That it is not a resource-bidding exercise.
How we began to develop the scheme in practice: Small working group set up consisting of
representation from 4 member FE colleges and one 6th form college
Draft framework prepared Emphasis on practical, real-world
approach Written up for CoFHE Bulletin Moved forward quickly to first peer
evaluation
The nitty gritty bit…..
So much for the theory, how does peer evaluation actually work?
First: select your evaluators Evaluations are initiated by a formal
request to a Circle Meeting Two volunteers are then requested who
have no connection with the college to be inspected.
First-timers are always paired with an experienced evaluator
We also call upon colleagues from outside the area, with whom we have reciprocal links
Preparation for the visit:
Request advance documentation eg. SAR, Business plan
Make notes/draft questions arising out of the advance documentation
Set a realistic timetable for the day Arrive at a division of labour Make necessary travel
arrangements
On the day:
Ensure we speak to a range of staff and to students
Strike a balance between interviews and legwork. Go and see for ourselves and be very hands-on
Stick to our own part of the brief and ask the right questions
Writing Up:
This is more time-consuming than you may think. Need to allow adequate time for drafting and re-drafting
Be prepared to re-check points of fact Be helpful to the Librarian, but always
tell the truth Link to local and national standards
Peer evaluation and benchmarking : towards a synthesis
University sector has always had reliable benchmarks against which to assess services
FE sector until recently had only dated/over-ambitious guidelines
Three years ago, as a response to this CMCL began its own annual benchmarking round
Now augmented by realistic new CoFHE national guidelines
SouthportCollege
KnowsleyCommunity College
St.HelensCollege
Hugh Baird College
Wirral Metropolitan College
King George V
West CheshireCollege
Liverpool CommunityCollege
Total expenditure per FTE(resources,stationery,etc)
£9.05 £14.28 £15.91 £16.46 £13.33 £16.36 £18.25 £8.43(£14.50 inc Excellence in Cities Budget)
Resource expenditure per FTE
£7.44 £12.86 £14.29 £11.54 £9.89 £15.56 £13.30 £4.60(£10.72 inc.EIC budget)
Stock per FTE 7.9 7.14 9.06 11.4 8.66 21.1 15.95 7.02
Issues per FTE 8 7 7.9 6.4 6.33 5.8 9.46 8.4
Study spaces per FTE 1:31.5 1:13.25 1:0.03 1:8 1:62 1:19 1:7.94 1:28
Non-professional staff per FTE
1:1928 1:350 1:0.00131 1:457 1:1065 1:428 1:471.91:840.0 IT Support
1:712
Professional staff per FTE 1:1735 1:250 1:0.00067 1:1600 1:1500 1:1222 1:804.3 1:979
Total staff per FTE 1:913 1:146 1:0.00198 1:355 1:372 1:317 1:219.7 1:412
Benchmark
For Recommendation Actual
CoFHE Floor space 1m2 per 10 students 0.5m2 per 10 students
ESTYN Floor space 235m2 per 1000 FTE 228m2 per 1000 FTE
CoFHE Study Places
1 per 90 students 1 per 176 students
ESTYN Study Places
40-50 per 1000 FTE 25 per 1000 FTE
Example of Benchmarking Statistics
On entering the library the overall impression on the ground floor is of a pleasant but ratherCramped space, dominated by a large counter area. The low ceiling height and requirement To keep access routes clear to fire exits reinforces this impression, and there is a need to Review the ration of staff to customer space, given that the library has, by the standard ofMost colleges, generous back-of-house work space.
FEEDBACK:
Completed draft to the Librarian Opportunity to correct matters of
fact
Final Report:
Remember, it will be read by a wider audience
Remember to include an executive summary
Copies to: Librarian, Quality Manager, Principal, Circle archive
What those who have been through the process think: “The peer evaluation provides an assessment of your service by
fellow professionals greater in scope than other reviews or inspections, providing a substantial report useful for QA, external validations and service reviews” (Patrick Cox, Learning Resources Manager, Liverpool Community College)
“ A valuable and encouraging experience, with confidence that the assessors and the process are informed and supportive” (Catriona Martin, Librarian, West Cheshire College)
“The library has been inspected by those qualified to pass judgement and I welcomed the valuable report which has led to improvements” (Cathy Bennett, Librarian, St Helens College)
The Peer Evaluation report is a quality document produced by fellow professionals whose judgment I respect(Sue Haydock, Librarian, Southport College)
In conclusion, the peer evaluation process is: A response to an inadequate inspection
regime that did little to drive up quality Appropriately rigorous, but also
evaluative and supportive Seeks to re-assert the knowledge and
experience of library professionals in the inspection process
An achievable and practical means of assessing the strengths and weaknesses of college library services
What have we gained from all this? We now have reliable, robust quality
procedures that are standard across all Merseyside Colleges
Our activities in this area have given a focus to thinking about quality issues and raised our profile internally and externally
We have benefited from the process and we now know what standards we should aspire to.
Ten Years on, where next?
Evaluation framework continually revised to reflect changes in inspection regime
Evaluation is a continual quadrennial cycle, to show distance travelled
Towards a toolkit for English colleges?
email: [email protected] reading/References
Peer Evaluation : the Merseyside Experience. CoFHE Bulletin, Issue 99, Spring 2003
Resources and Services Supporting Learning : a Service Quality Development Toolkit. SFEFC, November 2003, text available at: http://www.slainte.org.uk/Files/pdf/FEnet/toolkit03.pdf
EYNON, Andrew (ed.) Guidelines for Colleges : Recommendations for Learning Resources. London: Facet, 2005
fforwm. Services Supporting Learning in Wales. fforwm, 2005.