Keynote 2: Combining programme and
institutional aspects in QA: U.K.
Nick Harris
QAA - UK
+ some thoughts on the BEST to TEST
dimension 1: the HE ‘system’ - does size matter?
a bit of UK geography
Scotland 19
Northern Ireland 2
Wales 13
England 134
London
Manchester
and also - does diversity within the system matter ?
National variants of QA within UK
England (c 134 HEIs)Institutional audit (6 year cycle with mid point check)
Scotland (c 19 HEIs)Enhancement-led institutional review (ELIR)peer review panel includes students
Wales (c 13 HEIs)Developmental review in Wales
Northern Ireland (2 HEIs)as England
‘review’ is used to indicate less emphasis on ‘accountability’ but more on improvement
dimension 2: the legal context matters
Higher Education in the UK is ‘different’
• no national degrees .. but use of word regulated• no requirement for accreditation .. but .. • formal external QA since 1988
degrees are awarded by autonomous HE institutions• HE institutions (universities and univ colleges are responsible for
• the standard of their degrees and • the quality of their study programmes
Bachelors and Masters system VERY well established– Bachelors (3 or 4 years) are employable– Masters (1 or 2 years) for specific purposes
dimension 3i: maturity mattersthe agencies we are where we are because of the journey travelled
pre-1988 University quality & standards unregulated
1990 CVCP (UUK) Academic audit
1992 Further and Higher Education Act
HEQC (audit) + HEFCE (subject review)
1996 Major review of HE (Dearing / Garrick reports)
1997 i) QAA (combining HEQC, HEFCE unit, Wales, Northern Ireland and Scotland QA offices
ii) development of explicit basis for quality and standards to be shared by HE and others
2000/1 introduction of revised QA ‘framework’
(external subject review ended; ‘new QAA’)
dimension 3ii: maturity and diversity mattersinstitutions are not all the same (they are where they are because of the journey travelled)
pre-1960s the ‘old’ and the ‘ancient’ universities
1960s expansion phase 1 .. ‘conversions’ and ‘new’ ‘universities’ still a ‘binary system +’
- polytechnics; teacher training colleges; art colleges;
the QA universities – External Examiners + some internal QA systems polys – External QA system requiring internal + Ex Ex
1970s expansion phase 2 Polys converted to autonomous universities
1990s expansion phase 3 colleges etc become university colleges
2000s expansion phase 4 degree awarding powers for private sector lot of HE delivered by Further Education Colleges
Dimension 4: the purposes of QA
In the UK the purposes of QA are to:• secure academic standards of qualifications• secure quality of learning opportunities• accountability (including for public funding of HE)• provision of information about quality and standards• enhance students’ learning opportunities and
experience
QAA’s mission:… is to safeguard the public interest in sound standards
of higher education qualifications and to inform and encourage continuous improvement in the management of the quality of higher education.
Dimension 5: QA procedures
a brief history of QA in UK HE
internal programme scrutinyexternal examiners at programme/subject level – appointed by institution
there since ? for ever .. and still a key part of QA
external institutional audit through different bodies / different guises since 1980s
external programme scrutiny through different bodies / different guises from 1980s to 2001
phase i) judgement as descriptionphase ii) judgement as numerical score
Dimension 5: QA procedures and frameworks
Current UK Quality Assurance Framework
• regular internal institutional arrangements including• student surveys (at programme / module level)• external examiners (programme / module level)
• occasional external audit / review .. by peers• published information about quality and standards in
individual institutions
• ‘Academic Infrastructure’ ‘the reference points’ for ALL academic standards and QA
procedures
• some accreditation linked to professional practice
• published national level student surveys (NSS)
• national collation of statistical returns (HESA)
Academic Infrastructure (AI) used across the UK:applied irrespective of a specific QA procedure
Framework for HE qualifications (national agreement)
subject benchmark statement
(subject communities)
programme specification(institutional documents)
diploma supplement (UK transcript)(formal record of each student’s achievement)
setting out standards
using the Academic Infrastructure (AI)
Framework for HE qualifications (national agreement)
subject benchmark statement
(subject community)
programme specification(institutional document)
diploma supplement (UK transcript)(student record)
Code of Practice
quality m
anagement
the Code of Practice
the sections cover quality management for ‘key stages’:
• Programme design, approval, monitoring and review ( 7)
• Admissions to higher education (10)
• Collaborative provision and flexible and distributed learning (including e-learning) ( 2)
• Students with disabilities ( 3)
• Work-based and Placement learning ( 9)
• Assessment of students ( 6)
• External examining ( 4)
• Academic appeals and student complaints on academic matters ( 5)
• Career education, information and guidance ( 8)
• Postgraduate research programmes ( 1)
Also
Accreditation of Prior and Experiential Learning - guidelines
[ Credit guidelines ]
Briefing visit
Briefing paper, SWS, TQI, published reports
Audit visit
Audit trails
Report
and its Annex
Between visits
Last day of audit visit
Current QA procedure in England:Institutional Audit
Audit trails
Sampling trails – • normally based on the institution’s own way of managing and
reviewing its academic standards and quality of provision
Supplementary trails – • of particular topics proposed by the institution and accepted by
the audit team• there is no expectation that an audit will necessarily include a
supplementary trail• is made by the audit team at the briefing visit, following
discussion with the institution
Selection of audit trails• will bear in mind the value added to the audit process by
undertaking the trail• will also bear in mind that trails are a means of gathering
evidence, not an end in themselves• their findings are not reported separately but they contribute to
the evidence on which the team draws for its findings, judgements and comments
On the audit trail ...
A
B
Documents about the audit trail
Use made of AI Code of practice, FHEQ, Benchmark Statements Programme Specifications
Student evaluation Student liaisonInternal review – how well
does it work for assurance and enhancement ?
Handbooks - good information?
Evaluation of learning support & resources
Students’ understanding of expectations
Assessment policies in practice? Match with level of award?
Progression data – used well?
View of institution’s policies and processes in operation
The judgements
• 'the confidence that can reasonably be placed in the soundness of the institution’s present and likely future management of the academic standards of its awards‘
• 'the confidence that can reasonably be placed in the soundness of the institution’s present and likely future management of the quality of the learning opportunities available to students‘
3 levels• Confidence most common• Limited confidence occasional – serious • No confidence rare – VERY SERIOUS
Features of good practice & Recommendations
Features of good practice - in the context of the institution• that make a particularly positive contribution to the
institution’s approach to the management of the security of academic standards and the quality of provision
Recommendations• essential – currently putting quality and/or academic
standards at risk – require urgent corrective action
• advisable – have the potential to put quality and/or academic standards at risk – require preventative or corrective action
• desirable – have the potential to enhance quality of learning opportunities and/or further secure the academic standards of awards
dimension 6: the ethos .. (avoiding the ‘quality wars’)
in the UK ethos underpinning QA (now) encompasses:• collegial / partnership• respectful of mission• respectful of institution’s autonomy• prepared to celebrate good practice• honest and open• judgements based on evidence
encapsulated in the six questions…• what are you trying to do? purposes• why are you doing it? reasons• how are you going to do it? methods• why are you doing it that way? optimisation• how do you know it works? effectiveness• how can you improve it? enhancement
The reports … are not ‘the end’ • Summary - judgements, commentaries, recommendations,
features of good practice
• Report – seven section headings
• Annex to the report- same section headings as in the report but comprising
‘technical’ details
sharing the information in the audit reports and QAOutcomes from ..
thematic papers drawing on evidence in audit reports
ELIR themes reports in Scotland
Different contexts .. different proceduresbut a similar ‘direction of travel’
‘self regulation’
[ external
quality ‘control’ ]
external quality assurance
internal QA with externality
‘risk based’ QA
Austria
Germany
Norway
Switzerland
UK
Netherlands
/ Flanders ?
Different contexts .. different proceduresbut a similar ‘direction of travel’
‘self regulation’
[external]
quality ‘control’
external quality assurance
internal QA with externality
‘risk based’ QA
Austria
Germany
Norway
Switzerland
UK
Netherlands
/ Flanders ?
TRUST / RESPONSIBILITY
Building trust .. gaining responsibility shared purposes and principles context dependent procedures
Purposes – must be clear and shared• Public accountability for standards and quality• Responsibility for improvement
Principles …. must be related to ESG• Same principles apply to all types and sizes of institutions• Transparency is essential• Must underpin policies and procedures
Procedures• Depend upon context but• MUST produce clear evidence – available to all
Some practical issues ..
Building trust in self regulation– has to be progressive, to an agreed plan, – with clearly identified ‘benefits’ – and ‘checks’
a quality culture– changing the ethos and hard work – progressive but to a clear plan
from quality assurance to enhancement– doesn’t mean a lower level of assurance– does mean a different emphasis– should build to a ‘virtuous circle’
‘Light touch’– doesn’t mean ‘soft touch’– is really about ‘shift of touch’ .. less external / more internal !
Some questions?
What are the real (shared) purpose (s)?
How are the (necessary) standards agreed ?
Can / should reports be public?
Who selects / how are ‘the experts’ selected ?
How do procedures deal with heterogeneity?
Can we avoid procedure become predominant?
What about ‘ranking’?
Why bother? Data, information (?) and knowledge
do I stop ..
or ..
do I go .. ?
Top Related