Keynote 2: Combining programme and

institutional aspects in QA: U.K.

Nick Harris

QAA - UK

+ some thoughts on the BEST to TEST

dimension 1: the HE ‘system’ - does size matter?

a bit of UK geography

Scotland 19

Northern Ireland 2

Wales 13

England 134

London

Manchester

and also - does diversity within the system matter ?

National variants of QA within UK

England (c 134 HEIs)Institutional audit (6 year cycle with mid point check)

Scotland (c 19 HEIs)Enhancement-led institutional review (ELIR)peer review panel includes students

Wales (c 13 HEIs)Developmental review in Wales

Northern Ireland (2 HEIs)as England

‘review’ is used to indicate less emphasis on ‘accountability’ but more on improvement

dimension 2: the legal context matters

Higher Education in the UK is ‘different’

• no national degrees .. but use of word regulated• no requirement for accreditation .. but .. • formal external QA since 1988

degrees are awarded by autonomous HE institutions• HE institutions (universities and univ colleges are responsible for

• the standard of their degrees and • the quality of their study programmes

Bachelors and Masters system VERY well established– Bachelors (3 or 4 years) are employable– Masters (1 or 2 years) for specific purposes

dimension 3i: maturity mattersthe agencies we are where we are because of the journey travelled

pre-1988 University quality & standards unregulated

1990 CVCP (UUK) Academic audit

1992 Further and Higher Education Act

HEQC (audit) + HEFCE (subject review)

1996 Major review of HE (Dearing / Garrick reports)

1997 i) QAA (combining HEQC, HEFCE unit, Wales, Northern Ireland and Scotland QA offices

ii) development of explicit basis for quality and standards to be shared by HE and others

2000/1 introduction of revised QA ‘framework’

(external subject review ended; ‘new QAA’)

dimension 3ii: maturity and diversity mattersinstitutions are not all the same (they are where they are because of the journey travelled)

pre-1960s the ‘old’ and the ‘ancient’ universities

1960s expansion phase 1 .. ‘conversions’ and ‘new’ ‘universities’ still a ‘binary system +’

- polytechnics; teacher training colleges; art colleges;

the QA universities – External Examiners + some internal QA systems polys – External QA system requiring internal + Ex Ex

1970s expansion phase 2 Polys converted to autonomous universities

1990s expansion phase 3 colleges etc become university colleges

2000s expansion phase 4 degree awarding powers for private sector lot of HE delivered by Further Education Colleges

Dimension 4: the purposes of QA

In the UK the purposes of QA are to:• secure academic standards of qualifications• secure quality of learning opportunities• accountability (including for public funding of HE)• provision of information about quality and standards• enhance students’ learning opportunities and

experience

QAA’s mission:… is to safeguard the public interest in sound standards

of higher education qualifications and to inform and encourage continuous improvement in the management of the quality of higher education.

Dimension 5: QA procedures

a brief history of QA in UK HE

internal programme scrutinyexternal examiners at programme/subject level – appointed by institution

there since ? for ever .. and still a key part of QA

external institutional audit through different bodies / different guises since 1980s

external programme scrutiny through different bodies / different guises from 1980s to 2001

phase i) judgement as descriptionphase ii) judgement as numerical score

Dimension 5: QA procedures and frameworks

Current UK Quality Assurance Framework

• regular internal institutional arrangements including• student surveys (at programme / module level)• external examiners (programme / module level)

• occasional external audit / review .. by peers• published information about quality and standards in

individual institutions

• ‘Academic Infrastructure’ ‘the reference points’ for ALL academic standards and QA

procedures

• some accreditation linked to professional practice

• published national level student surveys (NSS)

• national collation of statistical returns (HESA)

Academic Infrastructure (AI) used across the UK:applied irrespective of a specific QA procedure

Framework for HE qualifications (national agreement)

subject benchmark statement

(subject communities)

programme specification(institutional documents)

diploma supplement (UK transcript)(formal record of each student’s achievement)

setting out standards

using the Academic Infrastructure (AI)

Framework for HE qualifications (national agreement)

subject benchmark statement

(subject community)

programme specification(institutional document)

diploma supplement (UK transcript)(student record)

Code of Practice

quality m

anagement

the Code of Practice

the sections cover quality management for ‘key stages’:

• Programme design, approval, monitoring and review ( 7)

• Admissions to higher education (10)

• Collaborative provision and flexible and distributed learning (including e-learning) ( 2)

• Students with disabilities ( 3)

• Work-based and Placement learning ( 9)

• Assessment of students ( 6)

• External examining ( 4)

• Academic appeals and student complaints on academic matters ( 5)

• Career education, information and guidance ( 8)

• Postgraduate research programmes ( 1)

Also

Accreditation of Prior and Experiential Learning - guidelines

[ Credit guidelines ]

Briefing visit

Briefing paper, SWS, TQI, published reports

Audit visit

Audit trails

Report

and its Annex

Between visits

Last day of audit visit

Current QA procedure in England:Institutional Audit

Audit trails

Sampling trails – • normally based on the institution’s own way of managing and

reviewing its academic standards and quality of provision

Supplementary trails – • of particular topics proposed by the institution and accepted by

the audit team• there is no expectation that an audit will necessarily include a

supplementary trail• is made by the audit team at the briefing visit, following

discussion with the institution

Selection of audit trails• will bear in mind the value added to the audit process by

undertaking the trail• will also bear in mind that trails are a means of gathering

evidence, not an end in themselves• their findings are not reported separately but they contribute to

the evidence on which the team draws for its findings, judgements and comments

On the audit trail ...

A

B

Documents about the audit trail

Use made of AI Code of practice, FHEQ, Benchmark Statements Programme Specifications

Student evaluation Student liaisonInternal review – how well

does it work for assurance and enhancement ?

Handbooks - good information?

Evaluation of learning support & resources

Students’ understanding of expectations

Assessment policies in practice? Match with level of award?

Progression data – used well?

View of institution’s policies and processes in operation

The judgements

• 'the confidence that can reasonably be placed in the soundness of the institution’s present and likely future management of the academic standards of its awards‘

• 'the confidence that can reasonably be placed in the soundness of the institution’s present and likely future management of the quality of the learning opportunities available to students‘

3 levels• Confidence most common• Limited confidence occasional – serious • No confidence rare – VERY SERIOUS

Features of good practice & Recommendations

Features of good practice - in the context of the institution• that make a particularly positive contribution to the

institution’s approach to the management of the security of academic standards and the quality of provision

Recommendations• essential – currently putting quality and/or academic

standards at risk – require urgent corrective action

• advisable – have the potential to put quality and/or academic standards at risk – require preventative or corrective action

• desirable – have the potential to enhance quality of learning opportunities and/or further secure the academic standards of awards

dimension 6: the ethos .. (avoiding the ‘quality wars’)

in the UK ethos underpinning QA (now) encompasses:• collegial / partnership• respectful of mission• respectful of institution’s autonomy• prepared to celebrate good practice• honest and open• judgements based on evidence

encapsulated in the six questions…• what are you trying to do? purposes• why are you doing it? reasons• how are you going to do it? methods• why are you doing it that way? optimisation• how do you know it works? effectiveness• how can you improve it? enhancement

The reports … are not ‘the end’ • Summary - judgements, commentaries, recommendations,

features of good practice

• Report – seven section headings

• Annex to the report- same section headings as in the report but comprising

‘technical’ details

sharing the information in the audit reports and QAOutcomes from ..

thematic papers drawing on evidence in audit reports

ELIR themes reports in Scotland

Different contexts .. different proceduresbut a similar ‘direction of travel’

‘self regulation’

[ external

quality ‘control’ ]

external quality assurance

internal QA with externality

‘risk based’ QA

Austria

Germany

Norway

Switzerland

UK

Netherlands

/ Flanders ?

Different contexts .. different proceduresbut a similar ‘direction of travel’

‘self regulation’

[external]

quality ‘control’

external quality assurance

internal QA with externality

‘risk based’ QA

Austria

Germany

Norway

Switzerland

UK

Netherlands

/ Flanders ?

TRUST / RESPONSIBILITY

Building trust .. gaining responsibility shared purposes and principles context dependent procedures

Purposes – must be clear and shared• Public accountability for standards and quality• Responsibility for improvement

Principles …. must be related to ESG• Same principles apply to all types and sizes of institutions• Transparency is essential• Must underpin policies and procedures

Procedures• Depend upon context but• MUST produce clear evidence – available to all

Some practical issues ..

Building trust in self regulation– has to be progressive, to an agreed plan, – with clearly identified ‘benefits’ – and ‘checks’

a quality culture– changing the ethos and hard work – progressive but to a clear plan

from quality assurance to enhancement– doesn’t mean a lower level of assurance– does mean a different emphasis– should build to a ‘virtuous circle’

‘Light touch’– doesn’t mean ‘soft touch’– is really about ‘shift of touch’ .. less external / more internal !

Some questions?

What are the real (shared) purpose (s)?

How are the (necessary) standards agreed ?

Can / should reports be public?

Who selects / how are ‘the experts’ selected ?

How do procedures deal with heterogeneity?

Can we avoid procedure become predominant?

What about ‘ranking’?

Why bother? Data, information (?) and knowledge

do I stop ..

or ..

do I go .. ?