ENQA – QAA meeting8-9 December 2005Birmingham, UK

8 December, 10.00 – 12.30

Presentations:

Peter Williams, President, ENQA

Nicola Channon, Head of Operations, Institutional Review, QAA

The development of institutional-level quality

assurance in Europe

Peter Williams

President, ENQA

European QA in HE

• History• State ownership and control of HE• Increasing autonomy of institutions • Challenges for governments• Different approaches to QA• Predominantly programme-based accreditation• Some institutional accreditation • Some ‘enhancement/development-based approaches

Key principles for mature quality assurance systems

• Purposes should determine processes• Autonomy means responsibility and power• QA is a means, not an end• Every € spent on quality assurance is a € not

spent on teaching

European quality assurance dimension: what are the purposes

• Accountability?• Academic standards?• Enhancement?• Information?• Control?• Standardisation?

In the UK the purposes of quality assurance are

• Security of academic standards of qualifications

• Enhancement of students’ learning opportunities and experience

• Accountability• Information

Quality assurance in the UK 1990-2001

‘Academic Audit’1990 -

HE sector-owned evaluation of institutions’

management of academic

standards and quality

‘Quality Assessment’

(Subject Review)1993-2001

Inspection at subject level

undertaken by HE funding bodies

Two separate streams

In 2001 subject reviews were stopped

Why was subject review stopped in the UK?

• subject reviews had shown generally high quality and standards: no point in repeating the exercise

• unnecessary burden on institutions • high cost• institutions had learned how to ‘win the game’• diminishing returns• institutional resistance• alternative ways of providing information and

identifying weaknesses

Institutional-level reviews• Require autonomy to be effective• Place responsibility for quality and standards

in institutions • Encourage internal quality assurance • Do not stifle innovation• Are less burdensome than programme

reviews• Are cheaper than programme reviews

Quality assurance is an evolutionary, not a static process:• From subject review to institutional audit• From inspection to quality assurance • From external prescription to internal rigour• From process to outcome• From the implicit to the explicit• From assertion to verified information • From accountability towards enhancement • From suspicion towards trust

Institutional audit- where next?

Nicola Channon

Head of Operations, Institutional Review

QAA

Background• 2001: new Quality Assurance Framework

- institutional audit including discipline audit trails (DATs)- developmental engagements- collaborative provision audit- Teaching Quality Information (TQI )- National Student Survey (NSS)- Academic Infrastructure

• 2002-2005: transitional period – all higher education institutions audited in England and Northern Ireland

• July 2005: report on transitional period, ‘Burslem Report’ published

What did the higher education institutions think of audit?

• Not bad – if we have to have something then it might as well be this

• ‘QAA is now the least unwelcome of our external visitors’

What would the higher education institutions like to change?

• change discipline audit trails to reduce the domination of the process and the unnecessary additional work

• reduction in paperwork• a formal contact between the team and the institution• oral feedback at the end of the audit• change the ‘broad confidence’ judgement

What are we being asked to do? 1

• retain the existing structure• replace discipline audit trails with flexible audit

trails• replace ‘broad confidence’ judgements with

‘confidence’• implement mid-cycle reviews• retain the ability to review particular elements if

there is a ‘cause for concern’

What are we being asked to do? 2

• QAA to develop feedback and dissemination mechanisms - for individual institutions and for the sector

• more enhancement focused • student participation to be promoted and supported• QAA to explicitly discourage ‘gold-plating’ by

institutions

Aims of the audit• ensuring that the awards and qualifications in higher education are of

an appropriate academic standard reflecting the levels of achievement set out in The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) and, thereby, the EHEA European Qualifications Framework and are, where relevant, exercising their powers as degree awarding bodies in a proper manner

• providing learning opportunities of a quality which enables students, whether on taught or research programmes, to achieve those higher education awards and qualifications, and

• enhancing the quality of their educational provision by building on information gained through monitoring and internal and external review, and feedback from stakeholders.

Judgements

• the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the academic standards of its awards

• the confidence that can reasonably be placed in the soundness of the institution's present and likely future management of the quality of the learning opportunities available to students.

Comments• the institution’s arrangements for maintaining appropriate

academic standards of postgraduate research programmes

• the institution’s approach to developing and implementing institutional strategies for enhancing the quality of its educational provision, both taught and by research

• the reliance that can reasonably be placed on the accuracy and completeness of the information that the institution publishes about the quality of its educational provision and the standards of its awards.

Information • a briefing paper by the institution • any key documents that the institution wishes to submit• a briefing paper prepared by representatives of students of

the institution on behalf of the student body • reports on the institution or its provision produced by QAA

and other relevant bodies within the six years preceding the audit

• the information placed by the institution on the TQI site, and the residual TQI information held by the institution

• a desk-based analysis by QAA the institution’s TQI set, with a commentary on the completeness and currency of this information

Reports

• 3 sections:- a short summary intended for the public, especially potential

students- concise information on the findings of the audit of use to

both a lay and professional readership- an annex intended for a professional audience, giving in

more detail the analysis on which the findings of the audit are based

• Publication- the whole report will be published on QAA’s website- only the summary and the findings will be printed

Audit trails

• a means of gathering evidence • tools for audit, not ends in themselves• themes to be followed

- Some ‘core themes’ to provide evidence for basic judgements on standards and quality

- Some ‘local themes’ chosen following discussion with HEI

• use of ‘off the shelf’ documentation• not necessarily reported separately

Reference points• Academic Infrastructure

- FHEQ- Benchmark Statements- Code of Practice in general but particular attention

on section 1 post-graduate research programmes- programme specifications

• European standards and guidelines for quality assurance (ESG)

Timetable

Jan 06 Feb Mar Apr May June Dec 06 Dec 11

Consultation ends(13 Jan)

Audit scheduleproposed

Draft handbookpublished

Preliminaryinstitutional meetingsfor 2007 audit visits

Auditors & auditsecretaries recruited

Auditors & auditsecretaries training

First auditvisits

Audit cycleends

Jan 07 Feb/Mar

Other points

• collaborative provision • mid-term reviews• relationship to European standards and

guidelines (ESG)• responding to the expectation of the inclusion of

student team members• responding to the expectation of the inclusion of

international team members• follow-up