1

Handbook for Internal Subject Review Team Members

2013/14

2

ContentsPurpose of this handbook and ISR training 4The Purpose of ISR 8Key features of the process 14Team Roles and Responsibilities 18Managing the process 31How to ask questions 38How to make commendations and recommendations 40

3

Contents (cont.)Appendices:1. List of key documents about the ISR process2. ISR Strategic Themes for 2012/133. Suggested ISR Review Visit schedule4. ISR Timeline5. Template for meeting preparation 6. Commentary exercise7. How to ask questions during a review Visit8. Commendations and Recommendations9. Recommendations exercise10. Previous examples of features of exemplary practiceAppendices can be found at:-http://www.ncl.ac.uk/quilt/assets/documents/qsh-isr-handbook_appendices.pdf

4

Purpose of the Training and Handbook

After reading this handbook and/or attending training ISR team members should be able to:

• Articulate the purpose of ISR• Appreciate what the key features of the

process are• Understand how ISRs are planned, and, with

support, how to carry out and follow up an ISR• Understand the role of the team members

5

Purpose (cont.)

• Understand the special roles of the Chair and the Secretary

• Understand how to use the commentary and other evidence

• Pose suitable questions in an appropriate way• Word commendations for good and exemplary

practice and recommendations appropriately

6

Accentuate the positive

• We can learn more from what works well than from what works badly

• What works well in one context can often be transferred to another

• Messages which are constructive and positive in tone are likely to be better received

• Makes the event a more positive experience for everyone

7

Key documents

• The key documents are in the ISR section of the Quality and Standards Handbook at

http://www.ncl.ac.uk/quilt/resources/monitoring/internal

• See also Appendix 1

8

The Purpose of ISR

9

Institutional responsibility

• Primary responsibility for academic standards and quality in UK HE rests with individual universities, each of which is independent and self-governing

• QAA checks how well they meet their responsibilities, identifying good practice and making recommendations for improvement

10

QAA requirements

• Formal and effective procedures for the design, approval, monitoring and review of programmes

• Processes should foster creativity, and encourage a culture of continuous enhancement of provision

• Institutions should have means of assessing the effectiveness of their programme design, approval, monitoring and review practices

11

How we fulfil QAA requirements

• Mechanisms for annual and periodic review (Annual Monitoring and Review – AMR; and Internal Subject Review – ISR). Two parts of the same process of continuous improvement

• Programme Approval• Internal Audit

12

How ISR can add value

• Self-reflection on whether programmes remain current and are effectively taught

• External and student input• Constructive and positive engagement• Opportunity to bring matters to the attention

of Faculty and University• Highlighting areas of exemplary practice

13

Outcomes of the ISR process

• A positive experience!• Confirmation that standards are secure, there is a

high quality learning experience• Re-approval of degree programmes for a further

six year period• Formal report considered by faculty and

University• Recommendations to assist the subject area with

continuous improvement of provision

14

ISR – key features

15

The key features are...

• Enhancement-led - emphasis on ISR as a developmental process which encourages enhancement of teaching and learning processes as well as disseminating exemplary practice

• A Review Team visit to the School/subject area of one day

• The inclusion of a student representative on the Review Team

16

Key features(cont.)

• Role for the Chair as critical friend

• Compliance review - the secretary to the Review Team conducts a review of local implementation of quality assurance and enhancement processes, and of programme documentation

• Priority review themes for each year (Appendix 2)

17

Key features (cont.)

• Training for Review Team members

• Annual evaluation of the process involving Review Team members

18

The role of the members of the team

19

What does a team consist of?

• Chair – usually an academic from another faculty (ULTSEC nominee)

• An academic from another faculty (ULTSEC nominee)• An academic from the same faculty, but a different

school (FLTSEC nominee)• A student rep nominated by the Students’ Union• An external expert nominated by the subject• ISR Secretary from QuILT• There may be other observers

20

Team’s responsibility

• The whole team is responsible for the conduct of the ISR and agreement to the report

• This is a collective task

• However, some members of the team have special roles

21

Shared tasks

• Reading the commentary and reviewing other evidence before the visit

• Preparatory meeting (external remotely)• Preparing for meetings• Debriefing after meetings• Agreeing exemplaries, commendations and

recommendations• Commenting on the draft report

22

Role of the external

• Only subject specialist in the team• Of key importance in judging the curriculum,

standards, currency, compliance with benchmarks and PSRB requirements, and how the provision compares with that available elsewhere

• Vitally important and required by QAA• Will lead on curriculum issues

23

Role of the Chair

• Also vitally important, but in a different way

BEFORE THE VISIT• Meeting the subject area before the visit• Chairing the preparatory meeting• Ensuring the schedule is agreed (Appendix 3)

24

Role of the Chair (cont.)

DURING THE VISIT• Keeping to schedule• Keeping focused• Ensuring there is preparation for meetings and

debriefing afterwards at which the key points are recorded

• Giving oral feedback at the end of the day

25

Role of the Chair (cont.)

AFTER THE VISIT• Final say on the draft report

26

Role of the Student Member

• A full member of the ISR team• Selected from another area and not there to

represent the students in the subject • Likely to bring some different perspectives• Will take part in preparatory meeting• Will ask questions in all meetings• May chair a session

27

Role of other academic members

• Full members of the team• Will ask questions in all sessions• May chair a session

28

Secretary

GENERAL• Take a proactive approach to advising the Review Team • Help to foster a collaborative culture within the team and to support an

enhancement-led approach• Ensure that all Review Team members and the subject area understand the

purpose of ISR and how the process works

BEFORE THE VISIT• Make arrangements and liaise with the subject starting many months before• Liaise over appointment of external and fixing the date of the visit• Carry out compliance review• Negotiate with the subject area over the schedule and other logistical

arrangements (See Appendix 3)

29

Role of the secretary (cont.)

DURING THE VISIT• Liaise with School contact about arrangements,

document requests etc.• Take notes during sessions• Help team devise questions for sessions and

debrief afterwards, recording agreed key points• Helps chair and team agree exemplaries,

recommendations and feedback

30

Role of the secretary (cont.)

AFTER THE VISIT• Draft the report• Liaise with team members• Send draft report to subject for factual

checking• Send final report to subject and to relevant

FLTSEC secretary

31

Managing the process

32

Planning the visit and designing the questions

• ISR – Timeline (Appendix 4)• The commentary and other documents provided

(e.g. handbooks; externals’ reports; programme data; programme specifications etc.) form the initial basis for identifying areas to follow up and planning the visit

• The compliance review carried out by the ISR secretary may also contribute useful pointers

• The meeting with students may reveal new issues

33

Evidence-based process

• This is an evidence-based process• Where the documentation provides sufficient

evidence to reach a conclusion, then there is no need for oral questions

• Sometimes additional documents may (within reason) be requested

• Oral evidence from one source ideally needs confirmation

34

Using scarce time profitably

• One day is an incredibly short period of time to gather evidence

• It has to include time for reviewing some documents and for discussion and reflection as well as for meetings

• Questioning has to be very strategic and focused on the most important issues emerging from the team’s review

35

Using time profitably

• Sessions must be prepared for identifying a chair, topics to pursue, who will ask what and rough time scales

• Sometimes you have to drop something in a session because time runs out, so it is important to know what matters most

• Session chair needs to keep team and interviewees focused and prevent digression

• Useful to have a rough schedule (see Appendix 5i/5ii)

36

Follow up

• After each session the team should agree a small number of bullet points relating to the key issues identified in the session

• Be on the look-out for commendations and recommendations

37

Exercise 1

• Exercise on reviewing providers’ commentaries - See Appendix 6

38

How to ask questions• Clarity of purpose• “Etiquette” of questioning• Structuring lines of questioning– what are you trying to do?– why are you doing it?– how are you doing it?– why is that the best way to do it?– how do you know it works?– how can you improve it?

See also appendix 7

39

Exercise 2In the meeting with students, concerns were raised by some PGT student representatives about their experience of personal tutoring. They indicated that they had raised this at the Staff Student Committee but had not received a response. This does not seem to support the assertion made by the School that in the last three years they have reformed personal tutoring and have fully implemented the new student representation policy.

In groups, construct a line of questioning that explores this with members of staff

40

Commendations and Recommendations

• What is exemplary?• What is a commendation?• Recommendations• SMAR(T) goals• See also appendix 8

41

Exercise 3

• Exercise on wording a recommendation using scenarios provided and a review of some actual exemplaries to identify which ones work better [See appendices 9 and 10]

• Feedback at the end

42

And finally...

…thank you for attending this workshop

• Questions• Contacts• Feedback