www.derby.ac.uk/ehs

Dr DOUG CARR, Curriculum Development Manager

Using Nominal Group Technique(NGT) to Foster Dialogues withStudents about Enhancement

www.derby.ac.uk/ehs

Evaluation in Context

How knowledgecaptured is used

to effect enhancement

What weknow about

the student’sexperience

INSTITUTIONAL

VIEW OF

EVALUATION

NSS / PTES

‘Feedback’

Module Evaluation

Prog Evaluation

formalised/ systematic

‘capture’

informal / secondary‘capture’

NGT

focus groups

discussion groups

other ad hoc

www.derby.ac.uk/ehs

Origins of NGTDubecq & Van De Ven in 1970s

change management; clinical guidelines creation; needs assessment & item generation for questionnaires

course / programme evaluation = main current usage

www.derby.ac.uk/ehs

Characteristics of NGTdialogue based group activity

pooled judgements based on knowledge

skills & experience of group participants

used where judgements with creative

solutions are required

www.derby.ac.uk/ehs

Typical Steps in NGToverview / introduction to task

nominal / silent phase

idea generation

discussion of ideas

voting / quantifying

reflections

www.derby.ac.uk/ehs

OVERVIEW / INTRODUCTIONYour question = what sort of developmental / support activities should HEIs promote to ensure that outputs from surveys can lead to enhancement

1

www.derby.ac.uk/ehs

NOMINAL / SILENT PHASEworking on your own, write down some responses to question posed2

www.derby.ac.uk/ehs

IDEA GENERATION identify a group spokesperson spokesperson prompts each

group member for single response

place post-its in convenient place

3

www.derby.ac.uk/ehs

DISCUSSION OF IDEASarrange ideas into categories / groups (ensuring that it’s a group task)4

www.derby.ac.uk/ehs

VOTING / QUANTIFYINGIdentify the TOP THREE most important categories,1 = most important and 3 = least important (of top three) 5

www.derby.ac.uk/ehs

REFLECTION / ACTIONSWhat enhancement actions will be required as a result of voting / quantifying 6

www.derby.ac.uk/ehs

synchronous completion

issues can get ‘smoothed out’

discussions can get picky

scope / focus needs to be limited

doesn’t work well when comparisons required

process can be stifling if used too often

Limitations of NGT

www.derby.ac.uk/ehs

Strengths of NGT

dialogue basis

empowers students

gives authority to student voice

useful vehicle for learning about learning

involvement of students as co-producers

www.derby.ac.uk/ehs

Key Factors in Usagesize of group

who leads

simple or more complex voting

multiple groups / extended discussions

www.derby.ac.uk/ehs

the formal & explicit opportunity for having dialogues with students

Five reasons why people like NGT

www.derby.ac.uk/ehs

Five reasons why people like NGTstudents feel involved-it doesn’t feel like being surveyed!

www.derby.ac.uk/ehs

the opportunity tomake connections about and between learning

Five reasons why people like NGT

www.derby.ac.uk/ehs

it’s potential use in peer review / moderation by academic colleagues

Five reasons why people like NGT

www.derby.ac.uk/ehs

how it can complement surveys & its enhancementfocus

Five reasons why people like NGT

www.derby.ac.uk/ehs

Some Further Reading on NGTO’Neil MJ & Jackson L, Nominal Group Technique: A process for initiating curriculum development in higher education, Studies in Higher Education, Vol 8 No 2, 1983

Chapple M & Murphy R, The Nominal Group Technique: extending the evaluation of students teaching and learning experiences, Assessment & Evaluation in Higher Education, Vol 21 No 2, 1996

Lloyd-Jones G, Fowell S & Bligh J, The use of the nominal group technique as an evaluative tool in medical undergraduate education. Medical Education, Vol 33 Issue 1, pages 8–13, January 1999

Perry J & Linsley S, The use of the nominal group technique as an evaluative tool in the teaching and summative assessment of the inter-personal skills of student mental health nurses, Nurse Education Today, Vol 26, Issue 4, May 2006, Pages 346–353

Kristofco R, Shewchuk R, Casebeer L, Bellande B & Bennett N, Attributes of an ideal continuing medical education institution identified through nominal group technique, Journal of Continuing Education in the Health Professions, Vol 25 Issue 3, pages 221–228, Summer 2005

Rohrbaugh J, Improving the Quality of Group Judgement: Social Judgement Analysis & the Nominal group technique, Organisational Behaviour & Human Performance, 28, 272-288 (1981)

Johnson R, The authority of the student questionnaire, Teaching in Higher Education, Vol 5 No 4, p419 – 434, 2000

Delbecq AL & Van De Ven AH, A group process model for problem identification and program planning, Journal of Applied Behavioural Sciences, July-August, 7(4), 1971

Delbecq AL , Van De Ven AH & Gustafson DH (1975) Group Techniques for Program Planning: A Guide to the Nominal Group and Delphi Processes, Glenview, Illinois, Scott Foreman & Company

Note resources posted on slideshare by Tunde Varga- Atkins focusing on Nominal Group Technique , Survey Fatigue & Using Technology to support NGT approaches