Giving All Students a Voice – Introducing Online Evaluation of Teaching and Learning within QUT

Queensland University of Technology

CRICOS No. 000213J

Giving All Students a Voice – Introducing Online Evaluation of

Teaching and Learning within QUT

Presented at

Association for Tertiary Education Management Queensland Branch Conference

“Taking it to the Streets”

By

Margot DuncanJude Smith

Sam Nielsen

1 June 2007

CRICOS No. 000213Ja university for the worldrealR

Aims of Presentation

• To overview QUT’s new approach to student evaluation

• To highlight challenges related to implementing the new approach to surveying

• To discuss lessons about implementation of a major reform learned from the process


The Need for ChangeQUT Senior Executive responded to recommendations from a review of Quality Teaching that:

• Teaching evaluation should be universal for all teachers every semester in every unit;

• Teaching and unit data should be provided routinely and form part of corporate reporting internally;

• Individual teaching results should be reported (minimally) to the teacher, supervisor and relevant managers;

• Data should form part of performance management and development discussions; and

• Advice and support should be provided to enable teaching data to be used more consistently, systematically and appropriately in both performance management and in responding to students.


The New Evaluation Scheme at QUT

QUT has been improving student evaluations processes and policies during 2007:

• a new suite of internal surveys encompassing teaching, units, courses and experience has been introduced:– Learning Experience Survey– First Year Experience Survey– Exit Year Experience Survey– Mid Year Experience Survey

• The surveys are fully administered and reported online• The surveys allow QUT students unprecedented opportunity to

provide feedback about individual teachers and units


Taking it to the Streets…

• The new systematic, online approach to student evaluation has raised concerns and involved challenges for students, professional staff, academics, and QUT’s senior executive.

To meet these challenges the project team had to “Take it to the streets”


Perspective

This presentation focuses upon the implementation from the

perspective that the outcomes are not yet fully known.


Challenges and Issues

STOP


40,000Students

In the streets:

• students don’t think surveys are ever used to improve teaching

• students feel they are being over-surveyed

• students don’t have time to complete lengthy surveys

• students doubt the confidentiality of their survey comments

Challenges: Students

QUT has almost 40,000 students.


In the streets:

• staff doubt the validity and accuracy of survey data

• staff are concerned about promotional implications of survey data

• staff have limited support in innovating their teaching practice based on survey data

• staff have issues with previous online surveying efforts

3,000Academic Staff

QUT has over 3,000 Academic Staff.

Challenges: Academic Staff


Professional staff: 2 divisions 3 campuses9 faculties

In the streets:• professional staff have been frustrated with previous complex systems & the demands of managing mandatory evaluations

Professional Staff from 9 faculties and 2 Divisions are involved.

• each faculty and division is structured differently & has different evaluation requirements

• evaluation communication channels between faculties and divisions have not been established

Challenges: Professional Staff


In the streets:

• Senior Executive feels pressure from AUQA and LTPF.

• Policy changes have union implications.

Challenges : QUT Senior Executive


3TechnicalSystems

In the streets:

• key data systems were missing from university practice

• existing technical systems were disconnected

Technical staff working with 3 university-wide systems are involved.

Challenges : Technical Staff


Project Challenges

In the streets:

• Demanding deadlines

• Limited funding

• Politics

• Uncertainty about roles

• Aware of challenges for others and need for solution satisfying all

Challenges: Project Management

Many limitations and conflicting issues


Moving ahead…Approach to and lessons learned

from implementing change


…to get all stakeholders working together effectively to ensure continuous improvement of teaching through listening to the student voice.

1. Keep the project goal in mind.


2. Consider All Perspectives

Knowledge from other implementations Use of available researchConstructive feedback


… do not seek to achieve more than is possible… do not over-promise… be willing to enhance the product in the future… effectively use project committees and project sponsorship

3. Manage Project Scope


4. Bring stakeholders on the journey


“Always be prepared to inform and be informed by stakeholders”

Human Resources Staff

IT Help desk staff

Undergraduate Students

Corporate Systems Managers

Learning & Teaching Consultants

T&L Support Staff

International Students

Senior StaffAcademic Staff

Faculty Admin Staff

Sessional Staff

Corporate Performance Staff

Post Graduate Students

Course Coordinators

Off-shore Coordinators

The Project Team

5. Enable Communication


6. Take every opportunity and use every method to communicate

OnlineResources

Forums

ReferenceGroups

Emails

Focus Groups


We used:• Steering groups• Reference groups• Working Parties• Committees

OnlineResources

Forums

ReferenceGroups

Emails

Focus Groups

We used:• Student Focus groups for survey questions & Interface design• Staff Focus groups for survey questions• Staff & students in survey trials

We used:• DVC & Senior Staff email communications to Academic staff• Staff information Packs• Staff Information Web Site• Staff & Student Information Web Site• Staff Teaching Resource Web site

We used:• HR workshops for Academic PPR• Teaching Portfolio Workshops• Faculty-based Workshops• Campus based Staff Forums• IT Helpdesk training• T&L Support Staff training

We used:• Student advertising campaign through postcards, posters, monitor displays, unit website messages.


7. Confront stakeholders’ concerns.


Student Confidentiality

Response Rates

PPR Processes

Seamless experience for students & staff

Closing the Loop Sustainability

of systems & processes

T&L Support


8. Find the shortcuts

… Cross organisational boundaries … Include and utilise expertise


9. Trust your Intuition.

•Many choices did not have ‘the’ obvious answer •Many discussions had an unspoken political aspect •Intuition as an indicator that there is a need for further ‘unpacking’ of a problem


10. Think outside the box.


11. Keep going.

Giving All Students a Voice – Introducing Online Evaluation of Teaching and Learning within QUT

Documents

Transcript of Giving All Students a Voice – Introducing Online Evaluation of Teaching and Learning within QUT