User-centred development of the Sakai survey tool

Fawei Gengfawei.geng@oucs.ox.ac.uk

Oxford University Computing Services

Software piloting: not just pressing & clicking

University of Oxford

Source: http://www.flickr.com/photos/earthworm/5889087243/

Demo: WebLearn Surveys tool

WebLearn Surveys tool pilot

Overview

What can the WebLearn Surveys tool do ?

• create an online survey • deliver to Oxford users or general public• view and export survey results • use for course experience feedback, research purposes, or general data gathering

Terminology

Survey creator

Survey respondents

Survey

WebLearn site

participants

Ad-hoc groups

General public

Template – a set of questions (items)

My Surveys

- Specifications such as dates etc

- All your surveys

Three steps in creating a survey

1Create a template

2

Assign the survey to recipients (unless public)

Configure survey settings

3

Survey dashboard

Create a new template

1

Add a question (a)

Question types

1

Add a question (b) 1

Edit template 1

prev

iew

edit

dele

te

Create a new survey (a) 2

Select your template

Create a new survey (b) 2

Assign survey to respondents3

For a public survey, just click “Next’ without choosing respondents

Recipients’ view of a survey

Scale question with a comment box

Optional text field for name

My surveys page

View partial results

View final results

Survey results

An example of a PDF summary report

A frequently requested IT application

Source: http://www.flickr.com/photos/jacobwhittaker/3388288370/

A free survey tool please

Oxford University strategic plan 2008–9 to 2012–13

“Create more effective processes for the

monitoring of student feedback”

“Review the mechanisms for ensuring that the

feedback of student views which it gathers is

systematically monitored and acted upon”

Intended outcomes:

• To test the functionality of the survey tool

• To improve the tool – identify and eliminate

WebLearn Surveys tool pilot project

WebLearn Surveys tool pilot project

Steve Jo

bs, 1997 W

WD

Ch

ttp://w

ww

.you

tub

e.co

m/w

atch

?v=

FF

-tKL

ISfP

E

Steve Jobs at WWDC 1997

Intended outcomes:

• To customise the tool – meet user needs and requests

• To help users to develop good surveys

WebLearn Surveys tool pilot project

Outline of the project

Time Activities Deliverables

11/09 Recruit participants

(40 people)

Workshop 1

Workshop 2

Workshop 3

Project survey

Focus group

Observation

§1

12/09

01/10

02/10

03/10

04/10

05/10

06/10

07/10

Phase I

Phase II

Gather U

ser Feedback

…see later

Recruit participants

• mailing lists and personal contact: expectations of the project team benefits to the participants

• 40 participants: academics researchers administrators

Mixed methods

• Workshops: demonstrate the tool observe participants creating their

own online surveys provide personalised support

• Email communication to follow up• WebLearn support site• Pilot project survey

‘walk the talk’ demonstrate good practice evaluate success of the pilot project

Three workshops

• Format: presentation demo hands-on practice Q&A

• Characteristics: similar activities for each workshop user requests shaped the hands-on and

Q&A working collaboratively with participants

Developing pilot project survey

• Survey design consult with participants design questions seek advice from experts to refine

questions test survey with a small number of

participants

• Survey delivery: July 2010, 22 responses (55%)

Survey follow-up

Focus group session:

• respond to comments in the survey results

• how to improve response rate design survey questions: useful to the survey

participants engage a small sample of survey participants

in the design process offer an incentive use reminders wisely: most people respond to

a survey either in the first couple of hours or just before the deadline

Evaluation of the project

Time Activities Deliverables

11/09 Recruit participants

(40 people)

Workshop 1

Workshop 2

Workshop 3

Questionnaire

Focus group

Observation

• an improved survey tool

• 66 completed surveys designed and delivered by participants

• project report

• training guides

• training course – WebLearn Surveys

• a set of public surveys used staff across the university, e.g. the College student experience survey

12/09

01/10

02/10

03/10

04/10

05/10

06/10

07/10

Phase I

Phase II

Gather U

ser Feedback

Contribution to Sakai community• Course evaluation + public

surveys

• Simplified for public survey

• View results of open surveys

• Fix bugs: Extension dates now work Caching problems fixed Results can be deleted Better error messages Notifications and reminders now work

Evaluation of the project

Evaluation of the project

• expanded user communities

• publicity for OUCS services, including WebLearn

• networking: pilot participants and experts in evaluation

Comparison of survey tools

Thank you, and…

More information: • http://www.weblearn.ox.ac.uk/info• fawei.geng@oucs.ox.ac.uk

http://ww

w.fotopedia.com

/items/

flickr-2200500024