UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability
40
Electoral Commission project TM
-
Upload
uxpa-uk -
Category
Technology
-
view
2 -
download
0
description
The Electoral Commission was concerned about whether the design of ballot papers was making it difficult for electors to vote accurately.Spurred by this, the Electoral Commission commissioned User Vision and Effortmark to conduct usability tests with a range of voters in Scotland, England, Wales and Northern Ireland.In their talk, Clare and Caroline will provide insight on how they approached the project and what they learned about ballots and about running paper testing.
Transcript of UKUPA Jan 10 Clare Barnett and Caroline Jarrett: Election Ballot Usability
Electoral Commission project
TM
Clare Barnett and Caroline Jarrett
Clare is a usability consultant with User Vision
Caroline is a usability consultant / forms expert
Other people involved: Louise Ferguson advised the Electoral Commission on
usability issues Will Reburn of the Electoral Commission was our main
contact for this project Chris Rourke & Rob Van Tol from User Vision worked with
us.
And now, a word from Caroline’s sponsor
Redish (2007)“Letting go of the words:writing web content that works”Morgan Kaufmann
Jarrett and Gaffney (2009)“Forms that work: Designing web forms for usability”
Morgan Kaufmann
How would you cast your vote?
Outline: we tested ballots in four countries
Background to the testing How we approached the testing A brief description of the main findings
What makes voting hard or easy The detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
This project was a response to problems in 2007 After the 2007 Scottish elections, and other
elections, concerns grew about the usability of election materials.
The Electoral Commission commissioned usability testing of: existing voting materials, the use of registered party descriptions on ballot papers.
Focus groups were also organised by Brahm. The end goal was to feed into the development of
draft design standards creating consistency.
The participants represented a cross-section of UK voters Location
Scotland, England, Wales, Northern Ireland rural, suburban, and urban locations
Age from 18 to 67
Voting experience never voted, occasional voters, frequent voters
Different voting methods Polling station & postal
Social classes Educational levels
from no qualifications to degrees
Nationality & first language Including Welsh speakers.
Outline: we tested ballots in four countries
Background to the testing How we approached the testing A brief description of the main findings
What makes voting hard or easy A selection of the detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
The test was an ordinary task-based test
Each participant had to complete a selection of voting tasks, varied by location. Two examples: In London: General election, Mayoral election, London
Assembly, European election, Local election by postal vote In Scotland: General election, Scottish Parliament,
European election, Local election
For each voting task: we asked who they were going to vote for we gave them the appropriate materials they voted we briefly discussed the experience of that particular vote we offered a short distraction task between votes
We had a ‘beauty competition’ of the materials to look at the finer details.
We co-ordinated our approach
We created a generic crib sheet that worked for all the different voting patterns
We used the first test in London as a pilot Caroline facilitated Rob and the clients watched We debriefed immediately
We passed our experiences along: Rob tested in Wales Caroline tested in Cornwall Rob and Clare worked together in Scotland Clare tested in Northern Ireland
We all met in Scotland to debrief later.
We used a distraction task between votes
As the voting process is a fairly repetitive task, we provided distraction tasks between voting.
Each session required skills in paper juggling
For a typical participant we had: 4 ballot papers 3 station posters 3 booth posters 1 set of postal instructions
Postal voting statement (or declaration of identity in N Ireland)
Textual instructions Illustrated instructions 2 envelopes
our generic crib sheet template for notes.
Outline
Background to the testing How we approached it A brief description of the main findings
What makes voting hard or easy A selection of the detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
Two big factors: voting experience and the voting task(s) Participants found many plausible (though
incorrect) interpretations of the materials. We found that:
Voting experience is important. The structure of the election has a major effect. The complexity of the postal voting process
caused difficulty.
Voting experience is crucial to voting success If offered the choice,
even new voters skip the instructions
If asked to read the instructions, new voters read more carefully
Even if they had read the instructions, new voters became confused and some made mistakes Extract of Ballot Paper completed by a new voter
Participant deals with a complex election as if it were an ordinary one Example
Voting is harder than you realise:He votes the way he has done for years
Am I voting for a candidate or a party?
This participant tried to vote for an individual
The instruction to ‘vote for a party or an individual’ can be misinterpreted to mean ‘vote for an individual within the party’.
Why do I have more than one vote?
London Mayor Voter can indicate first and second choice “Can I vote twice for Boris?” “Do I have to vote for someone else?”
Scottish Local Election You can vote for as many as you like “Why would I vote for more than one?”
Welsh (& English) Local Election “Why do I have so many votes?” What happens if the voter does not use them all?
Outline
Background to the testing How we approached it A brief description of the main findings
What makes voting hard or easy A selection of the detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
Why have the candidate’s addresses on the ballot?
Some participants thought this was unnecessary Others worried about the potential danger to the
candidate Some were concerned that it might sway floating
voters
Logos are important. Some of them are missing Some participants didn’t vote for parties without
logos as they didn’t look “official”.
Numbers next to candidates are not useful
Numbering on the ballot is unhelpful There is a risk of circling the number instead of voting
properly.
Numbering within the party is also unhelpful Potential for error in voting for a candidate rather than a
party.
Candidates names are in an unfamiliar format Some ballots have:
SURNAMEFirst names
Other ballots have: SURNAME
Full name
And other ballots have even more:
Party names are important, but sometimes hard to find Scottish Conservative & Unionist Party Conservatives Conservative Party Welsh Conservatives Welsh Conservative Party The Conservative Party
Scottish Labour Party The Scottish Labour Party Labour Party
Party names are included in the brackets with the candidates name and address.
Instructions on the ballot were often poor
Some ballots did not have important instructions: Title of the ballot How to vote
Some ballots had instructions, but not easy to see London elections white-on-black instruction
If participants saw this instruction then they liked it:
English and Welsh should be kept aligned
Welsh and English together double the information
There was no consistency over which came first Participants do not mind which, slight preference for
Welsh first
They do not want to see separate English and Welsh forms
Translations into Welsh were not exactly the same Mixing Welsh and English is confusing
Example of mixing Welsh and English
Speak in voters’ language
Give it a simple title. Don’t bunch important
instructions into one sentence.
Don’t double information by repeating it.
Polling station notices have instructions that are hard to understand What does ‘spoil a ballot’ mean? What’s a compartment? What’s this about showing your vote to the
presiding officer? Instructions have numbers alongside them (good)
but often more than one item included for a number (bad)
Why tell voters to leave the polling station immediately?
The poster in the booth should tell you what to do in the booth If they need it before, put it on a poster If they need it in the booth, tell them in the
booth What type of mark to make
“Do I use a cross or a tick?”
The rules about the numbers of votes you have What to do if you make a mistake.
If they need it after voting, put it after Fold/don’t fold, Show to member of staff.
If they don’t need it at all, scrap it! “Leave the voting station immediately”
Outline
Background to the testing How we approached it A brief description of the main findings
What makes voting hard or easy A selection of the detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
Collating analysis was a joint task
There were 3 testers, 45 participants and 4 countries. We entered our findings per country into an Excel
spread sheet Caroline came to Edinburgh
We each had 30 mins to talk about our particular test results for each country
This allowed us to see whether issues were country specific or UK wide
Each issue was written onto a post it and placed on a white board under country specific or UK wide
We then removed any duplicated issues from the board
The Electoral commission wanted a summary presentation of our main findings, followed up with a report detailing a full list of the issues and recommendations.
We wrote the full report with client involvement The Electoral Commission wanted to make the report a
public document. Due to experiences with previous reports, the Electoral
Commission did not want any figures referred to in the report. “In a small qualitative study, it is important to focus on the
problems that were identified by participants rather than the numbers who found those problems.“
“We also did not consider how difficult or easy our recommendations may be to act on for legislative or any other reason.“
We split issues into categories. These were agreed with the Electoral Commission.
The post-its from the analysis meeting were used, and split into categories, flowing as much as possible with the journey that voters take.
Full report
Electoral commission website: http://www.electoralcommission.org.uk/doc
ument-summary?assetid=77687
Outline
Background to the testing How we approached it A brief description of the main findings
What makes voting hard or easy The detailed design findings
How we analysed the results Offline and online testing, differences and
similarities
Offline and online testing: similarities
The process followed was the same for online testing.
Flapping about of paper is similar to users being unable to find information online: Users are having trouble finding something.
People skipped instructions, even if they were only a few words long.
Users seemed to ‘get into’ the idea of the test quickly and treat it as realistic.
Offline and online testing: differences
“Feel appeal” – you actually interact with the materials You can draw straight onto the materials. Paper is an physical product and can be bulky.
On paper, signatures really matter. Recording is harder:
One venue: good quality video – of the participant’s head
Other places: used a web cam, but quality not very good.
What next?
The Electoral Commission has published their guidelines – Making your Mark:
http://www.dopolitics.org.uk/making-your-mark
