Usability testing and redesign of Odeon cinemas...

28
Usability testing and redesign of Odeon cinemas website Coursework for Usability, principles and practice Course: User Experience Design, M.A. Jaroslav Sumbal B629626 January 2017

Transcript of Usability testing and redesign of Odeon cinemas...

Page 1: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

Usability testing and redesign of Odeon

cinemas website

Coursework for Usability, principles and practice

Course: User Experience Design, M.A.

Jaroslav Sumbal – B629626

January 2017

Page 2: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

2

Page 3: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

3

Abstract

This study tested and evaluated usability of Odeon cinemas website. Usability testing was

done in two rounds; first round tested usability of the original website, as it was in

November to December 2016. Tests were analysed and numerous issues were identified

in the design. These issues were then addressed in a redesign: an interactive prototype

done in Axure RP8. The prototype was tested in the second round. Analysis, both

qualitative and quantitative, confirmed that the redesign is more usable than the original

website and shed light on other areas that should be improved or changed in next iteration

of redesign if there were any.

Main recommendations from the study are to remove unnecessary visual clutter from the

homepage and promote finding films and cinemas and move the featured films element

down in the page. Next, in the cinema page, move show times up so that they appear

above the fold. When user clicks on a film name on cinema page, remember which

cinema it was so that they do not have to search it again on film page. Introduce the

movie modal as explained in the main text.

Axure share link: http://bit.ly/OdeonRedesign

Instructions for viewing prototype: prototype was designed to be viewed on full HD

screen. If viewed on a smaller screen or on a screen with scaling turned on, it may be

necessary to zoom out in the browser. To view page structure, variables and other data

about structure of the prototype, hover over top left corner of the prototype and click the

downward facing chevron that appears. Note that this will move everything to the right.

That is why it is hidden by default.

Page 4: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

4

Table of Contents 1 Introduction .................................................................................................................. 7

1.1 The website ........................................................................................................... 7

2 Sample description and user goals ............................................................................... 7

2.1 Users ...................................................................................................................... 7

2.1.1 User goals....................................................................................................... 7

2.1.2 Tested sample................................................................................................. 8

3 Methodology ................................................................................................................ 9

3.1 Tasks...................................................................................................................... 9

3.1.1 Task 1 ............................................................................................................. 9

3.1.2 Task 2 ............................................................................................................. 9

3.1.3 Task 3 ........................................................................................................... 10

3.2 Running and recording sessions .......................................................................... 10

3.3 Evaluation metrics ............................................................................................... 11

4 First round of testing – original website .................................................................... 11

4.1 Pilot ..................................................................................................................... 11

4.2 Spotted patterns ................................................................................................... 12

4.3 Eye tracking......................................................................................................... 18

4.4 Mobile version..................................................................................................... 18

4.5 Performance results ............................................................................................. 20

4.6 Subjective satisfaction results ............................................................................. 20

5 Second round of testing ............................................................................................. 21

5.1 Testing on prototype............................................................................................ 21

5.2 Redesigned pages and spotted behaviour patterns .............................................. 21

5.3 Performance results comparison ......................................................................... 24

5.4 Subjective satisfaction results comparison .......................................................... 26

Page 5: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

5

6 Conclusion ................................................................................................................. 27

6.1 Personal notes ...................................................................................................... 27

References .......................................................................................................................... 28

Table of Figures

Figure 1 Homepage ............................................................................................................ 12

Figure 2 Loughborough cinema page ................................................................................ 13

Figure 3 Loughborough seat selection ............................................................................... 14

Figure 4 Ticket selection .................................................................................................... 16

Figure 5 Seat reservation ................................................................................................... 17

Figure 6 Mobile - Find a cinema....................................................................................... 19

Figure 7 Mobile - Find a movie ......................................................................................... 19

Figure 8 Mobile - seat reservation ..................................................................................... 19

Figure 9 Redesigned homepage ......................................................................................... 22

Figure 10 Cinema page redesigned .................................................................................... 22

Figure 11 Movie modal ...................................................................................................... 23

Figure 12 Comparison of mean number of clicks per task across designs ........................ 26

Figure 13 Comparison of median number of clicks per task across designs ..................... 26

Figure 14 Comparison of means in questionnaire answers ............................................... 27

Figure 15 Comparison of medians in questionnaire answers ............................................ 27

Table of tables

Table 1 Participants overview .............................................................................................. 9

Table 2 Wrong clicks in first round ................................................................................... 20

Table 3 Questionnaire results - first round ........................................................................ 21

Table 4 Wrong clicks in redesign ...................................................................................... 25

Table 5 Comparison of questionnaire answers .................................................................. 27

Page 6: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

6

Page 7: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

1 Introduction

The task at hand was to choose any website, perform usability testing on that website,

analyse test results, redesign the website for better usability and finally test the new

design to see if it was better that the original and suggest changes for future redesign.

1.1 The website

After some searching and consideration, webpage of the Odeon cinemas was chosen for

the project (http://www.odeon.co.uk/). There were several reasons for this choice:

the site does not require any authentication for successful completion of user

goals, which is extremely important since there was no support from the company

for mock accounts or anything similar. There is some functionality hidden behind

authentication, but it is not very important.

it has a very large target user group available for research

there is a lot of room for improvement

2 Sample description and user goals

2.1 Users

Many people go to cinema, some less and some more often. Therefore, it was easy to find

intended users of the website for the testing. University students were asked to participate

in this study. The sampling method consisted of one question: “Have you ever used this

website before? If so, when was the last time?”

Reason for this is that usability of a system is best tested on first time users who did not

have the time to get used to it (Cooper, Reimann, and Cronin, 2014). Most of the

participants have never used the website before. While some have, it was at least a month

ago, so there was a good chance of forgetting how it worked, since they did not use it

frequently. There were no other restrictions except for age (over 18).

2.1.1 User goals

Because the initiative for this project did not come from the company running the

website, there was no access to any of their user research or business goals that might

help with identifying user goals that the website should support. Therefore, user goals

were extracted from the features of the website and using common sense.

Page 8: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

8

First goal identified was to look up what films are shown in the cinema for inspiration

when users just want to go out for a film. Second goal was booking a film. This was

further divided into two sub-goals:

Booking a film ticket when you know which cinema you want to visit and want to

see it there.

Booking a film ticket when you know you want to watch the film but don’t mind

the cinema selection so much as an area where the cinema should be.

The first sub-goal mimics the traditional way of buying tickets: you go to the cinema, see

what is there and buy a ticket. To test more variants, it is impossible to reserve seats

online in this cinema. The second sub-goal is made possible by technological

advancement, where you can check which cinemas show a specific film. Online seat

reservations are made in this cinema.

2.1.2 Tested sample

There were two rounds of testing. One participant attended both and others either the first

or the second. Their short description is shown in the TABLE 1. Recent study in OECD

countries measured IT skills of the general population using four skill levels. This study is

huge, but Nielsen (2016) pulled out the important information about IT skills and

explained them in easily digestible manner. However, it was not possible to properly

assess participants’ IT skills, so instead of using these four levels, their IT skills were

determined only by what they said about it and categorised into three levels. These levels

were used: poor, basic and good.

Page 9: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

9

Participant

No.

Sex IT skills Used the site Eye tracking Test

round

1 Male Good Never No 1

2 Female Poor Never No 1

3 Female Basic Never No 1, 2

4 Male Basic Never No 1

5 Male Good Never Yes 1

6 Male Good Never Yes 1

7 Female Good Within last month No 2

8 Male Good Within last month No 2

9 Female Good Never No 2

10 Male Good Within last month No 2

11 Male Good Never No 2

Table 1 Participants overview

3 Methodology

Method used to evaluate usability of the

website is moderated usability testing.

One reason for this choice is the time

and budget allowances for this project,

which, considering it is a student work,

were not great.

3.1 Tasks

Three tasks written as scenarios that

were linked into one overall scenario

were devised from identified user goals.

The core of each task remained the same

throughout the testing, but the wording

changed after the pilot test for better

clarity.

While task 2 and 3 may seem similar,

there are great differences in the booking

process when done in a small cinema

like the one in Loughborough compared

to a big London cinema.

3.1.1 Task 1

“You want to go watch a film tonight

that you might like at Odeon cinema in

Loughborough, that plays after 19:00,

maybe a comedy. You're not really

decided which film exactly you want to

watch, so you hope that the cinema page

will help you with that. Find such film

and watch the trailer.”

This task was following the first goal to

see how the site enables users to see

what is shown in the cinema and how

clear is the navigation.

3.1.2 Task 2

“After watching the trailer, you realise

that this film is not for you, so you check

Page 10: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

10

out other films after 19:00. Choose one

and book a ticket.”

This task was following the first sub-

goal of the second goal to see how the

site enabled users booking of a film

when they are in the cinema.

3.1.3 Task 3

“Just before checkout, you've

remembered that you're going to London

in 2 days and want to go watch Doctor

Strange with a friend in the evening. So,

you think that 2 films in one week would

be too much, so you don't buy these

tickets but book tickets to see Doctor

Strange with a friend in London in

Odeon cinema of your choosing.” – This

film was not played in every London

cinema, only big ones with seat

reservation. So, the choice was merely

an illusion.

This task was following the second sub-

goal of the second goal to see how the

site enabled users booking of a film in a

specified area with multiple cinemas

present.

While this task is plausible (cancelling

tickets in the beginning), as in it can

happen to users, it would be good to

know if the company wanted to make

this scenario easy. One could argue that

when a user is already on a path to

purchase, there should be as few ways

out as possible to “force” the purchase.

Therefore, the way out of the booking

flow was not redesigned, rather only the

subsequent part after user successfully

abandoned the booking path.

3.2 Running and recording sessions

Most testing sessions were done in a

standard environment on the same laptop

with consistent environment with

exception of the eye tracking sessions.

For better analysis of data, screen was

recorded, along with the face of the

participant using the laptop webcam.

At the start of a session, each participant

was welcomed with short conversation,

notified about the purpose of the

research and recording and was asked to

sign an informed consent form. After

that they were told the website was being

tested and not them no ease the possible

tension and asked to think aloud.

Tasks were printed out on pieces of

paper and handed to participants in order

after the previous task was completed.

Participants were asked to read the task

aloud and during the test encouraged to

look at it again, as often they lost focus

of their task.

After completion of all the tasks, they

were asked to fill in a short questionnaire

about their experience with using the

Page 11: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

11

website and then add any comments they

wanted.

Finally, they were thanked for their time

and rewarded with a little sweet treat as a

little surprise as they were not promised

anything.

3.3 Evaluation metrics

Decision was made to take a more

qualitative approach rather than

quantitative to measure the usability of

the website. This means that while

timing was recorded, it was not used as a

metric, because participants were

encouraged to think aloud and

sometimes asked questions about their

actions, which makes time measurements

distorted.

Another reason for this decision is the

number of participants and successfully

collected data points. With only 6

participants and sometimes fewer data

points (read more in section 4.5),

reliability of statistical tests is

questionable.

Therefore, all metrics recorded that

could be potentially used as quantitative

with a bigger sample were treated more

as qualitative. Nielsen (2001) makes a

point about this and talks about using

quantitative measures in usability testing

in much greater detail.

One metric considered was the number

of wrong clicks per task. Wrong click

was defined as follows: a click that

moved a participant further away from

completing the task at hand, or that did

not at all do what the participant

expected, for example clicking on an

item which did not do anything.

Other metrics were gathered by

questionnaire. These measured, on a

scale from 1 to 5

the overall perceived ease of use

as well as per task

how text and headings made

sense to participants

how using the site made

participants feel: sad to happy (or

just negative to positive)

how the site was organised,

whether it made sense

4 First round of testing –

original website

4.1 Pilot

The pilot test proved very useful as it

showed that wording of the tasks needed

to be changed to improve clarity and

participants’ understanding of the tasks.

Because of this, results from the pilot

were not considered greatly but were a

necessary step in the process.

Page 12: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

12

4.2 Spotted patterns

To analyse results, notes were taken

from looking at screen recordings. They

were then used to make affinity diagram

to see which problems occurred the most

and should be made priority for redesign.

There is a common problem for the first

and third tasks: the homepage (FIGURE

1) is not well laid out. It does not give

enough visual significance to important

features for successful completion of

these tasks, contains a lot of noise and

confuses users a lot. This was part of the

main focus of the redesign.

Figure 1 Homepage

Most users were confused by the menu,

because it reacted on hover. They

expected to click on it, and when they

moved cursor too fast and clicked,

sometimes it froze and they were stuck.

Another point of confusion was the film

page, which was substituted by movie

modal (more in section 5.2 on page 21).

Confusion arose when participants

clicked on the name of the film in

Loughborough cinema page, and

suddenly they just saw reviews. They did

not notice small dropdown to choose a

cinema. One participant even went back

and did it again as she thought she has

done something wrong. And when they

noticed the dropdown, they did not

understand, why they had to choose the

cinema again. The website, for some

reason, did not remember that they

Reacted on hover

Page 13: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

13

clicked on the film when in

Loughborough cinema page and did not

automatically choose it.

Second part of the main focus was the

cinema page, when user navigates to a

concrete cinema to check show times or

other information. FIGURE 2 shows what

user sees above the fold after navigation

to the page of Odeon in Loughborough

(on laptop with 15.6” screen and scaling

turned on). No show times, no relevant

information. Some users even thought

that they did not navigate to the right

page at all. It is necessary to scroll down

to see relevant content, but there is no

indicator of that.

Figure 2 Loughborough cinema page

Next, during booking process for a

smaller cinema such as Loughborough,

there is a confusing screen about seat

selection. It shows seat layout and over

that a message that one can choose seats

when one arrives to the cinema. Most

participants did not understand the

message and tried to reserve a seat and

payed attention to the message only after

unsuccessful attempts.

Page 14: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

14

Figure 3 Loughborough seat selection

Seat selection was very confusing for

bigger cinemas too, because users first

had to choose a section of seats (FIGURE

4) and then seats on next page (FIGURE

5). There were two big issues with this.

Firstly, majority of users (except for one)

did not even notice there were different

sections and therefore went with the

preselected and thus on the next page

they were confused when they could not

choose any seat they wanted. Secondly,

on the seat selection page, it was not

clear where the screen was and what

does each colour mean, as the legend

was below the fold and some participants

did not notice it at all. So, they first

chose seats thinking they were on the

other side of the room than they were

and then scrolled down to click continue

without noticing that the screen was on

the other side of the room.

The legend itself was not well designed,

because it was not explained what grey

seat means. It says which seat is

available (blue) and which is occupied

(blue with user icon in it). But it

neglected grey seats. Some participants

thought, that because there were also

user icons on grey seats, that grey were

available too. This comes hand in hand

with the fact that most users did not

notice that there were different sections

of seats and so expected to be able to

choose any seat they wanted.

Participants tried

to click on seats

Page 15: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

15

Recommendations for future redesign

are to abandon choosing section of seats

before user sees the seats and move it to

the seat selection page. That way user

would have direct connection between

chosen section and available seats.

Going back to ticket selection on

FIGURE 4 for 3D shows, users did not

appreciate that before they checked the

option to have 3D glasses, they did not

know they cost anything. When £1

appeared after selection, they were

unpleasantly surprised. Another common

comment was that: “Why are there two

checkboxes for the same thing? Why not

just one item with a quantity next to it,

just like tickets?” That is a valid

suggestion and would improve usability

of the site, as the current solution

increases cognitive load of users for no

good reason.

Page 16: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

16

Figure 4 Ticket selection

Should be moved

to next page

Prices should be

displayed up front.

Page 17: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

17

Figure 5 Seat reservation

This was not visible above

fold, mostly unnoticed

What are grey

seats?

Page 18: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

18

4.3 Eye tracking

The original website was tested using eye tracker system available with two participants.

There were unexpected problems. The system was not optimised to work with heavily

dynamic websites. Content on this website was dynamically changing based on user’s

actions, and the eye tracker showed only first screenshot of the page before any

interaction. For this reason, heatmaps are irrelevant, as it is not possible to determine why

would user look at a certain place, because the content has probably changed. Also,

videos with the eye movement behaved in the same way and were therefore completely

unusable for this project.

Another issue stemming from this is that it was not possible to ascertain number of wrong

clicks with these two participants, because it is not possible to determine where they

clicked and whether it was wrong.

4.4 Mobile version

The website is not responsive, but it has an alternate mobile version at

http://mobi.odeon.co.uk/. This version contains less clutter and visual noise than the

desktop version. It allows the completion of all goals defined before (see FIGURE 6 and

FIGURE 7), however, reserving a seat is extremely difficult.

At first, all seats are shown and it is impossible to accurately pick the one user wants with

a finger (see FIGURE 8 MOBILE - SEAT RESERVATION). There is an option to zoom in,

but then it is difficult to drag the seats to reveal the one user wants. Also, during the test

the page froze after zooming in, making any other action impossible. Other than that, less

screen space forces to show only important things and it is in a way easier to accomplish

some goals on this version than desktop version.

Page 19: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

Pag

e19

Figure 6 Mobile - Find a cinema

Figure 7 Mobile - Find a movie

Figure 8 Mobile - seat reservation

Page 20: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

20

4.5 Performance results

While there were six participants in the

first round, data collected from pilot

testing cannot be used as some

conditions and tasks changed afterwards.

Number of clicks was gathered by

watching back recorded videos. Last two

tests were done using eye tracker system.

Because of major issues mentioned

earlier, it was not possible to get the

number of clicks for these tests. Thus,

the number of data points was halved to

𝑁 = 3, so statistical analysis provided

should be taken with a grain of salt.

Participant No. Task 1 Task 2 Task 3

P2 4 2 6

P3 1 3 4

P4 10 2 4

mean 5 2.33 4.67

median 4 2 4 Table 2 Wrong clicks in first round

4.6 Subjective satisfaction results

At the end of section 3.3 on page 11, outline of questionnaire was mentioned. All

questions used Likert scale style of options, but were optimised for measuring usability.

Classical Likert scales have one disadvantage in general: they influence participants in the

question, as it is not open ended, but yes or no style of question (Cooper, Reimann, and

Cronin, 2014). Therefore, instead of statement: I found the site easy to use; with options

of strongly agree to strongly disagree, there was a statement: Overall, the site allowed me

to complete the tasks; with options on one end saying “hardly” and on the other “easily”.

Data points were collected for all but one participant, where technology failed and data

were lost.

These questions were asked (numbers are used in TABLE 3 and TABLE 5):

1. Overall, the site allowed me to complete the tasks (hardly to easily)

2. Using the site made me feel (sad or negative to happy or positive)

3. Text of headings and buttons (confused me to made sense)

4. The site was organized (badly to well)

5. The site allowed me to complete the first task (hardly to easily)

6. The site allowed me to complete the second task (hardly to easily)

7. The site allowed me to complete the third task (hardly to easily)

Page 21: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

21

Participant No. q. 1 q. 2 q. 3 q. 4 q. 5 q. 6 q. 7

P1 4 3 5 5 5 4 5

P2 4 3 4 3 4 5 3

P3 4 4 3 2 2 4 4

P4 4 3 5 5 2 5 5

P5 3 3 4 3 2 4 4

mean 3.8 3.2 4.2 3.6 3 4.4 4.2

median 4 3 4 3 2 4 4 Table 3 Questionnaire results - first round

5 Second round of testing

First and second round of testing were

done more than a month apart, so the last

task had to be slightly adjusted to reflect

changes in what film was currently

trending. Otherwise the tasks and

environment were the same.

5.1 Testing on prototype

Second round of testing was done on a

prototype of the redesign. This factor put

greater importance on the role of the

moderator as not every feature was

implemented or completed.

The prototype fully supported the

completion of all goals, with exception

of implementation of genre filter, which

proved to be a technical challenge in

Axure RP8, a prototyping tool chosen

for the job.

5.2 Redesigned pages and spotted

behaviour patterns

As mentioned earlier, major redesign

was done on the homepage and cinema

page, and some small things were

redesigned in the booking flow.

The redesigned homepage can be seen in

FIGURE 9. Featured films element was

moved down and does not span the

entire width of the page to avoid

overwhelming users. Search bar and

browsing options were given a priority.

Interestingly, only one participant used

the featured films element, others went

straight for search or browsing option. It

is questionable whether it is even

important.

Page 22: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

22

Figure 9 Redesigned homepage

Figure 10 Cinema page redesigned

FIGURE 10 shows redesigned cinema

page. Instead of being confused by

meaningless “Hello” taking up most of

the space above the fold, users can see

show times right away. This eliminated

the confusion that some users

experienced with the original design,

when they were not even sure that they

are on the right page. As Krug (2013)

highlights as the most important

usability rule, users did not have to think

where they are or what are they supposed

to do now.

Individual show times are aligned in a

grid to make finding relevant options

easier. To prevent visual clutter by

drawing a complete table, principle from

Page 23: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

23

Gestalt theory was used saying that

people complete patterns in their minds.

Movie titles and images are all

concentrated on the left side, because

that is how users, at least in our part of

the world, scan the page. Users scan the

page from left to right and from top to

bottom, creating an F shape. Most

important information should then be in

the top left corner (logo of the website

serving as a link to homepage, and then

the heading of the current website, so

that user always know where they are).

Whether the user will start scanning

more to the right and then down, or first

down and then right depends on the

layout. Design can lead the eyes where it

wants, to certain extent (Florentin,

2015).

One problem was detected in the tests:

difference between times for 2D

showings and 3D is not clear. Some

participants clicked on a time, then in

movie modal change the version and

then noticed that time has changed and

did not understand why it happened.

Then did it all over again and then

noticed. These times should be more

clearly differentiated, maybe by colour

or put them further away from each

other.

Another problem is that participants

expected to find time filter. Despite show

times being laid out in table like fashion

for easy navigation, users wanted to

make their life easier by filtering show

times within specified range. This

feature should be added in the next

iteration.

Figure 11 Movie modal

Page 24: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

24

FIGURE 11 shows new page element

introduced with the redesign as a

substitute for movie page, internally

called the movie modal. This modal

shows all details of a movie and enables

users to check their selection or change

it. One usability principle is

“Recognition rather than recall”, as

identified by Jakob Nielsen (1995).

Movie modal embodies this principle by

brining all important choices of movie

selection together in one place.

It can be accessed anywhere on the

website where user can choose to see

details of a film. Options available in

prototype are: click on item in featured

films carousel, click on returned result

from search, click on item when

browsing movies, click on movie

picture, name or show time on cinema

page or looking up movie in the main

navigation.

This element is the gateway to booking a

film and therefore easy discoverability is

very important. The ability to change

choice of cinema, version, date and time

helps users to recover from errors. While

completing third task, one user used

browser navigation to go back several

steps from booking confirmation page

back to Loughborough cinema page.

There she clicked on target film she

wanted to watch in London. Then, when

she realised she is still in Loughborough

cinema, she changed her choice to

London in the dropdown. When she was

asked whether she did this on purpose,

whether she knew she could do this

before she has done it, she said that she

clicked on the movie just randomly and

then noticed she could complete the task

by changing the options.

Very common problem was during

transition from task 2 to task 3. There

users found themselves in a situation

with unfinished booking, needing to

cancel it and start a new one. As

mentioned in section 3.1.3 on page 10,

this flow was not redesigned, but it could

be improved. Internet users today are

used to online shopping, and participants

often thought that those tickets were in

some basket. What often happened was

that users tried to find a way to cancel

the tickets before going back, or change

them. They did not realise that they

could just return to homepage and start

over. Therefore, it would be good to add

the option to cancel current tickets. It

does not have to be obvious, but a simple

underlined link in bottom left corner

somewhere would help ease the tension

in users.

5.3 Performance results comparison

In the second round, data points were

collected for every participant. Mean and

Page 25: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

25

median were calculated for this data set

as well (see TABLE 4) and one tailed

student’s t-test assuming equal variance

was applied to see if changes found were

statistically significant (note that with

such small samples, reliability of these

tests is not guaranteed, but it can provide

a picture). Significance value chosen was

standard 𝑝 = 0.05. Hypothesis was

always along the same lines: Users do

fewer wrong clicks in redesign than

original.

Participant No. Task 1 Task 2 Task 3

P3 0 0 1

P7 0 0 2

P8 2 0 2

P9 0 2 4

P10 0 1 1

P11 12 0 2

mean 2.33 0.5 2

median 0 0 2

p-value 0.226233439 0.006037575 0.005805456 Table 4 Wrong clicks in redesign

Tasks 2 and 3 have significantly

decreased the number of wrong clicks,

which in turn leads to less confusion and

faster completion of tasks. Task 1 was

not significantly different because of

participant 11. This value is an outlier

and could be ignored from the data set. If

it were ignored, changes would be

significant with 𝑝 = 0.03, and 𝑚𝑒𝑎𝑛 =

0.40.

Decrease in wrong clicks can be seen in

FIGURE 12 and FIGURE 13. Here, the

outlier value for task 1 was ignored to

provide different look on the data.

Median did not change (because it was

an outlier), but mean was affected

dramatically.

Page 26: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

26

Figure 12 Comparison of mean number of clicks per task across designs

Figure 13 Comparison of median number of clicks per task across designs

5.4 Subjective satisfaction results

comparison

Answers from all participants were

successfully recorded in the second

round. However, after applying the same

test for significance, only question 5 got

significantly better results, while

question 1 almost got it. These are

shown in FIGURE 14 and FIGURE 15.

For detailed values see TABLE 5

(question are written in section 4.6 on

page 20).

5

2.33

4.67

0.40 0.5

2

0

1

2

3

4

5

6

Task 1 Task 2 Task 3

Mea

n o

f w

rong c

lick

sComparison of mean number of clicks per task across

designs

original redesign

4

2

4

0 0

2

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Task 1 Task 2 Task 3

Med

ian o

f w

rong c

lick

s

Comparison of median number of clicks per task

across designs

original redesign

Page 27: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

27

Participant No. q. 1 q. 2 q. 3 q. 4 q. 5 q. 6 q. 7

P3 5 3 4 5 5 5 4

P7 5 5 5 5 5 5 5

P8 4 3 4 3 3 4 3

P9 4 3 5 5 5 5 3

P10 4 3 4 4 5 4 2

P11 4 4 3 4 4 5 5

mean 4.33 3.50 4.17 4.33 4.50 4.67 3.67

median 4 3 4 4.5 5 5 3.5

p-value

0.05194

4

0.24585

2

0.47302

5

0.14596

7

0.0280

6

0.21398

7

0.21398

7 Table 5 Comparison of questionnaire answers

Figure 14 Comparison of means in questionnaire

answers

Figure 15 Comparison of medians in questionnaire

answers

6 Conclusion

Both qualitative and quantitative analysis clearly show that the redesign is more usable

than the original design. Throughout the text were mentioned several suggestions on how

the redesign could be improved to increase its usability.

6.1 Personal notes

I learned a lot doing this project. My moderating skills have improved from session to

session. I also got better in prototyping and learned to appreciate more how different

people use the web differently and that one cannot rely purely on best practices. I would

have liked to learn to work more with the Eye tracker, which failed me in this project.

Word count: 4 333 (excluding abstract, tables and references)

0

1

2

3

4

5

Comparison of means

Original Redesign

0

1

2

3

4

5

6

Comparison of medians

Original Redesign

Page 28: Usability testing and redesign of Odeon cinemas websitejsumbal.rocks/resources_for_download/Usability report.pdf · The task at hand was to choose any website, perform usability testing

28

References

Cooper, A., Reimann, R. and Cronin, D. (2014) About face: The essentials of interaction

design. United States: John Wiley & Sons.

Florentin, T. (2015) Design for a perfect screen. Available at:

http://lp2.webydo.com/design-for-a-perfect-screen-ebook.html.

Krug, S. (2013) Don’t make me think, revisited: A common sense approach to web

usability. 3rd edn. Boston, MA, United States: Pearson Education (US).

Nielsen, J. (2001) Usability metrics. Available at:

https://www.nngroup.com/articles/usability-metrics/ (Accessed: 30 December 2016).

Nielsen, J. (2016) The distribution of users’ computer skills: Worse than you think.

Available at: https://www.nngroup.com/articles/computer-skill-levels/ (Accessed: 30

December 2016).

Nielsen, J. (1995) 10 Heuristics for user interface design. Available at:

https://www.nngroup.com/articles/ten-usability-heuristics/ (Accessed: 20 January 2017).