Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1
description
Transcript of Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1
![Page 1: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/1.jpg)
Webinar
Four Country Mobile Operator Study:a competitive benchmark of leading mobile operator sites
19th January 2011
![Page 2: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/2.jpg)
Overview
Introductions
Study Background
What do we mean by UX Benchmarking? Why is it important?
Methodology and Technology
Findings and Opportunities
Questions and Comments
UserZoom and UX Alliance Partners at a glance
![Page 3: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/3.jpg)
Introductions
Arthur Moan
Country Manager UK & Ireland
at UserZoom
Martje van der Linde
User Experience Consultant
at User Intelligence
Louiselle Morand
User Experience Consultant
at Telono
Javier Darriba
Managing Director
at Xperience Consulting
![Page 4: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/4.jpg)
Today’s Session Goals
See how users in four countries look for:
the monthly cost to surf the internet daily with an iPad
the fax number/address for a high street shop
To gain insight into the European mobile operator market
Introduction and review of UX Benchmarking methodology
Introduction and review of UserZoom software solution for conducting this type of research
![Page 5: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/5.jpg)
Study Background
![Page 6: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/6.jpg)
What is UX benchmarking and why is it important?
OK……. So how do we define UX benchmarking?
![Page 7: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/7.jpg)
What is UX benchmarking and why is it important?
Definition of Benchmarking: The process of comparing the
business processes and performance metrics including cost, cycle time, productivity, or quality to another that is widely considered to be an industry standard benchmark or best practice.
It provides a snapshot of the performance of your business and helps you understand where you are in relation to a particular standard.
![Page 8: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/8.jpg)
What is UX benchmarking and why is it important?
Definition of UX: It’s about how a person feels about
using a system. It highlights the experiential, affective, meaningful and valuable aspects of HCI and product ownership, but it also covers a person’s perceptions of the practical aspects such as utility, ease of use and efficiency of the system.
User experience is subjective in nature, as it is about an individual’s feelings and thoughts about the system. User experience is dynamic, because it changes over time as the circumstances change.
![Page 9: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/9.jpg)
What is UX benchmarking and why is it important?
Some may wonder: is UX benchmarking:
?
![Page 10: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/10.jpg)
Not really…..
Here’s how we’d define UX Benchmarking:
• The process of comparing a website’s performance, specifically in relation to UX metrics, including among others effectiveness, efficiency, overall satisfaction and quality perception (ISO-9241-11 standard), to another that is widely considered to be an industry standard benchmark or best practice.
• UX benchmarking provides a snapshot of the performance of your website and helps you understand where you are in relation to a particular standard.
![Page 11: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/11.jpg)
10 must-dos with UX Benchmarking
1. Define measurable & comparable UX-related KPIs (key performance indicators)
2. Select competitors to benchmark against
3. Define target audience
4. Identify common site activities/tasks (between-subjects design)
5. Conduct research using scenario-based testing
6. Gather both quantitative & qualitative data
7. Use the quantitative data to test for statistically significant differences between means or proportions
8. Use qualitative data to support your findings and understand the why
9. Research at least twice a year to measure design changes and trends in expectations
10.Use the right research solution/tool
![Page 12: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/12.jpg)
What is UX benchmarking and why is it important?
‘You can’t manage what you can’t measure’
Attributed to Peter Drucker, Mgt Guru
The question is:
How are you actually measuring UX and defining your benchmark?
![Page 13: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/13.jpg)
Methodology and Technology
![Page 14: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/14.jpg)
Common UX research challenges
Difficulties obtaining objective, statistically significant,
actionable data
Obtaining idea adoption from stakeholders
Geographic/demographic limitations
Research is costly and time-consuming
![Page 15: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/15.jpg)
How does UserZoom add value?
By allowing you to see the BIG picture
Testing large samples of geographically spread participants
Combining statistically-significant usability + user behaviour data
Measuring Customer Experience cost-effectively
Getting actionable insights
Usability Lab Studies
Web Traffic Analysis
Online UX
Studies
Online SurveysFocus
Groups
Qualitative
Quantitative
![Page 16: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/16.jpg)
Online Usability Testing Solution
UZ Self-Serve Edition
100% on-demand, web-based software solution
To manage & conduct various types of online UX research projects
![Page 17: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/17.jpg)
• Hundreds of users participate in a study
• In their natural context
• From geographically spread locations
• Users try to complete tasks (or goals) + answer questions
• No human moderation needed
• Our browser bar connects users with our secure servers
Our method
![Page 18: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/18.jpg)
Analyze data real-time using UZ Analytics
Effectiveness + efficiency ratios
![Page 19: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/19.jpg)
Analyze data using UZ Analytics
Qualitative feedback collected. Easy visualization through ‘text clouds’
![Page 20: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/20.jpg)
Analyze data using UZ Analytics
![Page 21: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/21.jpg)
UserZoom Results - Behavioural
![Page 22: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/22.jpg)
Study structure
Introduction:
For each study, participants were randomly assigned two tasks on two popular mobile operator websites (four tasks in total)
Task: Find a monthly iPad data plan
Task: Locate a store address/fax number
Post-task:
Mobile operator site evaluation ratings included -
Ease of use
Perceived complexity of sites
Perceived consistency of sites
Clear where to start
Confidence in using the sites
Usability Satisfaction (SUS score)
What was liked most about the sites
What was liked least about the sites
Wrap-up:
Mobile Operator Site Preference
Likelihood to recommend: (Net Promoter Score)
![Page 23: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/23.jpg)
Switzerland Study – Participant Demographics
Participants:N= 78 participants
Demographic data collected:• Gender• Current mobile operator• Age
• Individuals between 18-55 years of age were invited to complete the study.
GENDER45 females completed the study33 males completed the study
![Page 24: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/24.jpg)
Switzerland Study – Participant Demographics
AGEThe highest percentage of Swiss participants are 36 - 45 years of age.
• 14 participants are 18-25• 19 participants are 26-35• 25 participants are 36-45• 20 participants are 46-55
MOBILE OPERATORSwisscom, Sunrise and Orange were the most common current providers among participants.
• “Other”: yallo, vtx, Vodafone
![Page 25: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/25.jpg)
Netherlands Study – Participant Demographics
Participants:N= 100 participants
Demographic data collected:• Gender• Current mobile operator• Age
• Individuals between 18-55 years of age were invited to complete the study.
GENDER47 females completed the study53 males completed the study
![Page 26: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/26.jpg)
Netherlands Study – Participant Demographics
MOBILE OPERATORVodafone, KPN and T-Mobile were the most common current providers among participants.
•“Other”: hi, simyo, lebara, ziggo, rabo mobiel
AGEThe highest percentage of Dutch participants are 36 - 45 years of age.
• 21 participants are 18-25• 15 participants are 26-35• 41 participants are 36-45• 23 participants are 46-55
![Page 27: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/27.jpg)
Spain Study – Participant Demographics
Participants:N= 100 participants
Demographic data collected:• Gender• Current mobile operator• Age
• Individuals between 18-55 years of age were invited to complete the study.
GENDER45 females completed the study55 males completed the study
![Page 28: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/28.jpg)
Spain Study – Participant Demographics
AGEThe highest percentage of Spanish participants are 26-35 years of age.
• 27 participants are 18-25• 43 participants are 26-35• 22 participants are 36-45• 08 participants are 46-55
MOBILE OPERATORMovistar, Vodafone and Orange were the most common current providers among participants.
![Page 29: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/29.jpg)
United Kingdom Study – Participant Demographics
Participants:N= 101 participants
Demographic data collected:• Gender• Current mobile operator• Age
• Individuals between 18-55 years of age were invited to complete the study.
GENDER65 females completed the study35 males completed the study
![Page 30: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/30.jpg)
United Kingdom Study – Participant Demographics
AGEThe highest percentage of UK participants are 36 - 45 years of age.
• 21 participants are 18-25• 30 participants are 26-35• 31 participants are 36-45• 18 participants are 46-55
MOBILE OPERATORVodafone, O2 and Orange were the most common current providers among participants.
• “Other”: Asda, Tesco, Virgin
![Page 31: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/31.jpg)
Executive Summary
![Page 32: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/32.jpg)
Executive Summary
Orange UK and Orange Switzerland (CH) had highest task success rates for both iPad (UK 75%) and Address (CH 87%) tasks.
Orange Spain (SP) had the lowest success rate across tasks 24%, the lowest participants satisfaction scores, and substantially lower preference (25%) vs. Vodafone (75%).
All mobile providers tested in each country received low usability satisfaction scores by participants.
The lowest system usability scale scores (SUS) were received by Vodafone Spain (51) and Orange Spain (40).
All four mobile providers received low Net Promoter Scores (between -46% and -75%). There were more detractors than promoters in every instance.
![Page 33: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/33.jpg)
Findings and Opportunities
![Page 34: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/34.jpg)
Top level overview of Task 1 across all countries with deep dive into Switzerland & UK results
“Find the monthly cost to surf the internet daily with an iPad”
![Page 35: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/35.jpg)
Task Metrics: iPad Monthly Plan
![Page 36: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/36.jpg)
Switzerland: Swisscom Successful process
“Internet” “Internet with your
Notebook” “iPad tariffs”
Can be found in 3 clicks if browsing
![Page 37: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/37.jpg)
Heatmaps First Click: Swisscom - Switzerland (DE)
Overall 30% were successful- 6% used the search bar- 40% rolled over to the side navigation bar and then clicked on the links on the right - 15% clicked “Kombi-Angebote” on the side navigation bar
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 38: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/38.jpg)
Heatmaps First Click: Orange – Switzerland (DE)
Overall 24% were successful- 14% used the search bar- 16% clicked “Mobile” on the top navigation bar- 14% clicked “Internet” on the top navigation bar
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 39: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/39.jpg)
Heatmaps First Click: Orange - Spain
Overall 28% were successful- 21% used the search bar- 14% clicked “movil” on the top navigation bar- 65% clicked in other places
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 40: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/40.jpg)
Heatmaps First Click: Vodafone - Spain
Overall 61% were successful- 7% used the search bar- 31% clicked “Tienda Online” on the top navigation bar- 14% clicked in the center promotional area
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 41: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/41.jpg)
Heatmaps First Click: Vodafone - Netherlands
Overall 27% were successful- 4% used the search bar- 14% clicked “Internet” on the top navigation bar- 19% clicked in the center promotional area
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 42: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/42.jpg)
Heatmaps First Click: T-Mobile - Netherlands
Overall 40% were successful- 15% used the search bar- 13% clicked “Abonnement” on the top navigation bar- 27% clicked “Internet” on the top navigation bar
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 43: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/43.jpg)
Heatmaps First Click: Orange - United Kingdom
Overall 75% were successful- 13% used the search bar- 37% clicked “iPad” on the left navigation bar- 50% clicked in other places
Task: iPad Monthly PlanTask: iPad Monthly Plan
![Page 44: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/44.jpg)
• Only half of the participants were able to successfully find the monthly cost of an iPad subscription on the Vodafone website.
• Successful Participants spent an average of 5 minutes 24 seconds and 19 clicks completing the task.
Vodafone > iPad: UK
Task Effectiveness
![Page 45: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/45.jpg)
Orange > iPad: UK
Task Effectiveness
• The majority of participants were able to find the correct price for the plan (75%).
• Successful Participants spent an average of 1 minute 11 seconds and 7 clicks completing the task.
![Page 46: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/46.jpg)
Top level overview of Task 2 across all countries with deep dive into The Netherlands & Spain’s results
“Locate a store address/fax number”
![Page 47: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/47.jpg)
Task Metrics: Locating Store Address/Phone number
![Page 48: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/48.jpg)
Success rates for Netherlands
Task 2 : « Find the closest shop to the Amsterdam railway station and write down its address. »
The T-mobile site provided the best results regarding this task, 77% of the participants was successful.
On average it took slightly over one minute to find the right information, with an average of only 4 page views and 8 clicks.
On the Vodafone website 52% was successful in achieving this information.
Remarkable result is that it took an average of 3 minutes and 29 clicks to complete this task.
![Page 49: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/49.jpg)
Success questionnaire - Netherlands
Nevertheless, the rate of success regarding a task is not necessarily an indication of the ease of use:
The Vodafone website was less successful, but only 28% of the participants regarded it to be difficult to acquire the needed information.
On the T-mobile website, people were far more successful, but also less satisfied with the level of difficulty to get to the right URL: 42% of the participants thought this was (very) hard to achieve.
Vodafone
T-mobile
![Page 50: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/50.jpg)
Abandon questionnaire - Netherlands
Main reasons for abandoning a task were:
Page not loading: Especially the Vodafone website was unable to load the required page. Remarkable is that people wait up to 5 minutes!
Information not in expected location: Both Vodafone and T-mobile present information in an unexpected manner. Participants have certain expectations on where and how to find the required information. In case these expectations are not met, people easily abandon the website.
QUOTES“I was looking for it under Customer service and expected to be able to fill in my location and that it would automatically give the closest store location.”“ I was unable to open the page.”
![Page 51: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/51.jpg)
Final questionnaire - Netherlands
Users of both the Vodafone as well as the T-mobile website are neither positive nor negative about the websites:
They regard it as easy to use for inexperienced users and there is no need for extra help.
Nevertheless, the lack of success will have some influence on the use of the website.
Most users indicate that they will not likely be using these websites in the near future again.
![Page 52: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/52.jpg)
Task 2 Spain : Find the closest shop to the Atocha railway station and take note of its
fax number.
![Page 53: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/53.jpg)
Task 2 Spain – success ratios
• On Vodafone 68% of users completed the task successfully .
• On Orange 53% of users completed the task successfully.
![Page 54: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/54.jpg)
Vodafone analysis - Spain
• First click of users was mostly focused on the search engine and the help section. There were several users that tried to find the shops using the Vodafone logo.
![Page 55: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/55.jpg)
Vodafone analysis - Spain
• The reason users got the answer incorrect was due to a problem on the web. Although as happened in the previous task, the information that users found was not clear enough.
The site is not clear nor intuitive, you have to use the search engine, the information provided is not precise
The section with the Vodafone shops is easy, the problem is one you have to find one in Madrid
Too much information on the map
![Page 56: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/56.jpg)
Vodafone analysis - Spain
• The reasons participants abandoned the task was mainly due to the information being incomplete (67%). Other reasons were:• The information
presentation is not clear enough
• The information, options and menus are difficult to understand
• The information wasn’t in the expected location
• Few search options• As it has been seen in the
previous task users again had problems with the information provided.
I have entered province “Madrid”, city “Madrid” and no results appeared with this search options. I have tried with different options and it always happened the same.
![Page 57: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/57.jpg)
Orange Analysis - Spain
• Most of the users went to the tab “Orange shop” in order to look for the addresses of the physical shops.
![Page 58: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/58.jpg)
• The reason for the incorrect answer was ‘not being able to find the correct information due to options and menus being difficult to understand.’
• Users explained that the site mainly focuses on the iPhone whereas iPad information is quite difficult to find.
• The system could be improved using an accessible and easy to find search engine.
There is no way to understand what is going on here.
There is no way to find anything. What a fuss!
Orange Analysis - Spain
![Page 59: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/59.jpg)
• Those users who didn’t finish the task was mainly because they couldn’t find the information where they were expecting to find it.
• The links to the information were not clear enough.
• The site is very focused on selling products or services instead of giving clients the service that they need. I don’t have anything
against Orange but its website is a disaster
Orange Analysis - Spain
![Page 60: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/60.jpg)
Final Questionnaire – all countries: Which mobile operator do you prefer?
Netherlands Switzerland
Spain UK
![Page 61: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/61.jpg)
1. I think that I would like to use this system frequently. 2. I found the system unnecessarily complex. 3. I thought the system was easy to use. 4. I think that I would need the support of a technical person to be able to use this system. 5. I found the various functions in this system were well integrated. 6. I thought there was too much inconsistency in this system. 7. I would imagine that most people would learn to use this system very quickly. 8. I found the system very cumbersome to use. 9. I felt very confident using the system. 10. I needed to learn a lot of things before I could get going with this system.
Metrics: SUS ScoreThe System Usability Scale (SUS) is a set of 10 questions that is used to measure the perception of a systems overall usability. Participants rate each question with a 1 to 5 scale (5 being strongly agree).
![Page 62: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/62.jpg)
Metrics: SUS ScoreAfter completing the tasks on each mobile provider website participants were asked to complete the SUS questionnaire. The averages for each question are used to calculate an overall score (0-100).
SUS Score Guide*: 35=poor, 50=OK, 70=good, 85=excellent
Bangor, A. (2009, May). Determining What Individual SUS Scores Mean: Adding and Adjective Rating Scale. Journal of Usability Studies, 4(3), 114-123.
![Page 63: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/63.jpg)
Net Promoter Score (NPS®)
0 1 2 3 4 5 6 7 8 9 10 NPS
Orange (SP)16%
11% 15% 10% 7% 14% 9% 9% 2% 4% 3% -75%
Orange (UK) (n=101)
10%
3% 4% 7% 7% 21% 11% 10% 14% 9% 4% -50%
Orange (CH) (n=78) 9% 4% 12% 4% 10% 23% 9% 15% 12% 0% 3% -68%
Swisscom (CH) (n=78)
4% 1% 15% 6% 5% 15% 12% 22% 12% 3% 5%-51%
T-Mobile (NL)13%
4% 6% 4% 6% 27% 13% 10% 13% 3% 1% -69%
Vodafone (NL) 8% 3% 9% 8% 5% 22% 18% 11% 9% 3% 4% -66%
Vodafone (SP) 7% 1% 4% 6% 14% 13% 13% 18% 12% 8% 4% -46%
Vodafone (UK) 8% 1% 4% 8% 9% 22% 13% 16% 9% 5% 5% -55%
Promoters
NPS% of Promoters
(9s and 10s)Detractors
(0 through 6)= -
After completing the four tasks, participants were asked how likely they would be to recommend each mobile provider’s website.
Detractors
![Page 64: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/64.jpg)
Top 5 findings / Learnings The satisfaction is directly related with the success of the users in
obtaining the information that they are looking for. In general the websites for mobile operators in all countries are not performing optimal when consumers have a task in mind. The performance rate was dependent on the task, not the website.
Across the board the main difficulties were in participants finding specific information. The normal way users tried to find the information was through the menu structure. When they realized it was too complicated they tried to use ‘search’ although the results weren’t always clear enough. A much clearer architecture and clearer results in search engines are therefore needed. Furthermore the use of promotional ads made the navigation much longer and difficult for most of the users.
![Page 65: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/65.jpg)
Top 5 findings / Learnings
Analysing NPS, SUS, success on task and preference data it is clear that the brand halo effect has a bearing on site preference in each market irrespective of task success.
UK - although participants had notably higher task success rates on the Orange site (75% and 71%) vs. the Vodafone site (50% and 49%), when asked for site preference at the end of the study, the Orange website was selected only 8% more than the Vodafone site (54% vs 46%) respectively. Out of the Orange customers (n=17) more than half (9) preferred the Vodafone website.
Orange is not strong in Spain
Swisscom the national provider is stronger in Switzerland
Vodafone marginally outperforms T-mobile in the Netherlands
![Page 66: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/66.jpg)
Top 5 findings / Learnings When participants were asked whether they would recommend the
sites in each market all operators scored negatively showing room for improvement. Although these scores are low, previous research has shown that telecommunication companies have the lowest net promoter scores within the technology sector.
As Vodafone and Orange were both tested in 3 of the 4 markets we can benchmark their experiences.
Vodafone outperforms Orange on aggregated SUS (53 versus 51.3)
Vodafone outperforms Orange on aggregated NPS (-55% versus -64.3%)
Task 1 success aggregated : Vodafone outperforms Orange (46% versus 42%)
Task 2 success aggregated : Orange outperforms Vodafone (70% versus 56%)
![Page 67: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/67.jpg)
Q & A
![Page 68: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/68.jpg)
Information on UI
User Intelligence is a user experience consultancy and
was founded in 2002
Offices in Amsterdam and Helsinki, additional service
offering in Sweden and Norway
Our team consists of 17 consultants with a passion for
people and interactive technology
![Page 69: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/69.jpg)
About Telono
Telono is a Swiss consultancy specialising in usability, user experience and web accessibility.
We help companies implement a systematic user-centred design process, from user experience strategy and user research to interaction design and usability evaluations.
Telono is the Swiss representative of the User Experience Alliance (UXalliance)
![Page 70: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/70.jpg)
About Xperience Consulting
• 10 years of experience
• 2 offices (Barcelona & Madrid)
• 150 AI projects
• 300 usability studies
• 6000 users analyzed
• 150 clients (15 countries)
Usability and user experience experts, we
create interfaces that suit users’ needs and
objectives of your company. Intuitive and easy
to use websites.
![Page 71: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/71.jpg)
www.uxalliance.com
UX Alliance - Who are we?
The UX Alliance is an international network for usability testing and user
experience research consisting of core partners, associates, and affiliate
members.
Back in 2005 the leading user experience companies from France, Germany,
Spain, UK and US met in Barcelona with a single common goal in mind: how to
provide effective international user experience services to our clients.
Having worked together on international
projects for many years, we recognised
that we were ideally placed to lead
international user experience research.
Thus, the UXalliance was born…
![Page 72: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/72.jpg)
www.uxalliance.
Who are we?
20 Countries – 22 Companies Over 100 global projects per year together.
User Centric, Inc.United States
SirValUseGermany
User ExperienceChina
Xperience ConsultingSpain
User IntelligenceNetherlands/
Finland
UXa Partners, Associates, & Affiliates
Optimal UsabilityNew Zealand
Mitsue LinksJapan
Think UserKorea
AxanceFrance
UIDesign GroupRussia
Optimal ExperienceAustralia
Mercedes SanchezBrazil
SymetriaPoland
Snitker & Co.Denmark
ExperienceLabUnited Kingdom
Yu CentrikCanada
Ayer ViernesChile
TelonoSwitzerland
ExperienceUCzech Republic
![Page 73: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/73.jpg)
About UserZoom
Leading online (remote) user experience research software company
9 years experience in UX research & consulting
Offices in Sunnyvale (CA), London (UK) and Barcelona (Spain)
Created because traditional UX research methods are great but not enough
![Page 74: Four Country Mobile Operator Study: a competitive benchmark of leading mobile operator sites1](https://reader037.fdocuments.us/reader037/viewer/2022103111/54c8a0f24a7959fc278b4591/html5/thumbnails/74.jpg)
Sunnyvale (USA)
440 N. Wolfe Rd. Sunnyvale,CA 94085 Phone: +1 (408) 524 7445 Contact: Alfonso de la Nuez
Barcelona (Spain)
Av. Diagonal 419 3º 2ª 08008 Barcelona Phone: +34 93 414 7554 Fax: +34 93 209 8380 Contact: Xavier Mestres
Cheshire (UK)
Booths Park. Chelford Road, Knutsford, Cheshire WA16 8GSPhone: + 44 (0)1565 759890Cell: + 44 (0)7900 472 920Contact: Arthur Moan
Thanks so much for your time!
Check out our upcoming events at www.userzoom.com