Siri vs Alexa, Which is Better?

11www.metrostarsystems.com/digital

SIRI VS ALEXAWHICH ONE HAS THE SUPERIOR USER EXPERIENCE?

http://www.metrostarsystems.com/digital

http://www.metrostarsystems.com/



INTRODUCTION

Our Human Factors team conducted a

usability study of the user interface (voice

and graphical) of Apple’s Siri and

Amazon’s Alexa.



Participants

50%

50%



Previous Experience with Siri & Alexa

83% 17% 33% 67%

Percentage of users who

had used Siri “for many

years” prior to the test


had “played around” with

Siri prior to the test


had “played around” with

Alexa prior to the test


had “never used” Alexa

prior to the test



When conducting our usabilitytest, we chose tasks that wouldcover different aspects of a virtualassistant’s functionality. These arealso tasks that the average userswould be likely to ask a virtualassistant to complete. After eachtask, our participants were askedto rate their experience on a scaleof 1-5, 1 being total failure, 5being perfect.

TASKS



Task 1: Alarm & TimerAsk Siri and Alexa to complete a timer or clock related task

100% 100%

Distribution of participants

rating their experience on

a scale of 1-5 (5 being

perfect)


were able to complete the

task without any major

difficulties




perfect)




difficulties

SIRI ALEXA

0

1

2

3

4

1 2 3 4 5

Experience Satisfaction

0

2

4

6

8

1 2 3 4 5




Task 1

Summary

Although all participants were able to complete a timer- or

clock-related task without any major difficulty, Alexa (with

an average rating of 5) performed better than Siri (with an

average rating of 4.17) on the satisfaction rating. These

results are due to Alexa’s ability to set a timer/alarm down

to the second, whereas Siri can only do down to minutes.



Task 2: MusicAsk Siri and Alexa to perform a music related task, like “Play today’s hits on

Pandora”

16% 100%




perfect)




difficulties




perfect)




difficulties

SIRI ALEXA

0

1

2

3

4

5

1 2 3 4 5


0

2

4

6

8

1 2 3 4 5




Task 2

Summary

Alexa was able to understand our participants’ command

to play a certain type of music or music by a specific

artist. Siri was only able to open the music app and had

difficulty playing music, which resulted in low completion

rates. Participants on this task reported an average

satisfaction rate of 4.67 for Alexa, and an average rating

of 2 for Siri.



Task 3: Travel & CommuteAsk Siri and Alexa a travel or commute related question like, “What is traffic like right

now?”

100% 50%




perfect)




difficulties




perfect)




difficulties

SIRI ALEXA

0

1

2

3

4

5

1 2 3 4 5


0

0.5

1

1.5

2

2.5

1 2 3 4 5




Task 3

Summary

For this task, Siri performed slightly better than Alexa with

an average rating of 4 (Alexa was 2.28). Alexa requires

users to configure their commute, work, and home

locations, and is only able to provide general traffic

information. With Siri, no configuration is required,

however, many users still had to go through multiple

commands before receiving navigation instructions. Siri

still received a 4 overall because it provided adequate

information. One participant was able to ask Alexa the

“right” question and receive routing instructions.



Task 4: Random Question/ConversationAsk Siri and Alexa anything you want (up to three questions or two minutes,

whichever comes first)

66% 33%




perfect)

Percentage of users that



difficulties




perfect)

Percentage of users that



difficulties

SIRI ALEXA

0

1

2

3

4

5

1 2 3 4 5


0

0.5

1

1.5

2

2.5

1 2 3 4 5




Task 4

Summary

Participants asked Siri and Alexa any question they could

think of. Siri performed slightly better than Alexa with an

average satisfaction rating of 3.16 (Alexa’s was 2.67).

Participants noted that Siri was able to provide

informative answers to their questions, while Alexa was

not able to do that. Over 60% of participants thought their

conversation/questions with Siri was successful, while

only 33% of participants felt this way about Alexa.



Overall, our participants were slightly more satisfied with their experience with Alexa, which received a satisfaction rating of 3.83 (Siri received 3.17). When it came to ease-of-use though, Siri and Alexa received the same rating (3.83).

OVERALL

IMPRESSIONS



How Would You Describe Siri & Alexa?

SIRI

Funny

Helpful

Well-Meaning

Basic

ALEXA

Beta

Newbie

Cool

Competent

Inconsistent

Alexa was the only to receive a negative adjective, which is mostly likely because Alexa is very good at some tasks (like music). However, Alexa was

not very good (or did not work as expected) at other tasks, like giving directions or describing traffic conditions.

Easy to Use Cool



After aggregating all of the task statistics and overall satisfaction scores, Siri received an overall rating of 3.88 and Alexa, 3.71. These ratings were consistent with the qualitative data of the positive adjectives participants used to describe Siri. Alexa and Siri both demonstrated strengths, like Alexa’s music capabilities and Siri’s navigation capabilities.

Summary



About Us

LI LUUser Experience Architect

Li Lu is an User Experience Architect at

MetroStar Systems. She has a passion

for observing and interpreting human

behavior and translating business and

user needs into meaningful and pleasing

user experiences. Her diverse

background include human factors

engineering, user experience design,

and research. She holds a Human

Factors Master's degree from George

Mason University.

From the rough drawings on cave walls to the recap over coffee

about what happened in yesterday's business meeting, humans are

natural storytellers. Every organization, brand and product is built on

a narrative that is told through imagery, copy and multimedia.

Whether it’s the FCC’s promise to provide transparent information to

the American people or AAFMAA’s passion behind providing

premier military insurance, narratives are what connect with your

audience and help you meet business objectives. From copywriting

to graphic design and videography, our technical innovators and

user experience designers create the content for your foundational

narrative.

MetroStar’s Digital Practice


http://www.metrostarsystems.com/practice/digital/



Learn more about our Digital services, which include UI/UX, Human Factors, and Web & Mobile Design visit:

www.metrostarsystems.com






Siri vs Alexa, Which is Better?

Technology

Transcript of Siri vs Alexa, Which is Better?