Eye Tracking the UX of Mobile: What You Need to Know
-
Upload
jennifer-romano-bergstrom -
Category
Technology
-
view
1.050 -
download
4
Transcript of Eye Tracking the UX of Mobile: What You Need to Know
Eye tracking the UX of mobile: What you need to know
Jen Romano Bergstrom UX Researcher Facebook @romanocog
1
Jon Strohl UX Researcher Fors Marsh Group @jonstrohl
March 23, 2015 ModevUX | Arlington VA
We will be answering these questions!
2
Ø Why is mobile important?
Ø What makes mobile different?
Ø Do I need to test my design? • How do I get started? • What are the benefits?
Ø Do I need to include eye tracking? • How do I get started? • What are the benefits? • Can you show me how it’s done? • Are there any examples?
The rise of mobile
3
@romanocog @jonstrohl #modevux
Mobile internet usage continues to increase
4
Mobile retail usage increases
5
@romanocog @jonstrohl #modevux
App usage continues to increase
6
7
What we design for…and reality @romanocog @jonstrohl
#modevux
Krug, S. (2000) Don’t Make Me Think. Pearson Education *NEW 3rd Edition (2014)*
@skrug
8
8
Ø Controlled environment
Ø All par>cipants have the same experience
Ø Record and communicate from control room
Ø Observers watch from control room and provide addi>onal probes (via moderator) in real >me
Ø Incorporate physiological measures (e.g., eye tracking, EDA)
Ø No travel costs
LABORATORY REMOTE IN THE FIELD Ø Par>cipants tend to be
more comfortable in their natural environments
Ø Recruit hard-‐to-‐reach popula>ons (e.g., children, doctors)
Ø Moderator travels to various loca>ons
Ø Bring equipment (e.g., eye tracker)
Ø Natural observa>ons
Ø Par>cipants in their natural environments (e.g., home, work)
Ø Use video chat (moderated sessions) or online programs (unmoderated)
Ø Conduct many sessions quickly
Ø Recruit par>cipants in many loca>ons (e.g., states, countries)
> FMG Capabilities
Where to test
9
When to test
Iterative Process Throughout Development
@romanocog @jonstrohl #modevux
10
Research goals and
plan Form
hypotheses Will eye
tracking add value?
Design the study
Planning the test @romanocog @jonstrohl
#modevux
11
Who is involved
SATQ Items
Other materials
Debriefing
Reports
.
. .
.
. .
.
“Make it a spectator sport” @skrug
@romanocog @jonstrohl #modevux
Project Managers
UX Designers
Executives
Engineers
Researchers
User Experience (UX) Defined
12
@romanocog @jonstrohl #modevux
Adapted from Morville, P. Semantic Studios. http://semanticstudios.com/user_experience_design/
A combination of measures are used in UX research.
@romanocog @jonstrohl #modevux
Using multiple measures allows researchers to capture the most detailed and complete picture of the user experience.
14
OBSERVATIONAL First click accuracy
Task accuracy Time on page/task
Selection/click behavior Conversion rate
SELF-REPORT Satisfaction ratings
Difficulty ratings Think-aloud protocol Debriefing interview
UX Measures @romanocog @jonstrohl
#modevux
15 http://blog.thomsonreuters.com/wp-content/uploads/2013/01/smartphones1.jpg
Lots of sizes to consider… @romanocog @jonstrohl
#modevux
16 http://armodilo.com/support/supported-tablets/
Don’t forget about tablets! @romanocog @jonstrohl
#modevux
…our thumbs are only so big
17 http://gizmodo.com/how-to-design-for-thumbs-in-the-era-of-huge-screens-1636955848
…and people use them differently.
18
@romanocog @jonstrohl #modevux
Hoober (2013). UX Matters. http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php
Eye tracking and attention
19
@romanocog @jonstrohl #modevux
@agabojko
Bojko, Aga (2013). Designing the User Experience. Rosenfeld Media
User Experience (UX) Defined
20
Eye tracking helps to
explain how usable,
engaging, and desirable a product is.
@romanocog @jonstrohl #modevux
21
OBSERVATIONAL First click accuracy
Task accuracy Time on page/task
Selection/click behavior Conversion rate
PHYSIOLOGICAL Eye movements
Electrodermal activity (EDA) Pupil dilation
UX Measures @romanocog @jonstrohl
#modevux
SELF-REPORT Satisfaction ratings
Difficulty ratings Think-aloud protocol Debriefing interview
User Experience (UX) Measures • Self-report metrics tell us why participants focus on
certain site aspects.
22
• Eye tracking tells us what, how long, and how often participants focus on design elements.
• The combination of self-report, observational, and implicit data allows us to better understand perceptions and behavior.
• Observational metrics tell us how participants navigate and interact.
We do not use eye tracking in isolation.
@romanocog @jonstrohl #modevux
23
Eye Tracking Considerations @romanocog @jonstrohl
#modevux
@agabojko
Bojko, Aga (2013). Designing the User Experience. Rosenfeld Media
User Experience Testing + Eye Tracking
24
UX Testing UX Testing with Eye Tracking
Data while completing tasks
Mouse movements, clicks, comments
Scan paths, dwell time, first fixation, re-fixations. Eye-tracking helps determine what led to these behaviors and comments
Findings Usability problems are detected
A more thorough understanding of usability issues and the sources of these issues
Recommendations Quality recommendations based on available data
More insightful and precise recommendations
@romanocog @jonstrohl #modevux
Eye-Tracking Metrics ENGAGEMENT Number of fixations
Total dwell time Percentage of time on
an area
FINDABILITY Time to first fixation Number of fixations prior to first fixation
PROCESSING Fixation
durations
WORKLOAD/EXCITEMENT Pupil dilation
PROCESSING ORDER
Gaze path
COMPREHENSION Repeat fixations
25
26
Data Visualization– Heat & Opacity Maps @romanocog @jonstrohl
#modevux
27
Data Visualization– Gaze Plots @romanocog @jonstrohl
#modevux
28
Modern eye tracking @romanocog @dcaapor
Eye Tracking in the Past
29
Modern eye tracking
Modern Eye Tracking
Mobile Device Stand
30
Glasses
31
Shopping Research
32
Driving Research
33
Design Evaluation Research
34
Moderator & Participant
35 Slide from: Alayoubi, L. & Romano Bergstrom, J. (2014). Paying for Education: User-Centered Research on the Paying for College Website and Tools. Presentation at edUi, Richmond, VA, Sept 2014.
Example Methodology – Formative Evaluation
Eye Tracker
Participants: • N = 9 • Mix of gender, ethnicity, income. • Formative evaluation with a single
web prototype.
User Experience Testing session: • Participants read a description of
the study. • The moderator gave instructions
and calibrated the eye tracker. • Participants completed a set of
tasks provided individually by the moderator.
• End-of-session satisfaction questionnaire.
• Debriefing interview.
36
Formative Studies Ø Are <10 participants appropriate?
• Very often. Typical for finding large usability problems. • Problem occurrence is 25% or higher. • Problem discovery goal is 75% or less.
Ø Are >20 participants appropriate? • Not as common, but sometimes used when testing more mature
applications or trying to find smaller, less frequent issues. • Problem occurrence is 10% or higher. • Problem discovery goal is 90%.
Ø If eye tracking, over recruit by ~20% 1.Calculations derived from:
Sauro, J., & Lewis, J. R. (2012). Quantifying the user experience: Practical statistics for user research. Elsevier. Bojko, A. (2013). Eye Tracking the User Experience. Rosenfield Media.
Determining Sample Size1 @romanocog @jonstrohl
#modevux
UX Test with Eye Tracking Demo
37
@romanocog @jonstrohl #modevux
FLEET • Beta • Flyfleet.org Features: • Real-time flight status updates • Let friends and family track your flight • Get delay times at check-in and
security • Find stores and restaurants near you
Fleet was designed and developed by Rob Youmans, Daniel Gartenburg, Melissa Smith, Peter Lee, Jordan Higgins, Octavian, and Vincent Mamo
38
Example Methodology – Summative Evaluation Participants: • N = 74 | Average Age = 37 • Mix of gender, ethnicity, income. • Summative Evaluation with a between
groups design. • New, Old, Prototype, and Bilingual
media diaries.
User Experience Testing session: • Participants read a description of the
study. • The moderator gave instructions and
calibrated the eye tracker. • Participants completed Steps 1-5 in the
diary at their own pace. • End-of-session satisfaction questionnaire. • Debriefing interview.
Eye Tracker
Moderators worked from another room.
Control Room
Slide from: Walton, L., Romano Bergstrom, J., Hawkins, D. & Pierce, C. (2014). User Experience and Eye-Tracking Study: Paper Diary Design Decisions. Paper presentation at the American Association for Public Opinion Research (AAPOR) Conference, Anaheim, CA, May 2014.
39
1.Calculations derived from: Sauro, J., & Lewis, J. R. (2012). Quantifying the user experience: Practical statistics for user research. Elsevier. Bojko, A. (2013). Eye Tracking the User Experience. Rosenfield Media.
Summative Studies Ø Are <10 participants appropriate?
• Very rarely, but sometimes when trying to detect very large differences. • Difference between groups is 60% or more. • Assuming our confidence goal is 90%. • Assuming our power is 80%.
Ø Are >20 participants appropriate? • Yes, very often. Typical for detecting moderate differences with
confidence. • Difference between groups is 30% or more. • Assuming our confidence goal is 90%. • Assuming our power is 80%.
Ø If eye tracking, over recruit by ~20%
Determining Sample Size1 @romanocog @jonstrohl
#modevux
Example Methodology – Naturalistic environment
40
Participants: • N = 8 • Mix of gender, income. • Participants use their own
device and Internet connectivity.
User Experience Testing session: • Sessions occurred in participant’s home. • The moderator gave instructions. • Sessions began with an interview about
how participants use the Internet. • Participants completed a set of tasks
provided individually by the moderator. • Debriefing interview.
What can we learn from eye tracking?
42
Fixation count heat map of 10 seconds of interaction with the page. 20 participants included.*
Fixation count heat map of 10 seconds of interaction with the page. 10 Novice participants included.*
Fixation count heat map of 10 seconds of interaction with the page. 9 Experienced participants included.*
All Participants Novice Experienced
*Heatmaps include participants who located the link from the Refunds page only. Fixation count refers to the number of times the eye fixates, or directs itself towards, an area of the page.
Novice and experienced users focus on different elements.
43
Making something large does not mean people will notice it.
@romanocog @jonstrohl #modevux
44
New/Old DVR Grid New/Old DVR Grid Prototype Grid
Grids that match users’ mental model are easier to process.
Walton, L., Romano Bergstrom, J., Hawkins, D., & Pierce, C. (2014). User experience testing and eye tracking inform paper diary design. Proceedings from the Human Computer Interaction International Conference, June 2014, Crete, Greece.
@romanocog @jonstrohl #modevux
45
Gaze plot of 60 seconds of engagement from four participants on the Loan Rehabilitation page who scanned the text without reading it thoroughly.
Gaze plot of 60 seconds of engagement from four participants on the Contact Your Servicer page who read most of the text.
People do not read dense text.
@romanocog @jonstrohl #modevux
46
The text within the orange circle indicates the information to enter on the page. The red circled area is where the user enters the ID number. The participant skips the instructions and introductory text and moves on to the actionable areas.
People do not read instructions.
He, J., Siu, C., Chaparro, B. & Strohl, J. (2014). Mobile. In J. Romano Bergstrom & A. Schall (Eds.), Eye Tracking in User Experience Design. San Francisco, CA: Morgan Kaufmann.
@romanocog @jonstrohl #modevux
47
If the mandatory fields were not completed, participants received an error message that was not helpful. This gaze plot begins after the participant clicks “Ok.” The participant looks all over the screen, searching for the missing mandatory fields. The error message did not indicate which fields were missing.
People need to know what to do next
He, J., Siu, C., Chaparro, B. & Strohl, J. (2014). Mobile. In J. Romano Bergstrom & A. Schall (Eds.), Eye Tracking in User Experience Design. San Francisco, CA: Morgan Kaufmann.
@romanocog @jonstrohl #modevux
Activity
48
Awesome! I want to use eye tracking. What’s next?
50
Eye-Tracking Decision Progression
Research goals and plan Form hypotheses Will eye tracking
add value? Design the study
Collect data Clean & prepare data
Visualize qualitative data
Analyze quantitative data
51
Evidence: Younger and middle-age social media users access the sites/apps largely through smartphones, whereas older users access largely through desktop computers. It is important to test across user groups and across devices.
Older Adults Middle Age Adults Younger Adults
People fixate elements of displays differently across devices. Different user groups access via different interfaces
@romanocog @jonstrohl #modevux
52
Android App Homepage Tablet Web Homepage Tablet App Homepage
People fixate elements of displays differently across devices. People attend to elements of displays differently across devices
Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data Collection Tool. Paper presentation at the AAPOR Conference, Anaheim, CA, May 2014.
@romanocog @jonstrohl #modevux
Thank you!
Jen Romano Bergstrom UX Researcher Facebook @romanocog [email protected]
Jon Strohl UX Researcher Fors Marsh Group @jonstrohl [email protected]
March 23, 2015 ModevUX | Arlington VA
54
Website www.forsmarshgroup.com LinkedIn hXp://www.linkedin.com/company/fors-‐marsh-‐group Blog www.forsmarshgroup.com/index.php/blog TwiXer @forsmarshgroup