Eye Tracking for Surveys on Smartphones

May 14, 2016 AAPOR annual conference | Aus9n, TX

Eye Tracking for Surveys on Smartphones

Jen Romano-‐Bergstrom UX Researcher

[email protected]

Mobile Internet Users > Desktop

2 hIp://www.smar9nsights.com/mobile-‐marke9ng/mobile-‐marke9ng-‐analy9cs/mobile-‐marke9ng-‐sta9s9cs/

#AAPOR @romanocog

Mobile Internet Usage is on the Rise

3 hIp://www.smar9nsights.com/mobile-‐marke9ng/mobile-‐marke9ng-‐analy9cs/mobile-‐marke9ng-‐sta9s9cs/

#AAPOR @romanocog

Simultaneous Usage

4 Nielsen: The Cross-‐PlaTorm Report, Quarter 2, 2012-‐US

#AAPOR @romanocog

Simultaneous Usage is Frequent

5 Nielsen: The Cross-‐PlaTorm Report, Quarter 2, 2012-‐US

#AAPOR @romanocog

User Experience Metrics

6

SELF-‐REPORT •  Difficulty ra9ngs •  Sa9sfac9on ra9ngs •  Think-‐aloud protocol •  Moderator debriefing

interview

OBSERVATIONAL •  Time to selec9on •  Click/selec9on paIerns

#AAPOR @romanocog

Self-report data combined with observational data is very useful…

but there is a filter.

User Experience Metrics

8

SELF-‐REPORT •  Difficulty ra9ngs •  Sa9sfac9on ra9ngs •  Think-‐aloud protocol •  Moderator debriefing

interview


IMPLICIT •  Eye tracking •  Electrodermal ac9vity (EDA) •  Behavioral analysis •  Verbaliza9on analysis •  Pupil dila9on

What

Why

What, how long, how oden people focus on things

We do not use eye tracking in isola9on.

#AAPOR @romanocog

9

Qualitative QuantitativeSelf-ReportSatisfaction and difficulty questionnaires YES YES

Verbal think aloud YES NOModerator follow up YES NOObservationalTime to selection NO YESSelection/click behavior YES NOImplicitVerbalization analysis YES YESEye tracking YES YES

Data #AAPOR

@romanocog

Eye Tracking in the Past

10

#AAPOR @romanocog

Future DirecDons in UX

11

Modern Eye Tracking

Mobile Device Stand

12

Fors Marsh Group UX Lab -‐ Jon Strohl and I created a non-‐invasive mobile device stand.

Facebook UX Lab – Dave Poole and I aIempted to create a non-‐invasive mobile device stand.

#AAPOR @romanocog

Mobile Device Stand

13 Facebook UX Lab

Issues: -‐  Sight camera posi9on -‐  Not natural to hold -‐  Poor tracking quality

#AAPOR @romanocog

Glasses

14

#AAPOR @romanocog

Glasses

15

16

Lab-‐Based TesDng: Do not sit with the parDcipant – they will look at YOU.

Fors Marsh Group UX Lab

We maneuver the cameras, record, and communicate through microphones and speakers from the control room so we do not interfere

Live streaming close-‐up screen shot of the par9cipant’s screen

Par9cipant in the tes9ng room

Par9cipant in the tes9ng room

Large screens to display material during focus groups

#AAPOR @romanocog

17

“Lab” on the Road – Make a Lab!

Eye Tracker

Moderator & Participant

Fors Marsh Group – Lab on the Road

What can we learn from eye tracking?

People don’t read important parts of survey invitaDon leUers.

19

Olmsted-‐Hawala E., Wang, L., Willimack, D., Burke, E., & Lakhe, S. (2016). A pilot inves9ga9on of the associa9on between eye-‐tracking paIerns and self-‐reported reading behavior. Proceedings from the Human Computer Interac6on Interna6onal Conference, July 2016, Toronto, Canada.

#AAPOR @romanocog

Grids that match users’ mental models are easier to process.

20

Walton, L., Romano Bergstrom, J., Hawkins, D., & Pierce, C. (2014). User experience tes9ng and eye tracking inform paper diary design. Proceedings from the Human Computer Interac6on Interna6onal Conference, June 2014, Crete, Greece.

#AAPOR @romanocog

People read pages with quesDons on them differently than other pages.

21 JarreI & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

The F-‐shaped eye-‐tracking paIern of the block of text at the top of the page is completely different from the eye-‐tracking paIern on the ques9on and answer spaces at the boIom of the page.

#AAPOR @romanocog

People only read what they need to read.

22 He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

Error messages should suggest how to correct and should be near the error.

23 He, Siu, Strohl, & Chaparro (2014). Mobile. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

Gaze Plot: Ader gesng an error message, the par9cipant had to search all over the screen to find the missing field.

#AAPOR @romanocog

People consume content quickly on mobile.

24

% who looked = 80.0% Time looked = 7.789 sec.

Approx Time Ads on Screen = 14 sec.

% who looked = 96.4% Time looked = 3.45 sec.

Approx Time Ads on Screen = 7 sec.

N=70 per group

People consume mobile content on Facebook faster than on a desktop (1.7s v.2.5s).

hIp://adage.com/ar9cle/digitalnext/facebook-‐twiIer-‐mobile-‐content-‐consumed-‐differently/302397/

#AAPOR @romanocog

Certain elements are more salient than others: Faces aUract aUenDon

25 Burridge (2014). Social Media. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

Certain elements are more salient than others: Faces and color aUract aUenDon

26 Burridge (2014). Social Media. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

When NOT to track

28 JarreI & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

29

Slot-In Survey Answers

JarreI & Romano Bergstrom (2014). Forms and Surveys. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

30

Gathered Survey Answers


#AAPOR @romanocog

31

Created Survey Answers


#AAPOR @romanocog

32

Third-Party Survey Answers


#AAPOR @romanocog

33

Include eye tracking? – Summary


#AAPOR @romanocog

Future Directions

35

•  Romano Bergstrom, J. C., Olmsted-‐Hawala, E. L. & Bergstrom, H. C. (2014). Older adults fail to see the periphery during website naviga9on. Universal Access in the Informa6on Society, in press.

•  Romano Bergstrom, J. C., Olmsted-‐Hawala, E. L. & Jans, M. E. (2013). Age-‐related differences in eye tracking and usability performance: Web site usability for older adults. Interna6onal Journal of Human-‐Computer Interac6on, 29, 541-‐548.

Older adults interact differently than young

Older Adults Middle Age Adults Younger Adults

#AAPOR @romanocog

36

Older Adults Middle Age Adults Younger Adults

Need to assess

different user

groups

37 Bristol, K., Romano Bergstrom, J. & Link, M. (2014). Eye Tracking the User Experience of a Smartphone and Web Data Collec9on Tool. Paper presenta9on at the AAPOR Conference, Anaheim, CA, May 2014.

People aUend to elements of displays differently across devices

We must test across devices

Android App Homepage Tablet Web Homepage Tablet App Homepage

#AAPOR @romanocog

Improving Implicit Measures

38

Biometrics (EDA = Electrodermal

AcBvity)

Eye Tracking Facial Coding

SELF-‐REPORT •  Difficulty ra9ngs •  Sa9sfac9on ra9ngs •  Think-‐aloud protocol •  Debriefing interview


IMPLICIT •  Eye tracking •  Electrodermal ac9vity (EDA) •  Behavioral analysis •  Verbaliza9on analysis •  Pupil dila9on

#AAPOR @romanocog

Biometrics may tell a more accurate story of users’ percepDons.

39

Biometrics (EDA = Electrodermal

AcBvity)

Romano Bergstrom, Duda, Hawkins & McGill (2014). Physiological Response Measurements. In Romano Bergstrom & Schall (Eds). Eye Tracking in User Experience Design. Morgan Kaufmann.

#AAPOR @romanocog

Web Cam Eye Tracking

hIps://xlabsgaze.com/user-‐manual/ 40

#AAPOR @romanocog

AddiDonal Reading

41

Jen Romano-‐Bergstrom UX Researcher

[email protected]

Real-‐world examples from over twenty leading UX professionals

•  Tips on everything from gesng the most value out of your eye tracking research to incorpora9ng eye-‐tracking methodology into your organiza9on’s user-‐centered design process.

•  Ac9onable findings and recommenda9ons based on eye-‐tracking research

•  Insights into cusng-‐edge applica9ons of eye tracking for mobile, gaming, social media, e-‐commerce and more!

#AAPOR @romanocog

Eye Tracking for Surveys on Smartphones

Education

Transcript of Eye Tracking for Surveys on Smartphones