Which Eye Tracking Systems Are Best Suited for Usability ... · Eye tracking systems have a history...

UniversityofSouthDakotaHumanFactorsDepartment–414EastClarkStreet–Vermillion,SD57069 1

Which Eye Tracking Systems Are Best Suited for Usability Studies? University of South Dakota Denis Kozhokar 4/15/2018


TableofContents

EXECUTIVESUMMARY...........................................................................................................3

LOWCOSTSYSTEMTRADEOFFS.........................................................................................4EyetrackingTechniquesandOutput.........................................................................................4

HARDWAREDETAILS.............................................................................................................5Typicalrequirements......................................................................................................................5

SOFTWAREDETAILS...............................................................................................................5IdealRequirements..........................................................................................................................5

Hardware+SoftwareProducts...........................................................................................6Tobii......................................................................................................................................................6

Hardware Review........................................................................................................................................6Bundles..............................................................................................................................................................6Software Review..........................................................................................................................................7

Gazepoint.............................................................................................................................................8Harware Review...........................................................................................................................................8Bundles..............................................................................................................................................................8Software Review..........................................................................................................................................9

EyeTech...............................................................................................................................................9Hardware Review........................................................................................................................................9Software Review.......................................................................................................................................10

HARDWAREONLYPRODUCTS..........................................................................................11MirametrixS2..................................................................................................................................11MyGaze...............................................................................................................................................11PupilLabsEyeGlasses...................................................................................................................12

COMPATIBLESOFTWAREPRODUCTS............................................................................13iMotions&EyeWorks....................................................................................................................13

FULLSPECIFICATIONSOFHARDWARE.........................................................................14


EXECUTIVE SUMMARY This document summarizes the capabilities of seven low-cost eye trackers (most are under $3000) for their appropriateness in user research. Included in the review were the following systems: Tobii X2-30, Mirametric S2, MyGaze, Gazepoint GP3, Gazepoint GP3 HD, EyeTech VT3 Mini, and Pupil Labs Tracking Headset. A tabular comparison of eye trackers is provided for sampling rate, precision, price, eye capturing efficiency, portability, cross-platform capability, and head movement acclamation. Low cost solutions often involve tradeoffs to ensure a low price point. Reviews of the system hardware document any tradeoff made in developing each system. Also considered were the guidelines for software capabilities that were established based on fixation metrics, gaze metrics, screen tracking, flexibility, and ease of use. Recommendations and reviews were given on criteria for products within web and mobile technology. Tiers for recommendations: Tier 1 Price Tobii X2-30 with Enterprise Software $15,000

Tier 2

Gazepoint GP3 HD Usability Bundle $3,800

Gazepoint GP3 Usability Bundle $2,800

Gazepoint GP3 HD UX Edition $3,490

Gazepoint GP3 UX Edition $2,190

Pupil Labs Eye Glasses $1,230 - $2,650

Eyetech VT3 Mini with Mangold Vision $3,000 + software (Unavailable)

Tier 3

MyGaze $500

Mirametrix Unavailable

Due to Gazepoint eye trackers not being compatible with mobile devices, we recommend the Gazepoint GP3 HD Usability Bundle + Pupil Labs Eye Glasses as an alternative ($6,450).


LOW COST SYSTEM TRADEOFFS There are many tradeoffs to consider when choosing an eye tracking system. Eye tracking systems have a history of being overpriced and thus the cost benefit ratio has been historically low in user research and usability studies. Advances in computer tracking algorithms, improved camera capability, and hardware component availability have led to lower priced equipment. As high-end research grade tracking systems have expanded, eye tracking companies have developed UX and usability specific systems capitalizing on better software and affordable hardware. An important issue to consider is the amount of expertise and training necessary to gather accurate and useful information from the eye tracking systems. Eye tracking Techniques and Output

Although eye tracking systems has the capability of extracting many forms of eye movement data, two are of particular interest in user research. Fixations are locations where users are looking long enough for the information to process from the visual system. Saccades are movements of a user’s eyes between fixation points or prolonged gazes. The information from saccades demonstrates that the speed of the movement means the user is not processing what they are seeing. More importantly, visualizing saccades shows the path the eyes are taking between fixations. The most common method of measuring the attention of the user’s gaze is by the position of a near-infrared light (reflected in the eye) to the position of a pupil. A combination of this method with the position of the participant’s head, can extract information to determine the point at which the user’s eyes are focused on the corresponding screen coordinates. Therefore, UX and usability professionals tend to focus on these areas:

• The ordered list of fixations (and an unordered list of overlooked elements): First and foremost, eye tracking highlights both what the user sees and what they don’t see.

• The time taken to arrive at any given fixation: This can be related to how easy or difficult an element was to find.

• The length of any given fixation: This can be related to how engaging or

comprehensible an element might have been.

• The number of fixations per element: This can be related to how distracting, useful or contradictory an element might have been.


HARDWARE DETAILS The criteria that was used to assess the different eye trackers was based on price and usability research requirements. High-end eye trackers are not cost-effective for usability studies. Therefore, a limit of <$10,000 was set when comparing the eye trackers. 7 eye trackers were found that have been used for UX and usability purposes and met typical requirements. They are theTobii X2-30, Mirametric S2, MyGaze, Gazepoint GP3, Gazepoint GP3 HD, EyeTech VT3 Mini, and the Pupil Labs Tracking Headset. Typical requirements

• Measurement precision – From 0.5 – 1.0 degrees • Trackability – Tracks Eye position, Gaze point, and Pupil diameter • Sampling rate - From 30-60 Hz (more is preferred) • Headbox size - around 11 inches / 28 cm • Recapture rate - fast enough to recapture from blinks • Support – Company assistance on hardware malfunctions

SOFTWARE DETAILS Ideal Requirements

• Scalable and flexible to meet various research needs. • Accommodates both data recording and data analysis. • Tracks Screen-based stimuli (video, images, websites, screen capture, and mobile

devices) and survey stimuli for self-reports. • Simple and easy to adjust sampling rate, calibration, and gaze or saccade/fixation

detection. • Able to generate heat maps, select Areas of Interest (AOIs), descriptive statistics, and

place markers.


Hardware + Software Products Tobii

Hardware Review

Required Tobii X2-30 PROS CONS Measurement

precision 0.4° ✓ List List

Trackability

Eye position Gaze point

Pupil diameter

✓ Very accurate with head

motion and has a robust calibration

Good accuracy: Most accurate out of all

compared eye trackers (<0.5°)

Smallest and most

portable eye tracker Can be mounted to the largest range of screen

sizes (25”)

Can be used on Tablets, mobile devices,

and computers

VERY expensive: Can cost over 5x more than other

eye tracking hardware

Head needs to be relatively low with

respect to the monitor

Only Windows Accessible

Sampling rate 30 Hz ✓

Headbox size 50 x 36 cm ✓

Recapture rate Immediate ✓

Support

https://www.tobii.com /support/

✓

Bundles

Tobii UX Live Package

Price: $15,000 Pro X2-30 hardware for recording eye movements

Tobii Pro Studio (live edition) software for live viewing and replay

Mobile Device Stand accessory for mobile device testing (optional)

Two-hour online training session

Support available from Tobii


Software Review

Requirements Comparisons

Software Tobii

Studio Live

Tobii Studio Enterprise

Tobii Pro Studio Live

Tobii Pro Studio Enterprise

Scalable and Flexible X ✓

Price: Unknown (Expensive) Price: Unknown

(VERY Expensive)Data

Recording and Data Analysis

✓ ✓

Screen Capture Screen Capture

Screen-based stimuli X ✓ Eye Gaze Eye Gaze

Mobile Device Testing X ✓

Keystroke and mouse clicks Keystroke and mouse

clicks

Heat maps X ✓ Retrospective

Think Aloud (RTA) and Webcam Recording

Retrospective Think Aloud (RTA) and Webcam Recording

Easy to use ✓ ✓ Raw data export Raw data export

AOIs X ✓ Live Viewer Live ViewerDescriptive Statistics X ✓ Playback of gaze

and video clipsPlayback of gaze and video clips

Place Markers X ✓ Fixation Filer Fixation Filer

Heat Map

Gaze plot

Web page transitions

and scrolling

Screen Capture / Image / Video / Web Multiple User Data Aggregation

Dynamic Areas of Interest (AOIs) and automatic cluster AOIs


Gazepoint

Harware Review

Required Gazepoint PROS CONS

Hardware GP3 GP3 HD List List

Measure precision 0.5 – 1.0° ✓ ✓

One of the most intuitive usability and UX software on the

market

Best sampling rate to price point ratio

GP3 HD is an

alternative for a larger range of head

movement and a higher sampling rate

Low accuracy in gaze point estimation

Only windows accessible

Has issues with

glare from glasses

Restricted amount of head movement

Trackability

Eye position

Gaze point Pupil

diameter

✓ ✓

Sampling rate 60 Hz ✓ 150

Hz ✓

Headbox size 25 x 11 cm X

35 x 22 cm ✓

Recapture rate Immediate ✓ ✓

Support

https://www.gazept.com/support/

✓ ✓

Bundles

Gazepoint GP3 Gazepoint GP3 HD

UX Bundle Usability Bundle UX Bundle Usability Bundle

Price: $2,190 Price: $2,800 Price: $3,490 Price: $3,800

GP3 Eye Tracker GP3 Eye Tracker GP3 HD Eye

Tracker GP3 HD Eye Tracker

Gazepoint Analysis UX

Edition

Gazepoint Analysis UX Edition

Gazepoint Analysis UX

Edition

Gazepoint Analysis UX Edition

Gazepoint Laptop/Notebook Mount

Gazepoint Laptop/Notebook Mount

Gazepoint Vesa Mount Gazepoint Vesa Mount


Remote Viewer Software Module

Remote Viewer Software

Module

3 Year Warranty 3 Year Warranty

Software Review

Requirements Details Software Gaze Point

Analysis UX Edition

Gaze Point Analysis UX Edition

Scalable and Flexible ✓ Price: $1,495Data Recording and

Data Analysis ✓ Heat Map

Screen-based stimuli ✓ Gaze Fixation Path Mobile Device Testing X Screen Capture / Image / Video / Web

Multiple User Data AggregationHeat maps ✓ Dynamic Areas of Interest (AOIs)Easy to use ✓ Image, Video and Statistics Export

AOIs ✓ Thinkaloud Voice & Webcam Recording

Descriptive Statistics ✓ Live viewerPlace Markers ✓ 1 year software updates and technical

support

Eye Tech

Hardware Review

Required Eyetech VT3 Mini PROS CONS Measurement


Trackability Eye position Gaze point

Pupil diameter ✓

Strobing IR lights enable quick eye

detection.

Smallest range of operating distance between the other

alternatives.


Sampling rate 120 Hz ✓ Eye tracking software is

included in the price ($3,000)

Compatible with tablets, phones, and computers.

Creates hardware time-stamped data and has

high-speed data processing power.

Tracker automatically

switches between monocular and

binocular tracking.

Cannot capture larger screen

sizes (EyeTech VT3 XL is an

option) .

Windows only capability.



Support https://www.eyetechds.com/s

upport.html ✓

Software Review

Requirements Details

Software Mangold

Vision Software

Mangold Vision Software (Eyetech VT3 Mini)

Scalable and Flexible ✓ Price: Unknown (Expensive)Data Recording and Data

Analysis ✓ Simultaneous eye tracking capability

Screen-based stimuli ✓ Heat Map

Mobile Device Testing ✓ Gaze plot

Heat maps ✓ Focus maps / Heat maps

Easy to use ✓ AOI / AOI Gaze pattern

AOIs ✓ Descriptive statistics of gaze data with tables and charts

Descriptive Statistics ✓ Data export visualizations

Place Markers ✓


HARDWARE ONLY PRODUCTS Mirametrix S2

Required Mirametrix S2 PROS CONS Measurement

precision 0.5 – 1° ✓ List List

Trackability Eye position Gaze point X Takes a few minutes to

set up and calibrate

Portable and Affordable

Compensates for glasses, head tilting,

and excessive blinking

Only Windows accessible

Not as diverse in

features from software


Headbox size 25 x 30 cm X


Support No support X

MyGaze

Required MyGaze PROS CONS Measurement


Trackability


Pupil diameter

✓ Lengthy USB cable

Lowest price when compared to the other

eye trackers

Still precise and accurate with the low

price

Very portable, light, and can be attached to most

computer screens

The user interface is not as smooth

as others

Difficult to navigate for

analyzing data

Slow recovery time

Can only be used on computer

screens

Smallest range of screen size compatibility




Support No support X


Pupil Labs Eye Glasses

Required Pupil Labs Eye Glasses

PROS CONS Measurement


Trackability


Pupil diameter

✓ Robustly tracks markers in order to map gaze

onto any surfaces

Good tracking: High precision and accuracy

Fully cross platform: Windows, OSX and

Linux

Uses open source software with easy to

use software and many features

Low price relative to

sampling rate. Has complete head

motion capability

Can be used with tablets, mobile devices,

and computers).

Can go out of calibration quickly

with excessive facial movements

Needs to be

recalibrated every time

it is put on


Headbox size Free Head Motion ✓


Need to wear something on your head to keep the glasses in place.

Does not work well with uses

wearing glasses

Support Open Source X


COMPATIBLE SOFTWARE PRODUCTS iMotions & EyeWorks

Requirements Comparisons

Software iMotions Eyeworks iMotions Eye

Tracking Software

EyeWorks Software

Scalable and Flexible ✓ ✓ Price: Unknown

(Expensive)Price: Unknown

(Expensive)Data

Recording and Data Analysis

✓ ✓

Compatible with all of

the eye trackers (excluding MyGaze)

Compatible with all eye trackers

(excluding myGaze and Pupil Labs)

Screen-based stimuli

✓ ✓

Eye gaze Eye gaze

Mobile Device Testing

✓ ✓

Mouse clicksMouse clicks,

scrolling, and key presses

Heat maps ✓ ✓

Heat maps of AOI’s Scripting capabilities

Easy to use ✓ ✓

Time to first fixation (TTFF) / Task time /

Fixations / Ratio

Gaze position / fixations / blinks /

task time / questionnaire results

AOIs ✓ ✓

Pupil Size Metric / Distance metric

Live viewer with remote streaming

capability

Descriptive Statistics ✓ ✓

Gaze path and gaze plot / Gaze replay

Screen Capture / Image / Audio /

Video / Web Multiple User Data

Aggregation

Place Markers ✓ ✓

Individual and/ or aggregated data

export

Descriptive statistics tables and graphs

Static and dynamic AOIs Dynamic AOIs

Color coded heat maps

Retrospective Think Aloud


FULL SPECIFICATIONS OF HARDWARE

Spec. Tobii X2-30

Mirametrix S2 MyGaze

Gazepoint GP3

Gazepoint GP3 HD

EyeTech VT3 Mini

Pupil Labs

Sample rate 30 Hz 60 Hz 30 Hz 60 Hz 150 Hz 120 Hz 200 Hz Accuracy 0.4° 0.5 – 1° 0.5 ° 0.5 – 1° 0.5 – 1° 0.5° 0.6° Mount type On

screen On

screen On

screen On

screen On

screen On

screen Eye

Glasses Screen size Up to

25″ Unavailable Up to 22″ Up to 24″ Up to 24″ Up to 22″ Any

Operating Distance

40-90 cm Unavailable 50 – 75 cm

50 – 80cm Unavailable 50 – 70 cm

N/A

Head Movement

50 x 36 cm

25 x 11 x 30 cm

32cm x 21cm

25 x 11 cm 35 x 22 cm 32 x 20 cm

Free head

motion Pupil size Yes Unavailable Yes Yes Yes Yes Yes Tracking Technique

Dark and bright pupil

Bright pupil Bright pupil

Bright pupil

Bright pupil Dark pupil

Dark pupil

Tracking type

Binocular Binocular Binocular Binocular Binocular Both Both

Connection Type

USB 2.0 USB 2.0 USB USB USB USB 2.0 USBA or USBC cable

Operating System

PC PC PC PC PC PC PC, Mac, Linux

Weight 0.2 kg 0.3 kg 0.13 kg 0.14 kg 0.15 kg 0.21 kg N/A Dimensions 18.4 x

2.8 x 2.3 cm

35 x 4 x 3 cm

24cm x 3cm x 3.5cm

32 x 4.5 x 4 cm

32 x 4.5 x 4.7 cm

29 x 2.5 x 2.7 cm

N/A

Price $5,000 Low (Unknown)

$500 $700 $2,000 $3,000 $1,230 - $2,650

** It is important to note that there is free open-source eye tracking software. The most used are xLabs, GazePointer, Myeye, Ogama, openEyes, PyGaze, and etc. (the most popular being OGAMA)

Which Eye Tracking Systems Are Best Suited for Usability ... · Eye tracking systems have a history...

Documents

Transcript of Which Eye Tracking Systems Are Best Suited for Usability ... · Eye tracking systems have a history...