Robert M. Christopherson,Javier Gonzalez-Sanchez, Mustafa Baydogan,Maria Elena Chavez-Echeagaray, David-GibsonRobert Atkinson

This research was supported by Office of Naval Research underGrant N00014-10-1-0143 awarded to Dr. Robert Atkinson

COVERT AND OVERTmeasures of engagement within an educational multimedia environment.

lsrl.lab.asu.edu

Games can change the way we learn

Empirical research can change the way we game

surprise

happiness

flow/engagement

anger

delight

frustration

confusion

curiosity

anxiety

fear

boredom

Learning Gaming

Goals

GAMES: Engagement = Fun

EDUCATION: Engagement = Learning

Overt - observable Covert - hidden

Overt and Covert

Cognition

Emotion

Engagement

Motivation

Attitude

Flow

Performance

Posture

Physical interactions

Verbalization

Facial expressions

Behavior

What is engagement?

GAMING“concerned with all the qualities of an experience that really pull people in – whether this is a sense of immersion that one feels when reading a good book, or a challenge one feels when playing a good game, or the fascinating unfolding of a radio drama”

Benyon and colleagues (2005)

What is engagement?

LEARNING“the nexus of intrinsic knowledge and interest and external stimuli that promote the initial interest in, and use of a computer-based learning environment”

(Jones, 1998)

Interviews &Focus Groups

Measuring EngagementOBJECTIVE

SUBJECTIVE

QUANTITATIVE

QUALITATIVE

Observational AnalysisCognitive

walkthroughThink-aloud

Surveys &

Questionnaires

Physiological data

HeuristicEvaluation

Use of Physiological Data

1. Decide what you want to measure2. Choose the appropriate sensors3. Control your task and environment4. Process the data according to which

sensors were chosen5. Make inferences, evaluate and revise

1. Decide what you want to measure

• engagement• arousal• mental effort• attention• excitement• boredom• meditation• frustration

2. Choose the appropriate sensorsPHYSICAL PSYCHOLOGICAL

skin conductivity (GSR) arousal

pressure on controller frustration, arousal

pupil dilationpositive vs negative emotions,mental effort

brain waves (EEG) engagement, mental effort, frustration, boredom

gaze location attention

3. Control your task and environmentCon• 20 Users play

Guitar HeroTM

• easy and hard song• 15 mins of practice• Skin/Eye/Head/

Guitar sensors

Real Time Monitoring

4. Process the data according towhich sensors were chosen

0 100 200 300 400 500 600 700 800 900

Engagement

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

0 100 200 300 400 500 600 700 800 900

Engagement

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

0 100 200 300 400 500 600 700 800 900

Performance

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

0 100 200 300 400 500 600 700 800 900

Performance

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

0 100 200 300 400 500 600 700 800 900

Median engagement

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

0 100 200 300 400 500 600 700 800 900

Median engagement

P0009

P0010P0011

P0013

P0015

P0016P0017

P0019

5. Make inferences and iterate on game/instructional design

Datamining

Visualization

Statistics

Interpretation

0 100 200 300 400 500 600 700 800 9000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P0017

Engagement

Median EngagementNormalized Performance

High Scoring Performance in Guitar HeroTM

––––– Raw Engagement––––– Median Engagement––––– Normalized Performance

0 100 200 300 400 500 600 700 800 9000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

P0015

Engagement

Median EngagementNormalized PerformanceLow Scoring Performance in Guitar HeroTM

––––– Raw Engagement––––– Median Engagement––––– Normalized Performance

Engagement

Boredom

Meditation

Frustration

Long Term Excitement

Why measure engagement?

• Dynamic Difficulty Adjustment• Expand Demographics• Longer Time on Task

Dynamic Difficulty Adjustment

CHALLENGE

ABILITY

Anxiety

Boredom

Flow Zone

Expand Demographics

CHALLENGE

ABILITY

Expert

Novice

Flow Zone

Time on Task

FLOW

TIME

ROI in Games

• Competitive edge • Broader appeal• Micro and macro evaluation• Personalization• Improve gameplay

ROI in Education

• Increase Performance• Retention• Time on task• Attitude toward learning

Ongoing Work

• Seductive Details (Instructional Design)• Videogames and Engagement (Guitar Hero)• Emotions and Working Memory Capacity

(puzzles)• 3D Instructional training (US Navy, Save

Science)• Affective Meta Tutor

Questions?

lsrl.lab.asu.edu