ECOVAL: A Framework for Increasing the Ecological Validity in Usability Testing
-
Upload
jean-vanderdonckt -
Category
Software
-
view
151 -
download
0
Transcript of ECOVAL: A Framework for Increasing the Ecological Validity in Usability Testing
Suzanne Kieffer
ECOVAL: A Framework for Increasing the Ecological Validity in Usability Testing
Jean VanderdoncktUgo Braga Sangiorgi
Université catholique de Louvain
Louvain School of Management
Louvain-la-Neuve, Belgium
Ecological validity
Experimental design
Natural setting
x x
xxx
xxx
xx
xx
xx
xx
Environment
Tasks
Stimuli
Ecological validity
High
Low
Experimental design
Natural setting
≈
Experimental design
Natural setting
≠
Internal validityWas the study well done?
External validityCan we generalize the findings to particular persons, settings and times?
Ecological validityTo which extent can we generalize the findings?
What’s the problem with ecological validity?
No formal definition that enables
To objectively quantify the ecological validity of experimental designs
To capture the critical attributes of the real-world environment
Outline
The ECOVAL framework ECOVAL Guidelines GAMBIT
Case study Operability and usefulness of ECOVAL ROI of increased ecological validity Recommendations
GAMBIT – the supporting tool
Production of prototypes Easy Quick Cost-effective
Real user interface performances Interaction Navigation Responsiveness
• Q1: To which extent is ECOVAL applicable for controlling the ecological validity?
• Q2: What are the possible outcomes/benefits of increased ecological validity ?
Monitoring report
Task Control Result
Task Control Result
Artefact:paper checklist
Section > Subsection
Function – Name - Date – Shift
Section > Subsection
300 items a day
Monitoring report
Task Control Result
Task Control Result
Artefact:paper checklist
Section > Subsection
Function – Name - Date – Shift
Section > Subsection
Experimentations
Paper GAMBIT
5 key users x 2
10 participants (30’) 18 participants (1 hour)
5 key users x 3+ 3 user representatives
V16 participants
V212 participants
Paper GAMBIT
Ecologicalvalidity
1 1.17
Datacollection
Manual & face-to-faceDifficult & tedious Unstructured processObtrusive
Automated & remoteComfortableStructured processUnobtrusive
OutcomesCompliant withthe literature
Early detection of usability problemsIncreased user productivityDecreased late design changesIncreased organizational efficiency
Contact [email protected]
Thank you!