Languages
Pages
Legal
11
Remote Usability Testing
December 5, 2009
Elizabeth SnowdonUsability Analyst, Sage Software
22
About meSenior Usability Analyst at Sage Software
Previously worked in usability at high-tech company
Conducting usability studies since 2003
3
Principles of Scientific Management
4
Traditional Lab Testing
55
• Remote usability• Advantages / Disadvantages• Case study• Tools
What I will talk about today
6
What is Remote Testing?
• Testing sessions in which you and participant are not physically in the same place but are communicating via electronic technology
6
7
Types of Remote Testing
7
Remote User Research
Synchronous
Asynchronous
•One on one and real time•Shared technology
•No moderator•Self-paced •Special tools record activities & feedback
8
Current Usability Testing Trends
1. Movement away from lab testing
2. Increased interest in testing in users’ environments
3. Agile development methods
9
Advantages• Lab not required
9
University of Leeds Human Perception and Performance:Usability Laboratory
10
Advantages
• Expands sampling population
• Ease of recruitment
• Users in their own natural browsing environment
• Share their desktop with you
11
Advantages
• Save money– Travel
– Equipment
– Recruiting costs
– Compensation
11
12
Disadvantages
• Difficult to see participant’s screen
• Cannot see participants
• Problems downloading or installing desktop sharing
• Task scenarios disseminated in advance
• Participants interrupted or distracted
12
13
When not to use
• When security is important
• When reliable high speed internet connection is unavailable
• For eye tracking studies
• When it is important to attend to user’s physical movement
13
14
When to use
• Recruiting valuable individuals
• Minimal set-up
• For lone wolf consultants
• Rapid iterative development (Agile)
14
15
Recruiting Users
15
Invitation + incentive
Screener ScheduleConfirmation +
Consent
Session Thank YouTasks
16
Case Study
• Baseline study for Accpac
• Three personas – clerks, controllers, operations
• 24 participants – 20 remote
• US, South Africa, Australia, UK, Canada
• One moderator, one note taker, observers
17
Results
• Accpac study completed in 4 weeks• Cost savings
– Travel– Lab set-up
• Time savings– Recruiting – Scheduling – Less overtime
• Bottom line - $50,000 saved
18
Recruiting
• Internal recruitment, customer lists, Craigslist, email
• Interested parties must pass screener
19
Other issues to sort out
• Obtaining informed consent and confidentiality agreement
• What computing and communications equipment do candidates have?
• Are they willing and able to download and run collaboration software?
20
Typical Set-up
• Moderator / Note-taker – Screen sharing: WebEx– Recording: Morae – Speakerphone
• Participant– High speed internet access– Speakerphone or headset telephone
21
Meeting room set-up
22
What can go wrong?
• User forgets– Send reminder one day in advance
• No shows– Provide your cell # and get participant’s cell
and work #
• User cannot download plug in– Send participant WebEx compatibility test
check email in advance
23
What can go wrong?
• User takes a long time to log onto WebEx– Add extra 10 minutes for WebEx set-up– Log onto session 15 minutes in advance– Get participant to log onto WebEx 10 minutes
in advance
• User brings along his co-worker– Explain 1to1 test– Reschedule secondary participant
24
What can go wrong?
• Can’t see what the user is doing– Ask user to point with mouse
• Conversations– Mute your line
25
WebEx Limitations
• Test! Test! Test! – Dry run
• Delays– Explain that there is a slight delay between
what they are seeing and what you see
• Cannot view hover overs– Verbally tell users what they are hovering
over
26
Pre-test / Test checklist
Email scenarios Set-up system or web site Open up required documents Dial into conference bridge or participant Sign onto WebEx Facilitator script Start recording Hand WebEx control to participant
27
Incentives
• Challenging for international users
• Email local Amazon gift certificates
28
Resources
• IBM - Experience remote usability testing
www.ibm.com/developerworks/library/wa-rmusts1/• Remote Usability remoteusability.com/• User Experience magazine
ww.usabilityprofessionals.org/upa_publications/user_experience/past_issues/2008-3.html
• Moderating Usability Tests: Principles and Practices for Interacting by Loring and Dumas
• Usability Engineering by Jakob Neilsen
29
Remote tools
• Liz Bacon listing http://remoteusability.com/remote-testing-tools-round-up-by-liz-bacon/
• Bolt Peters http://remoteusability.com/tools/
Top Related