UX Sofia 2012 - DIY Mobile Usability Testing Workshop

126

description

These are the slides for our 4-hour workshop in UX Sofia 2012.

Transcript of UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Page 1: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 2: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Bernard, packet core engineer at NSN

Page 3: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Belén, interaction designer at Intel’s OTC

Page 4: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

how to plan and run usability tests with mobile devices in a laboratory setting

Page 5: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

assuming ...

1. you have experience planning and running usability tests with desktop computers

2. you are testing software or an interactive prototype running on a mobile device

3. you are testing in a usability laboratory or similar environment

Page 6: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

we will not focus on

1. the fundamentals of the usability testing technique

2. how to prototype for mobile devices

3. how to test in the field

Page 7: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

these are the golden rules you need to remember

Page 8: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 9: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

let us begin ...

Page 10: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

usability testinga process that employs people as testing participants who are representative of the target audience to evaluate the degree to which a product meets specific usability criteria.

Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell

Page 11: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

usability testing

watching people try to use what you’re creating/designing/building (...) with the intention of (a)making it easier for people to use or (b)proving that it is easy to use.

Rocket Surgery Made Easy, S. Krug

Page 12: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 13: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 14: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

dut vs mut

Page 15: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Product Desktop software

Mobilesoftware

Device Computer Mobile phone

Context Predictable Unpredictable

Connection DSL / Cable Mobile phone network

Page 16: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone?

which context?

which connection?

Page 17: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone?

which context?

which connection?

Page 18: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 19: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 20: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 21: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 22: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 23: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 24: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

web task success rates

feature phones 38%

smartphones 55%

touch phones 75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html

Page 25: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

handset usability affects test results

Page 26: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

test with participants’ own phones

Page 27: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

when not possible, include training and

warm-up tasks

Page 28: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

handset becomes a screening criteria when recruiting test subjects

Page 29: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 30: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 31: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 32: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 33: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://goo.gl/hmp8t

Page 34: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

mobile phone subscriptions per 100 people

http://data.worldbank.org

Page 35: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://gs.statcounter.com/

Page 36: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://gs.statcounter.com/

Page 37: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

WP iOS Android Blackberry

Page 38: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://developer.android.com/resources/dashboard/platform-versions.html

Page 39: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://us.blackberry.com/developers/choosingtargetos.jsp

Page 40: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

do not confuse target platform with target user

Page 41: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 42: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone would you use for testing?

who are the target users?

handset screening criteria

will you need warm-up tasks? (yes / no)

if yes, which ones?

Page 43: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

BNT are considering a mobile version of their website.

Before investing in the project, they would like to know how their existing desktop site performs on mobile devices.

Page 44: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

You are working on an Android application for Sofia Airport.

Together with your developers, you have created a first version of the app. This ‘alpha’ version is not available in Google Play, it is not fully tested and might be buggy.

Page 45: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

St. Ekaterina University Hospital has developed a Blackberry application that helps nurses and doctors access and update patients records during their daily shift.

They have asked you to help them test the application.

Page 46: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Evernote wants to uncover the main usability issues in their existing mobile applications for iOS, Android, Windows Phone and Blackberry.

Help them organise the necessary testing.

Page 47: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

what about emulators?

mas_fsx_ by Massimiliano Silipigni at http://www.flickr.com/photos/silipigni/6055803785/

Page 48: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 49: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 50: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

good for ...

using an emulator enables testers to thoroughly capture user behaviour (...) The captured data are generally informative and useful for analysing user performance

Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)

Page 51: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

good for ...

emulators are more suitable to be used for improving the interface design of applications such as the layout of menu structures during the development process.

Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)

Page 52: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

but bad for ...

using emulators alleviates the problems of latency, inefficient input mechanisms, and the changing wireless environment, potentially leading to untruthful user satisfaction

Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)

Page 53: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone?

which context?

which connection?

Page 54: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

field vs. lab

Page 55: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

field vs. lab

0 0

Page 56: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the FieldJ. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004

Page 57: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.

Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the FieldJ. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004

0 1Field Lab

Page 58: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005

Page 59: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported

Usability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005

0 2Field Lab

Page 60: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations

It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

Page 61: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations

It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

1 2Field Lab

Page 62: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

The analyses of the comparison between usability testing done in two different settings revealed that there were many more types and occurrences of usability problems found in the field than in the laboratory. Those problems discovered tend to be critical issues.

Usability Evaluation of Mobile Device: a Comparison of Laboratory and Field TestsH.B Duh, G. C. B. Tan, V. H. Chen, MobileHCI 2006

Page 63: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

The analyses of the comparison between usability testing done in two different settings revealed that there were many more types and occurrences of usability problems found in the field than in the laboratory. Those problems discovered tend to be critical issues.

Usability Evaluation of Mobile Device: a Comparison of Laboratory and Field TestsH.B Duh, G. C. B. Tan, V. H. Chen, MobileHCI 2006

2 2Field Lab

Page 64: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

field vs. lab

EXPERTS DISAGREE

It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

Page 65: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

... but they all agree

testing in the field requires double the time in comparison to the laboratoryUsability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005

Page 66: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

... but they all agree

evaluations in the field (are) more complex and time-consumingIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

Page 67: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

... but they all agree

field-based usability studies are not easy to conduct. They are time consuming and the added value is questionable.Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field, J. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004

Page 68: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

testing in the lab is better than no testing

Page 69: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 70: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

can you think of an example of mobile software that you must test in the field?

Page 71: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

do it late

plan an run pilot tests

be prepared (like the Scouts)

if doing field testing ...

Page 72: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

have I chosen an appropriate location?

which external factors could impact results?

is the equipment working as planned?

do I have enough time to run the tests?

does the protocol work? Is it well understood?

if doing field testing ...

Page 73: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

If we cannot go to the field, the field must come to us

Page 74: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Simulations

Scenarios

Take the lab out

Bringing the field into the lab

Page 75: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

simulations

Use of techniques to “reflect or recreate a mobile use situation” (...) their level of sophistication is measured by the extent to which the environmental and ergonomic factors of the actual use context are recreated.

Emerging Research Methods for Understanding Mobile Technology Use, P. Hagen, T. Robertson, M. Kan and K. Sadler, Proceedings of OZCHI 2005, Canberra (Australia)

Page 76: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field, J. Kjeldskov, M.B. Skov, B.S. Als and R.T.Høegh, MobileHCI 2004

Page 77: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Experimental Evaluation of Techniques for Usability Testing of Mobile Systems in a Laboratory Setting, E.T. Beck, M.K. Christiansen, J. Kjedskov, N. Kolbe and J. Stage, OZCHI 2003, Brisbane (Australia)

Page 78: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

scenarios

First you choose the tasks to test (...) Then you expand these tasks into scenarios (...) A scenario provides some context and supplies information the user needs to know

Rocket Surgery Made Easy, S. Krug

Page 79: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 80: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

a task:Find a hotel

a desktop scenario:You are planning a surprise weekend trip to Paris. You’ll be leaving on Friday and coming back on Sunday evening. Your partner’s favourite Paris area is Le Marais. Find a place to stay he/she will like.

Page 81: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 82: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

a task:Find a hotel

a mobile scenario:You were late to Orly airport and have missed your flight home. There are no more flights from Paris today, so you’ll have to spend the night. Since it’s already 10 pm, use your phone to look for a place to stay close to the airport.

Page 83: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

portable labs can be deployed anywhere

document camera

laptop

webcam

Page 84: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Jyske Bank in Roskilde by king_david_uk at http://www.flickr.com/photos/kingdavid/2926673466/

Vodafone, Croydon, London CR0 by Kate Pugh at http://www.flickr.com/photos/kake_pugh/6545344321/

Page 85: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 86: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

write a mobile scenario for your

group’s task

Page 87: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

You are testing:

http://www.tfl.gov.uk

Task:

Find a public transport route

A bit of context:

From Heathrow Terminal 5 to the Olympic Stadium. User must arrive on time for the Olympics opening ceremony (27th July at 7:30pm)

Page 88: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

You are testing:

http://www.toptable.co.uk

Task:

Find a restaurant

A bit of context:

Lebanese cuisine Around Shoreditch (an East London area)For tomorrow at 8:30pm For 5 people

Page 89: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

You are testing:

An Android app called ‘CycleStreets’, which helps you find cycling routes across London

Task:

Calculate a route distance

A bit of context:

From King’s Cross station to Tate ModernRoute with least traffic

Page 90: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

You are testing:

A task management app called Remember the Milk

Task:

Create a task

A bit of context:

The task is to buy flowers for a friend (make up a name)Next Tuesday morningIn her favourite flower shop, called Blooming

Page 91: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone?

which context?

which connection?

Page 92: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

how often are mobile users connecting their devices to WiFi networks instead of mobile networks?

Towering by Rachel Strohm at http://www.flickr.com/photos/rachelstrohm/4693016049/

Page 93: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

25% of households worldwide have WiFi networks set up

Globally, smartphones account for 36% of all public WiFi hotspot connections, compared to 48% for laptops and 10% for tablets

Out of 4 billion unique mobile phone users, 1 billion uses WiFi to access the Internet (there are only about 900 million smartphones)

http://www.lukew.com/ff/entry.asp?1537

http://communities-dominate.blogs.com/brands/2012/05/mobile-services-beyond-messaging-excellent-tns-global-survey-reveals-tons.html

Page 94: UX Sofia 2012 - DIY Mobile Usability Testing Workshop
Page 95: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://databank.worldbank.org

Page 96: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

do not test over WiFi

Page 97: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

cover your participants’ data costs

Page 98: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

and on the big day ...

Page 99: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

remote?

Remote Mania by Dramagirl at http://www.flickr.com/photos/dramagirl/2268235917/

Page 100: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

http://www.usertesting.com/mobile

Page 101: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

but ...I’d say remote testing gives you about 80% of the benefits of a live test with about 70% of the effort. You do lose 20% of something not being in the same room as the participant. The in-person experience is just richer, somehow.

Rocket Surgery Made Easy, S. Krug

Page 102: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

To be or not to be, that is the question by Mister_Jack at http://www.flickr.com/photos/machineman22/5130901135/

to record or not to record ...

Page 103: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

it is difficult ...We have used Morae in the past also for website testing, but could not easily use the system for mobile (...) Obviously it’s difficult to video something like that so we just sit next to them and observe.

http://belenpena.posterous.com/user-research-for-mobile-software-a-short-sur

Page 104: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

... but video is both a

memory aid

powerful communication tool

Page 105: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

reactions

actions

Page 106: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

actions

reactions

Page 107: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

which phone?

which context?

which connection?

which recording setup?

Page 108: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Mobiola Screen Capture for Blackberry 4.2+ and Symbian S60 v3http://www.shapeservices.com/en/products/details.php?product=capture&platform=none#

Page 109: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

document camera records mobile screen

USB webcam records participant’s face

both cameras connect to a laptop running screen

recording software

document camera range is marked with tape

Page 110: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Mobile Device Camera by Noldus http://www.noldus.com/human-behavior-research/accessories/mobile-device-camera-mdc#

Ovo Studios device camerahttp://www.ovostudios.com/devicecamera.asp

Page 111: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Little Springs Designhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/

Nick Bowmast http://www.bowmast.com/mob-device-cam/

Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)

Usability Scienceshttp://www.usabilitysciences.com/services/lab-based-usability-testing/mobile-usability-testing

Page 112: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Little Springs Designhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/

Nick Bowmast http://www.bowmast.com/mob-device-cam/

Usability Scienceshttp://www.usabilitysciences.com/services/lab-based-usability-testing/mobile-usability-testing

Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)

are blooming!!

Page 113: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

screen audio

Skype Skype

Screen sharingTelephone conferencing system

Mirroring

Telephone conferencing system

the observation room

Page 114: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Screen sharing

Mirroring

Telephone conferencing system

Page 115: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

pros cons

provides high quality screen recording

participants won’t appreciate you installing stuff on their phones

provides high quality screen recording no application will

support all platforms

provides high quality screen recording

in touchscreen phones, finger interactions are not captured

screen capture applications

Page 116: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Ovo Studios screen capture application for iOShttp://www.ovostudios.com/

Page 117: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Obviously,

think aloud is critical because I cannot see how the participant is interacting

with his fingers on the touch screen.

I can see using the mirroring in a lab setting to get the signal from the iPad

to the observation room though you still won’t see the physical interaction

with the device, like you would with a device mounted camera (e.g. Noldus).

... and that is a big deal

Why AirPlay mirroring is the Biggest Thing to Happen to User Research in 2011 - http://www.remoteresear.ch/airplay/

Page 118: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

Recently, we’ve questioned the value of capturing the device screen. It does

after all end up being a video of the screen changing but with no sense

of the participant interacting with it.

iPad usability testing - our equipment - http://www.cxpartners.co.uk/cxblog/ipad_usability_testing_-_our_equipment/

... and that is a big deal

Page 119: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

pros cons

good recording quality document cameras bring up the price

easy to set up participants must keep within the camera range

phone must lay on a desk or be hold at a flat angle

document cameras

Page 120: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

pros cons

allow natural interaction with the phone

ready-made devices will bring up the price

allow natural interaction with the phone

if bulky, they can prevent single-hand use

allow natural interaction with the phone

if heavy, they can become tiring during long tests

mounted devices

Page 121: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

the ideal recording setup

easy to put together

cheap

repeatable

allows holding the device

allows one-handed use

supports all form factors

runs tests with participants’ phones

captures screen, face & fingers

gives enough video quality

Page 122: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

the ideal recording setup screen capture

applications

document cameras

mounted devices

ready-made

mounted devices DIY

easy to put together

cheap

repeatable

allows holding the device

allows one-handed use

supports all form factors

runs tests with participants’ phones

captures screen, face & fingers

gives enough video quality

Page 123: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

no recording setup is perfect: just use the best tool for the job

Page 124: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

?

Page 125: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

test with participants’ own phones

do not confuse target platform with target user

testing in the lab is better than no testing

do not test over WiFi

cover your participant’s data costs

no recording setup is perfect: just use the best tool for the job

Page 126: UX Sofia 2012 - DIY Mobile Usability Testing Workshop

thanks!!!

www.DIYMobileUsabilityTesting.net