UX Sofia 2012 - DIY Mobile Usability Testing Workshop
-
Upload
belen-barros-pena -
Category
Technology
-
view
112 -
download
0
description
Transcript of UX Sofia 2012 - DIY Mobile Usability Testing Workshop
![Page 1: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/1.jpg)
![Page 2: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/2.jpg)
Bernard, packet core engineer at NSN
![Page 3: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/3.jpg)
Belén, interaction designer at Intel’s OTC
![Page 4: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/4.jpg)
how to plan and run usability tests with mobile devices in a laboratory setting
![Page 5: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/5.jpg)
assuming ...
1. you have experience planning and running usability tests with desktop computers
2. you are testing software or an interactive prototype running on a mobile device
3. you are testing in a usability laboratory or similar environment
![Page 6: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/6.jpg)
we will not focus on
1. the fundamentals of the usability testing technique
2. how to prototype for mobile devices
3. how to test in the field
![Page 7: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/7.jpg)
these are the golden rules you need to remember
![Page 8: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/8.jpg)
![Page 9: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/9.jpg)
let us begin ...
![Page 10: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/10.jpg)
usability testinga process that employs people as testing participants who are representative of the target audience to evaluate the degree to which a product meets specific usability criteria.
Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
![Page 11: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/11.jpg)
usability testing
watching people try to use what you’re creating/designing/building (...) with the intention of (a)making it easier for people to use or (b)proving that it is easy to use.
Rocket Surgery Made Easy, S. Krug
![Page 12: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/12.jpg)
![Page 13: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/13.jpg)
![Page 14: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/14.jpg)
dut vs mut
![Page 15: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/15.jpg)
Product Desktop software
Mobilesoftware
Device Computer Mobile phone
Context Predictable Unpredictable
Connection DSL / Cable Mobile phone network
![Page 16: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/16.jpg)
which phone?
which context?
which connection?
![Page 17: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/17.jpg)
which phone?
which context?
which connection?
![Page 18: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/18.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 19: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/19.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 20: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/20.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 21: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/21.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 22: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/22.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 23: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/23.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 24: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/24.jpg)
web task success rates
feature phones 38%
smartphones 55%
touch phones 75%
Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability-study-1.html
![Page 25: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/25.jpg)
handset usability affects test results
![Page 26: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/26.jpg)
test with participants’ own phones
![Page 27: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/27.jpg)
when not possible, include training and
warm-up tasks
![Page 28: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/28.jpg)
handset becomes a screening criteria when recruiting test subjects
![Page 29: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/29.jpg)
![Page 30: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/30.jpg)
![Page 31: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/31.jpg)
![Page 32: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/32.jpg)
![Page 33: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/33.jpg)
http://goo.gl/hmp8t
![Page 34: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/34.jpg)
mobile phone subscriptions per 100 people
http://data.worldbank.org
![Page 35: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/35.jpg)
http://gs.statcounter.com/
![Page 36: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/36.jpg)
http://gs.statcounter.com/
![Page 37: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/37.jpg)
WP iOS Android Blackberry
![Page 38: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/38.jpg)
http://developer.android.com/resources/dashboard/platform-versions.html
![Page 39: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/39.jpg)
http://us.blackberry.com/developers/choosingtargetos.jsp
![Page 40: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/40.jpg)
do not confuse target platform with target user
![Page 41: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/41.jpg)
![Page 42: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/42.jpg)
which phone would you use for testing?
who are the target users?
handset screening criteria
will you need warm-up tasks? (yes / no)
if yes, which ones?
![Page 43: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/43.jpg)
BNT are considering a mobile version of their website.
Before investing in the project, they would like to know how their existing desktop site performs on mobile devices.
![Page 44: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/44.jpg)
You are working on an Android application for Sofia Airport.
Together with your developers, you have created a first version of the app. This ‘alpha’ version is not available in Google Play, it is not fully tested and might be buggy.
![Page 45: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/45.jpg)
St. Ekaterina University Hospital has developed a Blackberry application that helps nurses and doctors access and update patients records during their daily shift.
They have asked you to help them test the application.
![Page 46: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/46.jpg)
Evernote wants to uncover the main usability issues in their existing mobile applications for iOS, Android, Windows Phone and Blackberry.
Help them organise the necessary testing.
![Page 47: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/47.jpg)
what about emulators?
mas_fsx_ by Massimiliano Silipigni at http://www.flickr.com/photos/silipigni/6055803785/
![Page 48: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/48.jpg)
![Page 49: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/49.jpg)
![Page 50: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/50.jpg)
good for ...
using an emulator enables testers to thoroughly capture user behaviour (...) The captured data are generally informative and useful for analysing user performance
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)
![Page 51: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/51.jpg)
good for ...
emulators are more suitable to be used for improving the interface design of applications such as the layout of menu structures during the development process.
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)
![Page 52: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/52.jpg)
but bad for ...
using emulators alleviates the problems of latency, inefficient input mechanisms, and the changing wireless environment, potentially leading to untruthful user satisfaction
Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, D. Zhang, B. Adipat, International Journal of Human Computer Studies (2005)
![Page 53: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/53.jpg)
which phone?
which context?
which connection?
![Page 54: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/54.jpg)
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
field vs. lab
![Page 55: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/55.jpg)
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
field vs. lab
0 0
![Page 56: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/56.jpg)
The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.
Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the FieldJ. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004
![Page 57: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/57.jpg)
The results show that the added value of conducting usability evaluations in the field is very little and that recreating central aspects of the use context in a laboratory setting enables the identification of the same usability problem list.
Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the FieldJ. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004
0 1Field Lab
![Page 58: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/58.jpg)
according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported
Usability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
![Page 59: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/59.jpg)
according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported
Usability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
0 2Field Lab
![Page 60: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/60.jpg)
evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
![Page 61: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/61.jpg)
evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
1 2Field Lab
![Page 62: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/62.jpg)
The analyses of the comparison between usability testing done in two different settings revealed that there were many more types and occurrences of usability problems found in the field than in the laboratory. Those problems discovered tend to be critical issues.
Usability Evaluation of Mobile Device: a Comparison of Laboratory and Field TestsH.B Duh, G. C. B. Tan, V. H. Chen, MobileHCI 2006
![Page 63: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/63.jpg)
The analyses of the comparison between usability testing done in two different settings revealed that there were many more types and occurrences of usability problems found in the field than in the laboratory. Those problems discovered tend to be critical issues.
Usability Evaluation of Mobile Device: a Comparison of Laboratory and Field TestsH.B Duh, G. C. B. Tan, V. H. Chen, MobileHCI 2006
2 2Field Lab
![Page 64: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/64.jpg)
field vs. lab
EXPERTS DISAGREE
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
![Page 65: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/65.jpg)
... but they all agree
testing in the field requires double the time in comparison to the laboratoryUsability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
![Page 66: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/66.jpg)
... but they all agree
evaluations in the field (are) more complex and time-consumingIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
![Page 67: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/67.jpg)
... but they all agree
field-based usability studies are not easy to conduct. They are time consuming and the added value is questionable.Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field, J. Kjeldskov, M. B. Skov, B. S. Als, R. T. Høegh, 2004
![Page 68: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/68.jpg)
testing in the lab is better than no testing
![Page 69: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/69.jpg)
![Page 70: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/70.jpg)
can you think of an example of mobile software that you must test in the field?
![Page 71: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/71.jpg)
do it late
plan an run pilot tests
be prepared (like the Scouts)
if doing field testing ...
![Page 72: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/72.jpg)
have I chosen an appropriate location?
which external factors could impact results?
is the equipment working as planned?
do I have enough time to run the tests?
does the protocol work? Is it well understood?
if doing field testing ...
![Page 73: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/73.jpg)
If we cannot go to the field, the field must come to us
![Page 74: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/74.jpg)
Simulations
Scenarios
Take the lab out
Bringing the field into the lab
![Page 75: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/75.jpg)
simulations
Use of techniques to “reflect or recreate a mobile use situation” (...) their level of sophistication is measured by the extent to which the environmental and ergonomic factors of the actual use context are recreated.
Emerging Research Methods for Understanding Mobile Technology Use, P. Hagen, T. Robertson, M. Kan and K. Sadler, Proceedings of OZCHI 2005, Canberra (Australia)
![Page 76: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/76.jpg)
Is it Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field, J. Kjeldskov, M.B. Skov, B.S. Als and R.T.Høegh, MobileHCI 2004
![Page 77: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/77.jpg)
Experimental Evaluation of Techniques for Usability Testing of Mobile Systems in a Laboratory Setting, E.T. Beck, M.K. Christiansen, J. Kjedskov, N. Kolbe and J. Stage, OZCHI 2003, Brisbane (Australia)
![Page 78: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/78.jpg)
scenarios
First you choose the tasks to test (...) Then you expand these tasks into scenarios (...) A scenario provides some context and supplies information the user needs to know
Rocket Surgery Made Easy, S. Krug
![Page 79: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/79.jpg)
![Page 80: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/80.jpg)
a task:Find a hotel
a desktop scenario:You are planning a surprise weekend trip to Paris. You’ll be leaving on Friday and coming back on Sunday evening. Your partner’s favourite Paris area is Le Marais. Find a place to stay he/she will like.
![Page 81: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/81.jpg)
![Page 82: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/82.jpg)
a task:Find a hotel
a mobile scenario:You were late to Orly airport and have missed your flight home. There are no more flights from Paris today, so you’ll have to spend the night. Since it’s already 10 pm, use your phone to look for a place to stay close to the airport.
![Page 83: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/83.jpg)
portable labs can be deployed anywhere
document camera
laptop
webcam
![Page 84: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/84.jpg)
Jyske Bank in Roskilde by king_david_uk at http://www.flickr.com/photos/kingdavid/2926673466/
Vodafone, Croydon, London CR0 by Kate Pugh at http://www.flickr.com/photos/kake_pugh/6545344321/
![Page 85: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/85.jpg)
![Page 86: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/86.jpg)
write a mobile scenario for your
group’s task
![Page 87: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/87.jpg)
You are testing:
http://www.tfl.gov.uk
Task:
Find a public transport route
A bit of context:
From Heathrow Terminal 5 to the Olympic Stadium. User must arrive on time for the Olympics opening ceremony (27th July at 7:30pm)
![Page 88: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/88.jpg)
You are testing:
http://www.toptable.co.uk
Task:
Find a restaurant
A bit of context:
Lebanese cuisine Around Shoreditch (an East London area)For tomorrow at 8:30pm For 5 people
![Page 89: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/89.jpg)
You are testing:
An Android app called ‘CycleStreets’, which helps you find cycling routes across London
Task:
Calculate a route distance
A bit of context:
From King’s Cross station to Tate ModernRoute with least traffic
![Page 90: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/90.jpg)
You are testing:
A task management app called Remember the Milk
Task:
Create a task
A bit of context:
The task is to buy flowers for a friend (make up a name)Next Tuesday morningIn her favourite flower shop, called Blooming
![Page 91: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/91.jpg)
which phone?
which context?
which connection?
![Page 92: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/92.jpg)
how often are mobile users connecting their devices to WiFi networks instead of mobile networks?
Towering by Rachel Strohm at http://www.flickr.com/photos/rachelstrohm/4693016049/
![Page 93: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/93.jpg)
25% of households worldwide have WiFi networks set up
Globally, smartphones account for 36% of all public WiFi hotspot connections, compared to 48% for laptops and 10% for tablets
Out of 4 billion unique mobile phone users, 1 billion uses WiFi to access the Internet (there are only about 900 million smartphones)
http://www.lukew.com/ff/entry.asp?1537
http://communities-dominate.blogs.com/brands/2012/05/mobile-services-beyond-messaging-excellent-tns-global-survey-reveals-tons.html
![Page 94: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/94.jpg)
![Page 95: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/95.jpg)
http://databank.worldbank.org
![Page 96: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/96.jpg)
do not test over WiFi
![Page 97: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/97.jpg)
cover your participants’ data costs
![Page 98: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/98.jpg)
and on the big day ...
![Page 99: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/99.jpg)
remote?
Remote Mania by Dramagirl at http://www.flickr.com/photos/dramagirl/2268235917/
![Page 100: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/100.jpg)
http://www.usertesting.com/mobile
![Page 101: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/101.jpg)
but ...I’d say remote testing gives you about 80% of the benefits of a live test with about 70% of the effort. You do lose 20% of something not being in the same room as the participant. The in-person experience is just richer, somehow.
Rocket Surgery Made Easy, S. Krug
![Page 102: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/102.jpg)
To be or not to be, that is the question by Mister_Jack at http://www.flickr.com/photos/machineman22/5130901135/
to record or not to record ...
![Page 103: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/103.jpg)
it is difficult ...We have used Morae in the past also for website testing, but could not easily use the system for mobile (...) Obviously it’s difficult to video something like that so we just sit next to them and observe.
http://belenpena.posterous.com/user-research-for-mobile-software-a-short-sur
![Page 104: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/104.jpg)
... but video is both a
memory aid
powerful communication tool
![Page 105: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/105.jpg)
reactions
actions
![Page 106: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/106.jpg)
actions
reactions
![Page 107: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/107.jpg)
which phone?
which context?
which connection?
which recording setup?
![Page 108: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/108.jpg)
Mobiola Screen Capture for Blackberry 4.2+ and Symbian S60 v3http://www.shapeservices.com/en/products/details.php?product=capture&platform=none#
![Page 109: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/109.jpg)
document camera records mobile screen
USB webcam records participant’s face
both cameras connect to a laptop running screen
recording software
document camera range is marked with tape
![Page 110: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/110.jpg)
Mobile Device Camera by Noldus http://www.noldus.com/human-behavior-research/accessories/mobile-device-camera-mdc#
Ovo Studios device camerahttp://www.ovostudios.com/devicecamera.asp
![Page 111: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/111.jpg)
Little Springs Designhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
Nick Bowmast http://www.bowmast.com/mob-device-cam/
Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
Usability Scienceshttp://www.usabilitysciences.com/services/lab-based-usability-testing/mobile-usability-testing
![Page 112: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/112.jpg)
Little Springs Designhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
Nick Bowmast http://www.bowmast.com/mob-device-cam/
Usability Scienceshttp://www.usabilitysciences.com/services/lab-based-usability-testing/mobile-usability-testing
Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
are blooming!!
![Page 113: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/113.jpg)
screen audio
Skype Skype
Screen sharingTelephone conferencing system
Mirroring
Telephone conferencing system
the observation room
![Page 114: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/114.jpg)
Screen sharing
Mirroring
Telephone conferencing system
![Page 115: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/115.jpg)
pros cons
provides high quality screen recording
participants won’t appreciate you installing stuff on their phones
provides high quality screen recording no application will
support all platforms
provides high quality screen recording
in touchscreen phones, finger interactions are not captured
screen capture applications
![Page 116: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/116.jpg)
Ovo Studios screen capture application for iOShttp://www.ovostudios.com/
![Page 117: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/117.jpg)
Obviously,
think aloud is critical because I cannot see how the participant is interacting
with his fingers on the touch screen.
I can see using the mirroring in a lab setting to get the signal from the iPad
to the observation room though you still won’t see the physical interaction
with the device, like you would with a device mounted camera (e.g. Noldus).
... and that is a big deal
Why AirPlay mirroring is the Biggest Thing to Happen to User Research in 2011 - http://www.remoteresear.ch/airplay/
![Page 118: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/118.jpg)
Recently, we’ve questioned the value of capturing the device screen. It does
after all end up being a video of the screen changing but with no sense
of the participant interacting with it.
iPad usability testing - our equipment - http://www.cxpartners.co.uk/cxblog/ipad_usability_testing_-_our_equipment/
... and that is a big deal
![Page 119: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/119.jpg)
pros cons
good recording quality document cameras bring up the price
easy to set up participants must keep within the camera range
phone must lay on a desk or be hold at a flat angle
document cameras
![Page 120: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/120.jpg)
pros cons
allow natural interaction with the phone
ready-made devices will bring up the price
allow natural interaction with the phone
if bulky, they can prevent single-hand use
allow natural interaction with the phone
if heavy, they can become tiring during long tests
mounted devices
![Page 121: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/121.jpg)
the ideal recording setup
easy to put together
cheap
repeatable
allows holding the device
allows one-handed use
supports all form factors
runs tests with participants’ phones
captures screen, face & fingers
gives enough video quality
![Page 122: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/122.jpg)
the ideal recording setup screen capture
applications
document cameras
mounted devices
ready-made
mounted devices DIY
easy to put together
cheap
repeatable
allows holding the device
allows one-handed use
supports all form factors
runs tests with participants’ phones
captures screen, face & fingers
gives enough video quality
![Page 123: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/123.jpg)
no recording setup is perfect: just use the best tool for the job
![Page 124: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/124.jpg)
?
![Page 125: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/125.jpg)
test with participants’ own phones
do not confuse target platform with target user
testing in the lab is better than no testing
do not test over WiFi
cover your participant’s data costs
no recording setup is perfect: just use the best tool for the job
![Page 126: UX Sofia 2012 - DIY Mobile Usability Testing Workshop](https://reader033.fdocuments.us/reader033/viewer/2022051210/54c712924a7959ae708b458a/html5/thumbnails/126.jpg)
thanks!!!
www.DIYMobileUsabilityTesting.net