HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me:...
-
Upload
jase-dipple -
Category
Documents
-
view
212 -
download
0
Transcript of HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me:...
![Page 1: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/1.jpg)
HUMAN TESTING OF MOBILE APPSFOR VISTACON 2013 CONFERENCE
25 APRIL 2013
JULIAN HARTYContact me: [email protected]: 29 April 2013
Creative Commons LicenseHow to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
http://creativecommons.org/licenses/by-sa/3.0/deed.en_US
![Page 2: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/2.jpg)
AUTOMATION
DRIVE FOR EFFICIENCY
![Page 3: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/3.jpg)
STATE OF AFFAIRS
The industry is driving towards automated tests
• ‘Manual testing’ is downgraded
• Agile projects need automated tests to provide fast & frequent feedback
• Development environments for Mobile Apps (SDKs) include Test Automation frameworks
![Page 4: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/4.jpg)
AUTOMATED TESTS
Unit Tests
Continuous Builds
UI Test Automation
Scale out to run tests on multiple devices
• Virtual devices, remote devices, local devices
Complex, sometimes fragile, ecosystems
• Challenges: device security models, limitations in frameworks, connectivity, immature domain, lack of support for multi-modal interfaces, etc.
![Page 5: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/5.jpg)
PEOPLE NEED PEOPLE TOO
HUMAN-CENTRIC TESTING
![Page 6: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/6.jpg)
BEYOND THE REACH OF AUTOMATED TESTSTest frameworks focus on code, API, or GUI,
Not:• Movement: acceleration, rotation, location
• Tactile, Haptic, audio Inputs or Outputs
• Aesthetics: look & feel, transitions
• User Experience (UX)
• Highly dynamic updates: video, audio output
• Complex inputs: Gestures, multi-touch, shaking
• Light: e.g. camera as an input device
So how do we test these?
![Page 7: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/7.jpg)
QOX
Quality Of Experience (QOX)
From the standpoint of a user’s holistic experience of using the mobile app.
Essentially subjective
http://en.wikipedia.org/wiki/Quality_of_experience
UX Research, Evaluation, and Testing: strategies, tips and tools Mark J. Williams
![Page 8: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/8.jpg)
I’VE GOT FEELINGS TOO
6 hats of software testing
• The ‘red’ hat – emotions, intuitions, feelings
I've Got a Feeling: Emotions in Testinghttp://www.stickyminds.com/s.asp?F=S17922_COL_2
Here's an article I wrote a few years ago on the six hats of software testing http://archive.newsweaver.com/qualtech/newsweaver.ie/qualtech/e_article000666359.html and if you really have lots of time on your hands, here's a video of a talk I gave on the same topic at StarWest 2008 http://www.stickyminds.com/Media/Video/Detail.aspx?WebPage=117
“An inconsistency between the product and the document might be surprising, frustrating, or annoying to that person, but what really matters is inconsistency between the product and the desire.”
![Page 9: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/9.jpg)
WHAT DO WE NEED TO DO?
SO…
![Page 10: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/10.jpg)
FAST, EFFICIENT INTERACTIVE TESTING
Bugfests
Move and rotate the phone
Be mobile
Bugfest: A group of people get together to test in parallel and in common. Developers on-hand for direct feedback.
![Page 11: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/11.jpg)
LUNCH OUTTESTING A VIDEO APP• Around 2 hours
• Around 10 testers
• Test the app while going out for lunch
Examples of what the testers did
• Tourist trip on the top of a double-decker bus
• London Underground
• Out for a walk with sandwiches
![Page 12: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/12.jpg)
ESTABLISH STRONG FOUNDATIONS
Learn:
• The platforms, the norms, what users expect
• How the platform was designed to be used
• The application lifecycle
• The devices
• Common, and unusual problems
![Page 13: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/13.jpg)
INSIGHTFUL TESTINGRAPID software
• Heuristics
• Mnemonics to guide the testing
Session-Based Test Management techniques
• Charter
• Time-boxed
• Simple, lightweight test reporting
• Debrief
http://moolya.com/blog/2012/04/18/test-mobile-applications-with-cop-who-flung-gun/
![Page 14: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/14.jpg)
EXTENDED TESTING
![Page 15: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/15.jpg)
EXTENDED TESTING
Remote devices
Remote testing
![Page 16: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/16.jpg)
REMOTE TESTERS
Credit: "Ken Banks, kiwanja.net"
http://www.kiwanja.net/gallery/texting/kiwanja_kenya_texting_17.jpghttp://qz.com/49916/indian-smartphone-users-are-gobbling-up-mobile-data/http://www.imedicalapps.com/2012/09/researchers-smartphone-spirometer-mobile-app/
![Page 17: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/17.jpg)
CHALLENGES OF EXTENDED TESTING
Loss of information
• Sensory deprivation
• Missing or incomplete data
Loss of fidelity
• Lags, delays
Loss of control
How do we test rich interactive apps unless you have the device in your hand?
“In theory, theory and practice are the same.
In practice, they are not.” A device in your hand is worth 2 in the cloud, and 100 virtual devices.
![Page 18: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/18.jpg)
MORE CONTEXT
![Page 19: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/19.jpg)
MORE CONTEXT
Additional considerations
• Cultural norms & differences
• Generations
• What’s important to them?
• How can we empathize?
Mobile Apps & Platforms
InternationalC
ultu
re?
Cu
lture
?
Culture?
GSMA REVEALS FEARS OVER MOBILE PRIVACY ARE HOLDING BACK THE GROWTH OF MOBILE APPS IN LATIN AMERICA
http://www.gsma.com/newsroom/gsma-reveals
A South Korean woman using her mobile phone to buy a product at the virtual retail shop in Seoul.www.guardian.co.uk
![Page 20: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/20.jpg)
EXAMPLES OF WHAT CAN GO WRONG“What this means is that unlike the iPhone, which is most certainly the closest competitor on the market to this phone, the Storm's UI is not custom built for touch navigation -- touch navigation is added after the fact. Things which flow naturally on an iPhone -- flicking through lists, scrolling for a contact, moving around in a webpage or looking through photos -- feel inelegant and uncomfortable on the Storm. There's no inertia to movement, no assurance that your finger is the lynchpin to control of the device. The screen is sensitive enough, surely, but how its software reacts to those touches makes all the difference, and here the feeling is that you're never completely in charge of the phone.”
http://www.engadget.com/2008/11/19/blackberry-storm-review/Image: http://worldwide.blackberry.com/gettingstarted/storm-9500.png
![Page 21: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/21.jpg)
! XOR[*]
PRACTICAL OUTCOMES
[*] Not Exclusive OR
![Page 22: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/22.jpg)
COMBINING STRENGTHS
Automated Tests
• Fast feedback, technology and developer centric
Interactive Tests
• Exploratory, rich interactions and dynamics
• Remote testing e.g. in other countries
Combined
• Augmented, Cyborg testing e.g. automated preparation, data collection, etc.
Learn to delegate & learn when to take control
![Page 23: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/23.jpg)
SOME FURTHER READINGS
http://www.wipconnector.com/download/GuideToTheParallelUniverse_3rdEdition.pdf
http://www.enough.de/fileadmin/uploads/dev_guide_pdfs/Guide_12thEdition_WEB.pdf
Testing and Test Automation for Mobile Apps
Julian Harty
Summer 2013CRC Press
Development Marketing Testing(2009) Testing(2013)
![Page 24: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/24.jpg)
Q & A ?
![Page 25: HUMAN TESTING OF MOBILE APPS FOR VISTACON 2013 CONFERENCE 25 APRIL 2013 JULIAN HARTY Contact me: julianharty@gmail.com Rev: 29 April 2013 Creative Commons.](https://reader034.fdocuments.us/reader034/viewer/2022051819/5519d8ef550346443e8b4d26/html5/thumbnails/25.jpg)
IF YOU WANT THE DRAFT BOOK
Draft book available for review & feedback
Email me [email protected]
• Agree not to share or distribute
• Comment to get the next available draft