Post on 15-Apr-2017
SIMPLE USABILITY TESTING(Fast & Lightweight Usability Testing Experiment)
Wayne PauApril 21, 2016 Internal
ABOUT MEWayne Pau is a well rounded, multi-disciplined member of the newly formed LoB Digital Asset and IoT group within SAP. Educated as a Systems Design Engineer from the University of Waterloo, he has been involved with wave after wave of ‘bleeding-edge’ technology, including RFID, Mobility and now IoT.
Wayne has a strong commitment to building the next generation of quality systems and applications. Over the years he has been involved with various parts of the development process, including app-dev, design, UX and testing. When not actively involved in building apps, he loves to engage with others to help promote adaptive innovation and building user-centric solutions using such approaches as Design Thinking and Agile/SCRUM.
WAYNE PAU Development Architect, LoB Digital Asset and IoT
Fast & Lightweight Usability Testing Experiment (FLUTE) - Part 1Curious on how to help ensure SAP builds the next wave of delightful applications? Wondering how to trouble-shoot those pesky user-interaction obstacles? Come and learn about a quick, low-cost, easy to implement Usability Testing framework that *any* team can implement. Perfect for those looking to start or enhance their usability testing endeavors.
No prior UX experience or training necessary! All are welcome.
4 Design Day 2016 @ SAP Montreal | Internal
WHO inspired FLUTE?WHAT is FLUTE? WHY (NOT) to do FLUTE?HOW to do FLUTE? WHEN to FLUTE?
01
02
03
04
05
Design Day 2016 @ SAP Montreal | Internal5
WHO?
Design Day 2016 @ SAP Montreal | Internal6
Design Day 2016 @ SAP Montreal | Internal7
8 Design Day 2016 @ SAP Montreal | Internal
WHO inspired FLUTE?WHAT is FLUTE? WHY (NOT) to do FLUTE?HOW to do FLUTE? WHEN to FLUTE?
01
02
03
04
05
Design Day 2016 @ SAP Montreal | Internal9
WHAT?
Design Day 2016 @ SAP Montreal | Internal10
FAST & LIGHWEIGHT USABILTY TESTING EXPERIMENT
(A.K.A. Simple Usability Testing…)
Design Day 2016 @ SAP Montreal | Internal11
“WE ARE ALL DESIGNERS BECAUSE WE MUST BE… WE MANIPULATE THE ENVIRONMENT TO BETTER TO SERVE OUR NEEDS.” › Norman (2004)
Design Day 2016 @ SAP Montreal | Internal12
“WE ARE NOT ALL DESIGNERS…A DESIGNER’S APPROACH TO CREATIVE PROBLEM SOLVING IS VERY DIFFERENT FROM HOW COMPUTER SCIENTIST, FOR EXAMPLE SOLVE PUZZLES.”
› Bill Buxton(2007)
Design Day 2016 @ SAP Montreal | Internal13
UXGestalt Psychology Affordances
Index of Difficulty
Fitt’s LawLevels of Abstraction
Ethnography
Heuristic EvaluationInformation Architecture
Likert ScaleMultivariate Testing (MVT)
Wizard of OzGolden PathGaze PlotSkeuomorphism10,000 Hours
Design Day 2016 @ SAP Montreal | Internal14
WHAT I CAN’T GIVE YOU…
• 10,000+ hrs of UX training (…on top of your tasks in 45 mins pres.)
• Predict User Intents, Reactions & Interactions
(…because understanding human is extremely difficult)
• Know if your specific Design will be Effective
(…again extremely difficult)
Design Day 2016 @ SAP Montreal | Internal15
WHAT I CAN GIVE YOU…
• METHOD to do Cheap & Effective UT(…because we are all strapped for resources)
• OBSERVE User Intents, Reactions & Interactions
(…because observing is far easier)
• OPPORTUNITY for team to act on User Feedback
(…and chance to make improvements if necessary)
Design Day 2016 @ SAP Montreal | Internal16
FAST & LIGHWEIGHT USABILTY TESTING EXPERIMENT
(A.K.A. Simple Usability Testing…)
17 Design Day 2016 @ SAP Montreal | Internal
WHO inspired FLUTE?WHAT is FLUTE? WHY (NOT) to do FLUTE?HOW to do FLUTE? WHEN to FLUTE?
01
02
03
04
05
Design Day 2016 @ SAP Montreal | Internal18
WHY (NOT)?
Design Day 2016 @ SAP Montreal | Internal19
TOP 5 EXCEUSESSpeaking subtitle
20© 2014 SAP AG or an SAP affiliate company. All rights reserved.
IT’S A TRAP…
Design Day 2016 @ SAP Montreal | Internal21
TOP 5X EXCUSES…
1.WE don’t have TIME.2.WE don’t have MONEY.3.WE don’t have EXPERTISE.4.WE don’t have a USABILITY LAB.5.WE don’t know HOW to interpret.
Design Day 2016 @ SAP Montreal | Internal22
1. WE DON’T HAVE TIME…
BUT…• Takes more time to fix issues after release• Wastes support and QA resources
Design Day 2016 @ SAP Montreal | Internal23
2. WE DON’T HAVE MONEY…
BUT …• Testing can be done for *almost* free• Fixing issues later can cause more Money
Design Day 2016 @ SAP Montreal | Internal24
3. WE DON’T HAVE EXPERTISE…
BUT …• FLUTE is easy to do• Can be done by *almost* ANYONE• You can reasonable results with *ANY* testing
Design Day 2016 @ SAP Montreal | Internal25
4. WE DON’T HAVE A LAB…
BUT …• Testing can be done for *almost* free• Fixing issues later can cause more Money
Design Day 2016 @ SAP Montreal | Internal26
5. WE DON’T KNOW HOW TO INTERPRET
BUT …• Important issues will be *EXTREMELY* obvious• Serious problems are HARD to miss
27 Design Day 2016 @ SAP Montreal | Internal
WHO inspired FLUTE?WHAT is FLUTE? WHY (NOT) to do FLUTE?HOW to do FLUTE? WHEN to FLUTE?
01
02
03
04
05
Design Day 2016 @ SAP Montreal | Internal28
HOW?
Design Day 2016 @ SAP Montreal | Internal29
USABILITY TESTING. ANYONE CAN DO IT. EVERYONE SHOULD BE DOING IT.
Design Day 2016 @ SAP Montreal | Internal30
Simple,Cheap,Effective
FAST & LIGHTWEIGHT USABILITY TESTING EXPERIMENT(F.L.U.T.E.)
31© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Simple,Cheap,Effective
Usability&&Testing
Design Day 2016 @ SAP Montreal | Internal32
SIMPLE
33© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Monitor
Couch
Camera
34© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Facilitator
Script
Subject
Design Day 2016 @ SAP Montreal | Internal35
https://jam4.sapjam.com/groups/ESDACJK4JQNIB1m3BlZ7tC/documents/3A8Mr7S5RCM16IzRoocgVF/
Design Day 2016 @ SAP Montreal | Internal36
https://jam4.sapjam.com/groups/about_page/ESDACJK4JQNIB1m3BlZ7tC
Design Day 2016 @ SAP Montreal | Internal37
CHEAP
Design Day 2016 @ SAP Montreal | Internal38
TOTAL EXPENSES…
1. User (Bribe: 5x$5 Timmy’s gift card)
2. Video Camera (FREE aka “Borrow”)
3. Room + Sofa (FREE aka “Borrow” again)
4. Snacks ($20 – also @ Timmy’s)
Total $45/mth (effect == “priceless”)
Design Day 2016 @ SAP Montreal | Internal39
EFFECTIVE
Design Day 2016 @ SAP Montreal | Internal41
THE RESULT… (SMALLEST CHANGE YOU CAN DO)
(Could even be disabled by user. On by default)
Design Day 2016 @ SAP Montreal | Internal42
TOP 5X EXCUSES…
1.WE don’t have TIME.2.WE don’t have MONEY.3.WE don’t have EXPERTISE.4.WE don’t have a USABILITY LAB.5.WE don’t know HOW to interpret.
Design Day 2016 @ SAP Montreal | Internal43
7X THINGS ABOUT FLUTE…
1. IF you want a GREAT app, you got to Testing2. Testing ONE User is 100% than testing NONE3. Testing ONE User is better than 50x Users at END4. Importance of recruit is OVER-RATED5. Testing does PROVE or DISPROVE anything6. Testing is ITERATIVE7. Nothing beats a LIVE audience
AppStore && Google Play “Ratings”
1x 7x
4.5
45© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Gather the Fellowship
46© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Not Clones…
But *MOST* Things are Obvious…
Get a Live Audience…
49 Design Day 2016 @ SAP Montreal | Internal
WHO inspired FLUTE?WHAT is FLUTE? WHY (NOT) to do FLUTE?HOW to do FLUTE? WHEN to FLUTE?
01
02
03
04
05
Design Day 2016 @ SAP Montreal | Internal50
WHEN?
Design Day 2016 @ SAP Montreal | Internal51
TEST *EVERY* SPRINT
Sprint 1
Sprint 2 … Sprin
t n
Usability
Test
Usability
Test…
Usability
Test
Usability
Test
start ship
*Always* something to Test…
Design Day 2016 @ SAP Montreal | Internal52
“THE PERFECT IS THE ENEMY OF GOOD”
Voltaire
User Tested App
Non- User Tested App>
WHEN YOU THINK YOUR
APP LOOKS THIS GOOD…
…IT COULD LOOK VERY
DIFFERENT TO OTHERS
SO DON’T BE AN …
THANK YOU.Simple Usability Testing - FLUTE
Wayne Pau, Development ArchitectApril 22, 2016
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes, BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries.
Oracle is a registered trademark of Oracle Corporation.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.
Java is a registered trademark of Sun Microsystems, Inc.
JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company.
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP company.
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.
The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.
This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice.
SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence.
The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.
© 2015 SAP SE. ALL RIGHTS RESERVED
Design Day 2016 @ SAP Montreal | Internal58