Usability and Usability Testing of Websites: An Example Redesign ...
Usability Tesng - cs.umd.eduatif/Teaching/Fall2009/Hyoungtae.pdf · Jakob Nielsen: the Magic Number...
Transcript of Usability Tesng - cs.umd.eduatif/Teaching/Fall2009/Hyoungtae.pdf · Jakob Nielsen: the Magic Number...
Whatisusability?• “How well users can learn and use a product to achieve their
goals and how sa5sfied they are with that process”
• Cannotbedirectlymeasured;quan9fiedbymeansofindirectmeasuresora<ributes:– Thenumberofreportedproblems
– Easeoflearning– Efficiencyofuse– Memorability
– Errorfrequencyandseverity– Subjec9vesa9sfac9on
Terminology• UsabilityTes9ng:toevaluateaproductbytes9ngiton
representa9veusersfromthetargetaudiencethroughspecifictasks
• ExpertReview:performedbyusabilityexpertstoiden9fypoten9alusabilityproblems
• ReturnOnInvestment(ROI)ofUsability:Thereturnsfromusabilityimprovementagainstaddedefforts/costs(9me&money)
UsabilityTes9ngvs.ExpertReviewUsabilityTes,ng ExpertReview
Whoconducts? Representa9vepeoplefromtargetaudience
AcoupleofUsabilityExperts
Useful Whenfindingrealusabilityproblems
Whenfindingviola9onsofusability/designstandards
Length 4‐6weeks 1‐3weeks
Disadvantage ‐ ExpensiveandTimeconsuming‐ Heavilydependsoniden9fyingrighttargetgroup,accuracyoftes9ngprotocol
‐ FalsePosi9ves‐ Misstherealproblemsthatcauseuserstofailtasks
JakobNielsen:theMagicNumber5
• Usabilitytestsareverycostlyandcomplex?– NotTrue– “Elaborate usability tests are a waste of resources”
• Nomorethan5usersandrunningasmanysmalltestsasyoucanafford
Threetestsetswith5users
each
15testsetswith1user
Atestsetwith15users
misled by a spurious user
same findings
JakobNielsen:theMagicNumber5
• “Discountusability”model
• Thepropor9onofUsabilityproblemsfound(%)=– L:usabilityproblemsfoundbyasingleuser
– n:numberoftestusers
L=31%84.36%
5
Conflictsagainstthemagicnumber
• Arguingagainstthefive‐userguidelineintermsoftheclaimonsta9s9calmethods:– AtypicalL,31%exists?(variesfrom8%to51%)– Asthenumberofpar9cipantsisincreasing,isthepropor9onof
usabilityproblemsfoundincreasing?
• Someempiricalresearchesshowthattes9ngthefirstfiveuncoveredmuchlowerpercentages(Spool:35%,Faulkner:55%)ofusabilityproblemsthan85%asNielsenclaimed.
UsabilityTes9ng:WhatHaveWeOverlooked?
• Gi<eLindgaard&JarineeChar<ra9chart• CUE:Compara9veUsabilityEvalua9on,tocollectdatafor
usabilitymethodsandtechniques
• CUE‐4:17professionalteamstoevaluatewww.hotelpenn.com(9teams:usabilitytes9ng,8teams:expertreview)
ResearchQues9ons
• Thereisacorrela9onbetweennumberofusersandthepropor9onofproblemsfound.
• Thereisacorrela9onbetweennumberofusertasksandthepropor9onofproblemsfound.
Methods
• Think‐aloudMethod
• 9teamswithdifferentnumberofusers
• AnalyzeTasksandScenarios– Taskgoal,e.g.,Findanavailableroom
– Usertask,e.g.,Checkroomavailabilityofapar9cularroomtypeonacertaindate;checkroomavailabilityforthefollowingyear,…
– Usertasktoken,e.g.,goingbacktothehomepage;makingareserva9onforafamilyofthreefromJune28toJuly5
• Analyzetheproblemsreportedbyeachusabilitytestteam
Team A H J K L M N O S
#ofUsers 6 12 7 5 6 15 13 6 6
Results
• NumberofUsers&UsabilityProblemsfound– Correla9onexists
r=0.731,p<0.05(n=9) r=0.821,p<0.01(n=9)
AnalysisonResults
• Correla9onbetweenthenumberofusersandthepropor9onofproblemsfoundwasnotsupported
• Correla9onbetweenthenumberoftasksandsthepropor9onofproblemsfoundwasupheld
• 5userclaimwasnotsupported(85%oftheproblemsfound)• Roleofpar9cipantrecruitment(TeamA&TeamL)
Team A H J K L M N O S
#ofUsers 6 12 7 5 6 15 13 6 6
#ofusertasks 14 11 5 11 12 10 6 10 8
ProblemsFound(%) 42 43 7 22 27 29 23 24 30
%Newproblems 12 8 0 4 4 3 2 5 4
Summary
• Whatwehaveoverlooked:Othercontribu9ngfactorstoimproveusabilitysuchastaskcoverageandpar9cipantrecruitmentratherthansamplesize– Usabilityinves9ga9ononimportanttargetusers&themostcri9cal
tasks
• Ifthegoalofusabilitytes9ngistogatherqualita9veinsightstoimproveproducts,oneortwoindividualobserva9onscanprovidethem
• Nielsen’sclaimisuseless?– Inoverall,themagicnumber5canbeappliedasoneofquan9ta9ve
assessment– Todemys9fytheconceptthatusabilitytes9ngisverycostlyatthat
9me
References
• Gi<eLindgaard,JarineeCha<ra9chart,Usabilitytes9ng:whathaveweoverlooked?,ProceedingsoftheSIGCHIconferenceonHumanfactorsincompu9ngsystems,April28‐May03,2007,SanJose,California,USA
• Nielsen,J.Whyyouonlyneedtotestwith5users,JakobNielsen'sAlertbox,March19,2000,h<p://www.useit.com/alertbox/20000319.html
• Faulkner,L.Beyondthefive‐userassump9on:Benefitsofincreasedsamplesizesinusabilitytes9ng.BehaviorResearchMethods,Instruments&Computers,35,3,PsychonomicSociety(2003),379—383
• JaredSpool,WillSchroeder,Tes9ngwebsites:fiveusersisnowherenearenough,CHI'01extendedabstractsonHumanfactorsincompu9ngsystems,March31‐April05,2001,Sea<le,Washington