Some Findings from WorldCat Local Usability Tests Prepared ... · Some Findings from WorldCat Local...
8
1 Some Findings from WorldCat Local Usability Tests Prepared for ALA Annual, July 2009 Test goals 2 Test methods 2 Test partners 2 Combining local, group, and global scope 2 Journal article content 2 Simple search box 2 Advanced search options 2 Search result pages 3 • Order of search results 3 • What “relevance” means 3 • The interplay of location and relevance 3 • Details on search result pages 4 • Editions and FRBR 4 • Faceted browsing 5 Item Details pages 5 • Getting print items 6 • Links to electronic resources 6 • Rearrangement of high-value 7 bibliographic data Collaboration and personal workflow tools 7 • Reviews 7 • Recommendations 7 • Tags 7 • Ratings 7 • Lists 7 Navigation 8 213941 WorldCat Local provides locally branded access through a simple search box and advanced search options to local, group and global holdings, print, non-print and electronic resources (including full text from electronic subscriptions) for academic and public library users. Search results include relevance ranking, FRBR-based edition clustering, and faceted browsing. Item details include collaborative and workflow features: lists, tags, reviews, and ratings. Over the past two years OCLC has conducted close to a dozen tests of WorldCat Local, most with academic library users, some with public library users, and many in cooperation with pilot libraries. This summary assembles some key findings from our ongoing testing process. Contents
Transcript of Some Findings from WorldCat Local Usability Tests Prepared ... · Some Findings from WorldCat Local...
1
Some Findings from WorldCat Local Usability Tests Prepared for ALA Annual, July 2009
Test goals 2
Test methods 2
Test partners 2
Combining local, group, and global scope 2
Journal article content 2
Simple search box 2
Advanced search options 2
Search result pages 3
• Orderofsearchresults 3
•What“relevance”means 3
• Theinterplayoflocationandrelevance 3
• Detailsonsearchresultpages 4
• EditionsandFRBR 4
• Facetedbrowsing 5
Item Details pages 5
• Gettingprintitems 6
• Linkstoelectronicresources 6
• Rearrangementofhigh-value 7 bibliographicdata
Collaboration and personal workflow tools 7
• Reviews 7
• Recommendations 7
• Tags 7
• Ratings 7
• Lists 7
Navigation 8
213941
WorldCatLocalprovideslocallybrandedaccessthroughasimplesearchboxandadvancedsearchoptions
tolocal,groupandglobalholdings,print,non-printandelectronicresources(includingfulltextfrom
electronicsubscriptions)foracademicandpubliclibraryusers.Searchresultsincluderelevanceranking,
FRBR-basededitionclustering,andfacetedbrowsing.Itemdetailsincludecollaborativeandworkflow
features:lists,tags,reviews,andratings.
OverthepasttwoyearsOCLChasconductedclosetoadozentestsofWorldCatLocal,mostwithacademic
libraryusers,somewithpubliclibraryusers,andmanyincooperationwithpilotlibraries.Thissummary
assemblessomekeyfindingsfromourongoingtestingprocess.
Contents
2
Test goalsOftenourusabilitytestsaresummative:thegoalistovalidatedesignsorchangeswe’vemade.Sometimesourtestsareformative:thegoalistoprovokeorcorrectourthinkingaboutanemergingapproachordesignatapreliminarystage.
Test methodsWe’vedoneusabilitytestsinanumberofways:oftentask-basedsessionswithprototypeorproductionsystemsfollowedbyquestionnairesandinterviews;sometimescontextualinterviews,wherethetestparticipanttrainstheobserver.Someofthesewererelativelylargetests,withapproximatelyfifteenparticipants,andothersweresmalltestswithonlyfiveparticipants.Oftenweconducttestsatcolleges,universitiesorpubliclibraries.Sometimestheuseristhereorathome,butweconductthetestfromOhioorCalifornia.
AnagiledevelopmentmethodologyisusedforWorldCatLocal:wedesign,test,pilot,test,design.Itisaniterativeandongoingprocess.
Test partnersWedothesetestsincollaborationwithstaffatpilotsiteswhoareoftenveryinvolvedinplanningthetestsandanalyzingthefindingswithus.TestpartnershaveincludedtheUniversityofWashington,theUniversityofCalifornia(Berkeley,DavisandIrvinecampuses),theOhioStateUniversity,thePeninsulaLibrarySysteminSanMateo,California,theFreeLibraryofUrbanaandtheDesPlainesPublicLibrary,bothinIllinois.LaterinJulywe’llbetestingwithNortheasternIllinoisUniversityandtheUniversityofIllinois,Springfield.
Usabilitytestslikethesehavesomeadvantages.Weseewhatpeopleactuallydo(inanartificialsetting)ratherthanhearwhattheysaytheywoulddo.Wecanprobeinteractively,andgetparticipantstoreflect.Wecanobservebehavioracrossdifferentinstitutionsandovertime.
Thesetestshavesomelimitationstoo.Oursamplesareinsomewaysdiverse:theycomefromdifferentinstitutions,buttheyarenotreliablyrepresentative.They’rejustlooselyalignedwithourdevelopingpersonasandprofilesforWorldCatLocalbyrank,discipline,ageandgender.Testparticipantsarepredominantlyfromthesocialsciencesandhumanities;afewhavebeenfromthesciences,engineeringandnursing.
SurveysandfocusgroupsthatcangiveaviewofbroaderandmorereliablyrepresentativesamplesthantheusabilitytestssummarizedherearealsoconductedbyvariouspartsofOCLC.
Combining local, group and global scopeAcademicusershaveastronglyfavorableresponsetosearchinglocal,groupandworldwidecollectionstogether.
Forsomepubliclibrarypatrons,localmeansthisparticularbranch,nottheirwholecitysystem.
Publiclibrarypatronsareinterestedinwhat’s“nearby”thattheycanget,withoutalwaysrecognizingtheinterlendingpartnershipthatsupportsit.
Journal article contentBothundergraduateandgraduatestudenttestparticipantsrecognized,understoodandwelcomedthebasicconceptofcombiningarticlesformdifferentsourcesandcombiningthemwithbooks.Theyrepeatedlymentionedtheinclusionofjournalarticlecontentassomethingtheyvaluedhighly.However,mostacademicparticipantsinonetest(nineoffourteen)wronglyassumedthatjournalarticlecoverageincludesallthelicensedcontentavailableattheircampuses.
TheadditionofmorelicensedcontenttoWorldCatandsupportforfederatedsearchingofdatabasesoutsideWorldCatwillenlargethesetofsourcestosearch.Wehavetestedmetasearchprototypes;however,thoseresultsarenotincludedhere.Metasearchingandmulti-databasesearchinginasingleinterfaceintroduceusabilitychallengesandneedfurthertesting.
Simple search boxFacultypraiseditas“Googly.”Veryfewparticipantsdemonstratedanyconcernaboutwhethersurnamesshouldprecedeforenames.Repeatedly,academicusersexpressedappreciationforasearchboxwheretheycan“justtypeanything.”Mostfaculty,studentandpubliclibrarypatronsuseditoften,searchingfromwhateverpagetheyhappenedtobeoneitherwithinWorldCatLocaloralibrary’shomepage.
Advanced search optionsInatestwith14participants(sevenundergraduatesandsevengraduatestudents)onlyhalfofparticipantseverusedtheadvancedsearchscreen,andinfewerthanhalfofthetotalsearches—ninetimesin66searches.
Whenparticipantschosetouseadvancedsearch,theyoftenmentionedadesiretoavoidatoo-largeresult.Inthisandotherteststheystatedapreferenceforsearchingbyauthor
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
3
toavoidworksaboutaperson,ortitletoavoidfindingcommontitlewordsassubjects.
Whenjournalarticlecontentwasinvolved,academictestparticipantsoftenmentionedadesireforajournaltitleindex.Wehavenowbuiltthatjournaltitleindex.
Wehavealsomadesomelanguageadjustmentsforacademicusers,saying“journal”insteadof“serial,magazine,newspaper.”
We’vecausedsomeproblemswithpromptedvalues,andsolvedsomeproblems.Earlyonwemadesearchtermspersistonadvancedsearchscreen(becausewebsearchengineshavetheuser’ssearchtermspersistintotheadvancedsearchform),butreceivedmanycomplaints.Nowthosetermsarenotpersistent.Wearealsolookingatalternativesforrevisingasearchorstartinganewsearch.
Search result pagesWehadquestionsandgotfindingsabouttheorderofsearchresults,editionsandFRBR,whatdetailsweshow,andfacets.
Search results, May 2007
Search results, July 2009
Order of search results
Weinvestigatedwhatthedefaultordershouldbeinanacademicsetting,whetheritdiffersbetweenadvancedresearchersandundergraduates,whatthedefaultordershouldbeinapubliclibrarysetting,howwellthedefaultrankingsupportsfindingknownitems,andhowwellitsupportstopicalsearching.
Wefoundthatbothforstudentsandforscholars,inboththeknownitemcaseandthetopicalsearchcase,theexpectedandpreferredorderwasrelevance.Nearlyallparticipantsexpectedit;alargemajoritypreferredit.Asmallminoritypreferreddatedescending,andthoseparticipantswereabletochangethesortorder.MostacademictestparticipantsrecognizedtheorderofsearchresultsinWorldCatLocalas“relevance.”
Forpubliclibraryusers,itisnotyetclearfromourtestingwhattheexpectedorderis.Somepubliclibrarypatronshadahardtimedeterminingwhatordersearchresultswerein.
Whensearchingforknownitems,testparticipantsexpectedtheitemtobeonpageoneofthesearchresults.Whenitwasn’t,themostcommonbehaviorwastosearchagain,usingmorewordsorquotationmarks.
What “relevance” means
Fortopicalsearches,testparticipants—advancedresearchers,facultyandgraduatestudents—statedthatthe“best”itemswouldbeatthetopofthelist.Thisis“relevance”inaspecialsense,onethatincludessomethinglikerenown.
Academictestparticipantsdemonstratedmoretrustthancuriosityabouthowthismightbeaccomplished.Mostfacultyandgraduatestudentscouldsuggestseveralfactors—relevance,popularity,andlocation—thatmightbeinvolved.Butattentiontothatwassomethingtheysaidtheyregardedaslibrarians’expertise,nottheirs.Theytrusttheirlibrariestohavemadetherightchoicesabouthowtodothisforthem.
Someparticipantsremarkedwithsurpriseonthedesireditemnotbeingfirstinsearchresultsorbeingprecededbyanitemwheretheycouldn’tseethematchingtermsinthebriefsearchdisplay(becausetheywereincontentsnotesorelsewhere).Thisranking,however,wasnotusuallyanobstacletoparticipantsgettingthedesireditem.
The interplay of location and relevance
Thedefaultresultrankinginourfirsttestswasasubtleone.Localholdingswerelistedfirst,goingfrommosttoleastrelevant,followedbygroupholdingsgoingfrommosttoleastrelevant,followedbyworldwideholdingsgoingfrommosttoleastrelevant.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
4
Undergraduatetestparticipantsgenerallyreportfeelingwellservedbythisrankingthatputsitemsfromthelocalinstitutionatthetopofthelist.Participantsweregenerallynotawarethatitemsfromthatcollectionwerepromotedtothetopofthelist.
Facultyandgraduatestudenttestparticipantsgenerallydidnoticethatlocalmaterialswereatthetopofthelist.Buttheygenerallydidnotanticipatethatrankingbybothlocationandrelevancewouldunpredictablyputhighlyrelevantitemsoutofsight,afterlessrelevantones.Thiswasaconcernforscholarlytestparticipantswhowereoftennotinterestedonlyinlocalmaterials.
Inresponse,wecreatedasortoptionforrelevancewithoutlocation,stillweightedbyholdingsbutwithoutlocal,groupandglobalgrouping.Nowthatoptioncanbesetbyaninstitutionasthedefault.Atthesametime,weaddedamoreovertcontrolforlocation:apull-downmenunearthesearchbox.Insubsequenttests,someundergraduateparticipantsusedthepull-downmenutoscoperesults.Forthosewhodid,itdidwhattheyexpected.Allgraduatestudentsandfacultyusedthiscontroloftenandproficiently.
Details on search result pages
Manyparticipants,frombothacademicandpubliclibraries,statedthattheyexpectedtoseetheirsearchtermsintheitemsatthetopofthesearchresultpage.Matchesintitlesarevisible,butmatchesinsubjects,whichhappenoftenandcanbeequallyimportant,arenotvisiblenow.Testparticipantsexpectedtheevidenceofmatchingtobevisible,andtendedtodisregarditemsinwhichtheirsearchtermswerenotvisible.
Testparticipantsatpubliclibrariesoftenneededtoseeholdingsforthelocalbranch,notthewholecitysystem,onthesearchresultspage.Loadinglocalholdingsrecordsisaremedywhenthisisaproblem.
Anindicationofwhat’savailableisabasisforselection.Severalparticipantsinvarioustestsexpressedapreferencetoseeanindicationofcirculationstatusonthesearchresultspage.However,nearlyalltestparticipantswereabletodeterminewhetherlocallyheldmaterialswereavailablebyexaminingtheitemdetailspage.
Manyparticipantsexpressedadesiretoseeanabstractorsummaryonthesearchresultspage.Thisisachangeweareevaluatingnow.
Authorswereanimportantbasisforselectionforadvancedscholars.Facultyandgraduatestudentsindicatedthatrecognizingareputableauthorintheirfieldwassomethingtheyusedtodecidewhetheraniteminasearchresultshouldbelookedintofurther.Noneoftheundergraduate
studentsinourcontextualinterviewsindicatedthiswasafactorintheirselectionprocess.
Editions and FRBR
Weinvestigatedwhethertestparticipantsweremisledbyseeingonlyonemanifestationonsearchresultsrepresentingallothermanifestationsofthatwork;whetherweshowtherightmanifestationonsearchresultpages;whetherparticipantswereconfusedbyseeingmorethanonemanifestationonsearchresultpageswhenFRBRizationisimperfect;whetherparticipantsgetthelatesteditionofaworkwhentheyknowtheyneedthelatestedition,butdon’tknowtheyearofthatedition.
Wefoundthatingeneralwork-levelgranularityforsearchresultsbasedonFRBRratherthanedition-levelgranularityfittheexpectationsofbothacademicandpubliclibrarytestparticipants.Evenscholarshaddelegatedattentiontowhicheditiontostartwithtous.Puttingthedetailsofoneeditioninsearchresultsdidn’tstopstudentsorscholarsfromfindingothereditionswhentheywantedthem.IncompleteFRBRizationhasanegativeimpactonperformance,notunderstanding.
Undergraduatetestparticipants,primarilyinthehumanitiesandsocialsciences,reportedthattheyseldom(sevenparticipantsoftenparticipants)ornever(threeparticipants)lookedforaspecificeditionofabook.
Werepeatedlyheardeditionssometimesmattertoscholars—thelast,thefirst,thelastduringtheauthor’slifetime.Historiansexpressedapreferenceforthefirsteditionofprimarysourcesandthemostrecenteditionofsecondarysources.
Wefoundthatthedefaulteditionmatters.Evenadvancedresearchersacceptedthedefaultwhennotdirectedtofindaspecificedition,withoutrecognizingthatitwasnottheeditiontheylaterdescribedaspreferring(i.e.,theearliestorlatestedition).Wearenowinvestigatingwhetherthisdefaultshouldbethemostrecenteditionthat’savailablelocallyorthemostwidelyheldeditionthat’savailablelocally.
Whenparticipantsdidhaveaspecificeditioninmindandhadinformation(publisher,date,editor)toidentifyit,theygenerallysearchedwiththatinformationandsawthecorrespondingeditioninthesearchresult,sinceweshowthematchingedition.Nearlyallacademictestparticipants—bothundergraduatesandgraduatestudentsorfaculty—havebeensuccessfulatthissortoftask.
However,inourfirsttestsnoundergraduatesorgraduatestudentsweresuccessfulatfindingthelatesteditionofaworkiftheydidn’talreadyknowthedate.Thistaskisoneinstanceofidentifyingaparticulareditionwithoutknowingin
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
5
advanceanywordsthatcouldbeusedtofinditbysearching.Performancewasalmostuniformlypoorforfaculty,too.
Weaddeda“viewalleditionsandformats”link,firstatthetopoftheitemdetailspage,andthenalsoonthesearchresultspage.Afterthat,undergraduatesaswellasadvancedresearchersgenerallyfoundthelatesteditionwithoutknowinginadvancetheyearofthatedition(evenwhen,asoftenhappens,thateditionisnottheoneshowninbriefsearchresults).
Mostadvancedresearchersexpressedadesiretohaveallavailableeditionslisted(ontheeditionstab,asitthenwas)inreversechronologicalorderwithoutregardtolocation.Publiclibrarypatronsdidn’texpecttoseenon-Englisheditions,especiallynotatthetopofthatlist.Wecreatedaneweditionspage,whereuserscansortbydateandlocation,refinebyformat,seelanguagesandscopebylocation.Wechangedsortofeditionstoremovegroupingbylocal,thengroup,andthenglobalholdings.Weareconsideringaddingmoreeditioninformationforhighlyused/searchformatsandeditionsontheitemdetailspage.
Faceted browsing
Weinvestigatedwhetherfacetsarenoticedbytestparticipants,whethertheyworkasexpected,andwhetherthereareanyobstaclestouse.
Participantssaidanddemonstratedthattheynoticedfacets.Theysometimesusedfacets,andfacetsgenerallyworkedasexpected.
However,facetsweremoreoftenpraisedthanused.Insubjectsearchinginparticular,facetswerenotoftenusedbyundergraduateorpubliclibrarytestparticipants.Threeoutoftenusersinatestofsubjectsearchingusedfacets.Thefacetstheyusedwerelanguage,formatandyear.Participantsinthistestdidnotusethesubjectfacet(thencalledContent).Whenaskedwhy,severalsaidtheycouldn’tbesurewhattheywouldbeeliminatingiftheydid.
Subjecttermsatfirstappearedinafacetlabeled“content.”Nowthatfacetislabeled“topic.”Weaddedawaytoreversefacetchoicesundereachfacet.Wechangedtheorderoffacetstoputthemore-often-usedonesclosertothetopofthepage.
Item details pages
Item details, May 2007
Item details, July 2007
Item details, July 2009
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
6
Somethingsworkedwellevenwiththeearliestversionswetested,includinggettingprintitemsfromthelocalcollection,atleastforacademiclibraryusers.Somethingswereproblematic,includinglinkstoelectronicversionsandsupportingacademicworkflowsforcopingandpastingcitations.Weundertookamajorrevisionofthispage,testinghigh-fidelityprototypespartwaythroughthedesignprocess.
Getting print materials from the local collection
Canusersidentifywhat’savailableandwhere?
Foracademiclibraryusers,thisworkedwell.Whileafewtestparticipantsexpressedapreferenceforseeinginformationaboutavailabilityonthesearchresultspage,goingontotheitemdetailspageforthisinformationprovednottobeanobstacle.Locationandavailabilityinformationwasvisibleandrecognized.
Publiclibraryuserswantedtoseeholdingsfortheirlocalbranch,notthewholecitysystem.Thisgranularitycanbeanissue,asitwasinourearlypubliclibrarytests,dependingonhowholdingsarerepresented.Loadinglocalholdingsrecordsisaremedy.
Getting print materials from the group collection
Inourearlytestsparticipantscouldn’ttellwhethercopieswouldbeavailablefromtheconsortialcollection.
Weaddedcapacitytogetavailabilityinformationandsetthefulfillmentpolicyandworkflowforconsortialholdings,soWorldCatLocalcanofferabuttontogetacopyfromtheconsortialcollectionwhenitisavailableandproposeanotherapproachwhennoconsortialcopyisavailable.
Academictestparticipantsingeneralhadahighawarenessof—andrelianceon—thegroupcollectionsupportedbyaconsortiumorstatewidesystemtheirinstitutionmightbeapartof.
Somepubliclibraryuserswereconcernedalmostexclusivelywithwhatwasheldintheirlocalbranch.Othersregardedallmaterialstheycouldhavespeedyandsureaccesstoasequivalent.Notalltheseusersrecognizedtheconsortialarrangementsthatmadethatpossible.
Getting print materials from the worldwide collection
Undergraduatetestparticipantssawthewaytorequestitemsfromoutsidethelocalconsortiumbut,inearlytestsdidn’tseeminterestedindoingit.Thatwasprobablyanissuewiththetestdesign.Therearesomebarriersandsomeformstofillout.Theseparticipantswerenot,infact,deeplyinterestedintheitemswehadthemsearchfor,nordidtheyreallyexpecttogetthem.ActivitylogsshowthatinterlibraryloanisquiteaccessiblethroughWorldCatLocal.Interlibrary
loanwasoflimitedinteresttomostofourpubliclibrarytestparticipants.
We’vemovedthecontrolsforgettinglocal,groupandglobalmaterialstoclarifywherethematerialiscomingfrom.We’retestingtheminthesummerof2009inpublicandacademiclibrariesinIllinois,andwe’lltestthemagainwiththeUniversityofCalifornia.
Links to electronic resources
Inearlytests,accessingelectronicresourcesprovedtobeveryproblematic.Atfirst,halfofourtestparticipantsoverlookedthebigbuttonforviewingonlineorcheckingforelectronicresources.Wemovedthatbutton.
Inthenextroundoftests,participantsdemonstratedpatternsofmisunderstandingaboutthemanylinksandiconsassociatedwithelectronicresources.Thoselinkswerestilldispersedonvariouspartsofthepage.Participantswerenotverysuccessfulinknowingwhenelectronicresourceswereavailable,orretrievingthemwhentheywere.
Weredesignedtheitemdetailspagetogathertogetherandprioritizepreviouslyscatteredlinksforelectronicfulfillmentandinformationaboutavailability.Weremovedsomeconfusingicons.Also,WorldCatLocalnowallowsforcustomizationofthewordsshownontheopenURLresolverbuttonsotheyresonatewithwhatauserseesonotherpagesformtheirlibrary.
Wetestedjpegprototypesofthereviseddesignwithsixgraduatestudentsfromavarietyofdisciplines.Wesawimprovements.Thatdesignisinproductionnow,buttherearestillsomeproblems.Therecanstillbeamixoflinksonthispage:onesthatleadtofulfillmentbecausealibraryhasasubscriptionorbecausethey’reopenaccess,andonesthataredeadends,becauseneitherofthoseisthecase.Werecognizethatthisisausabilityissuethatcanmisleadusersintothinkingtheyhaveaccesstosomethingthatwedon’t,andwe’veinitiatedactiontoresolvetheissue.Wecontinuetolistentofeedbacktomakethedisplayofelectroniclinksaccurateforusers.Thisposesmanychallengesaswetrytoaccommodatedifferentlibrarycatalogingpracticesaswellasmeettheexpectationsoftheusersofthesystem.Inaddition,therehavebeencommentsabouthowweshowandhidelinksontheitemdetailspage.Bydefault,weshowthreelinksandiftherearemoretheyarecollapsedandtheusermustclickon“showmore”toexpandandseethem.We’renowinvestigatingwaystoindicatewhichlinkscarrythelocallibrary’sguaranteeofreliabilityand,whentherearemore,howtoshowmore.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
7
Rearrangement of high-value bibliographic data
Inourredesignofitemdetailspageswealsoaimedtopromotehigh-valuebibliographicdetails.
Whenparticipantsexamineditemdetailspages,thesummaries,abstractsandtablesofcontentsonthosepageswerehighlyvalued.Participantsoftenselectedthemwiththemousepointerandsometimesreadthemaloud.Subjectheadingssometimesplayedasimilarrole,supportingevaluationratherthandiscoveryofrelateditems.
Wemovedsummaryinformationtothetopofthepageintheredesignoftheitemdetailspage.Weareevaluatingmovingotherinformation–perhapscontentsandphysicaldescription–tothetopofthepage.Surveydatawillhelpusdeterminewhatinformationismostvaluabletousersinthisarea.
Collaboration and personal workflow toolsWedidsomeformativetestingofcollaborationandpersonalworkflowtoolswithjpegprototypesandgraduatestudents.Ourchallengesinintroducingthesecollaborativefeaturesarenotunique.IdentityandresponsibilityarecriticalpiecesforallWebsitesandnotuniquetoourtestsortolibrariesingeneral.However,academicsareanaudienceespeciallyawareofreputation,andacademicusershavespheresinwhichtheseidentitiesandresponsibilityarecreated,outsideofthelibrarycatalog.
User-contributedreviewsgotmixedreactionsfromgraduatestudents.Responsetoreadingreviewswasmostlypositive.Sixofsevenparticipantswouldreadone.Onlyoneofsevenwouldwriteone.Itmatterswhodoeswritethem.Wesawastrongpreferencewhatparticipantscalled“editorial”reviewsoveruser-contributedreviewsinseveraltests.Amongpubliclibrarypatrons(eightofthemintwoIllinoislibraries),halfsaidtheywouldcontributereviews.
WechangedthedesignoftheItemDetailspagesothatreviewsarecollapsedbydefault.
“Readersrecommend”(based,atthetime,onrecommendationsfromWeRead)gotlessmixed—largelynegative—reactions.Onegraduatestudenthadapositivereactionbecause,hesaid,theserecommendationsmightbebasedonconnectionshecouldnototherwisehavemade.Fivegraduatestudentparticipantshadnegativereactionsthatrangedfromdoubttoasenseofviolation.Negativereactionshadmoretodowithwhotheotherreadersmightbethanwiththeideaofrecommendations(whichwerepraisedinMedLine)orwiththespecificitemsrecommended(whichwereignored).Tobevalued,recommendationsneedtocomefromameaningfulsource.Whatweoftenheard
matteredwasnotjustlevel—facultyrecommendationsoftenmeantmorethanstudentrecommendationstotestparticipants—butalsoinstitutionanddepartment.Andnotonly—orevenmainly—thesearcher’sowninstitution,butratheraninstitutiontheparticipantrecognizesashighlyregardedinhisorherparticulardiscipline.
WeremovedWeReadrecommendations,andinsteadweaddedarecommenderthatlooksatauthor,subjectandseriestitletorecommenditems.Thesefindingspromptedustoconductfocusgroupswithlibrarianstoexplorepossiblesourcesofmoreauthoritativereviewsandrecommendations.
Graduatestudents’reactionstotagsasawayofdiscoveringrelateditemsweremixed,butmostlypositive.Fiveoutofsevenparticipantswoulduseothers’tagsasawaytosearch.(Threeparticipantsbeganwithanegativeviewandmovedtoapositiveone.)Testparticipantssaidtheyvaluedthewaytags“gobeyondofficialsubjects.”Theyhaddifferentideasaboutwhatthiswaymightbe:someparticipantsexpectingtagstobebroader,othersnarrower,ormorecurrent,thanlibraryterms.Twoofsevenparticipantswouldnotusetagsprovidedbyothers;bothquestionedtheexpertiseorviewpointofthoseunknownandtoovariousothers.Oneparticipantsaidshewouldaddtagsofherown,onesaidshewouldnot.Themostcriticalusers—scholars—sawmorepromiseintags.Theonetestparticipantwithastrongpreferenceforthe“official”subjectsprovidedbylibrarianswasanundergraduate.
Wetestedstarratings.Graduatestudentreactionstoratingswerealmostallstronglynegative.Sixparticipantswouldnotrelyonarating.Fiveparticipantswouldnotrateanitem.Themainobjectionherewasnotaboutwhoisspeaking,butoneaboutpurpose.Participantspointedoutthattheywerenotreadingforleisure(acontextinwhichtheymightrelyonratings)butratherconstructinganargumentordevelopingaposition.Thefactthatsomethingisdisagreeableormistakenwouldnotmakeituseless.Publiclibraryusers,whosegoalsincludeleisure,mighthaveverydifferentreactionstoratings.
WeremovedStarratingsfromthebibliographicareaforWorldCatLocal.
Graduatestudentsvaluedlists.Theywouldbevaluedmorehighlyiftheauthorwereknownorauthor’sinstitutionalaffiliationwererecognizedandrespected.Nevertheless,inonetestonlyoneoffouruserscreatedabibliographyinWorldCatLocalwithoutbeingtoldtodoso.Instead,userscutandpastedreferencesintoaMicrosoftWorddocument.Advancedscholarsexpressedthesamepreference.
Thenewdesignoftheitemdetailspagenowinproductionprovidesforahighlight/copy/pasteworkflow.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
8
NavigationNavigationpastthesecondorthirdpageofsearchresultsdidnothappenoften,evenforscholars.Althoughwehadparticipantsdotopicalsearchesintheirareasofexpertise,wecouldnotgetthemtolookpastthefirsttwopagesofsearchresults.Completenessofthesesearchresultsandwhatwasgoingonatthetailoftherankingwerenotaconcernforourtestparticipants.
Selectionisoftenmadefrombriefsearchresultpages,withoutgoingfurther.Whenparticipantsdogofurther,summary,abstractandcontentsnotesaremostvalued.
Allparticipantsreliedheavilyonthebrowser’sbackbuttonfornavigation.Someparticipantsexpressedadesireforamoreexplicitwaybacktosearchresults.Weaddeda“returntosearchresults”linktotheitemdetails.
Werecognizethatacademiclibraryusersandpubliclibraryusershavedifferentneeds,goalsandexpectations.Manychangeswehavemadesofarhavebeenbasedontheneedsofacademicusers.Aswedevelopanddifferentiatepersonasandprofilesforacademicusersontheonehandandpubliclibraryusersontheother,weexpecttocreatedifferentdesignthemestoreflectthesedifferences.So,forexample,reviewsandratingscanbemoreprominentforpubliclibrariesusingWorldCatLocalthanforacademiclibraries.
Some Findings from WorldCat Local Usability TestsPrepared for ALA Annual, July 2009
