Quick Search It Is Not:

Quick Search It Is Not: Testing Response Times of Traditional and NextGen CatalogsNina McHaleMargaret Brown-SicaLITA Forum 2010Esteemed Researchers

Our ResearchForthcoming: Margaret Brown-Sica, Jeffrey Beall, and Nina McHale, Next-Generation Library Catalogs and the Problem of Slow Response Time, Information Technology and Libraries, Volume 29/4, December 2010, 207-216. Not-So-Quick-Search

Our Research QuestionsAre NextGen catalogsor traditional catalogs that add NextGen contenttoo slow?Do 2.0/NextGen features slow them down too much?

Our ConclusionsYup.Features such as cover art, reviews, tagging, etc., can significantly increase the amount of data, and therefore time, required to return a catalog record page.Performance factors, particularly speed, should be required criteria for librarians and vendors evaluating and designing products.Speed Standards?W3C does not set forth standardsJakob Nielsen0.1 of a second: feels instantaneous1.0 second: feels uninterrupted10 seconds:About the limit for keeping the users attention focused on the dialogue.Give the user time-remaining feedback

Jakob Nielsen, Usability Engineering (San Francisco: Morgan Kaufmann, 1994) 135.Our MethodDuring a busy time during the semester, we recorded response times in seconds of permalinks for three catalog recordsTested our classic/NextGen catalogs and three others3 books, 5 catalogs, 3 times per day for 13 days=585 data pointsCollecting several data points in this way using www.websitepulse.com ensured that data was consistentAdditional Catalogs TestedLibrary of Congress Catalog VoyagerTraditional catalogUniversity of Texas at AustinInnovative Interfaces Traditional catalog with added NextGen elementsUniversity of Southern CaliforniaSirsi/DynixTraditional catalog with added NextGen elements

Books UsedHard Lessons: The Iraq Reconstruction Experience. Washington, DC: Special Inspector General, Iraq Reconstruction, 2009. (OCLC number 302189848)Ehrenreich, Barbara. Nickel and Dimed: On (Not) Getting by in America. 1st ed. New York: Metropolitan Books, 2001. (OCLC number 256770509) Langley, Lester D. Simn Bolvar: Venezuelan Rebel, American Revolutionary. Lanham: Rowman & Littlefield Publishers, c2009. (OCLC number 256770509)

Permalink Exampleshttp://aurarialibrary.worldcat.org/oclc/302189848http://skyline.cudenver.edu/record=b2433301~S0http://lccn.loc.gov/2009366172http://catalog.lib.utexas.edu/record=b7195737~S29 http://library.usc.edu/uhtbin/cgisirsi/x/0/0/5?searchdata1=2770895{CKEY}

Testing ToolsWebSite Pulsewww.websitepulse.comAllows testing on any web page/site; does not require server installationSimilar services:websiteoptimization.combrowsermob.comtools.pingdom.comyour favorite?WebSitePulse

WebSitePulse ResultsHorizontal bar:Gives visual representation of load time for each item (image files, javascript files, style sheets, etc.)Provides quick indication of sticking pointsTable: provides specifics about file size and delivery time for eachDNS, Connect, Redirect, First Byte, Last Byte, ErrorNumbers Crunched: Average Response Time in SecondsAurarias Skyline: 1.2930Aurarias WCL: 11.5734 Library of Congress: 2.1530University of Texas at Austin: 3.4997University of Southern California: 4.1085Individual Catalog Test ResultsAfter data was analyzed, we took a closer look at each individual catalog, using the Hard Lessons catalog recordWebSitePulse allowed us to take a glimpse at the inner workings of each catalogFindings confirmed that extra data and load times were from 2.0/NextGen contentSkyline, Auraria Library

Skyline Test Results: Graph

Skyline Test Results: Table

Skyline FindingsMissing favicon (item 4)0.9172 seconds uninterrupted per Nielsen14 items, for a total of 84.64 K:9 GIFs2 CSS1 JavaScriptGood performance, but an interface that only a librarian could loveWorldCat@Auraria

WorldCat@Auraria Results: Graph

WorldCat@Auraria Results: Table

WorldCat@Auraria FindingsReference & Instruction librarians observations corroborated10.3615 seconds31 items, for a total of 633.09 K, to load:10 CSS files10 JavaScript files8 GIFs/PNGsNo single NextGen feature slowed down load time, but multitude of files created unacceptable delayLibrary of Congress Catalog

Library of Congress Catalog Results: Graph

Library of Congress Catalog Results: Table

Library of Congress Catalog FindingsOverall, second fastest of all five catalogs tested1.2900 secondsOnly six items and 19.27 K to load:2 CSS files3 GIFsLike Skyline, fast, but has that legacy look

University of Texas at Austin

UT Austin Results: Graph

UT Austin Results: Table

University of Texas at Austin FindingsAdded NextGen features:Cover artLibraryThings Catalog EnhancementSupports recommendations, tag browsing, alternate editions/translations2.4674 seconds: user experience interrupted19 items, 138.84 KCover art nearly doubles response timeItem 14: script on ILS that queries Amazon for artUniversity of Southern California: HOMER

USC Results: Graph

USC Results: Table

USC FindingsSlowest among traditional catalogs; Sirsi/Dynix takes longer to make initial connection (Item 1 on graph)8.7295 seconds (though average was 4.1085 seconds)16 items, 148.47 KWhile attractive and well-integrated, Syndetic Solutions content (cover art, summary, author biography, and table of contents) adds 1.2 seconds to load timeIs the Content Worth the Wait? The new database seems based on Amazon.com. I dont need suggestions, and poor ones at that, of related books when I use the library. I dont need to see what other borrowers thought of the book. The information I need is poorly displayed. It is hard to cut and paste. It takes several screens to scan through, instead of the much quicker scroll in the traditional format. It supplies distracting, if not useless information (a picture of the cover, the distance to other librariesas if I need to know how far Provo is). -Auraria Campus Faculty Member Our ConclusionsMake performance testing part of evaluation process for vendor productsAdhere to industry standards for acceptable response times when testingOptimize delivery of 2.0/NextGen features as much as possibleConduct user testing to ensure that the content is worth the wait to their mindsQuestions? Comments?Nina McHale@ninermacmilehighbrarian.net