134

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

Blackwell Publishing Ltd

Usability evaluation of an NHS library website

Catherine

Ebenezer Camberwell London UK

Abstract

Objectives

To carry out a usability evaluation of the recently launched SouthLondon and Maudsley NHS Trust library website

Methods

A variety of standard methodologies were employed content anddesign evaluation of selected comparable sites focus groups a questionnairesurvey of library and Web development staff heuristic evaluation observationtesting card sortingcluster analysis and label intuitivenesscategory member-ship testing All test participants were staff of or providers of services to thetrust Demographic information was recorded for each participant

Results

Test participantsrsquo overall responses to the site were enthusiastic andfavourable indicating the scope and content of the site to be broadly appropri-ate to the user group Testers made numerous suggestions for new content Usa-bility problems were discovered in two main areas in the organization of thesite and in the terminology used to refer to information services and sourcesBased on test results proposals for a revised menu structure improved acces-sibility and changes to the terminology used within the site are presented

Conclusion

Usability evaluation methods appropriately scaled can be advanta-geously applied to NHS library websites by an individual Web editor working alone

Introduction

Web-based information has become increasinglyimportant in health There is by now a large bodyof professional literature in the mental healthcommunity health and social welfare fields relat-ing to Web-based resources and their importancefor the development of evidence-based practice

1ndash8

The Department of Health and other statutorybodies use the Web extensively as a vehicle forofficial communications There are also severaljournals devoted to health informatics and tohealth information on the Internet Web pages arenow easily produced but there is a pressing needto evaluate whether the pages are meeting theusersrsquo needs The evaluation process tends to be

ignored in the production of web pages but itshould be core The purpose of this study was tocarry out a usability evaluation of the recentlylaunched South London and Maudsley NHSTrust library website The scope of the evaluationincluded not only design and navigation (thenarrower meaning of usability) but also thesuitability of the sitersquos content for its intendedpurpose and readership

Background

The South London and Maudsley NHS Trust

During the spring and summer of 2001 a websitewas built for the South London and MaudsleyNHS Trust (SLAM) Multidisciplinary Library inStockwell London SW9 The site was officiallylaunched and publicised in late June with the

Correspondence Catherine Ebenezer 8 Malfort Road CamberwellLondon SE5 8DQ UK E-mail ebenezercmsupanetcom

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

135

URL httpstlisthenhscomsthameslswmain(nowhttpstlisthenhscomhlns_londonlswmain)

The Trust is a large geographically dispersedorganization covering four London boroughs(Lambeth Southwark Lewisham and Croydon)with about 4500 staff based at over 100 sites Men-tal health and social services are now fully inte-grated within the borough of Lambeth and thelibrary is contracted to provide an informationservice to the staff of Lambeth Social Services Thelibrary also provides a service to the local com-munity health trust Community Health SouthLondon NHS Trust (CHSL)

dagger

While staff haveaccess to all the Kingrsquos College libraries and to theLewisham Hospital library SLAM Multidiscipli-nary Library is the only library managed directlyby the Trust and hence has an important role inthe provision of and co-ordination of informationservices to SLAM The staff it serves vary widelyin educational level in professional backgroundand in information and computer literacy

Health professionalsrsquo access to and use of the Web

During the data collection period for this evalua-tion web connectivity within SLAM was very poorOnly about 120 PCs had access to the NHSnetwith most staff having to depend on externalaccess from home or via NHS academic or localauthority libraries There is every indication thatthis connectivity level has improved significantlysince then Web connectivity in CHSL was muchbetter with most clinical teams having access attheir team bases In Lambeth Social Services webconnectivity within the workplace was availableto about 30 of staff

Anecdotal evidence (conversations betweenlibrary staff and readers) suggests that many read-ers (estimated gt 50 and increasing steadily) haveInternet access at home

9

but that their ability tosearch for information effectively is limited Awebsite can be useful also only insofar as it fitswith and relates to existing patterns of informa-tion seeking and use among its intended usersappropriate training and support also needs to be

provided to them in respect of information sourcesand services

10

The library website origins and rationale

Palmer in her study of information for communitymental health workers

11

suggested the develop-ment of lsquosignpostsrsquo to information as a method ofimproving awareness and use of informationresources A signpost should aim to function as

bull

a guide to the availability of sources and servicesof learning resource library and informationproviders

bull

a way to inform people of their rights to accessto these sources and services

bull

a channel through which users could be directedto the appropriate initial point of access toresources

bull

a device to educate and develop users as inde-pendent learning resource usersThus a full-scale library website might be able

to function as an lsquoelectronic signpostrsquo in this wayand also

bull

publicise the library and its services beyond itshistorical user base and immediate geographicalcatchment area (Lambeth and North Southwark)

bull

provide considerably enhanced access to libraryservices and other electronic sources from homefor staff who are geographically isolated or whohave difficulty accessing conventional libraryservices during their working day

bull

provide a window to selected quality-filteredinformation geared to the specific needs andinterests of practitioners working in mentalhealth and in community and primary carewhich is not readily available elsewhere

bull

offer an effective customised means of access tothe large variety of electronic journals which thelibrary provides

bull

make available internal resources and productssuch as bibliographies and search guides

1213

The initial design of the SLAM library site wasbased largely on intuition and first-hand know-ledge of the intended user group with some refer-ence to the websites of similar libraries Somecontent was drawn from existing material such asguides to services forms new book announce-ments and lists of web-based resources the restwas created de novo

dagger

Since the writing of this paper three PCTrsquos have taken the place ofCHSL These are Lambeth Southwark and Lewisham

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

135

URL httpstlisthenhscomsthameslswmain(nowhttpstlisthenhscomhlns_londonlswmain)

The Trust is a large geographically dispersedorganization covering four London boroughs(Lambeth Southwark Lewisham and Croydon)with about 4500 staff based at over 100 sites Men-tal health and social services are now fully inte-grated within the borough of Lambeth and thelibrary is contracted to provide an informationservice to the staff of Lambeth Social Services Thelibrary also provides a service to the local com-munity health trust Community Health SouthLondon NHS Trust (CHSL)

dagger

While staff haveaccess to all the Kingrsquos College libraries and to theLewisham Hospital library SLAM Multidiscipli-nary Library is the only library managed directlyby the Trust and hence has an important role inthe provision of and co-ordination of informationservices to SLAM The staff it serves vary widelyin educational level in professional backgroundand in information and computer literacy

Health professionalsrsquo access to and use of the Web

During the data collection period for this evalua-tion web connectivity within SLAM was very poorOnly about 120 PCs had access to the NHSnetwith most staff having to depend on externalaccess from home or via NHS academic or localauthority libraries There is every indication thatthis connectivity level has improved significantlysince then Web connectivity in CHSL was muchbetter with most clinical teams having access attheir team bases In Lambeth Social Services webconnectivity within the workplace was availableto about 30 of staff

Anecdotal evidence (conversations betweenlibrary staff and readers) suggests that many read-ers (estimated gt 50 and increasing steadily) haveInternet access at home

9

but that their ability tosearch for information effectively is limited Awebsite can be useful also only insofar as it fitswith and relates to existing patterns of informa-tion seeking and use among its intended usersappropriate training and support also needs to be

provided to them in respect of information sourcesand services

10

The library website origins and rationale

Palmer in her study of information for communitymental health workers

11

suggested the develop-ment of lsquosignpostsrsquo to information as a method ofimproving awareness and use of informationresources A signpost should aim to function as

bull

a guide to the availability of sources and servicesof learning resource library and informationproviders

bull

a way to inform people of their rights to accessto these sources and services

bull

a channel through which users could be directedto the appropriate initial point of access toresources

bull

a device to educate and develop users as inde-pendent learning resource usersThus a full-scale library website might be able

to function as an lsquoelectronic signpostrsquo in this wayand also

bull

publicise the library and its services beyond itshistorical user base and immediate geographicalcatchment area (Lambeth and North Southwark)

bull

provide considerably enhanced access to libraryservices and other electronic sources from homefor staff who are geographically isolated or whohave difficulty accessing conventional libraryservices during their working day

bull

provide a window to selected quality-filteredinformation geared to the specific needs andinterests of practitioners working in mentalhealth and in community and primary carewhich is not readily available elsewhere

bull

offer an effective customised means of access tothe large variety of electronic journals which thelibrary provides

bull

make available internal resources and productssuch as bibliographies and search guides

1213

The initial design of the SLAM library site wasbased largely on intuition and first-hand know-ledge of the intended user group with some refer-ence to the websites of similar libraries Somecontent was drawn from existing material such asguides to services forms new book announce-ments and lists of web-based resources the restwas created de novo

dagger

Since the writing of this paper three PCTrsquos have taken the place ofCHSL These are Lambeth Southwark and Lewisham

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

136

Objectives of the usability study

In developing the website a literature review wascarried out and a number of references coveringwebsite management design content evaluationusability and navigation were retrieved These arelisted on the authorrsquos web page

14

All these refer-ences were studied in detail and evaluated for useand applicability to this site

Some further literature on usability was alsoconsidered According to Fowler

15

lsquousability is thedegree to which a user can successfully learn anduse a product to achieve a goalrsquo It is often assessedin terms of a range of aspects ease of learningretention of learning over time speed of task com-pletion error rate and subjective user satisfac-tion

16

A web designer aims to create a site that isuseful (enables users to achieve their particularends and meets their needs) easy to use (enablesusers to move around the site rapidly and with fewerrors) visually attractive and popular

17

A site should be user centred in that it is basedon knowledge of the sitersquos users in particular theirtechnological and physical capacities their cul-tural context and their information needs

18

A brief usability evaluation of the site wascarried out as a means both of evaluating theappropriateness of its design and content and ofestablishing what modifications might be necessary

It is suggested by Veldof

19

and by Marmion

20

that information on library websites is typicallyidentified and structured in a librarian-orientatedfashion which does not accord with the usersrsquoneeds preconceptions and mental maps accord-ing to Gullikson

et al

21

their organizing prin-ciple can be inappropriate Matylonek

22

identifiesfive typical sources of bias for the creators ofwebsites

bull

discipline jargonmdashprofessional terminology notunderstood by users

bull

hierarchical biasmdashconfusion of the organiza-tionrsquos structure with customer services

bull

expert proficiency design of layout that presumescomfort in a web environment

bull

lsquofolk classificationrsquo naturally preferred termi-nology sets among various cultures

bull

preferred and novel services developers oftenuse certain services and over-emphasize them intheir designs

Methods

This project aimed via a combination of appro-priate usability testing methodologies to answerthe questions

bull

Is the sitemdashreadily intelligible ie not confusing to the

readermdashintuitive and easy to navigate (with respect to

overall structure navigation labellingsearchingbrowsing general features)

mdashvisually attractivemdashconsistent in design and terminology

bull

Are the readers able readily to locate informa-tion about library services

bull

Are the readers readily able to locate through itthe sources of mental health and communityhealth information they need Is its scope andcontent as far as is possible adequate andappropriate to the needs of the readership

bull

Does the way in which it presents informationabout library resources accord with the readerrsquosmental maps In particular is the division clearbetween locally networked and web-basedresourcesOverall the aim was to generate a set of evidence-

based proposals for redesigning or modifying thesite not focusing narrowly on design and naviga-tion aspects of usability but going some waytowards assessing the usefulness value and appro-priateness of the site content in relation to the per-ceived role of the library within the Trust

23ndash25

Nielsen and Landauer

26

in 1993 put forward theview that the best results in usability evaluationscome from carrying out as many small tests aspossible While their conclusions have been dis-puted

27

this work involved several phases and com-bined several different methodological approachesIn the preliminary phase a succinct content anddesign checklist was developed and used on aselection of websites of NHS libraries similar toSLAM Multidisciplinary Library as a bench-marking and evaluation tool and as a source ofnew ideas Focus group meetings with differentgroups of staff were also conducted in order toidentify key issues from the usersrsquo perspective togather ideas for development of the site andto determine if there were any issues requiringmore detailed investigation

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

137

The main phase of the project consisted of for-mal observation testing card sorting and a com-bined label intuitiveness and category membershiptest

In the final phase of the project proposals wereput forward based on the findings for revisions ofthe site The activities of the preliminary phasecannot be described here in detail for reasons ofspace A complete account of the project is avail-able from the author

Thirty-two participants were recruited to act astesters for the preliminary and main phases viaTrust-wide e-mail within SLAM and through per-sonal contacts It was not unfortunately possibleto recruit CHSL or Lambeth Social Services par-ticipants via e-mail circulars as access to theseorganizationsrsquo address books was not availableDespite the provision of an incentive (the offer ofa free lunch in the staff canteen) it proved difficultto recruit participants for activities which requiredextended time to be spent in the library such as thefocus groups and usability tests In practice mostof those who took part were based on site or inpremises nearby

Demographic information was recorded foreach tester as recommended by Davis

28

Amongthe volunteers there was a preponderance of medi-cal staff and of professional non-clinical staffalso there were considerably more women thanmen It proved impossible to recruit participantsfrom certain groups (social workers health careassistants) There was however a reasonablespread of backgrounds and a wide variation inlevels of information and computer literacy Onlythree participants reported making extensive useof the site before taking part in usability testingactivity

1

A small sample of six NHS library sites wasselected for the benchmarkingcontent evalua-tion The libraries were chosen deliberately forthe range of approaches to navigation anddesign they represented and the range of theironline content

2

Nine participants in all were recruited for threeseparate focus groups run at lunch times eachgroup had three members All were given about15 min to lsquoplayrsquo with the site before each of thesessions Each group was facilitated by theauthor and lasted about 45 min

3

Seven testers were recruited for the observationtest At the start of each test the participantswere given a script and list of tasks The 15 taskssome of which had a number of different com-ponents were designed to address anticipatedusability problems The usability metrics derivedwere percentage of tasks completed number offalse starts for each task longest time taken foreach task number of prompts required per taskper user and user satisfaction ratings

2930

Vol-unteers for the card-sorting test were recruitedvia a Trust-wide e-mail Sets of paper slips werecreated one slip for each item on each of themenus Menu category headings were alsoincluded among the slips Subjects were asked tosort the slips into categories using either one ofthe menu headings as a label for the category ordevising their own heading if they preferredThe cluster analysis software USortEZSort asdescribed by Dong and co-workers

31

was usedto record and analyse the results

4

Respondents were asked to complete a detailedlabel intuitivenesscategory membership ques-tionnaire This provided screen shots illustratingthe main menu and sub-menus respondents wereasked what they would expect to be included ineach main category and what sort of informationthey thought each of the links would indicate

Results and discussion

Content and design evaluation

The content evaluation was not intended to becomprehensive merely indicative of current prac-tice and trends in the sites of the libraries that wereselected

There is considerable variation in the primaryand additional navigation systems used in the cate-gories of information given on the home page inscope and content and in the services providedSites tend to group information about library serv-ices on a single long page Most of the sites makelittle use of interactive features typically providingonly one or two these however varied considera-bly Glenfield has a message board a chat roomand online polls Exeter has an online membershipregistration form while Brighton has online bookand journal article request forms

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

138

Typically print and electronic journals are listedtogether on one indexed alphabetical list only onesite (Exeter) had separate lists for print and elec-tronic formats Lists of electronic journals typi-cally include links to the relevant publisherrsquos oraggregatorrsquos website One site (Chichester) did notlist any journals Provision of book catalogueaccess varied one site provided a link to a univer-sity Web OPAC two others to a consortium unioncatalogue another provided access only within theTrust network to the catalogue and the others hadno access at all Databases available within thelibrary were generally listed and described in var-ying levels of detail under the general heading oflsquodatabasesrsquo Direct log-in facilities were providedwhere applicable

Sites did not generally have any significantuniquely developed content One exception wasthe Knowledgeshare clinical knowledge manage-ment website at Brighton Several however hadput together extremely useful lists of resources notfound elsewhere eg MCQs (Brighton) librarytraining guides (Glenfield) electronic medicaltextbooks (Exeter) guides to health statistics andsources for tracing practice guidelines (Exeter) Allthe sites except Chichester provided some selectedlinks to external websites The Exeter and Glen-field lists are comprehensive and highly developed

Focus groups

There were some things focus group membersliked

bull

lsquoLibrary on the desktoprsquo aspect

bull

Navigation and design clear and intuitive use offrames one item per page little scrolling neededpage content fits well within frame body

bull

Local map

bull

Union catalogues

bull

Use of language

bull

Links to web logsmdashthese are very useful as theyincorporate information circulated via e-mailinto a conveniently accessible format

bull

Principle of selecting key sites rather than hav-ing long comprehensive listsSome things they disliked

bull

URL too long and cumbersome

bull

Date on introductory page in Americanformat

bull

Lack of full web OPAC functionality

bull

Not clear what is password protected and whatis not

bull

lsquoSearch this sitersquo page is much too plain wouldlike prompts on search techniques and examplesFocus group members also gave suggestions for

development and additional content

bull

Union catalogue of serials across SLAM libraries

bull

List of professions allied to medicine links

bull

Update clinical governance material to includelinks to CHI and NICE

bull

Additional specialist bibliographies of relevanceto Trust eg mental health informatics cognitive-behavioural therapy transcultural psychiatry

bull

Register of Trust research

bull

Database of clinical audits

bull

More community health information especiallycommunity profiling

bull

More on boroughsAccording to the canons of focus group

methodology

32ndash34

the groups were (a) too smalland (b) should not have been conducted by the sitedesigner In this instance however combining thetwo roles in the same person meant that thefacilitator knew the participants and had a closeknowledge of the site and of the library servicehence could set individual comments in contextwithin the discussion

Observation test

The results of this test are analysed see Appendix 1A learning effect was apparent as the test was

conducted as they worked through the tasks thetesters learnt their way around the site and by theend their comments often indicated that that wereable to find things relatively easily Some basicgaps in peoplersquos knowledge of basic browser func-tionality became apparent even with experiencedsearchers one of these was not aware of the Findfeature available in Internet Explorer 5

prime

s Editmenu Some testers would go straight to lsquositesearchrsquo to locate an item of information whereasothers clearly preferred to work their way throughthe menus

35

The lsquomenu browsersrsquo among the test-ers often went to the more general category head-ings lsquofacilities for readersrsquo or lsquogeneral informationrsquoif uncertain where to look for something Twelvefailures to complete tasks and 15 prompts were

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

139

recorded out of a total of 126 test events The aver-age unprompted completion rate per participantwas 92

Testers rated the site highly on comprehensibil-ity of terminology ease of use and likelihood ofsubsequent use somewhat less highly on structureand organization

The test results were difficult to record withtotal accuracy it was difficult to take notes whileconducting the tests and the tapes that had beenrecorded could not always be deciphered Again itwas not good practice that the author of the siteshould have been conducting and recording thetests

36

It was impossible to refrain entirely fromprompting testers These tests were successfulhowever in identifying a number of significantusability issues

Card sorting test

Card sorting and cluster analysis is a usabilityevaluation method that is often adopted early onin the design of a site The rationale for using it inthis project was that it was felt to provide a clearidea of usersrsquo categorization and association ofinformation sources this providing a corrective tothe (possibly) librarian-centred organization ofthe site as it currently exists

The lsquohardrsquo end which emphasize the quantita-tive data that is the subject of cluster analysis

3437

The method adopted in this study of administer-ing the test by post and using EZSort to analyseand graphically represent results was of the lsquohardrsquovariety It had evident advantages in terms of timeand convenience but precluded any informativecontact on my part with the subjects It becameapparent that the results were being affected byuser uncertainty caused by lack of intuitiveness ofthe item labels with hindsight fuller descriptionsshould have been given of the item contents

The following results were evident across allthree charts lsquoInterlibrary loansrsquo is associatedstrongly with lsquoother librariesrsquo and with lsquoSouthThames Librariesrsquo rather than with journal orbook categories lsquoPhotocopyingrsquo lsquotelephonesrsquo andlsquorefreshmentsrsquo are only loosely associated withother library facilities lsquoComputingrsquo is associatedwith online search facilities rather than with

general library facilities Participants appeared tohave difficulty classifying lsquoour holdingsrsquo it doesnot convey a clear meaning apart from the contextof information about books lsquoCurrent awarenessrsquois associated with lsquousing the literaturersquo and withlsquosubject guidesrsquo and lsquosearch requests Of the 13people who initially volunteered to take part onlyseven returned results Card sorting is consideredto be more effective and accurate with 20 users ormore The results did however provide some clearpointers for restructuring the menu system

Category membershiplabel intuitiveness test

Eight completed questionnaires were receivedThis test revealed significant lack of clarity

around the main category headings lsquogeneral infor-mationrsquo and lsquofacilities for readersrsquo Four respond-ents expected that lsquofacilities for readersrsquo wouldinclude book loans three mentioned interlibraryloans two mention user support two mentionedjournals two mentioned Internet access and onementioned other libraries within the trust Underthe heading lsquogeneral informationrsquo one respondentmentioned other libraries two mentioned booksand book loans and one mentioned journalsBoth headings evidently tend to be interpreted asreferring comprehensively to all aspects of thelibrary service Three respondents expected lsquojournalsrsquoto include information on literature searching

Some of the item labels also appeared to beambiguous or problematic With lsquolinksrsquo threepeople expected this to cover specifically linksto trust sites three expected it to provide links toother libraries and one assumed it would lead tobasic contact information The heading lsquosubjectguidesrsquo led people to expect more general guides tosubject information rather than guides specifi-cally to web-based resources lsquoATHENSrsquo mystifiedpeople who were not familiar with the ATHENSauthentication system for databases and journalsPeople seemed to be unsure of what lsquocurrentawarenessrsquo might refer to With lsquocurrent titlesrsquowhich refers to journals currently subscribed tothree respondents expected this to include infor-mation on new books and one expected it to leadto a book recommendation form Three peopleexpected lsquoour holdingsrsquo which refers to booksto lead to information about journals and other

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

140

library materials as well Three respondents high-lighted an ambiguity in lsquorecommend a bookrsquo itwas interpreted as meaning lsquorecommendations bythe libraryrsquo lsquoreviewsrsquo lsquoother readersrsquo recommen-dationsrsquo as well as an invitation to suggest a bookfor purchase lsquoSite searchrsquo led two respondents toexpect a web search facility rather than a facilityfor searching the site this label was actuallyamended in the course of testing to lsquosearch thissitersquo Three people professed not to understand themeaning of lsquolocally served databases (CD-ROM)rsquoand four were evidently unsure of the meaning oflsquocommercial online databasesrsquo (by which had beenmeant fee-based services available via an onlinehost) The other significant labelling ambiguityseemed to be occurring with lsquorequest a searchrsquoalthough this actually leads to a search requestform for library staff to carry out a mediatedsearch four people expected this to lead to infor-mation on carrying out their own literaturesearches or directly to search facilities mountedon the site

The test also highlighted some expectations forcontent which are not currently available Underlsquobook collections and loansrsquo six respondentsexpected self-service Web OPAC circulation func-tionality to be available One expected there to bea facility to search their own interlibrary loanrecords online while another mentioned the desir-ability of an integrated union catalogue and inter-library loan request facility

As a result of usability testing detailed lists ofthe proposed changes to the site were drawn upand a new structure for the website proposed Thechanges covered the following areas navigationreadability HTML validity content and language

Conclusion

Relatively few people were involved in each ofthese tests It is a commonplace of the usabilitytesting literature that useful and valid results canbe obtained with eight users or less indeed that80 of usability problems will become apparentwith five users

38ndash39

It became clear particularly inthe formal observation testing that later testerswere identifying a high proportion of repeatrather than new problems This suggests (paceSpool

40

) that although additional issues would

probably have been identified with a larger groupof testers the small sample sizes do not seriouslyinvalidate the results as they stand

It appeared that the main usability issues hadbeen correctly anticipated

The major problems encountered by the testersappeared to involve two main areas (a) the spe-cialized terminology used in referring to informa-tion sources and services and (b) the organizationand structure of some of the information aboutlibrary services

According to France

41

library users can becharacterized as lsquochronic beginnersrsquo this he saysis in large part due to their uneven demand forlibrary services which can involve periods ofintense research separated by long gaps Thispresents particular problems of usability forlibrary applications in that users may combinedomain expertise and serious and sophisticatedinformation needs with considerable naiumlveteacute ininformation-seeking behaviour

Many researchers have highlighted the classifi-cation of information systems and the labelling ofresulting categories as a problem of informationdesign generally and of web information servicesin particular a review of the literature is providedby McGillis and Toms

42

With respect to librariesSpivey

43

discusses in detail the problem of howlibrarians should communicate on the Web withpeople outside the profession and with an increas-ing number of remote end-users who may havediverse cultural backgrounds and little previousexposure to academic information seeking

According to Spivey experienced library usersbecome familiar with library jargon but can beconfused by new systems and terminology or bythe availability of multiple platforms and inter-faces for a single resource (eg

) leadingto frustration and a sense of helplessness Libraryjargon can include short descriptions and nounsfor library resources and services eg lsquocirculationrsquolsquoATHENSrsquo library acronyms (OPAC ILL) ven-dorsrsquo trade names (eg SilverPlatter OVID) andwhat he terms lsquoembedded explanationsrsquo such asphrases in apposition examples or descriptionscategory headings or prepositional phrases Hefound that terms such as lsquoreferencersquo lsquoreservesrsquolsquoindexesrsquo lsquocitationsrsquo as well as more obviousjargon such as lsquoproximity operatorsrsquo lsquoimplicit

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

137

The main phase of the project consisted of for-mal observation testing card sorting and a com-bined label intuitiveness and category membershiptest

In the final phase of the project proposals wereput forward based on the findings for revisions ofthe site The activities of the preliminary phasecannot be described here in detail for reasons ofspace A complete account of the project is avail-able from the author

Thirty-two participants were recruited to act astesters for the preliminary and main phases viaTrust-wide e-mail within SLAM and through per-sonal contacts It was not unfortunately possibleto recruit CHSL or Lambeth Social Services par-ticipants via e-mail circulars as access to theseorganizationsrsquo address books was not availableDespite the provision of an incentive (the offer ofa free lunch in the staff canteen) it proved difficultto recruit participants for activities which requiredextended time to be spent in the library such as thefocus groups and usability tests In practice mostof those who took part were based on site or inpremises nearby

Demographic information was recorded foreach tester as recommended by Davis

28

Amongthe volunteers there was a preponderance of medi-cal staff and of professional non-clinical staffalso there were considerably more women thanmen It proved impossible to recruit participantsfrom certain groups (social workers health careassistants) There was however a reasonablespread of backgrounds and a wide variation inlevels of information and computer literacy Onlythree participants reported making extensive useof the site before taking part in usability testingactivity

1

A small sample of six NHS library sites wasselected for the benchmarkingcontent evalua-tion The libraries were chosen deliberately forthe range of approaches to navigation anddesign they represented and the range of theironline content

2

Nine participants in all were recruited for threeseparate focus groups run at lunch times eachgroup had three members All were given about15 min to lsquoplayrsquo with the site before each of thesessions Each group was facilitated by theauthor and lasted about 45 min

3

Seven testers were recruited for the observationtest At the start of each test the participantswere given a script and list of tasks The 15 taskssome of which had a number of different com-ponents were designed to address anticipatedusability problems The usability metrics derivedwere percentage of tasks completed number offalse starts for each task longest time taken foreach task number of prompts required per taskper user and user satisfaction ratings

2930

Vol-unteers for the card-sorting test were recruitedvia a Trust-wide e-mail Sets of paper slips werecreated one slip for each item on each of themenus Menu category headings were alsoincluded among the slips Subjects were asked tosort the slips into categories using either one ofthe menu headings as a label for the category ordevising their own heading if they preferredThe cluster analysis software USortEZSort asdescribed by Dong and co-workers

31

was usedto record and analyse the results

4

Respondents were asked to complete a detailedlabel intuitivenesscategory membership ques-tionnaire This provided screen shots illustratingthe main menu and sub-menus respondents wereasked what they would expect to be included ineach main category and what sort of informationthey thought each of the links would indicate

Results and discussion

Content and design evaluation

The content evaluation was not intended to becomprehensive merely indicative of current prac-tice and trends in the sites of the libraries that wereselected

There is considerable variation in the primaryand additional navigation systems used in the cate-gories of information given on the home page inscope and content and in the services providedSites tend to group information about library serv-ices on a single long page Most of the sites makelittle use of interactive features typically providingonly one or two these however varied considera-bly Glenfield has a message board a chat roomand online polls Exeter has an online membershipregistration form while Brighton has online bookand journal article request forms

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

138

Typically print and electronic journals are listedtogether on one indexed alphabetical list only onesite (Exeter) had separate lists for print and elec-tronic formats Lists of electronic journals typi-cally include links to the relevant publisherrsquos oraggregatorrsquos website One site (Chichester) did notlist any journals Provision of book catalogueaccess varied one site provided a link to a univer-sity Web OPAC two others to a consortium unioncatalogue another provided access only within theTrust network to the catalogue and the others hadno access at all Databases available within thelibrary were generally listed and described in var-ying levels of detail under the general heading oflsquodatabasesrsquo Direct log-in facilities were providedwhere applicable

Sites did not generally have any significantuniquely developed content One exception wasthe Knowledgeshare clinical knowledge manage-ment website at Brighton Several however hadput together extremely useful lists of resources notfound elsewhere eg MCQs (Brighton) librarytraining guides (Glenfield) electronic medicaltextbooks (Exeter) guides to health statistics andsources for tracing practice guidelines (Exeter) Allthe sites except Chichester provided some selectedlinks to external websites The Exeter and Glen-field lists are comprehensive and highly developed

Focus groups

There were some things focus group membersliked

bull

lsquoLibrary on the desktoprsquo aspect

bull

Navigation and design clear and intuitive use offrames one item per page little scrolling neededpage content fits well within frame body

bull

Local map

bull

Union catalogues

bull

Use of language

bull

Links to web logsmdashthese are very useful as theyincorporate information circulated via e-mailinto a conveniently accessible format

bull

Principle of selecting key sites rather than hav-ing long comprehensive listsSome things they disliked

bull

URL too long and cumbersome

bull

Date on introductory page in Americanformat

bull

Lack of full web OPAC functionality

bull

Not clear what is password protected and whatis not

bull

lsquoSearch this sitersquo page is much too plain wouldlike prompts on search techniques and examplesFocus group members also gave suggestions for

development and additional content

bull

Union catalogue of serials across SLAM libraries

bull

List of professions allied to medicine links

bull

Update clinical governance material to includelinks to CHI and NICE

bull

Additional specialist bibliographies of relevanceto Trust eg mental health informatics cognitive-behavioural therapy transcultural psychiatry

bull

Register of Trust research

bull

Database of clinical audits

bull

More community health information especiallycommunity profiling

bull

More on boroughsAccording to the canons of focus group

methodology

32ndash34

the groups were (a) too smalland (b) should not have been conducted by the sitedesigner In this instance however combining thetwo roles in the same person meant that thefacilitator knew the participants and had a closeknowledge of the site and of the library servicehence could set individual comments in contextwithin the discussion

Observation test

The results of this test are analysed see Appendix 1A learning effect was apparent as the test was

conducted as they worked through the tasks thetesters learnt their way around the site and by theend their comments often indicated that that wereable to find things relatively easily Some basicgaps in peoplersquos knowledge of basic browser func-tionality became apparent even with experiencedsearchers one of these was not aware of the Findfeature available in Internet Explorer 5

prime

s Editmenu Some testers would go straight to lsquositesearchrsquo to locate an item of information whereasothers clearly preferred to work their way throughthe menus

35

The lsquomenu browsersrsquo among the test-ers often went to the more general category head-ings lsquofacilities for readersrsquo or lsquogeneral informationrsquoif uncertain where to look for something Twelvefailures to complete tasks and 15 prompts were

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

139

recorded out of a total of 126 test events The aver-age unprompted completion rate per participantwas 92

Testers rated the site highly on comprehensibil-ity of terminology ease of use and likelihood ofsubsequent use somewhat less highly on structureand organization

The test results were difficult to record withtotal accuracy it was difficult to take notes whileconducting the tests and the tapes that had beenrecorded could not always be deciphered Again itwas not good practice that the author of the siteshould have been conducting and recording thetests

36

It was impossible to refrain entirely fromprompting testers These tests were successfulhowever in identifying a number of significantusability issues

Card sorting test

Card sorting and cluster analysis is a usabilityevaluation method that is often adopted early onin the design of a site The rationale for using it inthis project was that it was felt to provide a clearidea of usersrsquo categorization and association ofinformation sources this providing a corrective tothe (possibly) librarian-centred organization ofthe site as it currently exists

The lsquohardrsquo end which emphasize the quantita-tive data that is the subject of cluster analysis

3437

The method adopted in this study of administer-ing the test by post and using EZSort to analyseand graphically represent results was of the lsquohardrsquovariety It had evident advantages in terms of timeand convenience but precluded any informativecontact on my part with the subjects It becameapparent that the results were being affected byuser uncertainty caused by lack of intuitiveness ofthe item labels with hindsight fuller descriptionsshould have been given of the item contents

The following results were evident across allthree charts lsquoInterlibrary loansrsquo is associatedstrongly with lsquoother librariesrsquo and with lsquoSouthThames Librariesrsquo rather than with journal orbook categories lsquoPhotocopyingrsquo lsquotelephonesrsquo andlsquorefreshmentsrsquo are only loosely associated withother library facilities lsquoComputingrsquo is associatedwith online search facilities rather than with

general library facilities Participants appeared tohave difficulty classifying lsquoour holdingsrsquo it doesnot convey a clear meaning apart from the contextof information about books lsquoCurrent awarenessrsquois associated with lsquousing the literaturersquo and withlsquosubject guidesrsquo and lsquosearch requests Of the 13people who initially volunteered to take part onlyseven returned results Card sorting is consideredto be more effective and accurate with 20 users ormore The results did however provide some clearpointers for restructuring the menu system

Category membershiplabel intuitiveness test

Eight completed questionnaires were receivedThis test revealed significant lack of clarity

around the main category headings lsquogeneral infor-mationrsquo and lsquofacilities for readersrsquo Four respond-ents expected that lsquofacilities for readersrsquo wouldinclude book loans three mentioned interlibraryloans two mention user support two mentionedjournals two mentioned Internet access and onementioned other libraries within the trust Underthe heading lsquogeneral informationrsquo one respondentmentioned other libraries two mentioned booksand book loans and one mentioned journalsBoth headings evidently tend to be interpreted asreferring comprehensively to all aspects of thelibrary service Three respondents expected lsquojournalsrsquoto include information on literature searching

Some of the item labels also appeared to beambiguous or problematic With lsquolinksrsquo threepeople expected this to cover specifically linksto trust sites three expected it to provide links toother libraries and one assumed it would lead tobasic contact information The heading lsquosubjectguidesrsquo led people to expect more general guides tosubject information rather than guides specifi-cally to web-based resources lsquoATHENSrsquo mystifiedpeople who were not familiar with the ATHENSauthentication system for databases and journalsPeople seemed to be unsure of what lsquocurrentawarenessrsquo might refer to With lsquocurrent titlesrsquowhich refers to journals currently subscribed tothree respondents expected this to include infor-mation on new books and one expected it to leadto a book recommendation form Three peopleexpected lsquoour holdingsrsquo which refers to booksto lead to information about journals and other

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

140

library materials as well Three respondents high-lighted an ambiguity in lsquorecommend a bookrsquo itwas interpreted as meaning lsquorecommendations bythe libraryrsquo lsquoreviewsrsquo lsquoother readersrsquo recommen-dationsrsquo as well as an invitation to suggest a bookfor purchase lsquoSite searchrsquo led two respondents toexpect a web search facility rather than a facilityfor searching the site this label was actuallyamended in the course of testing to lsquosearch thissitersquo Three people professed not to understand themeaning of lsquolocally served databases (CD-ROM)rsquoand four were evidently unsure of the meaning oflsquocommercial online databasesrsquo (by which had beenmeant fee-based services available via an onlinehost) The other significant labelling ambiguityseemed to be occurring with lsquorequest a searchrsquoalthough this actually leads to a search requestform for library staff to carry out a mediatedsearch four people expected this to lead to infor-mation on carrying out their own literaturesearches or directly to search facilities mountedon the site

The test also highlighted some expectations forcontent which are not currently available Underlsquobook collections and loansrsquo six respondentsexpected self-service Web OPAC circulation func-tionality to be available One expected there to bea facility to search their own interlibrary loanrecords online while another mentioned the desir-ability of an integrated union catalogue and inter-library loan request facility

As a result of usability testing detailed lists ofthe proposed changes to the site were drawn upand a new structure for the website proposed Thechanges covered the following areas navigationreadability HTML validity content and language

Conclusion

Relatively few people were involved in each ofthese tests It is a commonplace of the usabilitytesting literature that useful and valid results canbe obtained with eight users or less indeed that80 of usability problems will become apparentwith five users

38ndash39

It became clear particularly inthe formal observation testing that later testerswere identifying a high proportion of repeatrather than new problems This suggests (paceSpool

40

) that although additional issues would

probably have been identified with a larger groupof testers the small sample sizes do not seriouslyinvalidate the results as they stand

It appeared that the main usability issues hadbeen correctly anticipated

The major problems encountered by the testersappeared to involve two main areas (a) the spe-cialized terminology used in referring to informa-tion sources and services and (b) the organizationand structure of some of the information aboutlibrary services

According to France

41

library users can becharacterized as lsquochronic beginnersrsquo this he saysis in large part due to their uneven demand forlibrary services which can involve periods ofintense research separated by long gaps Thispresents particular problems of usability forlibrary applications in that users may combinedomain expertise and serious and sophisticatedinformation needs with considerable naiumlveteacute ininformation-seeking behaviour

Many researchers have highlighted the classifi-cation of information systems and the labelling ofresulting categories as a problem of informationdesign generally and of web information servicesin particular a review of the literature is providedby McGillis and Toms

42

With respect to librariesSpivey

43

discusses in detail the problem of howlibrarians should communicate on the Web withpeople outside the profession and with an increas-ing number of remote end-users who may havediverse cultural backgrounds and little previousexposure to academic information seeking

According to Spivey experienced library usersbecome familiar with library jargon but can beconfused by new systems and terminology or bythe availability of multiple platforms and inter-faces for a single resource (eg

) leadingto frustration and a sense of helplessness Libraryjargon can include short descriptions and nounsfor library resources and services eg lsquocirculationrsquolsquoATHENSrsquo library acronyms (OPAC ILL) ven-dorsrsquo trade names (eg SilverPlatter OVID) andwhat he terms lsquoembedded explanationsrsquo such asphrases in apposition examples or descriptionscategory headings or prepositional phrases Hefound that terms such as lsquoreferencersquo lsquoreservesrsquolsquoindexesrsquo lsquocitationsrsquo as well as more obviousjargon such as lsquoproximity operatorsrsquo lsquoimplicit

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

138

Typically print and electronic journals are listedtogether on one indexed alphabetical list only onesite (Exeter) had separate lists for print and elec-tronic formats Lists of electronic journals typi-cally include links to the relevant publisherrsquos oraggregatorrsquos website One site (Chichester) did notlist any journals Provision of book catalogueaccess varied one site provided a link to a univer-sity Web OPAC two others to a consortium unioncatalogue another provided access only within theTrust network to the catalogue and the others hadno access at all Databases available within thelibrary were generally listed and described in var-ying levels of detail under the general heading oflsquodatabasesrsquo Direct log-in facilities were providedwhere applicable

Sites did not generally have any significantuniquely developed content One exception wasthe Knowledgeshare clinical knowledge manage-ment website at Brighton Several however hadput together extremely useful lists of resources notfound elsewhere eg MCQs (Brighton) librarytraining guides (Glenfield) electronic medicaltextbooks (Exeter) guides to health statistics andsources for tracing practice guidelines (Exeter) Allthe sites except Chichester provided some selectedlinks to external websites The Exeter and Glen-field lists are comprehensive and highly developed

Focus groups

There were some things focus group membersliked

bull

lsquoLibrary on the desktoprsquo aspect

bull

Navigation and design clear and intuitive use offrames one item per page little scrolling neededpage content fits well within frame body

bull

Local map

bull

Union catalogues

bull

Use of language

bull

Links to web logsmdashthese are very useful as theyincorporate information circulated via e-mailinto a conveniently accessible format

bull

Principle of selecting key sites rather than hav-ing long comprehensive listsSome things they disliked

bull

URL too long and cumbersome

bull

Date on introductory page in Americanformat

bull

Lack of full web OPAC functionality

bull

Not clear what is password protected and whatis not

bull

lsquoSearch this sitersquo page is much too plain wouldlike prompts on search techniques and examplesFocus group members also gave suggestions for

development and additional content

bull

Union catalogue of serials across SLAM libraries

bull

List of professions allied to medicine links

bull

Update clinical governance material to includelinks to CHI and NICE

bull

Additional specialist bibliographies of relevanceto Trust eg mental health informatics cognitive-behavioural therapy transcultural psychiatry

bull

Register of Trust research

bull

Database of clinical audits

bull

More community health information especiallycommunity profiling

bull

More on boroughsAccording to the canons of focus group

methodology

32ndash34

the groups were (a) too smalland (b) should not have been conducted by the sitedesigner In this instance however combining thetwo roles in the same person meant that thefacilitator knew the participants and had a closeknowledge of the site and of the library servicehence could set individual comments in contextwithin the discussion

Observation test

The results of this test are analysed see Appendix 1A learning effect was apparent as the test was

conducted as they worked through the tasks thetesters learnt their way around the site and by theend their comments often indicated that that wereable to find things relatively easily Some basicgaps in peoplersquos knowledge of basic browser func-tionality became apparent even with experiencedsearchers one of these was not aware of the Findfeature available in Internet Explorer 5

prime

s Editmenu Some testers would go straight to lsquositesearchrsquo to locate an item of information whereasothers clearly preferred to work their way throughthe menus

35

The lsquomenu browsersrsquo among the test-ers often went to the more general category head-ings lsquofacilities for readersrsquo or lsquogeneral informationrsquoif uncertain where to look for something Twelvefailures to complete tasks and 15 prompts were

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

139

recorded out of a total of 126 test events The aver-age unprompted completion rate per participantwas 92

Testers rated the site highly on comprehensibil-ity of terminology ease of use and likelihood ofsubsequent use somewhat less highly on structureand organization

The test results were difficult to record withtotal accuracy it was difficult to take notes whileconducting the tests and the tapes that had beenrecorded could not always be deciphered Again itwas not good practice that the author of the siteshould have been conducting and recording thetests

36

It was impossible to refrain entirely fromprompting testers These tests were successfulhowever in identifying a number of significantusability issues

Card sorting test

Card sorting and cluster analysis is a usabilityevaluation method that is often adopted early onin the design of a site The rationale for using it inthis project was that it was felt to provide a clearidea of usersrsquo categorization and association ofinformation sources this providing a corrective tothe (possibly) librarian-centred organization ofthe site as it currently exists

The lsquohardrsquo end which emphasize the quantita-tive data that is the subject of cluster analysis

3437

The method adopted in this study of administer-ing the test by post and using EZSort to analyseand graphically represent results was of the lsquohardrsquovariety It had evident advantages in terms of timeand convenience but precluded any informativecontact on my part with the subjects It becameapparent that the results were being affected byuser uncertainty caused by lack of intuitiveness ofthe item labels with hindsight fuller descriptionsshould have been given of the item contents

The following results were evident across allthree charts lsquoInterlibrary loansrsquo is associatedstrongly with lsquoother librariesrsquo and with lsquoSouthThames Librariesrsquo rather than with journal orbook categories lsquoPhotocopyingrsquo lsquotelephonesrsquo andlsquorefreshmentsrsquo are only loosely associated withother library facilities lsquoComputingrsquo is associatedwith online search facilities rather than with

general library facilities Participants appeared tohave difficulty classifying lsquoour holdingsrsquo it doesnot convey a clear meaning apart from the contextof information about books lsquoCurrent awarenessrsquois associated with lsquousing the literaturersquo and withlsquosubject guidesrsquo and lsquosearch requests Of the 13people who initially volunteered to take part onlyseven returned results Card sorting is consideredto be more effective and accurate with 20 users ormore The results did however provide some clearpointers for restructuring the menu system

Category membershiplabel intuitiveness test

Eight completed questionnaires were receivedThis test revealed significant lack of clarity

around the main category headings lsquogeneral infor-mationrsquo and lsquofacilities for readersrsquo Four respond-ents expected that lsquofacilities for readersrsquo wouldinclude book loans three mentioned interlibraryloans two mention user support two mentionedjournals two mentioned Internet access and onementioned other libraries within the trust Underthe heading lsquogeneral informationrsquo one respondentmentioned other libraries two mentioned booksand book loans and one mentioned journalsBoth headings evidently tend to be interpreted asreferring comprehensively to all aspects of thelibrary service Three respondents expected lsquojournalsrsquoto include information on literature searching

Some of the item labels also appeared to beambiguous or problematic With lsquolinksrsquo threepeople expected this to cover specifically linksto trust sites three expected it to provide links toother libraries and one assumed it would lead tobasic contact information The heading lsquosubjectguidesrsquo led people to expect more general guides tosubject information rather than guides specifi-cally to web-based resources lsquoATHENSrsquo mystifiedpeople who were not familiar with the ATHENSauthentication system for databases and journalsPeople seemed to be unsure of what lsquocurrentawarenessrsquo might refer to With lsquocurrent titlesrsquowhich refers to journals currently subscribed tothree respondents expected this to include infor-mation on new books and one expected it to leadto a book recommendation form Three peopleexpected lsquoour holdingsrsquo which refers to booksto lead to information about journals and other

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

140

library materials as well Three respondents high-lighted an ambiguity in lsquorecommend a bookrsquo itwas interpreted as meaning lsquorecommendations bythe libraryrsquo lsquoreviewsrsquo lsquoother readersrsquo recommen-dationsrsquo as well as an invitation to suggest a bookfor purchase lsquoSite searchrsquo led two respondents toexpect a web search facility rather than a facilityfor searching the site this label was actuallyamended in the course of testing to lsquosearch thissitersquo Three people professed not to understand themeaning of lsquolocally served databases (CD-ROM)rsquoand four were evidently unsure of the meaning oflsquocommercial online databasesrsquo (by which had beenmeant fee-based services available via an onlinehost) The other significant labelling ambiguityseemed to be occurring with lsquorequest a searchrsquoalthough this actually leads to a search requestform for library staff to carry out a mediatedsearch four people expected this to lead to infor-mation on carrying out their own literaturesearches or directly to search facilities mountedon the site

The test also highlighted some expectations forcontent which are not currently available Underlsquobook collections and loansrsquo six respondentsexpected self-service Web OPAC circulation func-tionality to be available One expected there to bea facility to search their own interlibrary loanrecords online while another mentioned the desir-ability of an integrated union catalogue and inter-library loan request facility

As a result of usability testing detailed lists ofthe proposed changes to the site were drawn upand a new structure for the website proposed Thechanges covered the following areas navigationreadability HTML validity content and language

Conclusion

Relatively few people were involved in each ofthese tests It is a commonplace of the usabilitytesting literature that useful and valid results canbe obtained with eight users or less indeed that80 of usability problems will become apparentwith five users

38ndash39

It became clear particularly inthe formal observation testing that later testerswere identifying a high proportion of repeatrather than new problems This suggests (paceSpool

40

) that although additional issues would

probably have been identified with a larger groupof testers the small sample sizes do not seriouslyinvalidate the results as they stand

It appeared that the main usability issues hadbeen correctly anticipated

The major problems encountered by the testersappeared to involve two main areas (a) the spe-cialized terminology used in referring to informa-tion sources and services and (b) the organizationand structure of some of the information aboutlibrary services

According to France

41

library users can becharacterized as lsquochronic beginnersrsquo this he saysis in large part due to their uneven demand forlibrary services which can involve periods ofintense research separated by long gaps Thispresents particular problems of usability forlibrary applications in that users may combinedomain expertise and serious and sophisticatedinformation needs with considerable naiumlveteacute ininformation-seeking behaviour

Many researchers have highlighted the classifi-cation of information systems and the labelling ofresulting categories as a problem of informationdesign generally and of web information servicesin particular a review of the literature is providedby McGillis and Toms

42

With respect to librariesSpivey

43

discusses in detail the problem of howlibrarians should communicate on the Web withpeople outside the profession and with an increas-ing number of remote end-users who may havediverse cultural backgrounds and little previousexposure to academic information seeking

According to Spivey experienced library usersbecome familiar with library jargon but can beconfused by new systems and terminology or bythe availability of multiple platforms and inter-faces for a single resource (eg

) leadingto frustration and a sense of helplessness Libraryjargon can include short descriptions and nounsfor library resources and services eg lsquocirculationrsquolsquoATHENSrsquo library acronyms (OPAC ILL) ven-dorsrsquo trade names (eg SilverPlatter OVID) andwhat he terms lsquoembedded explanationsrsquo such asphrases in apposition examples or descriptionscategory headings or prepositional phrases Hefound that terms such as lsquoreferencersquo lsquoreservesrsquolsquoindexesrsquo lsquocitationsrsquo as well as more obviousjargon such as lsquoproximity operatorsrsquo lsquoimplicit

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

139

recorded out of a total of 126 test events The aver-age unprompted completion rate per participantwas 92

Testers rated the site highly on comprehensibil-ity of terminology ease of use and likelihood ofsubsequent use somewhat less highly on structureand organization

The test results were difficult to record withtotal accuracy it was difficult to take notes whileconducting the tests and the tapes that had beenrecorded could not always be deciphered Again itwas not good practice that the author of the siteshould have been conducting and recording thetests

36

It was impossible to refrain entirely fromprompting testers These tests were successfulhowever in identifying a number of significantusability issues

Card sorting test

Card sorting and cluster analysis is a usabilityevaluation method that is often adopted early onin the design of a site The rationale for using it inthis project was that it was felt to provide a clearidea of usersrsquo categorization and association ofinformation sources this providing a corrective tothe (possibly) librarian-centred organization ofthe site as it currently exists

The lsquohardrsquo end which emphasize the quantita-tive data that is the subject of cluster analysis

3437

The method adopted in this study of administer-ing the test by post and using EZSort to analyseand graphically represent results was of the lsquohardrsquovariety It had evident advantages in terms of timeand convenience but precluded any informativecontact on my part with the subjects It becameapparent that the results were being affected byuser uncertainty caused by lack of intuitiveness ofthe item labels with hindsight fuller descriptionsshould have been given of the item contents

The following results were evident across allthree charts lsquoInterlibrary loansrsquo is associatedstrongly with lsquoother librariesrsquo and with lsquoSouthThames Librariesrsquo rather than with journal orbook categories lsquoPhotocopyingrsquo lsquotelephonesrsquo andlsquorefreshmentsrsquo are only loosely associated withother library facilities lsquoComputingrsquo is associatedwith online search facilities rather than with

general library facilities Participants appeared tohave difficulty classifying lsquoour holdingsrsquo it doesnot convey a clear meaning apart from the contextof information about books lsquoCurrent awarenessrsquois associated with lsquousing the literaturersquo and withlsquosubject guidesrsquo and lsquosearch requests Of the 13people who initially volunteered to take part onlyseven returned results Card sorting is consideredto be more effective and accurate with 20 users ormore The results did however provide some clearpointers for restructuring the menu system

Category membershiplabel intuitiveness test

Eight completed questionnaires were receivedThis test revealed significant lack of clarity

around the main category headings lsquogeneral infor-mationrsquo and lsquofacilities for readersrsquo Four respond-ents expected that lsquofacilities for readersrsquo wouldinclude book loans three mentioned interlibraryloans two mention user support two mentionedjournals two mentioned Internet access and onementioned other libraries within the trust Underthe heading lsquogeneral informationrsquo one respondentmentioned other libraries two mentioned booksand book loans and one mentioned journalsBoth headings evidently tend to be interpreted asreferring comprehensively to all aspects of thelibrary service Three respondents expected lsquojournalsrsquoto include information on literature searching

Some of the item labels also appeared to beambiguous or problematic With lsquolinksrsquo threepeople expected this to cover specifically linksto trust sites three expected it to provide links toother libraries and one assumed it would lead tobasic contact information The heading lsquosubjectguidesrsquo led people to expect more general guides tosubject information rather than guides specifi-cally to web-based resources lsquoATHENSrsquo mystifiedpeople who were not familiar with the ATHENSauthentication system for databases and journalsPeople seemed to be unsure of what lsquocurrentawarenessrsquo might refer to With lsquocurrent titlesrsquowhich refers to journals currently subscribed tothree respondents expected this to include infor-mation on new books and one expected it to leadto a book recommendation form Three peopleexpected lsquoour holdingsrsquo which refers to booksto lead to information about journals and other

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

140

library materials as well Three respondents high-lighted an ambiguity in lsquorecommend a bookrsquo itwas interpreted as meaning lsquorecommendations bythe libraryrsquo lsquoreviewsrsquo lsquoother readersrsquo recommen-dationsrsquo as well as an invitation to suggest a bookfor purchase lsquoSite searchrsquo led two respondents toexpect a web search facility rather than a facilityfor searching the site this label was actuallyamended in the course of testing to lsquosearch thissitersquo Three people professed not to understand themeaning of lsquolocally served databases (CD-ROM)rsquoand four were evidently unsure of the meaning oflsquocommercial online databasesrsquo (by which had beenmeant fee-based services available via an onlinehost) The other significant labelling ambiguityseemed to be occurring with lsquorequest a searchrsquoalthough this actually leads to a search requestform for library staff to carry out a mediatedsearch four people expected this to lead to infor-mation on carrying out their own literaturesearches or directly to search facilities mountedon the site

The test also highlighted some expectations forcontent which are not currently available Underlsquobook collections and loansrsquo six respondentsexpected self-service Web OPAC circulation func-tionality to be available One expected there to bea facility to search their own interlibrary loanrecords online while another mentioned the desir-ability of an integrated union catalogue and inter-library loan request facility

As a result of usability testing detailed lists ofthe proposed changes to the site were drawn upand a new structure for the website proposed Thechanges covered the following areas navigationreadability HTML validity content and language

Conclusion

Relatively few people were involved in each ofthese tests It is a commonplace of the usabilitytesting literature that useful and valid results canbe obtained with eight users or less indeed that80 of usability problems will become apparentwith five users

38ndash39

It became clear particularly inthe formal observation testing that later testerswere identifying a high proportion of repeatrather than new problems This suggests (paceSpool

40

) that although additional issues would

probably have been identified with a larger groupof testers the small sample sizes do not seriouslyinvalidate the results as they stand

It appeared that the main usability issues hadbeen correctly anticipated

The major problems encountered by the testersappeared to involve two main areas (a) the spe-cialized terminology used in referring to informa-tion sources and services and (b) the organizationand structure of some of the information aboutlibrary services

According to France

41

library users can becharacterized as lsquochronic beginnersrsquo this he saysis in large part due to their uneven demand forlibrary services which can involve periods ofintense research separated by long gaps Thispresents particular problems of usability forlibrary applications in that users may combinedomain expertise and serious and sophisticatedinformation needs with considerable naiumlveteacute ininformation-seeking behaviour

Many researchers have highlighted the classifi-cation of information systems and the labelling ofresulting categories as a problem of informationdesign generally and of web information servicesin particular a review of the literature is providedby McGillis and Toms

42

With respect to librariesSpivey

43

discusses in detail the problem of howlibrarians should communicate on the Web withpeople outside the profession and with an increas-ing number of remote end-users who may havediverse cultural backgrounds and little previousexposure to academic information seeking

According to Spivey experienced library usersbecome familiar with library jargon but can beconfused by new systems and terminology or bythe availability of multiple platforms and inter-faces for a single resource (eg

) leadingto frustration and a sense of helplessness Libraryjargon can include short descriptions and nounsfor library resources and services eg lsquocirculationrsquolsquoATHENSrsquo library acronyms (OPAC ILL) ven-dorsrsquo trade names (eg SilverPlatter OVID) andwhat he terms lsquoembedded explanationsrsquo such asphrases in apposition examples or descriptionscategory headings or prepositional phrases Hefound that terms such as lsquoreferencersquo lsquoreservesrsquolsquoindexesrsquo lsquocitationsrsquo as well as more obviousjargon such as lsquoproximity operatorsrsquo lsquoimplicit

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website

Catherine Ebenezer

copy Health Libraries Group 2003

Health Information and Libraries Journal

20

pp134ndash142

140

library materials as well Three respondents high-lighted an ambiguity in lsquorecommend a bookrsquo itwas interpreted as meaning lsquorecommendations bythe libraryrsquo lsquoreviewsrsquo lsquoother readersrsquo recommen-dationsrsquo as well as an invitation to suggest a bookfor purchase lsquoSite searchrsquo led two respondents toexpect a web search facility rather than a facilityfor searching the site this label was actuallyamended in the course of testing to lsquosearch thissitersquo Three people professed not to understand themeaning of lsquolocally served databases (CD-ROM)rsquoand four were evidently unsure of the meaning oflsquocommercial online databasesrsquo (by which had beenmeant fee-based services available via an onlinehost) The other significant labelling ambiguityseemed to be occurring with lsquorequest a searchrsquoalthough this actually leads to a search requestform for library staff to carry out a mediatedsearch four people expected this to lead to infor-mation on carrying out their own literaturesearches or directly to search facilities mountedon the site

The test also highlighted some expectations forcontent which are not currently available Underlsquobook collections and loansrsquo six respondentsexpected self-service Web OPAC circulation func-tionality to be available One expected there to bea facility to search their own interlibrary loanrecords online while another mentioned the desir-ability of an integrated union catalogue and inter-library loan request facility

As a result of usability testing detailed lists ofthe proposed changes to the site were drawn upand a new structure for the website proposed Thechanges covered the following areas navigationreadability HTML validity content and language

Conclusion

Relatively few people were involved in each ofthese tests It is a commonplace of the usabilitytesting literature that useful and valid results canbe obtained with eight users or less indeed that80 of usability problems will become apparentwith five users

38ndash39

It became clear particularly inthe formal observation testing that later testerswere identifying a high proportion of repeatrather than new problems This suggests (paceSpool

40

) that although additional issues would

probably have been identified with a larger groupof testers the small sample sizes do not seriouslyinvalidate the results as they stand

It appeared that the main usability issues hadbeen correctly anticipated

The major problems encountered by the testersappeared to involve two main areas (a) the spe-cialized terminology used in referring to informa-tion sources and services and (b) the organizationand structure of some of the information aboutlibrary services

According to France

41

library users can becharacterized as lsquochronic beginnersrsquo this he saysis in large part due to their uneven demand forlibrary services which can involve periods ofintense research separated by long gaps Thispresents particular problems of usability forlibrary applications in that users may combinedomain expertise and serious and sophisticatedinformation needs with considerable naiumlveteacute ininformation-seeking behaviour

Many researchers have highlighted the classifi-cation of information systems and the labelling ofresulting categories as a problem of informationdesign generally and of web information servicesin particular a review of the literature is providedby McGillis and Toms

42

With respect to librariesSpivey

43

discusses in detail the problem of howlibrarians should communicate on the Web withpeople outside the profession and with an increas-ing number of remote end-users who may havediverse cultural backgrounds and little previousexposure to academic information seeking

According to Spivey experienced library usersbecome familiar with library jargon but can beconfused by new systems and terminology or bythe availability of multiple platforms and inter-faces for a single resource (eg

) leadingto frustration and a sense of helplessness Libraryjargon can include short descriptions and nounsfor library resources and services eg lsquocirculationrsquolsquoATHENSrsquo library acronyms (OPAC ILL) ven-dorsrsquo trade names (eg SilverPlatter OVID) andwhat he terms lsquoembedded explanationsrsquo such asphrases in apposition examples or descriptionscategory headings or prepositional phrases Hefound that terms such as lsquoreferencersquo lsquoreservesrsquolsquoindexesrsquo lsquocitationsrsquo as well as more obviousjargon such as lsquoproximity operatorsrsquo lsquoimplicit

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

141

Booleanrsquo lsquouser authenticationrsquo were obstacles toreaders44 Naismith and Stein45 in their detailedstudy of student comprehension of technical lan-guage used by librarians found that readers mis-understood library terms in reference interviewsand library handouts about 50 of the timeUnsurprisingly given the rapid changes takingplace in the information market most readers donot have in their minds a clear taxonomy of elec-tronic information sources

Naismith and Stein suggest that a continuumof strategies such as the use of explanatory phras-ing the provision of glossaries etc should beemployed in written and verbal communication tobridge the gaps in understanding These haveobvious application to library website design Alibrary website needs to be seen in the context ofthe libraryrsquos user education and support strategyand indeed of its lsquoinformation architecturersquo as awhole with appropriate guidance provided tousers on identifying appropriate information toolsand sources

Regarding the provision of guides to webresources this study had sought to answer thepressing question lsquohow much is too muchrsquo Whilelibrarians frequently compile web resource guidesperceiving them to be an extension of library serv-ice functions46 there are evident limits to what canreasonably be included on the website of a smallspecialist library without undue duplication ofeffort Interestingly the participants in the studyappeared in their own use of the Web to be adopt-ing an lsquoanchor strategyrsquo of making regular use ofa few authoritative sites as observed by Westbergand Miller47

Participants in the focus group discussionsappeared to think that for them the key role of alibrary website in relation to external web-basedresources is not only to act as a form of quality fil-ter but also to provide readers with jumping-offpoints for their information seeking The focusgroup participants emphasized as well the valueof local content or content of immediate local rel-evance this appears to be an appropriate niche forthe SLAM Multidisciplinary Library site

Usability testing being limited to what can bereadily observed and measured is necessarilysomewhat artificial It is also limited in the type ofquestions it can answer it is unrealistic to expect

small-scale usability testing to answer questionsabout a sitersquos overall quality and effectiveness48 orto establish objective standards of usability42 Thisproject identified a number of significant usabilityproblems within the site it also afforded to a limi-ted extent the opportunity of evaluating emenda-tions The questionnaire and demographic dataobtained in the study gave indications of existinghabits of professional information seeking on theweb Focus group data and testersrsquo comments pro-vided indications of the potential value and useful-ness of the site to professional staff within thetrust and yielded many valuable suggestions forimproving it and developing it further

References

1 Barak A Psychological applications on the Internet a discipline on the threshold of a new millennium Applied and Preventive Psychology 1999 8 231ndash46

2 Bremer J amp Beresein E V Computers in psychiatry today Academic Psychiatry 2000 24 168ndash72

3 Kramer T amp Kennedy R Useful websites for psychiatrists Academic Psychiatry 22 141ndash3

4 Tantam D A guide to the Internet for psychotherapists Psychiatric Bulletin 2001 25 29ndash30

5 Holden G Rosenberg G amp Meenaghan T Information for social work practice observations regarding the role of the World Wide Web Social Work in Health Care 2000 32 1ndash8

6 Ward R amp Haines M Donrsquot get left behindmdashget online Practice Nurse 1998 16 164ndash6

7 Aitchison J amp Miller C Favourite web sites of UK GPs Available from httpwwwinpharmcomnetfocustrendsarticles_010html

8 Fleck E amp Levy S Internet support for nurses and midwives Professional Nurse 2000 14 280ndash2

9 Anthony D The Internet lsquohavesrsquo and lsquohave notsrsquo in nursing Presentation given at Healthcare Computing Conference Harrogate March 20ndash22 2000

10 Yeoman A Cooper C J Urquhart C amp Tyler A (eds) The Value and Impact of Virtual Outreach Services Report of the VIVOS Project Aberystwyth DIL University of Wales 2001

11 Palmer K Signposts to information for mental health workers A Research Project Funded by South and West Health Care Libraries Unit Bournemouth Bournemouth University Library and Information Services 1999

12 Diaz K The role of the library web site a step beyond deli sandwiches Reference and User Services Quarterly 1998 38 41ndash3

13 Stover M The mission and role of the library web site Available from httpwwwlibraryucsbeduuniversestoverhtml

Usability evaluation of an NHS website Catherine Ebenezer

copy Health Libraries Group 2003 Health Information and Libraries Journal 20 pp134ndash142

142

14 Ebenezer C Literature review Available from httpmemberslycoscoukebenezer1954hilj_literature_reviewhtm

15 Fowler S Appendix B usability tests In GUI Design Handbook [Online Monograph] New York McGraw-Hill 1997 Available from httpwwwfast-consultingcomappbhtm

16 Levi M D amp Conrad F G Usability testing of World Wide Web sites 1998 Available from httpstatsblsorgorehtm_papersst960150htm

17 Veldof J R Prasse M J amp Mills V A Chauffeured by the user usability in the digital library Journal of Library Administration 1999 26 115ndash40

18 Monash University ITS usability workshop 2001 Available from httpwwwitsmonasheduauwebslideshowsusabilityallhtm

19 Veldof J R Building user-centered library web sites on a shoestring Presentation given at WILS World 2000 Available from httpwwwwilswiscedueventsww2kww2kpresveldof

20 Marmion D Library web page design are we doing it right Information Technology and Libraries 2001 20 2ndash3

21 Gullikson S Blades R Bragdon M McKibbon S Sparling M amp Toms E G The impact of information architecture on academic web site usability Electronic Library 1999 17 293ndash30

22 Matylonek J Cluster analysis in Web design Presentation given at the ASIS PNC Bridging the Gap Innovations in Information Services Conference September 17ndash18 1999 Available from httposulibraryorstedustaffmatylonjclustersld001htm

23 Halub L P The value of Web-based library services at Cedars-Sinai Health System Bulletin of the Medical Library Association 1999 2001 87 256ndash9 Available from httpwwwshefacukuniprojectsimpact

24 Drezner J L A study of Internet use by physician treating HIV patients Medscape HIVAIDS 1998 4 Available from httpwwwmedscapecommedscapeHIVjournal1998v04n03

25 Harrison J M Tod A MorrisndashDocker S Black R amp Millen K The impact of access to the World Wide Web on evidence-based practice (of nurses and professions allied to medicine) Available from httpwwwimpactshefacukimpactpdf

26 Nielsen J amp Landauer T K A mathematical model of the finding of usability problems Proceedings of ACM INTERCHIprime93 Conference Amsterdam the Netherlands 24ndash29 April 1993 pp 206ndash13 Cited in Nielsen J 2000 Why you only need to test with five users Available from httpuseitcomalertbox20000319html

27 Spool J Five users is nowhere near enough Available from httpwwwwinwriterscomdownloadchi01_spoolpdf

28 Davis L M 2000 Effect of systematic usability testing on web site development and facilitation Available from httpwebmissouriedu simImd9f6ResearchUsabilityhtml

29 Rhodes J S 2001 Usability metrics Available from httpwwwwebwordcommovingmetricshtml

30 Nielsen J Usability metrics Available from httpwwwuseitcomalertbox20010121html

31 Dong J Martin S amp Waldo P A user input and analysis tool for information architecture Available from httpwww-3ibmcomibmeasyeou_extnsfPublish410$FileEZSortPaperpdf

32 Young V Focus on focus groups College and Research Libraries News 1993 54 391ndash4

33 Valentine B Undergraduate research behaviour using focus groups to generate theory Journal of Academic Librarianship 1993 19 300ndash4

34 Gafney G Usability techniques card sorting Available from httpwwwinfodesignau

35 Nielsen J 2001 Search visible and simple Available from httpuseitcomalertbox20010512html

36 Head A Web redemption and the promise of usability Online 1992 23 21ndash32

37 Hennig N Card sorting usability tests of the MIT Librariesrsquo web site categories from the userrsquos point of view In Hennig N (ed) Usability Assessment of Library-Related Web Sites Methods and Case Studies Chicago LITA 2001 88ndash9

38 Nielsen J 2000 Why you only need to test with five users Available from httpwwwuseitcomalertbox20000319html

39 Campbell N Chisman J Diller K R amp Walbridge S Discovering the user a practical glance at usability testing Electronic Library 1999 17 307ndash11

40 Spool J amp Schroeder W Testing web sites five users is nowhere near enough Available from httpwwwwinwriterscomchi01htm

41 France R K Nowell L T Fox E A Saad R A amp Zhao J Z Use and usability in a digital library search system Available from httpopac3ccvteduPapersUse_usabilityhtml

42 McGillis L amp Toms E G Usability of the academic web site implication for design College and Research Libraries News 2001 62 355ndash67

43 Spivey M A The vocabulary of library home pages an influence on diverse and remote end-users Information Technology and Libraries 2000 19 151ndash6

44 Dewey B In search of services analyzing the findability of links on CIC University libraries web pages Information Technology and Libraries 1999 18 210ndash3

45 Naismith R amp Stein J Library jargon student comprehension of technical language used by librarians College and Research Libraries News 1989 50 543ndash52

46 Sowards S A typology for ready reference web sites in libraries First Monday 1998 3 Available from httpwwwfirstmondaydkissues_3 minus5sowards

47 Westberg E E amp Miller R A The basis of using the Internet to support the information needs of primary care Journal of the American Medical Informatics Association 1996 6 6ndash25

48 Bernstein M Judging web sites usability or criticism Available from httpwwweastgatecomHypertextNowarchivesMerithtml