User behaviour in large-scale resource discovery contexts

Dick Hartley and Helen Booth

CERLIM

Manchester Metropolitan University

User behaviour: introduction

• Aims

• Methods

• Results

• Comments

User behaviour: aims

CERLIM’s role was to study user behaviour in the searching of union catalogues whether physical or virtual

User behaviour: methods

• Data collection consisted of three distinct phases– End-user searches– Interviews– Focus groups

• Data collection took place in three different university libraries: LSE, Leeds and Strathclyde

User behaviour: end-user searching

• Manufactured but realistic queries developed with the advice of staff from clumps and COPAC, 4 types– Find a common item in a library close by– Lot of information provided (what used?)– Find an obscure or unique item– Subject query giving large results

User behaviour: example queries

• You urgently need to get hold of a copy of ‘Clinical Medicine’ edited by Parveen Kumar and Michael Clark. You need to get the most up to date edition you can find in the nearest library.

• You are doing some research into monasticism and are interested in books on monks behaving badly. Someone has told you about a book that was published some time in the late 90s. They aren’t sure of the title but think that the author might be Justice or Jestice.

User behaviour: searching• Each searcher undertook 4 queries, 2 on COPAC

and 2 on local “clump”• No training but exploration time• Stop when

– searcher was satisfied that the appropriate search result had been achieved,

– searcher was not satisfied but did not want to proceed further,

– searcher was not able to proceed further,– searcher was fed up and wanted to go to the next

task.• Searches logged

User behaviour: interviews

• Post search interviews explored– Search options used– Selecting libraries– Results– Error messages and feedback– Problems encountered– Search session– Features liked, disliked and desired– Overall feelings

User behaviour: focus groups

• Took the view that librarians are users of union catalogues

• Given their knowledge and experience, used focus groups to enable discussion and ideas to flow

• Discussion based on a checklist of topics

• Taped and notes

Data analysis

• Search logs analysed into a large spreadsheet with a search as a row and a feature used as a column– Enabled easy analysis of features used

• Interviews and focus groups transcribed and manually analysed.

Results

• Profile of the end users

• Briefly consider results of searches

• Concentrate on attitudes and perceptions of end users

• Briefly pick up some points from focus groups of library staff

Profile of end users (1)

• Mix of genders– 20 female and 14 male

• Spread of ages– 21-30 19– 31-40 04– 41-50 05– 51-60 05– 61-70 01

Profile of end users (2)• Range of academic levels, FT and PT FT PTPGCE 1Taught Masters 4 3Research student 9 2Research staff 8 2Academic staff 3 1Admin staff 1

Profile of end users (3)

• Wide range of disciplines represented– Accounting and finance, Archaeology,

Biology, Counselling, Economics, Education, English, Environmental science, Genetics, Information management, Linguistics, Marketing, Media and communications, Medicine, Neuroscience, Philosophy, Politics, Psychology, Sociology, Statistics

Searchers’ use of e-resources

• Internet search engines

– Daily 28– 4-5 times per week 1– Once or twice per week 5

Searchers’ use of e-resources

• Bibliographic databases– 3-4 times per week 3– Once or twice a week 13– Every other week 6– Once or twice a month 5– Less than once a month 6– Have not used yet 1

Searchers’ use of e-resources

• OPACs– Daily 3– 3-4 times per week 3– Once or twice per week 15– Every other week 6– Once or twice a month 4– Less than once or twice a month 2– Never 1

Results from searches

• 30 out of 228 searches were abandoned without getting results

• Numerous spelling errors, some of which were never noticed and some of which produced results egs– Econimics, Bryon, Liguistics, Parlement,

Shapespeare

Results from searches: frequently used features

• CAIRNS– Refine search– Select libraries

• COPAC– Help– Language=English– Select libraries

• InforM25– Map– List of libraries

Results from searches

• Searches for Books– Almost equal use of author, title and isbn as

search criteria

• Searches for periodicals– ISSN followed by title most used search criteria

• But different searchers used a wide range of search criteria both singly and in combination

End-user perceptions of Union Catalogues(1)

• The term was almost completely unknown even by those who were using them

• Two or three users hazarded a guess that they were associated with Trades Unions.

End-user perceptions of Union Catalogues(2)

• End-user expectations of the performance of a union catalogue are heavily influenced by the Web

• Predictably this referred largely to Google but also to Amazon– If cannot learn to use a search tool very rapidly

then users would go to Google or Amazon– Some demand for greater information about book

content

End-user perceptions of Union Catalogues(3)

• End users expect search tools to be easy to master

– A typical comment was “If I can’t get the hang of what I am doing in the first half an hour then I’d reject the package and look for something else”.

End-user perceptions of Union Catalogues(4)

• Whilst there was the odd exception, most searchers were unwilling to wade through a large number of retrieved items

• Various approaches to dealing with large number of hits – Narrow search– Find out more about topic then try again– Give up and use the Web instead

End-user perceptions of Union Catalogues(5)

• End users have problems with on-screen language that is taken for granted by information professionals.

• Examples quoted to us included– Miniclumps– Holdings– Tag– Z39.50– Anything (COPAC search option)

End-user perceptions of Union Catalogues(6)

• End users do not like the presence of duplicate records in search output

• (fortunately for them they do not have to develop matching algorithms to remove dupes and only dupes!)

• ( how do they react to duplicate retrieval of the same website by Google?)

End-user perceptions of Union Catalogues(7)

• Many users expect “Google-like” speed of response.

• This was evidenced by the number of abandoned searches.

Librarians’ perceptions of end-users and union catalogues

• End-user awareness“I would say that I don’t think students are

particularly aware but some researchers might be”

• Researchers use– Location– Bibliographic details

• Researchers willingness to travel

Librarians and union catalogues

• Well aware of the concept– Quoted a far wider range at us than those which were the

focus of our study eg RLIN, WorldCat, even BUCOP!

• Limited trust in search output– Uptodateness of physical union catalogues– Doubts about efficacy of Z39.50 searches for virtual union

catalogues

Librarians’ views on scope of union catalogues

• Serials union catalogue is the most important

• After serials concentrate on rare books

• Some support for regional union catalogues

• No enthusiasm for subject based union catalogues

Facilities wanted in union catalogues

• Ability to sort by date• Ability to sort by author, title• Ability to place own library first• Ability to sort geographically• Ability to link search output to inter-

library loan software• Ability to pass search output to

reference management software

Comments and observations (1)

• COPAC usage data shows that there is clearly a demand for union catalogues

• How much greater could that demand be if there was greater awareness amongst potential users of the existence of such tools?

Comments and observations (2)

• There is a need for a major awareness-raising effort to increase use of union catalogues and other tools into which there has been considerable investment

Comments and observations (3)

• Designers need to take into account user expectations

• Simple interface• Speedy response• Manage expectations eg indications of

processing

Comments and observations (4)

• Investment in SUNCAT appears to be the correct decision

• Is it worth investigating the notion of what is a rare book and concentrating on adding those to union catalogues of books?

Comments and observations (5)

• Perhaps useful to recall Mooers Law– an information retrieval system will tend not

to be used whenever it is more painful and troublesome for a customer to have information than for him not to have it

The full report is• Helen Booth and R.J. Hartley

User behaviour in the searching of union catalogues: an investigation for Work Package C of CC-interop. Manchester: CERLIM, February 2004.

(http://ccinterop.cdlr.strath.ac.uk/documents/finalreportWPC.pdf )

• Thank you to all colleagues at CAIRNS, COPAC and InfoM25 and Tracey Stanley at University of Leeds for help in organising the searching, interviews and focus groups and devising the questions

• Thank you for listening!