Leabharlann UCD

An Coláiste Ollscoile, Baile

Átha Cliath,

Belfield, Baile Átha Cliath 4,

Eire

UCD Library

University College Dublin,

Belfield, Dublin 4, Ireland

Using a consultancy to assist in developing the UCD vision for the future online library environment

What we found out and was it worth it?

Ros Pan and

Caleb Derven

UCD Library

Presentation format summary

• Background - the basic facts about why we involved a consultant and what was done

• The report and recommendations

• Two points of view on the process and the findings

• Pros and Cons of the consultancy and suggestions for anyone thinking about following this approach

Section OneBackground - the basic facts, why we did it and what was done

What was the need?• We had been considering the need to re-think our

Website for some time – sprawling size, statistical snapshots, informal user feedback

• We were acutely aware that the Library website was just one of the main online environments UCD provides for its users, others being the VLE, UCD Connect portal and School websites, catalogue

• As we mused about all this, all the Web 2.0 options began to materialize as well and we started to experiment with blogs, Facebook, 2Life, chat

• It seemed clear that rather than just a website re-think we need to move up to a much higher level of thinking about the entire online Library platform choices and think strategically – currently we are involved in all these environments somewhat piecemeal

Why use a consultant and how to get one

• To help Library staff think outside the box with help of a fresh pair of eyes

• To resource time-consuming activities

• To bring in some expertise that we lacked

• Being a fairly small scale project we produced a specification of what we wanted and issued ITT via Irish mailing lists and lis-link

• Responses were not numerous and no Irish firms tendered

• 2 shortlisted companies came and did presentations for us and on the basis of that senior management chose a UK-based company

What took place

• A project team was formed up in the Library:Ros Pan (Chair); Julia Barrett; Caleb Derven; Valerie Kendlin; Hugh Murphy; Aoife O'Brien; Kathryn Smith; Anne Vernet – Deputy Heads, Sub-Librarians and E-service team members

• A Wiki was set up as project team support tool

• A blog was set up to keep library staff in general informed on progress

• Main activities of the consultancy were: kick off meeting; 1-1 phone interviews; various workshops with library staff and user groups; survey of library staff (74 responses); survey of users (1689 responses = about 7%)

• Main deliverable was report and a final workshop

What took place – the timing• Kick off meeting held Sept 18

• Signed off detailed terms of project Oct 16

• Workshop with library staff held Oct 22

• Workshops with academic staff (2) held Oct 23

• Workshops with students (2), held Oct 28

• 1-1 stakeholder interviews, Oct 13 - Nov 24

• Survey of all library staff November

• User Survey, November 6 - 24

• Final workshop with policy making library staff, Jan 20 2009 and report delivered end of January including 37 recommendations

• Our internal follow up actions and decisions are to follow starting with a management workshop in early April

Section TwoReport and Recommendations

Recommendations

• The future role for the Library Website and UCD Connect content

• Meeting the needs of different audiences

• Link with Blackboard

• Promoting services

• The Library brand

Recommendations, continued

• Communication strategy and channels

• The use of Web 2.0 and management of innovation

• The balance between the traditional physical Library and the online Library

• Other opportunities

Section ThreeOne view on the process and findings – Ros Pan, Head of Electronic Strategy & Innovation

Overall view of it• I have a more positive view of the project than most of my project

team colleagues

• We certainly did not get the overall roadmap that we were hoping for – but I am not sure how viable that ambition was given the lack of clarity on where the University as a whole is going with website, intranet, portal, CMS, VLE and so on

• I feel strongly that the final report was based too much on the user survey and the workshops and stakeholder interview that cost us a lot do not figure enough – not balanced

• I am critical of the 37 recommendations because they vary from top level strategic concepts to highly specific things in a mix that shows signs of hasty production, not careful insightful thought

• Not enough desk research was done to enable more insightful recommendations to be made

• Nevertheless, I think the deliverables useful in various ways – there is more there than meets the eye

• It is UP TO US TO TAKE THINGS FORWARD and the findings are useful AT VARIOUS QUITE DIFFERENT LEVELS

Level 1 – the specific user survey findings to feed into specific operational planning

Examples of individual findings to act upon

• Users overwhelmingly want an online room booking system

• Users are very happy with university e-mail as the main way for the Library to communicate with them, with no great demand for texting overall – we need to get better access to large mailing lists

• Users are not at all interested at this point in the social Web 2.0 initiatives of the Library – we need to control our resource going into this

• Users really want staffed help points to be maintained to assist in use of the online Library, that mix provides the library value proposition – we should look into when, who, where and consider change

Which of the following new services would be of interest if they were available: (Tick all that apply)

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0%

Online booking of library study rooms

Datasets &amp statistics support – expert help in using datasetsand expanded cataloguing and training in this area

GIS support service – expanded and specialised expert helpw ith GIS and mapping

Video streaming support to enable video and DVD to be hostedand view ed on the w eb

Delivery of Library resources and services to your mobile

Response Percent

How would you like to be updated or notified of events in your Library account e.g. overdue books? (Tick all that apply)

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%

UCD email

Other email account

SMS message

Instant message (IM)

Response Percent

Level 2 – to support projectsMetalib/SFX implementation as an example

• Importance of the online library to our users

• Desire for simplification of discovery and access

• Desire for single google style search box

• Lack of awareness of some of our tools

• Recommendation to have 6-12 month cycle of user promotion and publicity not just 1-off announcements influencing us

• Recommendation to have a full library staff awareness and training cycle influencing us

• Clear warning that with this investment we have to do better in terms of library staff and user awareness and expertise

Level 3 - the stakeholder interview comments, and individual free text comments users

• Not much used by the consultants who went for the mass user response as the key trigger

• They were done on the basis of anonymity so this limits their use

• Still a single comment from a senior member of the university can act as a useful trigger for discussion and thought – very different to the critical masses seen in the user survey

• There were 1000s of free text comments offered

• Some examples follow…

A single comment can be a useful trigger

“I use Google a huge amount - it’s just the best. I teach them to use Google scholar rather than the citation index.”

“They need a clearer articulation of the services they provide to the existing user base.”

“Group coursework - in a group of 5 you will have 2 who are atrocious online. That makes it unfair on the others.”

“The library people are highly accomplished, recurrent problem is that they're not strategies that connect well

with other parts of the University. So what comes out really needs to join up with other parts of the Uni or at

least have potential..”

Level 4: the high level themes

Some high-level themes emerged from the research:

• The Library needs more structure around how it communicates

• There should be much more focus on raising user awareness of what is available

• There are some pronounced gaps between views of users and expectations of Library staff

The Big themes 1- the communication context

The key outcome of the consultancy is not what I expected it to be – but it is useful for management

We did not get the “follow this roadmap” solution that we wanted

The most significant overall findings however seem to me to be that the CONTEXT in which we develop our online library service is more important than the specifics of the platform/platforms themselves

Our communication, branding, training and awareness strategy and activities emerged as a most critical area that we need to address

If we do not address those contextual issues then no matter what digital initiatives we introduce, we will not progress within the university beyond where we are now

Question 25 - Which of the following Library tools do you use when seeking information for study or research?

Answer OptionsUse

regularly

Use Occasionall

y

Aware of but don’t

use

Not aware of

itResponse

Count

Library catalogue 1025 324 92 27 1468

Library A-Z e-journal portal

660 348 210 173 1391

Library Citation Linker tool

85 175 306 683 1249

Library Cross Search@UCD

98 177 285 700 1260

Abstracts & indexes e.g. Web of Knowledge, ERIC, CINAHL

194 182 244 646 1266

Online journal collections e.g. Science Direct, PubMed, JSTOR

747 309 157 180 1393

Search engines e.g. Google

975 310 80 44 1409

Find it@UCD button 331 278 267 427 1303

Library subject portals 301 369 344 280 1294

         

Other (please specify)

21

         

answered

question 1509

         skipped

question 180

The Big themes 2- the library/user perception gap, how can we be both innovative and user-focused?

• Users have a conservative view of the Library

• How then can the Library be user-focused and yet at the same time innovate and try to move our user view of the Library service on?

• All comes to a head round the social Web 2.0 area

• UCD has done a lot of experimenting in this area

• 85% of our users are not interested in our social Web 2.0 activities

• Recommendation to keep resourcing going into this area under tight control and also adopt a robust phase-gate go/no go project approach to Web 2.0

• Only 40% of users wanted to be updated through the website or UCD Connect, yet 85% of Library staff felt this route was effective.

• There was very little user interest in the use of Web 2.0 tools for updates – only 8.4% wanted the Library to use Blogs or RSS feeds in this way, whereas Library staff generally seemed to over-state their utility, with 47% believing them to be effective.

Level 5 – arguing against ideas forces you to organize your thoughts and strategy to repudiate the suggestions being made

• the recommendations that you do not agree with are useful in themselves IF you put in the time to think about them

• These are useful because you have to sort out your thinking around why you do not agree – and develop an alternative strategic vision

Difficulties with the overall vision 3-5 years

1. There will be a single main entry point to Library services for all users.

2. The entry-point will be a set of pages within a UCD-wide intranet (which will also be the entry point for University services, Blackboard etc.)…..

Some issues with the vision – is it too simplistic, and one-dimensional?

• If we integrate our offering but then hide it as a tab in a university portal we lose branding, we lose control, we become invisible – users may find it good, not good for us though?

• Does not take into account large % of links into our website that come in via referrals - deep links and search engines – they ignore the need for a more strategic approach to inbound links

• Similarly the whole idea of breaking up key content and making it into packages that people can embed where they wish or we can embed in other online platforms is not considered

• my colleagues feel that this vision is one-dimensional and outdated, more fitted to an enterprise behind a firewall

• That it lacks real insight into the UCD context

Lacks insight and a deep understanding

“We cannot expect the user to come to the library any more; in fact, we cannot expect the user even to come to the library Web site any more.” Lorcan Dempsey, ARIADNE, Issue 48 July 2006

“The position of the library as a functionally integrated, discrete presence, whether on the Web or as a physical place, becomes diffused through various manifestations (a physical place to meet, a toolbar, a set of services in the course management system, a FaceBook application, a set of RSS feeds, office hours in a school or department, and so on)

Lorcan Dempsey, First Monday, Volume 14, Number 1 - 5 January 2009

Section Three A second view on the process and findings – Caleb Derven, Systems Librarian

Overall View of Consultancy

• Useful in pointing towards cultural themes in the Library, less useful in delineating and clarifying where we should be going with our online platforms

• Some of the tensions the report uncovered (traditional vs. online environments, single-search vs. library presence in heterogeneous environments, differing staff and reader views on Library services) could prove useful for future business decisions

• Too much reliance on survey as a data collection instrument, report outcomes too much conditioned by bias of consultant

Experiences with other consultancies

• Private sector, public sector, and consortial

• Myopia vs. hyperopia– Consultant may have too specific knowledge of the

domain or no background in the area

• Timing and deliverables– Main metric of consultancies is value for money

Context of Current Information Environment

• 2005 OCLC report, Perceptions of Libraries and Information Resources: “It has become increasingly difficult to characterize and describe the purpose of using libraries […] The relationships among the information professional, the user and the content have changed and continue to change.”

• Dissonances most obvious in self-service, satisfaction, and seamlessness

• Role of information in a networked environment (Yochai Benkler)

Context of Current Information Environment, continued

• Ubiquity of open source solutions– Apache, PHP (general context)– Koha, PINES, Open Library Environment Project

(library-specific)

• Proliferation of discovery interfaces– Primo, Encore, VuFind, etc.– Not just a new face to the library catalogue

• Modularization of services– Disconnect database end from front end– Exposing library resources through a variety of web

services

Highlights of the Report

• With regard to the UCD Library online environment, users overwhelmingly support refinements to our resource discovery tools– Users’ top three wishes– Catalogue / electronic resources are still main

access points– Current LMS is end-of-life regarding functionality

and delivering additional refinements to users

• There is an apparent tension between the catalogue as the main discovery tool and the prevalence of Google.

Most Commonly Used Library Tools

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Library catalogue

Library A-Z e-journal portal

Library Citation Linker tool

Library Cross Search@UCD

Abstracts & indexes e.g. Web of Knowledge, ERIC, CINAHL

Online journal collections e.g. Science Direct, PubMed, JSTOR

Search engines e.g. Google

Find it@UCD button

Library subject portals

Response Count

Which of the following Library tools do you use when seeking information for study or research?

Use regularly Use Occasionally Aware of but don’t use Not aware of it

Library access points and usability

How do you typically access the Library catalogue?

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0%

Link to catalogue on UCD connect

Link to catalogue on the Library w ebsite

Link to catalogue on a subject portal

Personal Bookmark

Type in the URL in a brow ser

Catalogue terminal provided in a Library building

Response Percent

How easy is it to search the catalogue efficiently?

20%

65%

12%

1%

2%

Very easy

Quite easy

Quite difficult

Very difficult

Don’t know

Reader Top 3 wishes

• Online resources – 16% for maintenance or increases in availability of online resources

• Enhanced publicity, training, and support for the Library – 11% enhanced support, training, and awareness in how to use the online library

• Resource discovery tools – 9% for simpler, easier resource discovery

• What do these tell us?

Staff Top Three Wishes for the Catalogue

• Functionality (relevancy ranking, spelling, etc.)

• Facets (subject, year, etc.)

• Better integration with the physical collection (help on locating resources in the library)

Consultancy Wish List

• More quantitative data analysis– Server and LMS log files are a rich source

of data about who uses what in the online environment

– Might provide a corrective view to that of the survey

– Would provide solid metrics of what is used and how

• Usability study – How do readers actually get from point A

to Point B?– Heuristic evaluation of platforms

Consultancy Wish List, continued

• How is the Library affected by the networked environment– Modularity– Context-dependent re-use

• Multiple access points for heterogeneous users– Intranet or single-search access point

would only accommodate the needs of a small group of readers

Suggestions for Future Consultancies

• A useful consultancy fully exploits existing data– Server logs, LMS search logs– Metrics collected by other units– Correlate interview findings to overall

findings

• Partner with units in institution– Information Technology– Usability studies

Section FourPros and Cons of the consultancy and suggestions

6 out of 10?

Suggestions

• Original Scoping of Project/ Invitation to Tender

• Selection of Company

• Agreeing to Consultancy’s Terms of Reference

• Project Management

• Methodology

A slight difference of viewpoint existed

The objectives of the consultancy - extract

• We want to get a consensus on how the website should work over a period of perhaps 2-3 years from being re-built before we start

• That involves developing in addition a concerted view about the Library role in the other available online environments and moving beyond viewing these as optional extras in our work

Would we do it again?

• No

• It yielded data of tremendous value - but was not in itself value for money in our view

• We have a high expectation of what you should get for a high consultant fee and we don’t think we got it

• “we could have done it ourselves” – maybe, but would we have done it?

• A better approach for such a broad-ranging objective may perhaps be to have a contract post for 3-6 months to focus intensively on the same issues, working with the permanent library staff

Thank you

Any questions ??