Analyzing Usage Statistics of Electronic Resources

Jagdish AroraDirector, INFLIBNET Centre

Not everything that counts can be measured. Not everything that can be measured counts.

– Einstein

Why do we measure Usage? Usage statistics provide essential evidence:

• for extent of usage of e-resources

• to showcase the trends in usage over a period of time, Patterns of usage can be helpful guide for future collection development decisions

• to take informed decisions regarding renewal / cancellation of resources

• to demonstrate value for money / return on investment

Why Collect Usage Statistics?

• To make best and justified use of financial resources

• Calculating Return on Investment (RoI)

• Accountability

• To find out emerging subject disciplines

• Reporting usage to administration, funding agencies, etc.

• Strategic planning

• Comparison with other libraries / institutions

Why Collect Usage Statistics?

• Justifications for change in document format– From print to electronics– Lesser number of users visiting library physically– Lesser no. of books being issued / Less re-shelving– Increase in usage of e-resources

• Bench Marking– Top-cited journals available in e-format as compare to other libraries

– Results of usage of e-resources by existing libraries can serve as a bench mark for other libraries

Why Collect Usage Statistics?:

Additional Information

• Journals that are used heavily

• Journals not being used at all

• Number of denials in case of limit of simultaneous usage

• Preference for formats: PDF, HTML

• Breach of license agreement: Heady downloads or

systematic downloads; How to handle it?

Acquiring Usage Statistics• Content Provider (Publishers / Database Vendors)

– Some publisher do not provide usage data (eg.

MathSciNet, ISID, etc.)

– Data inadequate and inconsistent

– Data retained on the publisher’s web site only

– Inconsistency in usage not reflected

– Server caching not reflected

What do libraries want from usage data?• Reliable usage Report in consistent format

• Usage at journals titles level

• Usage by subject area

• Analyse trends over time

• Ready access for reporting

• Evidence of value for money

• Benchmarking (comparative usage)

Adding More Value

• Cost-benefit analysis and RoI

• Impact of usage on research output

• Benchmarking

Why Evaluate at the Consortia Level?

Evaluation is Necessary

• Negotiation for renewal• Cost / Benefit analysis

Evaluation is Possible

• Relativity• Comparability• Generalizibility

Why Evaluate Why Evaluate at the Consortia Level ? ?

Review of current & prospective contracts

• Continuing price escalation not sustainable

• Evaluate prices to consortia and members

• Review contracts with additional criteria

• Promote models for quality not just quantity

• Plan for future

Problems with Manual Collection of Usage Statistics• The usage statistics has to be gathered manually from different

publishers

• Each publisher has

– Different formats for data and delivery– Different access methods– Different availability dates

• Cost has to be calculated separately

• Data needs to be cleaned up and aggregated manually

• It is a labor-intensive and cumbersome process prone to data loss and errors

Harvesting Usage Statistics using SUSHI• Automated import of consortia stats

• Consortium can track statistics for each member

• Data can be retrieved across a series of dates, e.g. period of months

• Member logins are pre-populated

• The library can access all COUNTER compliant usage stats across their serials holdings

• The library can obtain a top level view of their most and least-viewed publishers and titles

Negotiate More Effectively

• With COUNTER-compliant costs-per-view in hand, negotiate with publishers to realize more realistic cost models

• Uncover previously hidden cost information

• Utilize consortium-wide data to negotiate optimal terms for the group as a whole

• Obtain a better understanding of our consortium members’ usage patterns and collection needs

INFLIBNET Usage Portal

Benefits of Portal for Usage• Usage statistics for every e-journal package for

every member institutions is automatically collected

• Consortia-wide data readily available to the whole group for analysis and reporting

• The usage data can be exposed completely or partially to member institutions / consortium Administrators

Consortium Usage Analysis

3153

628

4687

776

6093

511

7479

060

1276

5817

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

No

of

Do

wn

load

s

2007 2008 2009 2010 2011

Year

Increase in Number of Fulltext Downlaoads

Cost Incurred Vs Downloads

0

2000

4000

6000

8000

10000

12000

14000

2011 2010 2009 2008

Th

ou

sa

nd

s

Year

No

of

Do

wn

load

s

0

100

200

300

400

500

600

700

800

900

Mill

ion

sE

xp

en

dit

ure

on

Su

bscri

pti

on

in

Rs.

No of Downloads

Cost Incurred

Cost Incurred vs Cost Recovered (2008 - 2011)

0

2000

4000

6000

8000

10000

12000

2011 2010 2009 2008

Millio

ns

Year

Am

ou

nt

in R

s.

Cost Recovered

Cost Incurred

Average Cost per Download (Year wise)

2.061.81

1.351.19 1.14

0

0.5

1

1.5

2

2.5

2007 2008 2009 2010 2011

Cost

in U

SD

Cost Incurred vs Cost Recovered in 2011

Usage Trend Analysis for a Single Publisher

(ACS)

Year-wise Download for ACS (2007 - 2012)

5605

06

5949

60

1031

237

1196

487

1398

741

0

200000

400000

600000

800000

1000000

1200000

1400000

1600000

2007 2008 2009 2010 2011

Year

No

of D

ownl

oads

Top Ten Journals of ACS in 2011

Sl. No.Journal Name

Number of Downloads

1 Journal of the American Chemical Society 194693

2 The Journal of Organic Chemistry 149561

3 Organic Letters 90272

4 Journal of Medicinal Chemistry 84866

5 Inorganic Chemistry 83202

6 Chemical Reviews 65255

7 Langmuir 58367

8 The Journal of Physical Chemistry B 57635

9 The Journal of Physical Chemistry C 56186

10 Journal of Agricultural and Food Chemistry 49706

MEASURING RESEARCH OUTPUT

AND

IMPACT OF E-RESOURCES

Measuring Research OutputThe Science Citation Index (SCI), Social Science Citation Index (SSCI) and Arts and Humanities Index (A&HI) are internationally recognized database that works as a filtering mechanism as it indexes qualitative research output from selected journals.

The source articles appeared in three indices for 50 first-phase universities of the Consortium was searched in blocks of five years from 1975 to 20010 with an aim to compare the research output in the last block year, i.e. 2005 – 2009.

A un-precedental increase in research productivity in terms of number of research articles is evident during 2005-2009 as compared to previous block of five years, i.e. 1975-1979 to 2000-2004.

Increase in no. of articles in past 35 Years (In block of Five Years)

Correlation Usage Vs. Publishing Output

Pearson’s Ranks order Coefficient Correlation = 0.75

Usage analysis for A Single Institution

Does this institution need Complete Collection or Selected Subject Collections

No. of Titles Fulfilling the User needs of the Library

Correlation Usage Vs. Publishing Output

Banaras Hind University

Pearson’s Ranks order Coefficient Correlation = 0.98

Banaras Hindu UniversityAnnual Average Growth Rate

Relative Specialization Index

Banaras Hindu University

Banaras Hindu University

Contribution of BHU as Compared to the World and India’s Total Publications

Publications Output of BHU

BHU’s Citation Impact in Nine Subject Areas

Contribution of BHU to the World’s Most Productive Areas of Research