How researchers share articles: impact on library resources and services | December 3, 2015

Suzie Allard Associate Dean for Research, Director of the Center for Information & Communication Studies, Professor University of Tennessee, Knoxville Lorraine Estelle Director Project COUNTER Wouter Haak Vice President, Research Data Management Solutions Elsevier

#LCwebinar

The secret life of articles: From download metrics to downstream impact

Suzie Allard

Associate Dean for Research Director of the Center for Information & Communication Studies

Professor University of Tennessee, Knoxville

CIBER Research Ltd.

Library Connect Webinar Dec. 3, 2015 How researchers share articles: impact on library resources and services

Beyond Project COUNTER

• What are download counts missing?

• How much do scholars share and what do they share?

• How might sharing be measured?

• What is a more complete value of articles?

14 focus group participants 15 interviews

Interviews / Focus Groups Demographics

International Survey with 1000 responses

Main takeaways 1. A means to an end 2. Function drives form 3. Email is #1 4. Version matters 5. The library is key

1. A MEANS TO AN END

“If I have got it, I will just share it. It is easier than having them go track it down, even if I have got the citation.”

“It’s not like I’m giving away something that they could not get anyhow. But it never dawned on me that in effect, you know in theory, I am violating copyright.”

When working in a research group …

N=771

Perc

ent

share more 67%

sharing habits would not

change 30%

share less 3%

2. FUNCTION DRIVES FORM

Interviews / Focus Groups

Two types of saving and sharing 1. Metadata (citation) When other researchers could easily access articles. 2. Saving document (pdf). When access was an issue.

Full-text vs. reference/link sharing

Their most recent link or reference is shared …

Email Learning Management Software an average of 10.5 (8.2

minus outliers) times per download.

an average of 13.9 (5.4 minus outliers) times per download.

Their most recent full-text is shared …

Email Research social networks an average of 10.5 (8.4 minus outliers) times per download.

an average of 10.8 (5.3 minus outliers) times per download.

3. EMAIL IS #1

Top preferred methods of sharing for research

Email ranked 1st with 73.8% respondents listing as most preferred method of sharing.

Cloud services 2nd with 11.6% respondents listing as most preferred method of sharing.

Internal networks ranked 3rd with 10.7% respondents listing as most preferred method of sharing.

Top preferred methods of sharing for teaching

Email ranked 1st with 82.8% respondents listing as most preferred method of sharing.

Internal networks ranked 3rd with 8% respondents listing as most preferred method of sharing

Learning management systems ranked 2nd with 9.6% respondents listing as most preferred method of

sharing

However, researchers feel more people are reached through …

General social networks: 96-614 people potentially reached

Internal networks: 40-189 people potentially reached

Research social networks: 559-4066 people potentially reached

4. VERSION MATTERS

9.7%

77.3%

9.3% 3.8% 5.5%

83.9%

7.3% 3.3%

Preprint Published Accepted MSS Other

When sharing scholarly articles, are you more likely to share

Own Work Other's Work

My feeling is that I’ve written this, this is my work, they’re publishing it. I’m not getting paid for this, they’re benefiting from this. I’m a little fish . . . Let them come for me.

The main point should be that I share a version that is evidentially the peer-reviewed version (with the publisher and the journal stamp on it).

5. THE LIBRARY IS KEY

Average number of downloads for last research project:

65

Average number of downloads for last teaching term:

26

54.1%

9.4% 8.8%

18.9%

7.0% 6.1%

35.4%

5.8% 5.7% 8.5%

3.4% 6.1%

0%

10%

20%

30%

40%

50%

60%

Library Otherdatabases

Institutionalrepositories

Publisherwebsites

Research socialnetworks

Other websites

What percentage of your downloads were from the following sources?

Research Teaching

But, library download counts may underestimate use, because when returning to an article …

61.3%

11.7%

57.3%

9.4%

Saved copy Download again

Research Teaching

Understanding our users • Representation of objectives and behaviors of

real users • Fictional but synthesized from data collected

with surveys, focus groups, interviews • Focuses on users instead of focusing only on

tasks • Helps us understand human interactions with

systems

Senior Faculty: “Charles Todd”

o Sharing Behaviors

• Heavily favors informal sharing methods such as email and internal networks.

• Will use formal methods to share more than his peers who work independently of research groups.

• Makes use of a wider variety of sharing resources than researchers in non-academic settings.

• PhD in Social Sciences. • Over ten years as a

faculty member. • Most of his work is done

in a research group.

“Charles Todd”

“When I share papers I do it with other colleagues that have access to the same databases as I have, it happens mainly in the following scenarios: I am doing literature search on a subject we are working as a group and find some papers that are interesting to my fellow colleagues working with me, I send them the pdf papers so they can also read them...”

Junior Faculty: “Leslie Bryant”

o Sharing Behaviors

• Reaches a large number of individuals through email or internal network.

• Relies on Research Social Networks such as ResearchGate or Academia.edu more than any other demographic.

• Rarely uses Reference or Learning Management Software to share. She is not likely to share with her students.

• PhD in Mathematics. • Less than ten years as a non-

tenured faculty member. • Most of her work is done

independently of a research group.

“Leslie Bryant”

“Pay gates to research content is frustrating when it isn't accessible via the university. The whole point of academic research is to share knowledge.”

Doctoral Student: “Hikaru Midorikawa”

o Sharing Behaviors

• Sharing is limited almost entirely to email.

• Mostly unaware of Creative Commons and other copyright restrictions.

• When he does share, he can potentially reach a wide range of individuals with a small number of posts.

• PhD student in engineering. • Less than ten years as

adjunct faculty. • Most of his work is done in a

research group.

Persona 4: “Hikaru Midorikawa”

“There is a lot of pressure to make research publically available, especially federally funded research, and also at the same time, not share full-text articles from the journals unless you order reprints from them directly. I want to honor both of these, but it is very difficult. There is much confusion on how to do this and the need to share research quickly, usually by email, ends up winning.’”

Sharing is a natural behavior

• Sharing supports research & disseminating results

• Convenience is a key concern

• Sharing channels chosen for fit to

work style • New policies and measures must

fit preferences & likely behavior

Center for Information and Communication Studies

Calculating sharing metrics: Possible approaches

Lorraine Estelle Director, Project Counter

Library Connect Webinar Dec. 3, 2015 How researchers share articles: impact on library resources and services

Center for Information and Communication Studies

Range of Systems Key findings include: • Lack of global usage data standards makes quantifying sharing

problematic.

• Difficult to obtain data on authors’ sharing via email and cloud services.

• Data obtained is likely to be out of date rather quickly.

Center for Information and Communication Studies

Two suggested approaches

1. Data-based approach employing usage data from publishers and citation data – where such data is comprehensive and reliable.

2. Survey-based approach using an online community as a basis for monitoring sharing behavior over time.

Center for Information and Communication Studies

A data-based approach to measuring the extent of scholarly materials sharing

1. Data on article sharing by formal and informal methods will be difficult to collect reliably owing to the large number of channels used and the lack of global standards

2. The mix of sharing methods used varies from discipline to discipline and from year to year

3. It would make sense to supplement a data-based approach to measuring post-download usage with regular surveys of the scholarly community

Center for Information and Communication Studies

Full-Text Sharing Calculator • Can a calculator be derived from the

findings? • If not, can a suggested range of

“sharing” be derived from the findings?

Problems: • Ranges and SE

are high • Outliers skew

results • Demographics

matter a lot

Center for Information and Communication Studies

A range of sharing, in lieu of a calculator, may be the answer. This range can come in the form of an overall range (derived from all respondents) or categorized ranges (derived from selected identifying categories).

Center for Information and Communication Studies

Demographics sometimes matter…

• Researchers age 39 and younger share less through formal methods

• Researchers age 40-59 rely more on formal sharing methods and post slightly less, but to outlets that reach more people

Center for Information and Communication Studies

Demographics sometimes matter…

Social scientists share more via social media.

Center for Information and Communication Studies

Alternatives • Any estimate on number of times an article is shared per

download depends on many things.

• An alternative to an exact calculator would be to apply a Confidence Interval to the findings and provide an estimated “range of sharing,” with a lower bound and upper at a 95% level of confidence.

• This range could in turn be combined with download

numbers to estimate a level of actual, post-download usage.

Center for Information and Communication Studies

Collaborative initiatives with

• DOI Event Tracking – Open, common infrastructure to track activities

around DOIs recording user content interactions

• Distributed Usage Logging – Peer-to-peer system for the exchange of

usage data on user content interactions

From download metrics to downstream impact => implications for Elsevier

Wouter Haak w.haak@elsevier.com Library Connect Webinar Dec. 3, 2015 How researchers share articles: impact on library resources and services

• The graph shows how many users have joined Mendeley groups year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year

Source: Mendeley analytics team

49%

17% -15%

17%

165%

697%

At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (1)

• The graph shows how many documents were added to the groups by users year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year Source: Mendeley analytics team

148%

546%

46%

3% 24%

14%

At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (2)

1) For the researcher: sharing in

2) For the institution: collaborations in

3) With publishers and platforms: principles for sharing on SCN’s

Actions undertaken and next steps – based on this research

47

1) For the researcher: sharing in

48

49

50

51

2) For the institution: collaborations in

52

53

54

3) With publishers and platforms: principles for sharing on SCN’s

55

• SCNs are platforms that allow researchers to develop and maintain professional relationships

• Typically they enable

- Discussion or research and information exchange - Sharing of research results, articles and data

• SCNs have grown substantially* in recent years, particularly in the “big”

three – Academia.edu, ResearchGate, and Mendeley

• STM working group identified 40+ such networks

*CCC/CLA Report on Scholarly Collaboration Platforms, M. Ware, November 27,2014 (CCC/Outsell)

Social Collaboration Networks (SCNs)

56

STM assembled a working group in September 2014 to examine the current landscape of article sharing via SCNs and publishers’ role

Ten publishers, both for profit and not-for-profit, and some of whom owned or interacted with SCNs, took part

They produced the draft set of “voluntary principles for article sharing on scholarly collaboration networks” and posted it for a two-month STM Open Consultation in early 2015, and invited wide participation among all stakeholder groups

STM Consultation on Article Sharing

57

The principles are meant to: • clarify how, where and what content should be shared using

these networks and sites

• improve the experience for all stakeholders

• encourage publishers and SCNs to work together to facilitate sharing, which benefits researchers, institutions and society as a whole

• All comments and a summary posted on the STM website • Key recommendation: Principles should be clear, simple, uniform

Voluntary Principles for Article Sharing on SCNs

58

• Based on the community’s constructive feedback, the working group posted a revised set of principles August 28.

http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/

• Posting includes an explanation of the changes made to the principles as a result

Response to STM’s Open Consultation

59

• Publishers commit to facilitate the dissemination and discovery of their authors’ scholarly articles.

• Sharing should be allowed within a research collaboration group (to be defined by the group) - Allow article sharing between subscribers and non-subscribers

within the group - Include members of the wider public participating with the group - Include commercial researchers, subject to publisher policy or

appropriate licensing • Publishers and libraries should extend their collective use of standards

such as COUNTER to quantify article use on networks.

• Publishers and standards organizations should continue to work together on tools that facilitate sharing (article versioning and access rights metadata).

• Publisher policies on research collaboration group sharing and public posting of articles should be clear and easily discoverable.

Voluntary principles: key points

60

Publisher posting policy - Several publishers (e.g., NPG, Wiley, Elsevier) have issued policies that address posting via SCNs

Article tagging project - To support implementation a sub-working group is tasked with demonstrating a technical solution based on tagging xmp metadata in scholarly articles

The SCN working group continues to reach out for wider involvement, including non-publisher-owned networks

Comments/updates on progress, email scnproject@stm-assoc.org Updates will continually be posted on:

www.stm-assoc.org/stm-consultations/scn-consultation-2015/

Next steps

61

Seeking your involvement and/or endorsement http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/

Alternatively: Google for “STM principles” or “STM consultation”

Next steps

62

1) For the researcher: sharing in

2) For the institution: collaborations in

3) With publishers and platforms: principles for sharing on SCNs

From download metrics to downstream impact => implications for Elsevier

63

Questions & Thank You!

Suzie Allard Associate Dean for Research, Director of the Center for Information & Communication Studies, Professor University of Tennessee, Knoxville sallard@utk.edu Lorraine Estelle Director, Project COUNTER lorraine.estelle@counterusage.org Wouter Haak Vice President, Research Data Management Solutions Elsevier w.haak@elsevier.com

#LCwebinar