Bibliometrics: the black art of citation rankingsRoger MillsOULS Head of Science Liaison and Specialist Services

February 2010

These slides are available on http://www.ouls.ox.ac.uk/services/training/

wiser

Overview of Session• What are bibliometrics?• How are they calculated?• How can I use them?• How will they be used in the REF?

Why bother?• Because we have to!

http://www.hefce.ac.uk/research/ref/pilot/ref.pdf

http://www.hefce.ac.uk/research/ref/resources/REFguide.pdf

http://www.hefce.ac.uk/research/ref/resources/REFguide.pdf

http://www.hefce.ac.uk/research/ref/Biblio/

http://www.hefce.ac.uk/pubs/hefce/2009/09_39/

http://www.springerlink.com/content/v111750n14086384/fulltext.pdf

http://www.ischool.utexas.edu/~palmquis/courses/biblio.html

http://www.slideshare.net/guest633b30/bibliometrics-and-scientometrics-1065282

© http://ceochef.files.wordpress.com/2008/07/pinch-of-salt.jpg

The power behind

Citation indexing• Invented in 1961 by Eugene Garfield at the Institute for

Scientific Information (ISI)• Scientific abstracting/indexing services began in nineteenth

century, recording author/title/publisher/source etc for articles and indexing them

• Garfield added details of all references quoted in the article and indexed them too, publishing results as Science Citation Index (SCI) – originally only in printed form

• Allowed for many new ways of linking articles

so• For an article you’ve read:• Find earlier articles that one was based on• Find later articles which quoted it• Find related articles which quote some of the same

references as this one• So you can trace the progress of ideas backwards,

sideways and, uniquely, forwards in time• And generate statistics on how influential those ideas /

authors / journals have been

Computers save time• Science Citation Index – now supplemented by Social

Science Citation Index and Arts and Humanities Citation Index – online as Web of Science

• Delivered along with Journal Citation Reports on the platform Web of Knowledge.

• Built-in analysis tools available to all users• For REF, universities may have access to raw data and

specially-developed analysis tools

Analysis tools• Who is citing?• Which journals are citing?• What is the relation between frequency of publication and

frequency of citation?• The h-index • Citation maps• Impact factors

Analyzing highest cited article

Citing articles

Highest citing authors

Highest citing journals

Citation report

Citation map

Citation map

Impact Factors - Journal Citation Reports (JCR)

• For Sciences and Social Sciences (not humanities)• A measure of the frequency with which the "average

article" in a journal has been cited in a particular year. The impact factor will help you evaluate a journal's relative importance, especially when you compare it to others in the same field

• From within a record you can click on Journal Citation Reports to view the impact factor of the journal

• Or you can view and compare impact factors of all journals within your subject area

Link from record

Other services offering citation searching - SCOPUS

• Sciences and Social Sciences• Results include journal articles and web pages• Each reference to a paper shows the number of times an

article has been cited

Citation analysis in Scopus

Citation searching in Google Scholar• References include ‘cited by’ data based on articles known

to Google Scholar• Entries ranked by number of cites• Not possible to save sets or analyse• Still useful for tracking research

Specific article with link to citing works

Citing works

Track an author

Author’s works, highest cited first

So which database will be used for REF?• Initial plan to use Web of Science only• Pilot used WoS and Scopus• WoS data handled by Symplectic, subcontractor for

Evidence; Scopus by Hefce• Pilot showed need for ‘normalization’• Final decision not yet made

• In our example:• WoS finds1294 citations to 581 articles• Scopus finds 1314 citations to 553 articles• GS finds 927 citations to 1020 articles

What is truth….

Where will it all end…

• And how much will it all cost…

Watch this space!