Jisc online CPD service for further education and skills - Jisc Digifest 2016
Introducing the open citation experiment - Jisc Digifest 2016
35
/15 Introducing the open citation experiment Petr Knoth (@petrknoth) & Drahomira Herrmannova (@damirah) Knowledge Media institute, The Open University Verena Weigert, Jisc 1 Towards full-text based research metrics: exploring Semantometrics
Transcript of Introducing the open citation experiment - Jisc Digifest 2016
/15
Introducing the open citation experiment
Petr Knoth (@petrknoth) & Drahomira Herrmannova (@damirah)Knowledge Media institute, The Open University
Verena Weigert, Jisc
1
Towards full-text based research metrics: exploring Semantometrics
/15
What are Semantometrics?• A new class of metrics for evaluating research• Build on the premise that full-text is needed to
asses the value of a publication• Make use of the full-text features of a given
resource rather than relying on outside evidence
2
/15
Semantometric contribution measure• Based on the idea of measuring the progress
of scholarly discussion• The hypothesis states that the added value of
publication p can be estimated based on the semantic distance from the publications cited by p to the publications citing p
• Demonstrator at http://semantometrics.org
3
/15
Comparative study• Analysis carried out to investigate the
properties of the contribution measure• Experiments carried out on a dataset obtained
by merging data from the Connecting Repositories (CORE) system, the Microsoft Academic Graph (MAG) and Mendeley
• After merging the datasets over 1.6 million publications
4
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
/15
ApproachPremise: Full-text needed to assess publication’s research contribution.Hypothesis: Added value of publication p can be estimated based on the semantic distance from the publications cited by p to publications citing p.
5
/15
Contribution measure
6
/15
Contribution measure
p
6
/15
Contribution measure
p
6
/15
Contribution measure
p
6
/15
Contribution measure
p
A
6
/15
Contribution measure
p
A B
6
/15
Contribution measure
p
A B
6
/15
Contribution measure
p
A B
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
6
/15
Contribution measure
p
A Bdist(a,b)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
/15
Contribution measure
p
A Bdist(a,b)
dist(b1,b2)
Average distance of the set members
6
/15
Real examples• Contribution 0.8452
/15
Real examples• Contribution 0.9220
/15
Real examples• Contribution 0.8296
/15
Real examples• Contribution 0.9348
/15
Real examples• Contribution distribution
/15
Conclusions• First large scale analysis of the semantometric
contribution measure• As semantometrics should exhibit a number of
advantages over existing research metrics, we should continue studying this field to better understand which facets of research quality they can capture and how they can be applied
13
