Berlin 6 Open Access Conference: Julianne Nyhan
-
Upload
cornelius-puschmann -
Category
Education
-
view
1.415 -
download
1
description
Transcript of Berlin 6 Open Access Conference: Julianne Nyhan
www.esf.org/erih
Evaluation, Quality and Impact of
Open Access Publications
• Chair: Julianne Nyhan (European Science Foundation)
• Susan Murray (Director, African Journals online)
• Christiane Fritze (Berlin-Brandenburg Academy of Sciences and Humanities)
www.esf.org/erih2
European Reference Index for the Humanities – ERIH
Better access to and assessment of Humanities research in Europe and beyond
Dr Julianne Nyhan
www.esf.org/erih3
European Humanities research– the challenges
• Difficulty to identify and compare Humanities excellence- vis-à-vis other sciences- across all languages at a supra-national (European) level
• Poor international visibility of research output in national languages(considered as necessary vehicles for Humanities research)
• Multiplicity of formats for research output (monographs, edited volumes, journals [less than 1/3 !], conference proceedings, web-based content and data, outreach)
www.esf.org/erih4
ERIH – the origins
SCH Workshop on evaluation of scientific production in Humanities (Budapest 2001):
• Existing citation indices (AHCI, SSCI) have unsatisfactory coverage of European Humanities research
• Need for a European citation index for the Humanities as additional tool for research assessment
• ESF requested to compile lists of reference journals in the first instance
www.esf.org/erih5
ERIH – objectives / methods
• Benchmarking tool for comparisons at aggregate level
• Focus on format used in other sciences (journals)> achieve a degree of initial comparabilityBUT: methodology needed for other formats
• Encourage ’best practice’ in the publication of journals in the HumanitesPeer-review: discursive identification of excellence> across all fields of the Humanities> across Europe < national consultations
• European Research Area:Improve access to European Humanities research across all languages (basis for VLE?)
ERIH as a tool for the strengthening of Humanities scholarship in Europe created by scholars for scholars
www.esf.org/erih6
ERIH – process
Set up Steering Committee• Define the first 15 disciplines to be included• Define categories of journals: A, B, C• Compile guidelines for inter/national panels • Approves Expert Panels (members suggested by
MO’s, SCH, StComm, in the future: ENCoPs)• Validates work of expert panels
Produce and process material• Input from ESF MO’s (national panels / reference
tools) according to guidelines • Initially 15 international panels who compile
categorised lists of good peer-reviewed research journals in 15 disciplines
www.esf.org/erih7
ERIH – current coverage
Index Coverage (publication criteria):Peer-reviewed journals; ISSN; bibliographic datafor cited references; author data
Index coverage (discipline criteria):15 subject groups defined, some covering areasbetween Humanities and Social Sciences(anthropology; education; psychology)
Index coverage (internationality criteria):Cat. A (<25%)/B: international journals (defined by status:
reputation, readership, authors, citations),Cat. C: important European journals with more regional
circulation (ESF MO Europe: not Russia)
All journals included – whether A-B-C - to be considered good research journals
www.esf.org/erih8
ERIH – categories
Precision of criteria for international journals• a genuine, varied and regular international cohort of
contributors and readership• a consistently high-quality scholarly content• a broad consensus within the field concerning international
status and visibility, insofar as possible• a quality control mechanism, normally through peer-review
Plus some, not necessarily all, of the following characteristics:• Active international advisory board• Openness to unsolicited contributions• Highly discriminating and selective in the choice of articles
published• Publication on time and to an agreed schedule
Degree to which these characteristics are applied could be added to the criteria distinguishing journals in categories ‘A’ and ‘B’
www.esf.org/erih9
ERIH – current disciplines/panels
• ANTHROPOLOGY• Archaeology• Art and Art History• Classical Studies• Gender Studies• History• History & Philosophy
of Science• Linguistics
• Literary Studies• Musicology• Oriental & African Studies• Pedagogical & Educational
Research• Philosophy• Psychology• Religious Studies
& Theology
Possible Additions for updates:
•Archives, Library & Museum Studies•Film, Media & Cultural Studies•General Humanities
www.esf.org/erih10
ERIH – towards the « initial lists »
• 2003/04: MO’s provide input based on previous national consultation (panels; reference tools)
• 2005/II: Expert Panels work- define field / remit (“scope notes”),- analyse and assess input received,- suggest circles of consultation,- consult where gaps are identified (eliminate / add),- discuss categories (multiple listing; single listing),- produce draft list, explain methods and problems.
• 2006 (mid): Wide consultation of- ESF MO’s,- (European) subject associations,- specialist research centres
• 2007 / 2008: Publication of the lists in three batches • 2008: Update / feedback
Online form (quantitative information): contact publishers, editors, European subject associations, national subject associations (through ENCoPs)
www.esf.org/erih11
Peer review at work…
Layers:• Input: National panels / scientific communities• Selection: Expert Panels define scope, analyse and
assess input, produce lists• Consultation: MOs, subject associations (European
level and some national), specialist research centres• Calibrate/harmonise: ERIH Steering Committee• Feedback: open process leading to updates in 2008Challenges:• Wide differences in quality of lists received from MOs• Domain-specific (e.g.: cult./soc./evol. anthropology;
philosophy and ethics)• Language (e.g.: “forum languages” in art history;
“international languages” in Oriental & African studies
• Some panels more reluctant to overrule authority of (own) national panels
• Outside peer pressure during consultation phases
www.esf.org/erih12
ERIH insights – quality through peer review?
• Peer review recognised as the only practicable method in basic research (standard method used in scientific communications themselves)
• Peer review can overrule weight of numbers for better (detect originality) or for worse (defend conservatism)
• Peer review introduces comparability into discussions of national discourses in Humanities scholarship
www.esf.org/erih13
Evolution of ERIH Lists
1st batch of published initial lists are highlighted in yellow
DisciplineInitialsubmission
1st draft lists Consultation 1a Consultation 1b Initial lists
Mar-Apr 2006 Spring 2006 Summer 2006 2007
Anthropology (social and evolutionary) 752 154 133 17 242
Archaeology 1310 524 290 25 419
Art and Art History 938 445 292 16 472
Classical Studies 619 256 321 11 251
Gender Studies 155 181 37 6 119
History 1419 874 508 71 907
History and Philosophy of Science 806 145 64 4 166
Linguistics 1093 680 391 34 586
Literature 1453 1481 706 10 802
Music and Musicology 204 n/a 187 4 166
Oriental and African Studies 196 588 386 14 505
Pedagogical and Educational Research 666 404 271 92 470
Philosophy 658 320 153 22 305
Psychology 1198 1201 159 4 634
Religious Studies and Theology 745 n/a 580 10 371
www.esf.org/erih14
ERIH – Update in 2009
• Recomposition of Expert panels: panel rotation mechanism and inclusion of new experts, document setting out methodology of recomposition
• Summary and Process of Methodology that underpinned work on ERIH “initial Lists”
• Summary and Process of Methodology that underpinned work of ERIH Steering Committee
• Panel meetings to take place from October 2008 – February 2009. ERIH Lists to be published incrementally in 2008 and 2009
www.esf.org/erih15
ERIH – Update in 2009
Feedback on ERIH “initial List” Anthropology (2007/08)
• Of crucial importance in the updating of the “initial Lists”
• ERIH office has been contacting Editors and Publishers of relevant journals to ask that they submit quantitative data about their journals
• ERIH Feedback form: https://www2.esf.org/feedback.asp
• Difficult process because no central database of contact details exists ([email protected] etc)
• Only 26% of journals on “initial List” have been commented upon by Editors / Publishers
www.esf.org/erih16
ERIH – futures?
Open Source
• - ERIH is creating consensus on excellence
• (> priority list for inclusions)• - ERIH includes publications of
ESF MO’s, incl. learned societies • (> institutional incentive?)• - link of access / assess may
create incentive to use OA• [perhaps further stimulated by
universities through support for IR’s]
www.esf.org/erih17
ERIH – futures?
Open source futures?
• - ERIH aims to create a consensus on excellence
(> priority list for inclusions)• - ERIH includes publications of
ESF MO’s, incl. learned societies (> institutional incentive?) - link of access / assess may
create incentive to use OA [perhaps further stimulated by
universities through support for IR’s]
www.esf.org/erih18
ERIH – futures?
• Open Questions:• - Measuring influence of OA journals when
not cited: downloads and citation refer to different phases in research process
• - OA portal appropriate for a highly diverse field?
- How to link OA to IR’s [additional institutional stakeholders]?
• - OA as linked to budget model of learned societies?
• - How to integrate current knowledge about delayed impact in most Humanities [6-month-rule not applicable?]?