Excellence in Research for Australia 2015 Update

Ms Leanne Harvey

Executive General Manager

2

IntroductionThere have been several changes to the Submission Guidelines for ERA 2015. However, the ERA objectives are unchanged:

1. Establish an evaluation framework that gives government, industry, business and the wider community assurance of the excellence of research conducted in Australian higher education institutions

2. Provide a national stocktake of discipline level areas of research strength and areas where there is opportunity for development in Australian higher education institutions

3. Identify excellence across the full spectrum of research performance 4. Identify emerging research areas and opportunities for further

development and5. Allow for comparisons of research in Australia, nationally and internationally,

for all discipline areas.

3

ERA 2015 Reference Periods

We: arc.gov.au Email: era@arc.gov.au

Staff census date: 31 March 2014

Data type Reference period Years

Research outputs 1 Jan 2008-31 Dec 2013 6

Research income 1 Jan 2011-31 Dec 2013 3

Applied measures 1 Jan 2011-31 Dec 2013 3

Esteem measures 1 Jan 2011-31 Dec 2013 3

4

High Level Changes for ERA 2015• Gender Data • Open Access • Submission Stage 0• Treatment of Revisions, Reprints and Multiple Editions• Digital Storage of Research Outputs Nominated for Peer

Review• ERA Submission Publisher ID • ERA Submission Conference ID • Publication Association for Staff Employed < 0.4 FTE • New Category of Non-Traditional Research Outputs• Nomination of Research Outputs for Peer Review• Peer Review Reporting

5

Preparation for ERA 2015• ERA team – university visits – 38 so far!• New ERA 2015 Journal list contains 24,028 journals – 3,123

journals had FoR changes and 1,589 new journals were added• The Journal and Conference lists will be released at the end

of ERA and will include a full list of journals where ERA-eligible publications have been submitted

• Stage 0 Submissions open soon – 19 January 2015• You are already working on it

We are here to help - ERA Helpdesk

6

Preparation for ERA 2015Submission top tips• Know the submission stages and timetable• Use Submission Stage 0• Develop your ERA Peer Review samples• Develop your Explanatory Statements• Set up and internally test repositories• Think carefully about reassignment!

7

ERA 2015 Submission Timetable

Phase Activity Start Date Deadline Responsible

Submission

Stage 0

Stage 1

19 January 2015

23 February 2015

19 February 2015

13 March 2015

Institutions

Institutions

Stage 2 16 March 2015 7 April 2015 ARC, with Institutions

Stage 3 8 April 2015 13 April 2015 Institutions

Stage 4 14 April 2015 20 April 2015 Institutions

8

Submission Stage 0 • Upload trial submissions to test XML data structure

• Opens 19 January 2015 to 19 February 2015

• All business rules available - except BR024 and BR117 relating to EID tagging

• New report - 30% ERA peer review sample

• Units of evaluation will not be created

• Submissions uploaded will not carry forward to Stage 1

9

Submission Upload Stats ERA 2012 Institution Upload Attempts

Highest 132

2nd Highest 111

3rd Highest 105

4th Highest 97

5th Highest 92

5th Lowest 18

4th Lowest 18

3rd Lowest 16

2nd Lowest 14

Lowest 13

Grand Total 1924

10

Nomination of Outputs for ERA Peer Review

In the peer review disciplines, universities are required to provide a 30% sample of the research outputs in the 4 digit unit of evaluation for ERA peer review.

What is a ‘representative sample’?• Should be drawn from a representative sample of the eligible researchers • Must include 30% of apportioned outputs (rounded up to the nearest integer) for each

output type:• Books• Book Chapters• Journal Articles• Conference Papers• Non Traditional Research Outputs

• No minimum threshold - if only one NTRO is submitted that NTRO must be nominated for peer review.

11

Nomination of Outputs for ERA Peer Review cont.

Book Journal Article Conference Paper

Non-Traditional Research Output

Total Research Outputs

Research volume (apportioned value of

outputs)

10.00 26.66 31.00 0.7 68.36

Peer Review sample (number of outputs)

3 8 10 1 22

Peer review sample(% required of research

volume)

30% 30% (rounded up to the next

integer)

30% (rounded up to the next

integer)

30% (rounded up to the next

integer)

Greater than 30%

Peer Review Samples for Each Research Output Type – University X, FoR 1603 (Demography)

12

Nomination of Outputs for ERA Peer Review cont.

Finally, think of the peer reviewers when getting your sample together.

My picks from ERA 2012 were:

• “10 Stiff Undies”

• “Beyond 'Do No Ham'……”

• “Ooga Booga”

13

Explanatory StatementsDo:• Provide an Explanatory Statement for all two-digit units of evaluation likely to be evaluated• Provide contextual information about publication profile, research environment and capacity,

collaboration, range of research activity, and other information such as awards/prizes• Explain perceived anomalies or unusual patterns• Limit your statement to 10 000 characters including spaces

Don’t: • Make claims not supported by the submission data• Refer to individual researchers• Refer to previous ERA ratings• Refer to other university ranking systems and results• Include additional citation information not included in ERA• Simply repeat information that is already included in the submission• Include reference to information outside the reference period (including future directions)• Include embedded links

14

Explanatory Statements cont.

I do the checks on these and whilst I love a good laugh - proofread your statements and perhaps just perhaps ………. think twice about comments like:

(note these are live ones from ERA 2012)

• A hallmark of our approach is productive methodological eclecticism

• You have to understand that we, at this institution, suffer from small discipline syndrome

• Our research office is staffed by idiots and they forgot to claim our research income but you should add $X million into our discipline

15

Repository Testing Instructions

• Guide to testing for Repository Managers

• Will be distributed this week through ELOs

16

REC Member and Peer Reviewer Recruitment

Research Evaluation Committee (REC) Membership• 8 REC Chairs announced in November - bring considerable experience and ability to

ensure ERA evaluations are of the highest standard• The ARC is in the process of contacting other selected members of the RECs• The ARC will announce the full list of RECs members in the coming months after all

contracts have been finalised• ELOs will be advised

Peer Reviewers• The call for Peer Reviewer nominations closed on 26 November 2014• Nearly 1000 nominations – even better than 2012!• Unsuccessful REC nominees may be contacted to participate in the peer review process

17

ERA-HERDC AlignmentBackground• PhillipsKPA Review of Reporting Requirements for Universities• Department of Education and the ARC working together• A single research data collection

Aim• A more efficient streamlined process for the collection of data to inform RBG and evaluate

excellence

Potential Efficiencies • University admin staff and IT systems only need to cater for one data set• Reduces opportunities for error and misunderstanding - enhancing integrity, transparency and

utility of data• Auditing requirements in relation to the data will be streamlined

Sector Consultation• Discussion paper including possible options will be sent to universities soon

18

Questions?