MODULE 5: PEER REVIEW
-
Upload
elton-freeman -
Category
Documents
-
view
39 -
download
0
description
Transcript of MODULE 5: PEER REVIEW
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
OverviewOverview
This module highlights the importance of peer review in IEA
In particular, it emphasizes the need to ensure scientific credibility, policy relevance and legitimacy
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
The module also outlines:
what constitutes peer review in the context of the AEO process
the objectives of peer review
why peer review is critical to IEA
how to select peer reviewers
Overview (cont’d)Overview (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Peer review definition in AEO
Peer review is a documented critical look at the AEO report, conducted by qualified individuals and organizations.
experts are considered equivalent in technical capacity (thus peers) to the experts who draft the contents of the report.
The peer review is important for ensuring that research and conclusions of the report are technically adequate, competently performed, properly documented, and satisfy established quality requirements.
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
The AEO peer review is an in-depth assessment of the assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria, and conclusions of AEO reports and documentation supporting them.
The process is characterized by a limited number of interactions by peer reviewers.
It is undertaken during the whole AEO process, including selection of methods, research and drafting.
Peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Extent of peer review for the AEO Scientific, technical and policy review has
always been a key component of the AEO and other environmental assessment reporting processes
It involves hundreds of stakeholders both inside and outside government structures
The peer review process takes different forms – ranging from informal to formal
activities include sub-regional and regional consultations, collaborating centre network meetings, and targeted expert input.
AEO draft sections are sent to scientists and government experts for review and comprehensive comment
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Objectives of peer review
The objectives of the peer review are to check, among others, for:
adherence by the authors to the TORs provided by the coordinating agency at the start of the assignment
reliability and appropriateness of the scientific basis of the analyses, as well as the data and information used
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
reliability of the sources of information and citations, and if these are up-to-date
regional and sub-regional coverage of the issues
appropriateness of the conclusions/findings of the analyses
relevance and soundness of the indicators used in the analyses
Objectives of peer review (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
The peer review process
comments provided by the expert peer reviewers are documented, with every input being logged in a comprehensive database for follow-up, and assigned to different authors and staff to address
where conflict between/among review comments arises, coordinators contact the experts to discuss and determine an acceptable compromise
peer review builds upon previous activities by widening the scientific input
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
The peer review process (cont’d)The peer review process (cont’d)
the process aims to harmonize methodologies (and) to improve and expand the peer review system and UNEP networks - in line with the recommendations of the Scientific and Technical Meeting on strengthening the scientific base of UNEP (January 2004)
The peer review process is, therefore, systematic and adapts experiences from other processes
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Peer reviewers
have technical expertise in the subject matter for which they have been selected; they are expected to participate fully in the process and read all materials
should also protect confidentiality - by not sharing any AEO material with third parties
should respect the set deadlines, be objective and avoid bias
should report any problems encountered in finalizing the peer review process
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Some peer review activities
Advising on the treatment of the issues across the chapters
Commenting on the key issues as highlighted
Pointing out other key issues which have not been highlighted but would be considered a serious omission if they were left out in the final report.
Where such omissions are highlighted, the reviewer should present. in a paragraph or two, the issues, including full relevant references.
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt Where such omissions are
highlighted, the reviewer should present within a paragraph or two the issues, including the relevant full references
Indicating additional sources of relevant data
Ensuring that trend data have been presented in all chapters.
Ensuring that key indicators on the state of the environment and environmental policy performance are properly tracked
Some peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt Paying special attention to the
interlinkages between trends and policies (both specific to the issue under each chapter and more general policies that impinge on the issue)
demonstrating the environment as a driver of policy and vice versa, and evaluating policy in terms of environmental impact (effective, ineffective, unexpected, etc.)
Providing additional information and data, including boxes and other illustrations to enrich the sections, if necessary
Some peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt Pointing out inconsistencies and/or
contradictions of facts/data within and across the chapters and regional perspectives
Highlighting the major messages from the state of the environment both at the regional and sub-regional levels
Providing a list of the major messages emanating from the state of the environment both at the regional and sub-regional levels
Highlighting the major policy weaknesses clearly evident from both the regional and sub-regional thematic sections
Some peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt Highlighting the major policy strengths
clearly evident from both the regional and sub-regional thematic sections
Providing a list of both the strengths and weaknesses emanating from the state of the environment both at the regional and sub-regional levels
Advising on possible alternative policy initiatives that should be considered to ensure effective environmental management at both regional and sub-regional levels
Indicating any priority and emerging issues, or areas of outstanding vulnerability related to the issue as analysed
Some peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Providing an assessment of the treatment of IEA, both between the chapters and the regional sections, as well as across them.
Providing detailed comments on all the points listed above. Reviewers should avoid sending questions on the sections but rather provide substantive comments, which can be considered during revision of the sections.
Some peer review… (cont’d)
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Finding peer reviewers
Candidates may be recommended by:
government ministries/departments
scientific institutions
universities, collaborating centres
other stakeholders
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Determining the Peer Review schedule
The Peer Review schedule is a critical feature of the AEO process and should take the following into account:
availability of quality draft material
availability of appropriate experts
time available to use the review comments
deadlines for the AEO report and logistics
Div
isi o
n O
f E
ar ly
Wa
rnin
g A
nd
As s
ess
me
nt
Documents to send peer reviewers
Most recent and quality copy of the draft environmental assessment report - this could be the whole or section of the report
Clear TORs, including specific activities and deadlines and a clear reporting hierarchy
Clear guidelines on how comments should be provided