Office of Scientific Quality ReviewDr. Joyce Loper, Scientific Quality Review Officer

Dr. Michael Strauss, Peer Review Coordinator

In-house researchFarm-to-table scope18 National Programs1,000+ projects 2,000 scientists100 labs$1B annual budget

Agricultural Research Service

ARS Locations

• Stakeholder input

• Program planning cycle

Setting Research Priorities

Input

5 years of Research

2

1

5

4

3

Planning

of next 5-

years

Stakeholder Needs

National Needs

National Plan

Assignment of Objectives by National Program Leader

Researchers write 5-year plan for research

Outside Scientific Review Research does

not go forward5- year Research Program begun

What Makes Project Plans Unique?

Directed Research Objectives set through internal planning processes

Funds already allocated for researchMay be large and with collaborators Range of disciplines, locations, scientists

Long-term 5-year horizon with contingencies

Like Review of a ManuscriptDocument should present a logical, coherent

narrative with a clear path for the research.

- “Editor” = SQRO

- Three outcomes1. Publish as presented (no revision)

2. Publish after revision as monitoredby the “editor” (SQRO). Reviewers clear on whatresearchers are planning (minor gaps in info).(minor, moderate revision)

3. Publish after revision and reexamination by both reviewers and SQRO. Reviewers not at allclear about what researchers are planning (majorgaps in info). (major revision, not feasible)

1998 Farm Bill ARS research peer-

reviewed every 5 years

Most review panelists external to ARS

Satisfactory review before beginning research

Creation of OSQR

National ProgramsCrop

Production and Protection

Nutrition, Food Safety, and

Quality

301. Plant Genetic Resources, Genomics & Genetic Improvement

303. Plant Diseases

304. Crop Protection & Quarantine

305. Crop Production

308. Methyl Bromide Alternatives

107. Human Nutrition

108. Food Safety (animal & plant products)

306. Quality & Utilization of Agricultural Products

Animal Production and

Protection

Natural Res. & Sust. Agric.

Systems

101. Food Animal Production

103. Animal Health

104. Veterinary, Medical, and Urban Entomology

106. Aquaculture

211. Water Availability & Water Management

212 . Climate Change, Soils and Emissions

213. Bioenergy & Energy Alternatives

214. Agricultural & Industrial Byproducts

215. Pasture, Forage, Turf and Range Land Systems

216. Agricultural System Competitive-ness & Sustainability

Project Plans not Proposals

Not Proposals for researchSubject/objectives established by process

Funding decisions made by plan or mandatePanels do not evaluate budgets

Plan for ResearchPanel assesses if plan adequate to address problem

Assessment of ImpactWill research produce new information or understanding?

Title and Investigator(s)………….page 1Signature Page……………...........page 2Table of Contents……….………….page 3Project summary (250 words)...page 4Objectives...…..……..................page 5 Need for research ...................(1-2 p)Scientific Background ..............(5-7 p)Prior Accomplishments ….........(1-3 p)Approach & Procedures............(6-15 p)Milestone TableLiterature CitedPast Accomplishments of Project TeamIssues of Concern statementsAppendices (letters plus other material)

OSQR

Document Overview

Whole document on a flash

drive

Let us know if you want

paper copies instead.

Title and Investigators..………….page 1Signature Page……………...........page 2Table of Contents……….………….page 3Project summary (250 words)...page 4Objectives...………….................page 5 Need for research………………....(1-2 p)Scientific Background…………....(5-7 p)Prior Accomplishments………....(1-3 p)Approach & Procedures………...(6-15 p)Milestone TableLiterature CitedPast Accomplishments of Project TeamIssues of Concern statementsAppendices (letters plus other material)

Document Overview

This section is limited to maximum of 15 (<2) to 30 (>7)

pages depending upon number of

researchers.

Several Objectives or subobjectives with low page

limit may restrict detail.

See Peer Review

Guidelines in Red Folder for

page limits.

Panel FunctionsPanel is NOT reviewing

National Program direction, objectives or funding

ROLES

Chair Guides process, selects panel, and serves as a panel memberPanelists Serve as primary and secondary reviewers as designated by chair

and provide comments in discussion of all plans.Products

Advisory ComponentConsensus recommendations of panel

Assessment ComponentScore based on average of individual panelist ratings

Conflicts of Interest Guidelines

Collaboration with project scientist within last FOUR years.

Thesis, dissertation, advisor or graduate student/postdoc association within last EIGHT years.

Institutional or Consulting affiliation.

Financial gain from project.

If you feel you have a conflict concerning a particular project, you should not participate

in its discussion or rate it…

…but let us know!

Review Process (online)

Primary Reviewerbrief overview

Secondary Revieweradditional comment

Panel Discussion

Action Class ScoringBy Each Panelist

TO SQROFFICER

OSQR combinesall comments

received

Panel ChairValidates

Recommendations

Advisory Compon

ent

Assessment

Component

Panel discusses andedits comments

online

Review Criteria

Adequacy of ApproachAre the research plan and procedures appropriate? Is there sufficient information to understand the procedure proposed? Does the plan display understanding of the technologies and methodologies proposed? Are the roles of researchers and collaborators clearly presented. Does the overall plan present a clear, logical, experimental design? Is the plan well-written and clear?

Probability of SuccessIs the plan likely to lead to success or, if successful will it produce significant new knowledge (If there is a significant risk of failure, are the risks justified by the potential payoffs?)?

Merit and SignificanceWill this lead to new information, new findings, or new understandings? What would be the impact of this work on stakeholders? Society? Agriculture?

Action Class RatingsNo Revision

Excellent project. No changes or additions are required.

Minor RevisionApproach sound. Some minor changes required

Moderate RevisionSome change to an approach required but project is generally feasible.

Major RevisionRequires significant revision. Major gaps in plan or information.

Not FeasibleMajor flaws, omissions. Unfeasible or not possible to assess.

ProductsFor Project Team, ONP & Area

Summary Action Class Score

Panel (Consensus) Recommendations

Panelist Review Form

Primary and Secondary Reviewers complete this

Panelist Review Form

Panelist Review Form

Panelist Review Form

Before your meeting OSQR will edit/cut & paste these to

produce the a draft consensus

recommendation…

The Impact of Vertically Striped Voles (VSV) on Wheat, Rye, and Egg Production

R. U. Kidding 1321-38000-123-00D 1/5/2006

Frontiers of Vole Biology and Relativity Theory

What Happens After Review?(Researcher)

What Happens After Review?(Researcher)

No, Minor or Moderate RevisionLead Scientist responds to comments. Scientific Quality Review Officer certifies compliance with committee recommendations.

Major Revision or Not Feasible Lead Scientist revises and responds to comments. Examined by panel (Web-based meeting).Scientific Quality Review Officer certifies.

Projects are reviewed no more than two times(once after initial review)

Logistics Online Panels

Online Meeting Software

What you are using for this briefing…

Use same logon information.

No need to log-on to upload patches if you use the same computer.

Re-reviews for all panels …will also use this online system.

Honorarium

-File required paperwork.(we have this already)

-75-100% at end of initial review meeting.

-If needed, balance at conclusion of re-review meeting

Peer Review ResourcesOSQR Web Site

www.ars.usda.gov/osqr

Office of National Programs www.ars.usda.gov/research/programs

OSQR Staff:Mike Strauss – mike.strauss@ars.usda.govChris Woods – christina.woods@ars.usda.govLinda Daly-Lucas – linda.dalylucas@ars.usda.govGeneral email – osqr@ars.usda.gov

What Next?

Read through all projects you have/will receive making notes for discussion.

Complete a Review form (on flash drive)… …for those for which you are primary or secondary reviewer. When asked, email reviews to intramuralreviews@ars.usda.gov

Input

Implementation

Planning

Assessment

Input

Review