Post on 01-Jan-2016
description
Reviewing the 2014 AmeriCorps Applications and
Conducting the Review
2014 AmeriCorps External Review Training
2
Training Objectives
¨ Identify key changes from last year’s competition
¨ Understand the expectations of the Review¨ Ensure that you feel prepared to begin the
review¨ Explain how to navigate through the Review
Process¨ Understand how to use the Individual Review
Form (IRF)¨ Identify Resources to help with High-Quality
IRFs¨ Discuss considerations for participating in the
Panel Discussions¨ Reference the Close Out process
3
Key Changes from Last Year
¨ # of Reviewers: 3 Reviewers per panel, Role of Panel Coordinator
¨ Page Limit for Narrative = 15 pages¨ Review Products
IRF: Different structure, no scores (Ratings only) Applicant Feedback Summary (AFS): Incorporates every
significant strength and weakness from every Reviewer’s IRF Prepared by Panel Coordinator after the IRFs are final
Justification of Evidence Level and Quality Form: Prepared by Panel Coordinator based on panel discussion Informs Internal Review (not for Applicant Feedback)
¨ Assessing more than just Program Design¨ Selection Criteria and Standards¨ Tier 1 Education Applications
4
Selection Criteria and Standards
¨ The Selection Criteria, taken from the Notice, are the elements against which the applications are assessed. To better understand the intention of the Selection Criteria, they have been further clarified into “Standards.”
¨ The purpose of the Standard is to “set the bar” of expectations for each Selection Criterion. They serve as a clear indicator, or benchmark, of what applicants need to address when responding.
Tier 1 Education Applications
¨ Tier 1 Education Applications address “turnaround efforts” in elementary and secondary schools
¨ These applications have different and additional elements that need to be assessed when considering the Selection Criteria
¨ Tier 1 Education Reviewers have already been designated and notified. There are some separate materials specifically for Tier I Ed Reviewers.
5
Expectations and Steps
6
Ensure your
Participation is
Verified
Familiarity with
Materials
Receive Panel
Assignment email
7
Review Preparation
¨ Complete all training requirements 2014 AmeriCorps Review Handbook The Notice & Application Instructions Orientation Sessions
¨ Familiarize yourself with tools: www.nationalservice.gov/AmeriCorpsReviewerPage
¨ Make schedule accommodations¨ Receive Panel Assignments
Download assigned applications from eGrants Participate in Panel Intro Call (order of applications is
determined)¨ Review each application for COI¨ Begin Review!
8
Logistical Responsibilities
¨ Use your Resources
¨ Stay Connected
¨ Accountability for your work
¨ Submit Confidentiality and COI Statement and Participant Agreement
¨ Banking Information for Honorarium
9
CNCS Staff Liaisons
¨ Grant Application Review Process (GARP) Liaison Primary contact for process-related guidance and
logistical support Checks with each Reviewer and PC on review progress to
assist
¨ Program Officer Liaison (POL) Provides CNCS programmatic expertise and guidance Available as a resource for clarification of Criteria and
Standards Reviews the Applicant Feedback Summaries to ensure
comments relate to Selection Criteria
Send final IRFs (Word doc) to GARP Liaison
Enter IRFs and AFSs in eGrants
Complete evaluation
External Review Process
10
Read the application Provide a rating for each
element (R) Comment on significant
strengths and weaknesses (R) PC reviews & provides
Feedback
Edit Individual Reviewer Forms (IRFs) (R)
Compile Applicant Feedback Summaries (AFS) (PC)
Receive Feedback on AFS from Program Officer Liaison
Submit FinalIndividual Reviewer
Forms (IRFs)
Participate inPanel Discussions
Assess Applications
Review Applications for COI
Download Assigned Applications
Review Training Materials
Complete Close Out Process
11
Reviewing the Applications
¨ Applications generally will be reviewed and discussed in groups
¨ Read the full application (within the page limit) Applications should be evaluated against the Selection
Criteria, not other applications or outside information Apply your experience and expertise with balance
¨ Complete your assessment of the Selection Criteria and Standards within the IRF Select the appropriate Rating Comment on significant strengths and weaknesses Utilize your resources
Keep In Mind…
12
Please remember that not every organization has the resources to hire an experienced grant-writer. Applications should be assessed based upon how the narrative addresses the Selection Criteria; not how well the narrative was written.
13
IRF Development Process
Complete the IRF Draft
Submit to Panel Coordinator
Panel Discussion
Amend IRF
PC creates & POL Reviews AFS Comments, POL Provides Feedback that may impact IRF
14
Completing the IRF
¨ Heading Information: Fill in each text box¨ Instructions: Carefully read section and follow
closely¨ Selection Criteria: Make sure to read the
Criterion, as well as their corresponding Standard(s)
¨ Ratings: There is no numerical score; check the appropriate box as it corresponds (Reference the Review Rubric)
¨ Justification of Evidence Level¨ Significant Strengths and Weaknesses
¨ Remember to utilize the Example IRF provided on the Reviewer Resource Webpage
15
Review Rubric—Ratings
¨ Read the Individual Reviewer Rubric¨ Be familiar with the five ratings:
Greatly Exceeds the Standard Exceeds the Standard Meets the Standard Partially Meets the Standard Does Not Meet the Standard
Significant Strengths and Weaknesses
¨ Incorporate the application’s significant strengths and weaknesses in the appropriate Comments boxes
¨ Not every section of the IRF requires comments, but all comment boxes should receive a response
¨ Comments should be written in bullet form, not paragraph form, and in complete sentences
¨ Do not use any (+), (=), or (+/-) indicators
16
17
Tips for Producing High-Quality IRFs
¨ Address only the Selection Criteria and Standards¨ Address only the information found within the
application Do not consider outside information (i.e. Google,
Internet) Do not compare against another application
¨ Consider Performance Measures appropriately¨ Do not Address: Budget, suggestions for
improvement, caution with references to PMs, what “the panel thinks” etc.
¨ Avoid using the application’s statements as + or -¨ Avoid making generic comments that can be
applied to “any” application ¨ Reference the
Reviewer Tips: Writing Meaningful Comments & Sentence Starters
18
Tips for High-Quality IRFs Cont’d
¨ Language is evaluative and does not restate information from Application
¨ Appropriate comments (no suggestions for a “better proposal”)
¨ No inflammatory statements¨ Avoid referencing Page Numbers¨ Grammar and spelling are correct (Spell
AmeriCorps properly)
19
Panel Discussions
¨ There is a Panel Discussion for each application Led by the Panel Coordinator Estimate 30-45 minutes per application Assessments should not be shared with other Reviewers
in advance¨ Purpose of the Panel Discussion
Open forum to discuss and re-consider your assessment of an application
Ensure fairness in the review Ensure Selection Criteria/Standards are the basis for
assessment Discuss points of agreement and disagreement,
Justification ¨ Serving as an Effective Panel Member
Be Prepared, on time Responsive to Feedback
20
Applicant Feedback Summary
¨ Completed by Panel Coordinator¨ Incorporates all significant strengths and
weaknesses from each Reviewer¨ High-quality IRFs produce high-quality
AFSs¨ Documents will be reviewed by CNCS staff¨ Serve as a final Review Product to be
shared with the applicant
21
Expectations for Close-Out
¨ Documents are labeled, verified, clean Individual Reviewer Forms Applicant Feedback Summaries Justification of Evidence Level and Quality
¨ Confirm requirements are met with PC and GARP Liaison
¨ Complete Evaluations¨ Confirmation received of approval for
Honorarium
22
Assessment
¨ To check for understanding and verify that you have completed this orientation session, please complete the Assessment at the following link: https://www.surveymonkey.com/s/reviewapps