The Evaluation of SPS Applications - NATO - Homepage · The Evaluation of SPS Applications ... Tips...
Transcript of The Evaluation of SPS Applications - NATO - Homepage · The Evaluation of SPS Applications ... Tips...
October 20th, 2014 Lisbon, Portugal
The Evaluation of SPS Applications
ISEG - Independent Scientific Evaluation Group
Lisbon, Portugal 20 October, 2014
Horsehead Nebula
October 20th, 2014 Lisbon, Portugal
ISEG Primary Functions 1. Provide scientific and technical advice on applications for
bottom-up SPS activities;
2. Evaluate (peer-review) and rank bottom-up applications against SPS Key Priorities as set by the SPS Political and Partnership Committee (PPC);
3. Review bottom-up and top-down Science for Peace (SfP) project plans submitted by applicants;
4. Review Nationally-Funded Activities;
5. Have an horizon scanning role at the multi-disciplinary level;
6. Prepare and present reports to the PPC on the progress of the SPS activities;
7. Initiate applications or develop plans for new top-down projects that respond to identified key priorities.
October 20th, 2014 Lisbon, Portugal
ISEG Membership ISEG members are technical experts on
different fields, usually from NATO countries.
NATO Countries • Albania, Belgium, Bulgaria, Canada, Croatia,
Czech Republic, Denmark, Estonia, France, Germany, Greece, Hungary, Iceland, Italy, Latvia, Lithuania, Luxembourg, Netherlands, Norway, Poland, Portugal, Romania, Slovak Republic, Slovenia, Spain, Turkey, United Kingdom, United States
October 20th, 2014 Lisbon, Portugal
How ISEG works Submissions are sent electronically to all ISEG members!
For each proposal, several reviewers are selected by SPS staff • Based on proposal subject matter and reviewers expertise
Reviewers submit their evaluations electronically to SPS • All members can contribute to evaluation process;
• Submit evaluations
• Participate in discussion at meeting
Decisions taken at ISEG meetings with all members • Typically two or three ISEG meetings per year
• Decision on which projects to recommend for financial support
October 20th, 2014 Lisbon, Portugal
How ISEG works Panel is multidisciplinary and acts in a multidisciplinary
way!
Applications can be submitted anytime • For 2015 there are 3 deadlines (1 March, 2015; 1 July 2015; and 1
October 2015) to prepare for specific upcoming ISEG meetings.
• Note; a decision on funding may take up to 9 months
After a project is recommended for financial support • A “godfather/godmother” is selected from the ISEG members to follow
and assist project development.
Submission and evaluation process is fast, has little bureaucracy, is flexible and supportive
October 20th, 2014 Lisbon, Portugal
Support Mechanisms in SPS
• Projects • Multi-year Grants to collaborate on applied R&D projects
• Training • Advanced Study Institutes (ASI): grants to organize high-level
tutorial courses to convey the latest developments in a subject of relevance to NATO to an advanced-level audience.
• Advanced Training Courses (ATC): designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.
• Workshops • Advanced research Workshops (ARW): grants to organize expert
workshops where advanced-level, intensive discussions aim at finding solutions for today's security challenges.
October 20th, 2014 Lisbon, Portugal
Support Mechanisms in SPS
• Projects • Multi-year Grants to collaborate on applied R&D projects
• Training • ASI: grants to organize high-level tutorial courses to convey
the latest developments in a subject of relevance to NATO to an advanced-level audience.
• ATC: designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.
• Workshops • ARW: grants to organize expert workshops where
advanced-level, intensive discussions aim at finding solutions for today's security challenges.
October 20th, 2014 Lisbon, Portugal
Proposal Evaluation Criteria
Must address Key Priorities!!
Impact Scientific/Technical: important problem
Defense and Security
Partnership
Public Diplomacy
Young Scientists
Potential for Success Proposers
Schedule & Work Plan
Risks
Project value
End-users or stake-holders
October 20th, 2014 Lisbon, Portugal
Must address SPS Key Priorities
Mutually beneficial cooperation on issues of common interest, to meet emerging security challenges
• Counter-Terrorism • Energy Security • Cyber Defense • Defense against CBRN (Chemical, Biological, Radiological, and Nuclear
Defense ) Agents • Environmental Security • Enhance support for NATO-led operations and missions • Enhance awareness on security developments including through early
warning, with a view to preventing crises. Security-related Advanced Technology
• Border and Port Security • Mine and Unexploded Ordnance Detection and Clearance • Human and Social Aspects of Security related to NATO’s strategic
objectives
October 20th, 2014 Lisbon, Portugal
• Scientific/Technical • Important problem?
• Original idea?
• Defense & Security • Important issue?
• Impact on security?
• Impact on society: local or global?
• Partnership
• Public diplomacy
• Involvement of Young Scientists?
Evaluation Criteria: Impact
October 20th, 2014 Lisbon, Portugal
Estimate capability for Execution Expertise, Infrastructure
Resources (human, institutional support) are sufficient?
Schedule and work plan Proposed work to accomplish the goals?
Schedule realistic?
Division of labor and budget appropriate?
Expected outputs
Risk identification Technical, programmatic, political or other
Risk mitigation strategies (alternatives)
Partnership quality and management aspects
Evaluation Criteria: Success
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (1)
• Ensure you address key priorities! • One or multiple
• Select proposal topic carefully • Clear idea, clear problem, clear solution
• Relevance of topic should be very clear
• Select a team appropriate for the topic • Relevant expertise, experience
• Complementary expertise
• Appropriate end-users and stake-holders
• Must be committed, involved
Good team + relevant topic = good project
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (2)
• Program activities in timely manner • Describe context and relevance of problem
• Describe solution and plan, goals, how achieve them
• Clear set of tasks, clear schedule
• Task clearly linked to research objectives
• Identify who does what, when and what is the result
• Budget must be realistic, well justified • Reason for expenses should be clear
• Avoid unclear, undocumented expenses
• Budget should be in line with activity, schedule
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (3)
• Project should foster collaboration • Regular exchanges and meetings
• Joint work, joint publications
• Encourage involvement of young researchers • Teamed with experienced researchers
• Project results have to be disseminated • Present a dissemination plan: where to publish, when
• ISI Journals, conferences, book
• Highlight practical aspects, impact on partners, efficiency
• Consider budget for young researchers involvement
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (4)
• Identify risks, potential impact, alternatives • Relevant risks should be pointed out as well as impact
• Describe mitigating strategies, alternatives
• Do not hide potential risks, better to be up front
• Strong strategy to mitigate high risk better than minimize or ignore risk
• Project results should generate new ideas • Should have a positive, evident and durable societal impact
• Should generate further research: imagine the next step
• Must be achieved within the duration of the Project • Clear in schedule, in line with budget
• Clear end-users, stake-holders
October 20th, 2014 Lisbon, Portugal
• Be clear!!
• Revise proposal before submitting
• Check for all items
• Be convinced
• If you are not convinced, neither will the reviewers
• Be convincing
• Reviewers should feel compelled to fund you
Tips for proposal writing (5)
October 20th, 2014 Lisbon, Portugal
Support Mechanisms in SPS
• Projects • Multi-year grants to collaborate on applied R&D projects
• Training • ASI: grants to organize high-level tutorial courses to convey
the latest developments in a subject of relevance to NATO to an advanced-level audience.
• ATC: designed to enable specialists in NATO countries to share their security-related expertise with trainees from NATO's partner countries.
• Workshops • ARW: grants to organize expert workshops where
advanced-level, intensive discussions aim at finding solutions for today's security challenges.
October 20th, 2014 Lisbon, Portugal
Proposal Evaluation Criteria
Must address Key Priorities!!
Impact Event likely to have impact?
Will impact extend beyond participants?
Does it overlap other efforts?
Attendees Is it the right mix?
Depends on type of event
Organization Title appropriate? Comprehensible?
Well organized? Budget realistic?
Program focused, appropriate?
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (1)
• Ensure you address key priorities
• One or multiple
• Topic and Program appropriate
• Relevance of topic should be very clear
• Clear on focus, impact
• Duration appropriate
• Attendees and topic match type of event
• ASI: tutorial, advanced-level audience
• ATC: training, trainees
• ARW: experts, share advanced knowledge
October 20th, 2014 Lisbon, Portugal
Tips for proposal writing (2)
• Background and expertise of speakers
• Relevant expertise, experience, on topic
• Strong speakers, recognized leaders, practitioners
• Appropriate budget
• Realistic, efficient
• Location
• Appropriate?
• Local members involved?
• Output of Event
• Book, notes, course
• Course reusable/repeatable?