Post on 13-Jan-2016
1
IST programme
IST KA3: The Evaluation
Introduction & Contents
• Principles
• Outline procedures
• Criteria and Assessment
• What this means for proposers
2
IST programme
IST KA3: The Evaluation
•Independent
•Anonymous
•Confidential
•Impartial
•Objective criteria
•Human fallibility
Basic Principles
3
IST programme
IST KA3: The Evaluation
Procedures (1): before the Evaluation
•Pre-registration of proposals
•Appointment of expert panels
•NDAs and notification of Interests
•Reading the Manuals
•Proposal eligibility screening
4
IST programme
IST KA3: The Evaluation
Procedures (2): During the Evaluation
•Initial Briefing
•Reading, commenting and scoring
•Panel discussion
•Ranking of proposals
•Reporting and justification
•Consensus meeting of readers
5
IST programme
IST KA3: The Evaluation
Procedures (3): after the Evaluation•IST Committee approval
•Notification of Decision
•Contract Negotiation
6
IST programme
IST KA3: The Evaluation
Assessment Criteria1 Scientific/Technical Quality & Innovation
2 Community added value and contribution to EC policies
3 Contribution to Community Social Objectives
4 Economic Development, S&T prospects
5 Resources, Management and Partnership
7
IST programme
IST KA3: The Evaluation
Scoring
•Numerical points scale from 0 - 50 = Unsatisfactory, out of scope or incomplete1 = Poor2 = Fair3 = Good4 = Very Good5 = Excellent
8
IST programme
IST KA3: The Evaluation
Weighting and thresholds for RTD bids
Criterion 1, threshold 3; weighting 4
Criterion 2, threshold 2; weighting 1
Criterion 3, threshold - ; weighting 1
Criterion 4, threshold 3; weighting 2
Criterion 5, threshold 2; weighting 2
9
IST programme
IST KA3: The Evaluation
What evaluators look for - fast
•The Vision Thing
- Originality- Significant innovation
•Credibility
- Realism- Precision and clarity
10
IST programme
IST KA3: The Evaluation
What the evaluators assess1 Scientific/technological quality & innovation• Does it contribute significantly to Key Action S&T issues?• What is the overall quality, strengths and weaknesses?• How original is it?• How far does it claim to extend the ‘state of the art’?• How credibly will it achieve this?• How risky is it & what is the risk/benefit balance?• Are the approach, methodology and workplan credible?• Are there good self-assessment procedures?
11
IST programme
IST KA3: The Evaluation
What the evaluators assess2 Community added value and contribution to EC policies
• Does it clearly identify and address European issues?• Is there clear value in doing it at the European level?• What is the expected impact at the European level?• What is the European added value of the Consortium?• Does is give ‘critical mass,’ complementary expertise?• Does it further EC policy on (e.g.) Standards and SMEs?
12
IST programme
IST KA3: The Evaluation
What the evaluators assess2 Community added value and contribution to EC policies
• Does it clearly identify and address European issues?• Does it reinforce national and international actions?• Is there clear value in doing it at the European level?• Does it help solve European problems?• What is the expected impact at the European level?
13
IST programme
IST KA3: The Evaluation
What the evaluators assess3 Contribution to Community Social Objectives
• Will it contribute to Quality of Life, Health and Safety?• Will it improve employment prospects and skills?• Does it help the environment or resource conservation?
14
IST programme
IST KA3: The Evaluation
What the evaluators assess4 Economic Development S&T prospects• Are there good potential applications?• How good is the exploitation plan?• Can the partners exploit the results?• Will it have a strategic impact on the market?• Will it create new business for users?• Will it be widely disseminated to the right people?
15
IST programme
IST KA3: The Evaluation
What the evaluators assess5 Resources, partnership and management• Is there a good management plan?• Are there clear milestones and monitoring?• Is there a good decision process?• Are the partners good?• Can they do all the tasks, without duplication?• Are the resources right?• Is the budget reasonable?
16
IST programme
IST KA3: The Evaluation
What this means for Promoters•Be ‘in scope’
- get the right Action Line- discuss with the PO
•Be eligible
•Register the proposal
•Follow the Guide to Promoters
•Read the Guidelines for Evaluators
17
IST programme
IST KA3: The Evaluation
What this means for Promoters (2)• Stand out from the crowd
- make a clear pitch, quickly
- justify it precisely
- no promotional hype
- clear English
- well illustrated
18
IST programme
IST KA3: The Evaluation
What this means for Promoters (3)• Part B
- defined objectives
- domain expertise
- significant innovation
- research methodology
- the right work packages
19
IST programme
IST KA3: The Evaluation
What this means for Promoters (4)• Part C
- policy contribution
- avoid bullshit
- tight management
- strong partnership
- future business model
20
IST programme
IST KA3: The Evaluation
What this means for Promoters (5)• Part A
- reasonable budget
- committed partners
- summary information