Seminar on Mid Term Evaluation in Objective 1 and 2 Regions Lessons from the Mid Term Evaluation of...
-
Upload
jonathan-hines -
Category
Documents
-
view
215 -
download
2
Transcript of Seminar on Mid Term Evaluation in Objective 1 and 2 Regions Lessons from the Mid Term Evaluation of...
Seminar on Mid Term Evaluation in Objective 1 and 2
Regions
Lessons from the Mid Term Evaluation of
Merseyside Objective One
Overall Evaluation Approach
Review andUpdate ofEconomicBaseline
Review of SPDStrategy
Analysis ofProgramme
Monitoring Data
PartnerConsultations
Project Reviews
BeneficiarySurveys - SMEs
& individuals
Top-DownAnalysis
Bottom-upAnalysis
Analysis andIntegration of
Emerging Findings forProgramme and
Priorities
Conclusions on: -Programme and Priority
Progress- Emerging Impacts &
added value- Effectiveness of
management
Comprehensive approach, responding directly to the evaluation questions
Mix of top down and bottom up analysis
Evaluation highlighted value, but also limitations, of original research
Bottom up Analysis
Need to understand nature and performance of projects, in terms of: Benefiting target groups Programme performance Overall socio-economic performance
Analysis therefore included: Detailed project reviews Surveys of business and individual beneficiaries
Project Reviews
Purpose: Reality check for progress ‘on
ground’ Check validity of monitoring data Explore emerging impacts
140 reviews in total: Covering 50% of committed spend Mix by priority, measure, fund,
delivery agency, location and size Use of a structured questionnaire,
gathering both quantitative and qualitative data
Performance data collected before hand
Review coverage: Rationale and
purpose Value added by SF Project progress Financial
performance Output/Impact
performance Cross cutting issues Programme
management
Project Reviews (cont)
Reviewers trained to undertake a rigorous assessment, including judgements on: Adequacy of project rationale and approach Articulation of project with the Programme Objectives Appropriateness and realism of targets and outputs
achieved Value added by Structural Fund support
Surveys of Beneficiaries
Two separate surveys: Individual beneficiaries Business beneficiaries
Purpose: Explore the effectiveness of the assistance for the
individual or business (gross impact) Establish the extent of deadweight and displacement
(net impact upon the economy) Investigate participants views of the support and
opportunities to improve provision But there are limitations
The Surveys
Individual Beneficiary Survey Postal survey with 100%
coverage 1,500 completed
questionnaires – 18% response rate
Potentially powerful tool, but limited by: Ability to capture detailed
information on impacts Ability to secure a
representative sample
Business Survey Telephone survey of 200
SMEs (c20% of participants)
Richer source of data due to use of telephone survey
Survey issues: Exclusion of SMEs with
minor interventions Ability to locate most
appropriate contact Too early to draw firm
conclusions on impacts
Survey Issues
Suitability of survey methods Representativeness of sample Timing of survey Design and testing of questions Training of interviewers
Conclusions
Value of original research in answering evaluation questions
Success dependent upon: Appropriate techniques Well designed and tested research instruments Accurate beneficiary data Timing of research
But need to be aware of limitations of approach