Ian Clark – Director

Evaluation, Outcome Agreements and Regeneration.

Are the Lessons Being Applied?

Personal Perspective

• National programme evaluations & reviews • Outcome Agreement support programmes• Local programme & project evaluations• Other relevant studies e.g. Outcome Agreements How to

Guide and Review of Evidence in ROAs

Issues

• Outcome Agreements have been used to move regeneration issues from the periphery towards the centre of local policy making and service delivery– Formerly, stand alone regeneration programmes

– Now, more strategic commitments from partners

• Evaluation has played a part in this process• But how much of the lessons learnt has been applied and

how much hasn’t?

Outcome Agreements - LOAs

• ‘Developing Local Outcome Agreements for the Better Neighbourhood Services Fund’ 2004

• All Pathfinders & the Executive supported LOAs• Advantages:

– Local ownership

– Focused

– Evidence based

– Flexible

– Bring clarity and accountability

– Encourage real partnerships

• Wider use advocated by Pathfinders

Outcome Agreements - LOAs

• Some of the problems with LOAs:– Guidance

– tight timeframe

– incorporating diverse views

– choice of indicators

– programme-level monitoring - diverse issues & 300+ outcomes

– focussed programmes more measurable

– capturing capital expenditure difficult

– rural application

• Recommendations– menu of indicators

– avoid proliferation of Outcome Agreements

• Learning process

Outcome Agreements – ROAs

• More comprehensive than LOAs• Input and commitment from wider range of partners• Guidance • Menu of indicators included but nearly 1,000 indicators• Reality of measuring impact is very difficult• Excessive output information • Performance Management Framework • No final evaluation framework• Were ROAs only concerned with CRF funding?• ASBOA not well integrated

‘Approaches to Evaluation in Community Regeneration’

• Highlighted SIP/BNSF lesson: need for ‘simpler, more focused and more centralised approach’ to evaluation

• Key findings:– strong central leadership with guidance on locally

commissioned evaluations for consistency

– guidance on identifying baseline, input and outcome indicators

– framework for evaluation and format for reporting

– project management reporting systems informs (not replicates) programme evaluation

– Timely feedback to local stakeholders to inform developments

– Local flexibility within a structure of central support and co-ordination

The Role of Local Evaluations

• What influence do local evaluations have?• CPPs stated that SIP/BNSF evaluations did influence their

ROA• But to what degree:

– Overlapping timescales

– how widely is the evaluation circulated & read

– other factors in retaining projects

Current Regeneration Policy

• ‘People & Place’ integrated approach • Clearly, outcome based• Fairer Scotland Fund• Lighter touch• SOAs

– National Performance Framework

– Guidance

– Tied to funding streams

– Local authorities initially, CPPs by Apr 2009

– Seen as a developmental process

Some final thoughts

• Changes in FSF compared to CRF

• Are SOAs just about the Fairer Scotland Fund?

• Will regeneration initiatives survive beyond ring fenced funding?

• Is there a consensus on the role of community engagement?

• Are SOAs manageable?