Tensions of Measurement: community investment activities in housing associations

12
Hosted by: Funded by: Tensions of Measurement: community investment activities in housing associations David Mullins and Vanessa Wilkes, TSRC Housing Studies Association Conference 15 th April 2011

description

Tensions of Measurement: community investment activities in housing associations. David Mullins and Vanessa Wilkes, TSRC Housing Studies Association Conference 15 th April 2011. Summary. Sector Context Evidence Base Levels of Measurement 5 levels of measurement Questions and Issues - PowerPoint PPT Presentation

Transcript of Tensions of Measurement: community investment activities in housing associations

Page 1: Tensions  of Measurement:   community investment activities in housing associations

Hos

ted

by:

Fund

ed b

y:

Tensions of Measurement: community investment activities

in housing associationsDavid Mullins and Vanessa Wilkes, TSRC Housing Studies Association Conference

15th April 2011

Page 2: Tensions  of Measurement:   community investment activities in housing associations

Summary• Sector Context• Evidence Base • Levels of Measurement– 5 levels of measurement– Questions and Issues

• Tensions– External Drivers– Internal Drivers

• Conclusions• Further Research

Page 3: Tensions  of Measurement:   community investment activities in housing associations

Sector Context • Well established industry, large enterprises

(50% market share in social housing)• Entrenched hybridity (social & commercial) • Core business highly regulated & measured • Non-core ‘community investment’ (CI)

activities symbol of independence • Legitimacy & control issues brought to fore –

2008 Housing & Regeneration Bill & NHF Audit• New tools emerging to measure social impact

of CI activities (in-house & off the shelf)

Page 4: Tensions  of Measurement:   community investment activities in housing associations

Evidence Base• Scoping study – emerging approaches to

measurement of social performance (2010)– literature review & checklist (17 tools to set priorities, measure

outcomes or link these to corporate goals)– 7 Case studies (organisation/tool based: 3 in-house, 4 outsourced) – 2 workshops (now 3, plus follow up tracking e.g. of first sector specific

tool)

• NHF Neighbourhood Audits 2008, 2011– Sector overview of CI (64% coverage by stock) – Inputs/Outputs (£435mil CI in 6,800 projects 2008; leverage 1.6:2.7)

• Linked PhD – Impacts & outcomes, depth case studies – More on this later!

Page 5: Tensions  of Measurement:   community investment activities in housing associations

The levels of impact measurementOECD definition of impact measurement

“the impact of all changes, positive or negative, attributable to the intervention”

• Also – intended and unintended effects and long-term as well as short-term

Original argument (Zappala and Lyons 2009):• 3 levels of impact measurement

– Whole sector– Individual organisations– Individual projects

• Tools, frameworks and methodologies not easily transferred between levels• Framework developed in relation to housing study (Mullins 2010) to 5 levels

Page 6: Tensions  of Measurement:   community investment activities in housing associations

5 levels of impact measurement

1. Sector level

2. Organisational level

3. Team level

4. Project level

5. Individual

Transferability between levels is dependent on:

• How the organisation adopts and uses the tool

• Having the necessary skills and expertise

• Relationship between internal and external data needs

• A corporate awareness of what is required from impact measurement

• The understanding and support by staff

• Maximising the usage of the data at more than one level

Page 7: Tensions  of Measurement:   community investment activities in housing associations

Questions and IssuesWhole sector– Is it possible or feasible to measure impact?

• Dependent on collective approach and understanding• Expensive • Housing sector – disparate projects and circumstances

– Draw on information from level 2Organisation / Team / Project– Reason / motivation

• External requirement?• Internal legitimisation / accountability?• A learning tool?

– Availability of skills and expertise• Not necessarily new methodologies but alternative application• Interpretative and subjective decisions

– Draw on information from levels below

Page 8: Tensions  of Measurement:   community investment activities in housing associations

External Tensions

External Drivers

• Funding • Sector Promotion• Dances around regulation• Common outcomes /

community budgeting • Wider third sector policy

influences (e.g. promotion of SROI)

Emerging Tensions• Data for Funders

– Conflicts with internal needs– Multiple monitoring– Timescales – Mission Drift

• Sector promotion– ‘more than a good story’– Limited no of PIs obscure

complexities of context & impact

Page 9: Tensions  of Measurement:   community investment activities in housing associations

Internal Tensions

Internal Drivers • Accountability & Reporting

(variety of audiences)– Boards & Trustees– Management – Making sure people enjoy the

projects • Benchmarking• Organisational Power Play

– Promoting visibility of CI within corporate scorecards

– Aligning CI activities to corporate goals

Emerging Tensions Commercial & Social Goals• Social Dividend from

commercial success– Tendency toward harder

measures – leverage• Contribute to core business

– Attribution problems – leads to focus of CI on tenants

Corporate v Project Levels– Who sets the goals? – which CI outcomes get

reported?

Page 10: Tensions  of Measurement:   community investment activities in housing associations

Conclusions

• Early days but likely to grow• High Expectations in advance of evidence!• Tensions around types of measures and uses

made of the data (both external & internal)• Is it worth it? (practical issues, costs

motivation/training)• Underlying tensions reflect hybrid and large

scale of these organisations & competing interests (e.g. Core business v CI)

Page 11: Tensions  of Measurement:   community investment activities in housing associations

Further Research 1. The role of community investment measurement tools

within the organisation– How is community investment integrated and understood?– What measurement tools are used, what is their strategic fit

within the organisation? 2. Data considerations (internal and external)– How is data reported and used?– What level of understanding does it provide?

3. The practical implications of measurement– Does impact measurement cause organisations to rethink

their mission, goals or values? – What are the operational issues and practical considerations

which affect measurement, such as length of contracts

Page 12: Tensions  of Measurement:   community investment activities in housing associations

Thanks for Listening

Please Keep in Touch

David Mullins [email protected] Wilkes [email protected]