Proving the value of advice:A study of the impact of Bath and North East

Somerset Citizens Advice Bureau services

Michelle Farr¹, Peter Cressey¹, Sue Milner², Nick Abercrombie¹ and Beth Jaynes¹

Department of Social and Policy Sciences¹,Politics, Languages & International Studies (PoLIS)²

South West Forum Event 14th April 2014

ContextHouse of Commons Public Administration Committee

“Hull city council put its advice services contract out to tender … (an) effective citizens advice bureau in Hull lost its funding …The essence of why A4e won the competition was that the commissioners responsible for sorting out that contracting opportunity were not able to place any value on all the aspects of the citizens advice bureau beyond the simple provision of advice from adviser to individual” (1).

(1)http://www.publications.parliament.uk/pa/cm201012/cmselect/cmpubadm/902/90208.htm#n178

Context for advice services

• Citizens Advice Bureaux (CABx) facing a ‘double whammy’ • reduced funding sources and increasing client numbers (Morris & Barr, 2013)

• Advice service funding is increasingly insecure due to cuts in legal funding and local authority budgets (Hynes 2013)

• Unprecedented changes within the welfare system• Complexity of benefit issues often requires specialist advice to resolve (Morris

and Barr 2013)

• Correlations found between suicide rates and financial problems (Jacoby, 2002), suicidal thoughts and debt strongly associated (Meltzer et al, 2010).

• Evidence on socio-economic inequalities in health suggest that enabling access to material, social and financial resources should result in improved health (Haighton et al 2012)

Previous studies• Jones (2009): 3 year longitudinal study of 149 clients found total

benefit of £92.6K benefits, £374.5K debt; improved health and wellbeing

• Debt: Orton (2010) notes estimate of £1k cost to society of each individual debt problem; positive outcomes for 72% of CAB clients

• Wiggan & Talbot (2006) review of literature on benefits advice: take-up is sub-optimal and advice can boost take-up with positive outcomes; high impact and improved wellbeing (see also CAB review, 2012)

• Difficulties of quantitative measurement are widely acknowledged... (e.g. Allmark et al, 2013; Williams, 2004). Coventry study suggested gain of £9 for £1 invested (NEF, 2010; Williams, 2004)

• CAB sample of 3000 clients (2011) found gain of £3,271 per client

Aim: To understand, measure and value the long term impact of CAB services

• Previous research studies in advice have tended to use non-financial client evaluations of outcomes or short-term direct financial benefit of advice.

This research analyses:• The cause and impact of the problem before a client came to the

CAB• What has happened through the giving of advice• Outcomes and impact of the advice both for the client and any wider

outcomes for the household or other agencies such as the state, employers etc.

• Aim to financialise and give monetary values to the outcomes of advice

Research methods• Co-produced research with trained and supervised CAB

volunteers• Volunteers involved in client recruitment, study design,

interviewing clients and analysing data

• 80 clients took part in a qualitative longitudinal study to understand changes over 12-18 months

• Client interviews and additional quantitative measures to analyse client mental well-being using the Warwick Edinburgh Mental Well-Being Scale (WEMWBS Short)

• Clients consented to allow researchers to access CAB notes on their advice services

• Financialisation of outcomes using adapted social return on investment (SROI) approach

Strengths of the data collected• 80 clients interviewed

• previous studies in advice have interviewed 4 (NEF 2010); 10 (Windle et al 2010); and 30 planned (Haighton et al 2012)

• Longitudinal quantitative and qualitative data from clients enhanced by CAB advice notes

• 80 impact maps for clients created from analysis of this data, mapping outcomes, attribution, drop off and deadweight.

• Qualitative and quantitative data that explored the personal impact of problems, and why the advice made/ did not make a difference – perspectives from both client and CAB

SROI and advice services• Complexity of client’s situation, complexity of

service, complexity of outcomes• Bespoke nature of advice services in comparison

with generic services to all clients • SROI methodological complexity associated with

range of client specific outcomes• 260 different outcomes for our client group with diverse values

• CAB differentiates between financial outcome types so we developed disaggregated SROI ratio.

• Further set of findings to inform service redesign and development

Applying SROI to clients’ experiences• Client 20 had a degenerative long term health condition.

She could no longer work, her business ceased trading. “I had no money, I tried to survive for two years”.

• CAB supported benefit application• What would have happened without the CAB:“I don’t know, I really don’t know. I would have been in dire

straits, I would have lost my house”.• CAB helped her to manage and reschedule her debts • Debt outcomes are not included as they occurred before

the period of the project. Nor are the full value of benefits (e.g. ESA) included as some of these occurred previously.

 

Type of outcome value NPV for 5 yearsFinancial gain (Debts written off) £110,613Financial gain (Income gained) £205,615Financial gain (Income gain following repayments rescheduled) £41,654Financial gain (Debts wrongly charged) £2,254Financial gain (Re-imbursement or service or loan) £77,666Other (improved relationship, housing, sleep, employment or volunteering gained) £29,731Prevention (of depression, homelessness, bailiffs, utilities disconnected, hospitalisation) £40,533Grand Total £508,066

Net present value of outcomes for 80 clients over 5 years

Original financial value source Sum of valueClient actual £327,602Client estimate £68,414Proxy average local (BANES) £19,881Proxy average national £74,939Proxy Unit based £17,231

Grand Total £508,066

Policy implications• Advice mitigates the adverse effects of welfare reform

• 20% of the client sample sought advice to appeal decisions about Employment Support Allowance (ESA) and Disability Living Allowance (DLA).

• At least 75% of these ESA appeals and 67% of DLA appeals were successful, alleviating the pressures of poverty for people in times of ill-health or disability.

• Advice services occupy an important strategic position in the health and social care system

• In the context of austerity, reducing funding/ coverage of advice services to save money runs the risk of higher costs to the public purse. Advice can prevent further costs

Practical implications

• Demand among some of the most vulnerable clients for more systematic and intensive follow-up of their cases

• More tailored approach to clients’ needs• Additional support for some clients to implement advice

where needed• A reduction in the ‘social distance’ between the client

and the Bureau• Further linking with health and social care services

Conclusions

• Considerable complexity in applying SROI to advice services and wider casework organisations

• Importance of evidencing prevention (homelessness, depression, suicidal feelings)

• Development of SROI to advice• Using actuals rather than averages• Differentiating between different types of value• Combination of CAB’s financial reporting categories with SROI

concepts e.g. attribution, deadweight, duration, drop-off, NPV

• What SROI does not include• Outcomes according to advice domains, multiple theories of

change

• Full report: http://southwestforum.org.uk/advice-study