Bullion Community Resource Centre The Operational...

i March 2017: SROI of Bullion Hall 2015/16

Bullion Community Resource Centre

The Operational Research Society

Final Report on the Social Return on

Investment by Bullion Hall

March 2017

ii March 2017: SROI of Bullion Hall 2015/16

The Social Return on Investment by Bullion Hall A monetary estimate of the social value added by the activities hall at Bullion Hall is calculated. A

social return on investment framework is used for this analysis. The scope of the analysis is limited

to direct beneficiaries, although it is noted that a better resourced study would be likely to identify

additional impact across the wider community served by Bullion Hall. Outcomes identified in a

beneficiary study are valued using an equivalent value technique, with statistical calculations used to

determine the likely number of individuals experiencing outcomes. For the impact attributable to

the activity hall at Bullion Hall, a social return of investment of £4.71 for every £1 input has been

found.

Acknowledgements This report was commissioned through the Operation Research Society’s (ORS) Pro Bono O.R.in the

Third Sector initiative1. The authors would like to thank Felicity McLeister from the ORS, Belinda

Lowis from Bullion Hall, Stephen Davison from the Government Operational Research Service and

Aleen Murtaza from the Government Economic Service for their contribution to this analysis. The

relevant government departments of the authors are also acknowledged for enabling this analysis to

be completed as part of continuous professional development.

The Authors This analysis has completed by Dr Yasmine Ammar, Dr Jamie Douglas, Dr Kayleigh Ellis, Matthew

Miller , Dr Claire Murray and Dr Ian Thomas who are all Government Operational Research Service

(GORS) analysts working across different government departments in England and Scotland.

1 http://www.theorsociety.com/Pages/Probono/Probono.aspx

http://www.theorsociety.com/Pages/Probono/Probono.aspx

iii March 2017: SROI of Bullion Hall 2015/16

Contents The Social Return on Investment by Bullion Hall .................................................................................... ii

Acknowledgements ................................................................................................................................. ii

The Authors ............................................................................................................................................. ii

1. Introduction .................................................................................................................................... 1

2. Stakeholders and Data Collection ................................................................................................... 2

a. Stakeholders ............................................................................................................................... 2

b. Data ............................................................................................................................................. 2

3. Input and outputs ........................................................................................................................... 4

a. Inputs .......................................................................................................................................... 4

b. Outputs and Outcomes ............................................................................................................... 4

4. Valuing outcomes ........................................................................................................................... 6

a. Statistical analysis of outcome volumes ..................................................................................... 6

b. Duration ...................................................................................................................................... 7

c. Proxy values ................................................................................................................................ 7

5. Establishing impact ......................................................................................................................... 9

a. Factors which affect impact ........................................................................................................ 9

b. Calculating impact ....................................................................................................................... 9

6. Social Return on Investment ......................................................................................................... 12

a. The SROI ratio ........................................................................................................................... 12

b. Sensitivity analysis .................................................................................................................... 12

c. Quality assurance ...................................................................................................................... 14

7. Conclusions and recommendations .............................................................................................. 15

a. Conclusions ............................................................................................................................... 15

b. Merits of the method and future recommendations ............................................................... 15

Appendix I – List of proxy values and alternatives................................................................................... I

Appendix II – Impact Map ...................................................................................................................... III

Appendix III – Approaches for valuing outcomes .................................................................................. IV

1 March 2017: SROI of Bullion Hall 2015/16

1. Introduction In June 2016, Bullion Hall commissioned analysis to quantify the return on investment for the

community activities using its main activity hall. This report presents an estimate of the monetary

value of the social return on investment from these activities.

Bullion Hall is a Community Resource Centre, providing accommodation to a number of local

charities as well as being a thriving and much loved community centre. The main hall at Bullion Hall

is a multi-function activity room and can be divided into two rooms with a sliding partition. This

room is ideal for large groups or active classes, for example dance and exercise classes, indoor

bowls, martial arts and children’s parties. A wide range of activities take place in the activity hall at

Bullion Hall including knitting clubs, yoga, seated exercise classes and luncheon clubs.

A Social Return on Investment (SROI) methodology has been adopted to undertake this analysis.

This is a standard and widely accepted way of estimating a monetary value for the social impact of

charities and social enterprises2.

The SROI framework initially identifies the inputs at Bullion Hall. These are the costs, both direct and

indirect, of undertaking the activities at the community centre. These resulting activities are known

as outputs. The outputs are then linked to outcomes for the participants, or beneficiaries, of Bullion

Hall. The outcomes are valued, using suitable techniques, and the total value of all outcomes for all

beneficiaries is calculated. Before the total value can be compared to the associated costs, any part

of an outcome which is attributable elsewhere is deducted. The total value less any deductions is

known as the total impact.

The difference of the total impact of all outcomes and the total costs or inputs is called the net

present value and is one monetary measurement of social value. The ratio of the total impact of all

outcomes and the total costs is known as the social return on investment. This represents the

amount of value generated for each £1 invested in Bullion Hall. The scope of this analysis does not

attempt to provide the complete impact of Bullion Hall across all levels of society. Instead it is

restricted to including only those stakeholders for whom valid and reliable evidence exists for

outcomes achieved as a result of activities at Bullion Hall.

Once stakeholders were identified and inputs and outputs determined, data on the outcomes of

participants at Bullion Hall was gathered in the form of a beneficiary survey. Statistical techniques

were employed to extrapolate the evidence from this survey across all participants of the activity

hall at Bullion Hall. The outcomes for all beneficiaries were valued using equivalent values sourced

from a review of existing literature. This data was quantitatively analysed using the SROI framework

to determine the social return on investment of the activity hall in 2015/16.

The report proceeds with an analysis of the stakeholders and data collection in Section 2. This is

used to inform the inputs and outputs of the analysis in Section 3. Section 4 describes the valuation

of the outcomes of the activity hall at Bullion Hall. These valuations are used to establish impact in

Section 5. The final calculation of social return on investment is shown in Section 6. Conclusions and

future recommendations are proposed in Section 7.

2 https://www.bond.org.uk/data/files/Cabinet_office_A_guide_to_Social_Return_on_Investment.pdf

https://www.bond.org.uk/data/files/Cabinet_office_A_guide_to_Social_Return_on_Investment.pdf


2. Stakeholders and Data Collection

a. Stakeholders Table 1 includes stakeholders are those for whom a credible, direct link to Bullion Hall activities can

be established and measured.

Included stakeholders Level of impact Reason for inclusion

Bullion Hall suppliers Direct Costs associated with Bullion Hall, chiefly associated maintenance/running costs.

Bullion Hall Employees Direct Managing Bullion Hall and enabling the activities.

Volunteers Direct Ensuring activities are available for participants.

Participants Direct Primary beneficiaries of the charity.

Activity Tutors Direct Deliver activities at Bullion Hall.

TABLE 1: INCLUDED STAKEHOLDERS

Many excluded stakeholders will receive benefits from activities at Bullion Hall. The reason for their

exclusion is not because of a lack of impact, but that any impact could not be measured within the

resources available to this analysis and/or was likely to have a negligible influence on the final result.

A larger scale, albeit more resource intensive, analysis might potentially capture the impact on these

stakeholders. Excluded stakeholders are included in Table 2.

For example, it is possible to imagine that by helping participants improve their health and wellbeing

Bullion Hall is generating an impact for local NHS providers, or if by providing an increased sense of

local community in participants then crime may fall producing an impact for the police and other

local residents. These wider impacts are outside the scope of this work as the evidence for any

impact would be hard to obtain and even harder to convincingly demonstrate as the result of activity

at Bullion Hall. Accordingly, for the purposes of this analysis, the stakeholders are restricted to staff

and participants of activities hosted at Bullion Hall, and individuals involved in running or facilitating

the activities.

b. Data The data used for this evaluation came from a variety of sources including official accounts, ad hoc

data collection, a beneficiary survey (in the form of participant interviews) and secondary research.

The authenticity of this data is now critically appraised.

The salary and running costs of Bullion Hall are taken from the charity’s published accounts for

2015/16 to generate costs for the inputs of staff and hall maintenance3. These accounts are

published, in line with statutory requirements of the Charity Commission4.

Data on the number of participants taking part in each activity at Bullion Hall have been provided by

the centre management. A full discussion of the number of participants is provided in Section 4a.

These were not formal registers, but informal counts. Similar counts were used for the number of

3 http://apps.charitycommission.gov.uk/Accounts/Ends05/0001084105_AC_20160331_E_C.pdf

4 https://www.gov.uk/guidance/prepare-a-charity-annual-return

http://apps.charitycommission.gov.uk/Accounts/Ends05/0001084105_AC_20160331_E_C.pdf

https://www.gov.uk/guidance/prepare-a-charity-annual-return


volunteers supporting each activity at Bullion Hall. It is acknowledged that there is no way of

assessing the level of repeat attendees in the numbers of participants, and this has been accounted

for this in Section 4a. Published timetables of activities at Bullion Hall have been used to verify the

activity programme, and the centre management have advised regarding changes to the programme

of activities5.

Excluded stakeholders Level of impact Reason for exclusion

Bullion Hall governors Indirect Cannot confidently state that Bullion Hall has a direct impact on the lives of the charity’s governors unless captured by participation.

Families of participants Direct/Indirect Beyond resources of analysis, recommend further study. Local community Indirect Evidence of impact unavailable/beyond resources of analysis.

Local health providers Indirect Evidence of impact beyond resources of analysis/unlikely to prove link to Bullion Hall.

Local Schools Limited Impact likely to be difficult to separate from other factors.

Local government/ council Limited Outside of resources of analysis.

Other charities/ community initiatives

Limited/none No competing organisations identified in initial scope of project.

Local employers Indirect Need to avoid double counting any impact on local labour market/ economy.

Investors Indirect Investment costs captured under Bullion Hall above, no evidence to support a separate stakeholder group.

Local/regional media Direct/limited Not identified during scoping of project.

Transport/bus/taxis Indirect/limited Identified that many participants live locally/any impact likely to be outside of resources of analysis.

TABLE 2: EXCLUDED STAKEHOLDERS

Attendance costs, course fees or subs for attending activities at Bullion Hall have been provided by

the centre management. These costs are also published in Bullion Hall literature and have been

available on the Bullion Hall website, so a level of independent verification has been possible6.

In order to value the outcomes of activities at Bullion Hall secondary data has been relied on to

ascribe financial proxies to each outcome. While this necessarily carries a level of subjectivity a wide

range of potential sources for these proxies has been assessed for each value to ameliorate this risk.

Only those proxies with the greatest validity and least risk have been selected. This is discussed

further in Section 4c.

5 http://bullionhall.btck.co.uk/RegularActivities

6 http://bullionhall.btck.co.uk/RegularActivities/ActivityPricelist

http://bullionhall.btck.co.uk/RegularActivities

http://bullionhall.btck.co.uk/RegularActivities/ActivityPricelist


3. Input and outputs The social cost-benefit analysis undertaken depends on identification and quantification of: the

financial inputs, i.e. expenditure by both Bullion Hall and individuals; the outputs that directly result

from this expenditure, i.e. the activities on offer; and, most challengingly, the resultant beneficial

outcomes that are experienced by participants and/or society.

a. Inputs From the published accounts of Bullion Hall items of input expenditure such as maintenance and

other running costs were identified. Calculations also include volunteer time, subscriptions and fees

paid. These are all fundamental input costs, without which the activities on offer would not take

place.

The café and office space at Bullion Hall are not be considered within scope, so these input costs

were not included. This was to isolate, as much as possible, the social value of the activity hall, at

the request of Bullion Hall in their commission.

There are also specific aspects of centre activity that were deliberately excluded from the analysis,

largely due to a lack of available data. The most significant was the school holiday programme of

activities; see our recommendations for future work in Section 7b.

b. Outputs and Outcomes The outputs are the full programme of time term activities only that Bullion Hall delivered in

2015/16 resulting from the input costs. The outcomes are the benefits experienced by participants

that can be attributed to the undertaking of the activity. The causal link between output and

outcome is important.

To identify the outcomes beneficiary interviews were used as a starting point. In these, the

participants themselves made the direct link between the activity (output) that they had undertaken

and the positive benefit (outcome) that they themselves had felt or observed. This has the benefit

that it is reliant on beneficiaries’ own perceptions rather than the centre management or other

interested groups. The principle drawback was the limitation in the number of interviews and the

non-complete coverage of all activities. To account for this the list of outcomes was expanded by

associating outcomes with particular similar outputs where there were no case-studies to make a

direct causal link.

In total, 14 different outcomes from the full range of outputs were identified. The outputs and

associated primary outcomes are listed in Table 3. Note that each activity (output) often has multiple

benefits (outcomes) and the same outcome can be associated with multiple outputs.

It should be noted that these are only the mappings that could be identified from the available data

within the scope of this analysis. It is likely that in the event of a more thorough investigation of all

outputs and outcomes, additional mappings could be found. Furthermore, although all the

associations between outputs and outcomes listed in Table 3 were identified as being positive, it

should not be assumed that this will always be the case.


Output Outcome(s) Evidence Effect

Skills based sessions, e.g. Furniture upcycling

Stress/mental health

Beneficiary survey Positive

Sense of purpose/achievement Beneficiary survey Positive

Socialising/isolation Beneficiary survey Positive

Quality food/drink Beneficiary survey Positive

IT knowledge/digital participation Beneficiary survey Positive

Family relationships Beneficiary survey Positive

Recovery from injury/illness Beneficiary survey Positive

Confidence Beneficiary survey Positive

Sense of community Beneficiary survey Positive

Lifelong learning Beneficiary survey Positive

Entrepreneurship Beneficiary survey Positive

Sociable sessions, e.g. Ladies that Lunch

Stress/mental health Beneficiary survey Positive


Quality food and drink Beneficiary survey Positive




Encouraging others Beneficiary survey Positive


Exercise sessions e.g. Ballet Exercise Assumed Positive

Hobbies, e.g. patchwork quilters


Sense of purpose/achievement Beneficiary survey Positive





Encouraging others Beneficiary survey Positive


Lifelong learning Beneficiary survey Positive

Parent and child sessions, e.g. parent and toddler classes


Improved confidence Beneficiary survey Positive


Other sessions, e.g. health action groups

No outcomes identified. n/a n/a

TABLE 3: MAPPING OUTPUTS AND OUTCOMES

Alternative datasets that could have been used to identify broader outcomes include data from the

local NHS, Police figures on crime rates or anti-social behaviour, or data from the Department for

Work and Pensions on local employment. For example, an increased sense of community in

participants at Bullion Hall could be expected to result in a reduction in violence or crime in the

surrounding area. Similarly activities at Bullion Hall that encourage good fitness and diet could lead

to a lower burden on local NHS services. Job club or entrepreneurial activity at Bullion Hall could

lead to an improvement in the local employment rate. However, it was decided that extending the

analysis to these broader outcomes was not feasible due to the challenge of a very small signal to

noise ratio and the risk of contamination of the outcome by other external factors beyond the scope

of this analysis, such as local and national programmes to improve these outcomes.


4. Valuing outcomes

a. Statistical analysis of outcome volumes The beneficiary survey took the form of interviews conducted with a random sample of Bullion Hall

participants by the Development Manager at Bullion Hall. In total 22 in depth interviews were

undertaken. The beneficiaries’ responses were transcribed and redacted before being used for

analysis. Statistical techniques were used to distribute the outcomes identified in Section 3 across

the total number of participants attending similar activities at Bullion Hall. The total number of

participants for each activity at Bullion Hall was obtained from registers provided by the Hall’s

management.

Activities at Bullion Hall were grouped into themes related to: learning new skills; sociable sessions;

exercise sessions: hobbies; and, parent and child activities. Each theme of activities was mutually

exclusive, so an activity appears in only one grouping.

Interviews with the beneficiaries referring to participation at activities from each theme were used

as the sample of all participants for that grouping. From this sample the proportion of case studies

which reported a particular outcome was used as the proportion of all participants in the theme who

experienced that outcome, as shown in Figure 1: Statistical Sampling. The total number of

participants was obtained by summing the registers of each activity in a theme.

FIGURE 1: STATISTICAL SAMPLING

It is incumbent on this analysis to highlight the strengths and weaknesses of this approach.

Examining a random sample of the participants is a valid way of determining the probable

distribution of outcomes for participants of a themed group of activities. In this way the method has

statistical rigour. An effort has also been made not to assume that all activities at Bullion Hall

delivers the same outcomes to all participants, which would have been the assumption if the case

studies had been taken as being proportionally representative of all participants of all activities.

Given the relatively small sample size of the case studies, reliability could be an issue in this

approach. To account for this standard confidence intervals have been calculated for each estimate


of the number of participants in each group experiencing an outcome. Sensitivity analysis using

these confidence intervals on the number of beneficiaries is shown in Section 6.

It is recognised that some individuals attending Bullion Hall could attend multiple activities. While

this is acknowledged as a risk there is no data to support a quantitative deduction based on such an

assumption. To justify this approach, an alternative assumption could be that the same group of

individuals attend every available course at Bullion Hall. This would be rejected as improbable.

There was also a methodological challenge in grouping the activities into those likely to have similar

outcomes. By basing the analysis on groups that have similar intended outcomes for participants,

for example exercise classes, any potential bias has been minimised.

b. Duration The outcomes participants of activities at Bullion Hall experience vary in their duration. Many

outcomes rely on the continued attendance of participants at Bullion Hall and where this is the case,

for example the outcome of improved mental health, this outcome is assumed to only be

experienced for the 39 weeks of the year where the term time activity timetable is in effect.

Exceptions to this assumption include outcomes where the proxy value is taken to be a subscription

which would necessitate an ongoing fee, for example a gym membership or home broadband.

Equally, where an outcome could be achieved with a discrete or finite number of sessions, for

example an improvement in family relationships through family therapy course, it has been assumed

that standard course duration would have the same impact as the continued attendance of a

participant at Bullion Hall. There are also examples of where the outcome could only be achieved

through a one-off setup fee which is assumed to apply only once, for example in purchasing IT

equipment to develop digital participation.

All of the assumptions about the duration of outcomes obtained by participants at Bullion Hall are

listed in the proxy section in Appendix I – List of proxy values and alternatives.

No assumptions about the long-term benefit participants might experience by attending activities at

Bullion Hall has been made. In the absence of evidence it would be challenging to make such an

assumption, and the on-going nature of activities at Bullion Hall suggests that outcomes for

participants could cease in the absence of their continued attendance.

c. Proxy values To establish the unit value of outcomes related to activities at Bullion Hall an approach of equivalent

value was used. This approach relies on selecting an equivalent activity that participants at Bullion

Hall could undertake to achieve the same outcome as we have identified through the analysis.

The equivalent values were selected by examining each outcome and considering a range of possible

alternative activities to achieve the same. This included values used for the same outcome in other

analyses, internet searches and assumptions about alternative activities made by the authors in

collaboration with government economists. From this range of possible equivalent values the

strengths and weaknesses of each alternative was assessed. Proxies previously used in peer

reviewed academic journals carried more weight than individual opinions of report writers. The

timeliness of each potential proxy was examined to give more credibility to more recent options.


Ultimately, overall value judgements about the relative strengths and weaknesses of each equivalent

value under consideration were made, and the equivalent value considered the most authentic

proxy for each outcome was selected. The full list of potential equivalent values is found in

Appendix II.

As equivalent values were selected their unit cost was determined. This allowed us to select, based

on the examination of the duration of a benefit in Section 4b, the correct number of units of

equivalent value per person for each outcome. The total outcome value for each outcome is

calculated as the product of the unit cost, duration and number of beneficiaries, as shown in Figure

2: Calculating total present value.

FIGURE 2: CALCULATING TOTAL PRESENT VALUE

Although an equivalent value technique has been applied here, it is reasonable to justify that

decision through consideration of alternative approaches. This is discussed further in Appendix III.


5. Establishing impact This section examines the impact of Bullion Hall with reference to the other factors that influence it.

a. Factors which affect impact In order to accurately calculate the Social Return on Investment, the outcomes identified must be

reviewed to estimate how much of each outcome would have happened anyway and what

proportion of the outcome can be isolated as being added by the activities of Bullion Hall.

There are four components to consider:

Attribution measures how much of the outcome is attributed to other activities. Was there another

service that the individual was using, alongside Bullion Hall, which could have also had a positive

contribution?

Deadweight identifies how much of the outcome would have happened anyway in the absence of

the beneficiary attending activities at Bullion Hall.

Displacement assesses how much of the outcomes from Bullion Hall have displaced other outcomes

that are likely to have arisen in the absence of the centre.

Drop Off calculates the decline in the outcome over time, once participation is complete.

There is a lack of evidence in the available data to individually determine values these four

components for all participants. Accordingly the beneficiary interviews and similar calculations from

other published SROI reports have been used to estimate overall values. Attribution, deadweight

and displacement components were combined into one overall discount rate. This was considered

appropriate given the lack of data available to assess the individual components. The drop off rate

was used to represent the period of the reporting year 2015/16 for which a benefit would continue

for a participant.

Table 4 shows the impact rates and drop off rates for each outcome and our rationale. 100%

represents that Bullion Hall contributed fully to the outcome whereas 0% represents no outcome

contribution. Similarly for drop off, a 100% represents that the outcome lasted a full year.

b. Calculating impact The impact and drop off rates have been applied to the proxy values identified in Section 4. The

calculation of the total impact of each outcome is shown in Table 5.


Outcome Impact Rate

Rationale Drop off rate

Rationale

Stress/mental health 90% Assumed that other services such as the NHS may contribute to this outcome but as this was not mentioned in any case study we have left the rate quite high.

100% Assume this would continue for period after course ends.

Sense of purpose/ achievement

100% No evidence to suggest that any other activity contributed to this outcome.

75% Assume that this would only continue for the length of the course (39 weeks).

Socialising/tackling isolation



Quality food and drink



Exercise 50% Assume that participants are not inactive for rest of week.


IT knowledge/digital participation



Family/relationship improvement



Overcoming injury/illness

50% Assumed that other services such as the NHS may contribute to this outcome.


Improved confidence 100% No evidence to suggest that any other activity contributed to this outcome.


Encouraging others 66% Some case studies indicated that this outcome may also be linked to other organisations.


Sense of community 100% No evidence to suggest that any other activity contributed to this outcome.


Lifelong learning 14% Assume that individual would continue to learn on rest of the week (6 days) when not at Bullion Hall.


Entrepreneurship 100% No evidence to suggest that any other activity contributed to this outcome.


TABLE 4: DETERMINING IMPACT AND DROP OFF


Outcome Proxy Value (A)

Estimated Number of Participants (B)

Total Outcome (C = A X B)

Impact Rate (D)

Drop off rate (E)

Impact (F = C X D X E)

Stress/mental health £2,340 169 £394,658 90% 100% £355,192

Sense of purpose/achievement £2,564 20 £52,517 100% 75% £39,388

Socialising/tackling isolation £2,223 80 £177,649 100% 75% £133,237

Quality food and drink £312 41 £12,759 100% 75% £9,569

Exercise £342 121 £41,382 50% 75% £15,518


£509 7 £3,799 100% 75% £2,849


£300 46 £13,934 100% 100% £13,934

Overcoming injury/illness £270 30 £8,028 50% 75% £3,011

Improved confidence £360 155 £55,838 100% 75% £41,878

Encouraging others £33 26 £845 66% 75% £418

Sense of community £1,747 125 £219,024 100% 100% £219,024

Lifelong learning £210 61 £12,886 14% 75% £1,353

Entrepreneurship £135 4 £502 100% 100% £502

TOTAL £835,875

TABLE 5: CALCULATING IMPACT


6. Social Return on Investment

a. The SROI ratio The Social Return on Investment (SROI) Value is expressed as a ratio of return and is derived by

dividing the total value of the impact by the value of the investment, i.e. the costs of running Bullion

Hall.

In this analysis, the total impact calculation is as follows:

SROI = Total present value

Total input value

The estimated impact for the activities run from the Bullion Hall in 2015/16 was £835,875, see Table

5. This represents an estimate of the social value created by Bullion Hall.

As the benefits of the activity hall at Bullion Hall have been calculated for only one year and not

projected benefits into the future, the Total Present Value is the total of all impacts, £835,875. This

is now compared to the input costs.

The total input costs at Bullion Hall are the Bullion Hall staff costs, the maintenance costs for Bullion

Hall, the associated cost of volunteer time and the sum of the attendance fees for each activity at

Bullion Hall across the 39 weeks of the term time activity programme. This is shown in Appendix II,

and is calculated to be £177,390 in 2015/16.

Figure 3: Net present value shows the Net Present Value of the activity hall at Bullion Hall in

2015/16.

Figure 4: SROI shows the social return on investment of the activity hall at Bullion Hall 2015/16.

This means that for every pound of investment in Bullion Hall and its activities, £4.71 of social value

is created. This is a comparable social return on investment to those found in other reviews of

social investment, for example Vineburgh7 and Craft Café8.

b. Sensitivity analysis The results presented in this report are based on variables and assumptions according to available

evidence, including qualitative data on the experience of members of Bullion Hall. This calls for

further analysis and validations to ensure that results are robust.

7 http://www.socialvalueuk.org/app/uploads/2016/03/Vineburgh%20SROI%20Report%20Final%20Assured.pdf

8 http://www.socialvaluelab.org.uk/wp-content/uploads/2013/05/CraftCafeSROI.pdf

Net Present Value £835,875 – £177,390 = £658,485.

SROI £835,875

£117,390 = £4.71 : £1.

FIGURE 3: NET PRESENT VALUE

FIGURE 4: SROI

http://www.socialvalueuk.org/app/uploads/2016/03/Vineburgh%20SROI%20Report%20Final%20Assured.pdf

http://www.socialvaluelab.org.uk/wp-content/uploads/2013/05/CraftCafeSROI.pdf


Sensitivity analysis is used to test variables and assumptions given that there are areas of the base

case that could be derived from imperfect evidence. Table 6 shows which areas were chosen as

those with the biggest potential to affect the results.

Outcome Base Case New Case SROI (value per £1 input)

Stress/mental health outcome

169 participants experienced this outcome

No participants experienced this outcome

£2.94

Sense of purpose/achievement outcome



£4.49

Socialising/tackling isolation outcome



£3.97

Quality food and drink outcome



£4.66

Exercise outcome 121 participants experienced this outcome


£4.62

IT knowledge/digital participation outcome



£4.70

Family/relationship improvement outcome



£4.63

Overcoming injury/illness outcome



£4.70

Improved confidence outcome



£4.50

Encouraging others outcome



£4.71

Sense of community outcome



£3.48

Lifelong learning outcome 61 participants experienced this outcome


£4.71

Entrepreneurship outcome 4 participants experienced this outcome


£4.71

Drop Off Rate Some outcomes last for the full

year No outcomes last for more than course duration (39 weeks)

£3.88

Impact Rate Depending on outcome ranging

from 50% impact rate for all outcomes

£2.54

Stress/mental health outcome



£6.49

TABLE 6: SENSITIVITY ANALYSIS


This sensitivity analysis shows that if the values used in the analysis are reduced from their

estimated values, the Social Return on investment is unlikely to fall below £2.54 : £1. This value

would be reached if the assumptions around each outcome’s impact rate were reduced to 50%.

Since reducing the equivalent value proxy to zero produces the same outcome as setting the number

of participants to zero, they will produce the same SROI. As such the values presented in Table 6 for

each outcome can be taken as the lower

estimate for fewer participants experiencing

each outcome.

If more data could have been obtained at an

earlier stage, particularly in relation to the

number of participants experiencing each

outcome, then the social return of Bullion Hall

could increase. The analysis presented here

indicates that 169 participants experienced a

stress or mental health improvement as a result

of attending activities at Bullion Hall. If this was

to increase to 318 participants, which is the

upper bound of the confidence interval, then the

SROI ratio would increase to £6.49 : £1.

c. Quality assurance The work was undertaken by a team of experienced analysts who cross checked and questioned

each other’s assumptions as the methodology was developed. Importantly, there was new and

external input to the project at the final stages of quality assurance.

Throughout the course of the work, particularly at the stages which involved selection of monetary

values, consideration was given to the sensitivity of the final result to the parameter in question.

Some of the sensitivity testing, though not all of the checks undertaken, is described in Section 7b.

Confidence intervals

Confidence intervals are a way of indicating

uncertainty when using samples of data.

For each outcome identified in the sample of case

studies, the confidence interval indicate that if

participants were resampled many times, it would

be expected that 95% of computed confidence

intervals would contain the true mean average

proportion of participants exhibiting that outcome.

It would therefore be considered unlikely for the

true proportion to lie outside out confidence

interval.


7. Conclusions and recommendations

a. Conclusions This report was commissioned to determine the social value generated by the activity hall at Bullion

Hall Community Resource Centre. The scope was limited to identifiable outcomes for participants at

Bullion Hall, and excluded wider social outcomes such as health, employment or criminal activity

benefits. This analysis was conducted on the basis of the programme of activities at Bullion Hall

during the 39 weeks a year that the term-time timetable is in operation. The reporting year is

2015/16.

By using a social return on investment framework, the social return on investment is assessed to be

£4.71:£1, so that for every £1 invested a social return of £4.71 is generated. This lies within a

potential range of values between £2.54:£1 and in excess of £6.49:£1. The total net present social

value of the activity hall in 2015/16 is assessed to be £658,485.

This analysis indicates that the activity hall at Bullion Hall generates a valuable social return to its

community, as well as providing a much loved local resource. The benefit of activities to the

beneficiaries exceeds the cost of delivery as measured in this analysis, and this report recommends

the continued investment in the activity hall to continue providing these outcomes.

The outcomes included in this analysis were restricted to those highlighted by beneficiaries in their

interviews. It is likely there are other potential outcomes of Bullion Hall that were not captured

through this data gather, so are not included in the analysis. While it should be recognised that

outcomes are not necessarily positive, the inclusion of any omitted benefits is likely to have a

positive impact on the social return on investment in only one direction. The value of social return

on investment in this report is therefore considered to be a conservative estimate.

b. Merits of the method and future recommendations Throughout the analysis a robust and recognised framework was used. This analysis indicated a

social return on investment of the activity hall at Bullion Hall of £4.71:£1 that has been presented in

Section7. This result is considered robust and fit for the purpose of showing potential funders an

estimate of the value added by the hall and the social return that might be expected from their

investment

The uncertainties and limitations of the analysis have been elucidated. These can broadly be split

into three categories: lack of data and data limitations; uncertainties and subjectivities inherent in

this kind of analysis; resource constraints.

There are areas which could be improved in any future analysis. One of the main challenges related

to the amount of data that was available in the beneficiary interviews. Whilst these were an

invaluable source of evidence, they were collected prior to the involvement of the analysts. In future

projects it would be beneficial for analysts to be involved at an earlier stage, allowing design of the

data collection activities to target it for specific analytical purposes.

Higher quality data would enable analysts to generate more options for proxy values, identify wider

stakeholders and develop a better understanding of the deadweight and attribution. Bullion Hall

could further assist in producing a useful dataset by retaining a register of participants per activity


per week so that this could be easily extrapolated across the year. The single biggest

recommendation would be to investigate ways by which the school holiday activities, a significant 13

week section of each year, could be incorporated into the analysis to give overall return on

investment over the course of a whole year.

I March 2017: SROI of Bullion Hall 2015/16

Appendix I – List of proxy values and alternatives Proxies highlighted are those which have been selected for our analysis.

Outcome Proxy Source Date Value Per Pros Cons

Counselling course https://counselling-newcastle.co.uk/faqs/what-are-your-fees/ 2016 £60.00 per session

Counselling courses are a well used proxy for

improving stress or mental health, with continuous

treatment frequently used.

A local counselling course may underestimate the

value associated compared to treatment through the

NHS.

Counselling course GVE | SROI real jobs evaluation 2010 £1,085.00 course Same outcome Not the same activity as Bullion hall

Counselling course GVE | Mental healh reduced stress in self employed people 2010 £630.00

9 session

course Not self employed participants

Value of having no problems wit hanti social behaviour Vineborough Regenerative initiaive 2014 £6,403.00 Year Improved stress/mental health not related to ASB

Stress management course GVE | SROI 2015 £245.00 course One-off course

Hospital treatment of stress related disorders GVE | SROI 2008 $4257 treatment Canadian, 2008

Counselling course Life cycle UK SROI summary report, Karen Bell 2013 £2,080.00 year

Same outcome and proxy as our study, restricted

our assumption to 5 hours

Counselling course Craft café creative solutions to isolation and lonliness 2011 £2,080.00 year



Part time job

http://www.ons.gov.uk/employmentandlabourmarket/peopleinwork

/earningsandworkinghours/datasets/ashe1997to2015selectedest

imates

2015 £6.70 per hours

Attending Bullion Hall requires the same

organisation and personal care that would be

expected by an employer. A part time job to proxy a

sense of achievement is commonly used.

There is a range of hours and hourly pay used to

describe a part time job in other studies.

Part time work, 5 hours Life cycle UK SROI summary report, Karen Bell 2013 £2,563.60 year


our assumption to 5 hours Value is not based on minimum wage

Part time work, 5 hours Craft café creative solutions to isolation and lonliness 2011 £2,563.60 year



Value is not based on minimum wage, but median

gross earnings

Club membership/social workerhttp://neweconomymanchester.com/our-work/research-evaluation-

cost-benefit-analysis/cost-benefit-analysis/unit-cost-database2015 £57.00 per hour

Social workers are commonly used in similar

analyses to proxy tackling isolation.

An alternative proxy could be a club membership,

however this would require a stated preference

technique if there are no clubs/organisations within a

similar distance to the community to Bullion Hall.

Average spending on cultural and recreational activities Life cycle UK SROI summary report, Karen Bell 2013 £769.00 Year

Not directly attributable to tackling isolation and

lonliness

Family spending Craft café creative solutions to isolation and lonliness 2011 £769.60 Year

Not directly attributable to tackling isolation and

lonliness

Value of one activity with friends per week Vineborough Regenerative initiaive 2014 £1,014.00 Year Subjective, revealed preference

12 week programme for unemployed men with mental health

issues GVE | MOJO SROI 2014 £2,337.00 course Unknown technique

Restaurant meal http://www.workgateways.com/working-in-the-uk/cost-of-living 2016 £8.00 per meal

We are making the assumption that only 1 meal a

week would be improved by attending a suitable

activity at Bullion hall. This is proxied by a

reasonably priced restaurant meal, and will be offset

by the cost of attending a suitable activity.

Invidiuals could reasonably be expected to spend

more time with family rather than attend Bullion Hall,

which would lead to them potentially getting a quality

meal an a lower cost. This is an unsubstanciated

assumption however.

Average weekly spend on food GVE | ONS 2011 £53.20 week Similar value Proxy is for 1 week

Average weekly spend on food and (non alcoholic) drinks GVE| ONS 2013 £56.80 weeks Similar value Proxy is for 1 week

Additional amount spend on food which produces long term

health implications due to a better diet GVE | ONS 2011 £2,941.00 year Similar proxy Proxy is for one year assuming food every day

Gym membership http://my.northtyneside.gov.uk/category/251/memberships 2016 £28.50 per month

It is commonly recommended that individuals do 20-

30 minutes of vigorous activity every day. This could

reasonably be replicated by a gym membership.

The vigorous activity could be done free of charge

by individuals, although this would require an

assumption to be made about individuals intrinsic

motivation

Guided healthy walk in the city Craft café creative solutions to isolation and lonliness 2011 £1,092.00 Year Probably a similar proxy

Weekly group exercises GVE | Age concern kingston 2010 £5.20 hour Similar proxy

Frequent mild exercise GVE | HCAT calculator 2016 £3,633.18 year Based in London

Frequent mild exercise GVE | HCAT calculator 2016 £3,537.12 year Not based in London Unknown frequency

Annual cost of an active hobby (gym) GVE | Yorkshire bank 2008 £480.00 year Similar proxy £2,008.00

Cost of equipment/training course/monthly internet cost https://www.uswitch.com/broadband/packages/ 2016 £17.40 per month

The UK government is committed to 95% of the

country getting high speed broadband and 10)% of

the country getting basic broadband. This is a

minimum retail cost of broadband.

Effective subsidies for broadband connections are

available for some low income households.

http://www.pcworld.co.uk/gbuk/computing/desktop-pcs/desktop-

pcs/1093_8225_71255_xx_xx/1_20/price-asc/xx-criteria.html2016 £300.00 one off

Individuals would require equipment to access the

worldwide web.

The cost could be amortised across several years,

and lower cost options are available, for example

tablet computers or smartphones.

Internet and media access in families GVE | Get IT (with BT) 2014 £1,064.00 year Not providing kit to inviduals home

Internet usage costs GVE | ONS 2012 £6.30 week Does not account for cost of equipment


Sense of purpose/achievement

Exercise


Socialising/tackling isolation


II March 2017: SROI of Bullion Hall 2015/16

Outcome Proxy Source Date Value Per Pros Cons

Relate counselling course http://www.relatenortheast.org.uk/page/faq 2016 £50.00 per session

Counselling courses are a well used proxy. Relate

are a nationwide relationship counselling service,

with local branches. A 6-course session has been

used in other SROI calculations.

If participants attend Bullion hall every week, a 6-

week course could underestimate the improvement

of their relationships. Equally, if their activity at

Bullion Hall is not necessrily leading to better social

and relationship skills, the proxy may overestimate

this.

Dig money/room and board Vineborough Regenerative initiaive 2014 £1,815.46 Year Relevant to apprentices, relvealed preference

Wellbeing valuation GVE | Catalysts for community action SROI 2010 £15,500.00 year

Increase from seeing friends and family once or

twice a week to everyday

Improved relationships from adult learning GVE | Fujiwara 2012 £658.00 year Proxy is cost of course only

Counselling course Craft café creative solutions to isolation and lonliness 2011 £510.00 year Same outcome and proxy Proxy uses 12 week session, not 6

Stated preference GVE | sandwell complex needs SROI - £26,600.00 year No date, no indication of the level of intervention

Family counselling Life cycle UK SROI summary report, Karen Bell 2013 £255.00 year Same outcome and proxy This proxy is used for friends and family

Family counselling Arvidson, Battye and Salisbury 2014 £270.00 year Same outcome and proxy This proxy is for community befriending

Physio sesions https://www.nuffieldhealth.com/physiotherapy/faqs 2016 £270.00 per course

Attending Bullion Hall encourages activity, widely

accepted to help with the recovery from injury and

illness.

Fails to give due weight to the associated costs o

the mental/emotional recovery from illness/injury.

Depression owing to long term illness - difference in cost

between those with and without depression GVE | centre for mental health 2012 £296.00 month Probably what participants refer to

Not an accurate calculation as fails to account for

cost of treating non-depressed patients

Average cost of treating stress related physical illness GVE | centre for economic performance 2012 £1,493.00 treatment

Physical illness is not necessarily caused by stress,

more an associated factor

Self-help course/life coaching http://www.judedauntcoaching.co.uk/specifications/ 2016 £45.00 per session

A confidence building course is a suitable proxy for

improved confidence and has been used in other

SROI studies.

This local proxy might not be representative of the

generally accepted cost of such a course.

Confidence course GVE | Coventry's local enterprise and growth initiative 2008 £995.00 year Similar proxy Not a similar activity

Referrers fee, Birtley lifestyle fitnesshttps://www.snapdda.co.uk/clients/Lifestyle/LifestyleSnapDDASig

nup2.aspx?branch=birtley2016 £32.50 per referral

Recruiting a friend or family member to a gym is

analogous to encouraging one to attend Bullion Hall,

as something with which the individual associates

value.

Different local organisations/fitness centres have

different prices and rewards for referrals.

Street partyhttp://www.tesco.com/direct/party-gifts-

flowers/bunting/cat19280016.cat2016 £10.00

food and

bunting

Does not seek to overestimate the value of a sense

of community, but suggests another activity that

could provide a similar outcome.

Significantly underestimates the cost of community

time to organise.

Cost of 2 state funded community workers Life cycle UK SROI summary report, Karen Bell 2013 £56,922.00 Year Recognises high value of difference Potentially overestimates the community developed

Value of living in a good neighbourhood Vineborough Regenerative initiaive 2014 £1,747.00 Year Recognises high value of difference Potentially overestimates the community developed

Feeling part of the community GVE | Guide to economic analysis living well in the west midlands 2010 £11,600.00 year Not the same as developing community spirit

Adult learning course/college course Newcastle City Learning 2016 £70.00 per courseAdult learning courses are frequently used as a

proxy for adult learning in other SROI calculations.

Lifelong learning has associated benefits, e.g.

employability, not captured in this proxy.

Access to learning GVE | DfT 2003 £370.00 year 2003, outcome not similar

Adult learning course GVE | social impact of housing providers 2013 £754.00 course Work related, not the same outcome

Willingness to pay GVE | BIS 2012 £897.00 course To facilitate progresion in to FE

Enterpeneurship course https://www.lynda.com/promo/trial/default.aspx?lpk35=8006 2016 £14.95 per month

A local course offering entrepenuership is a more

suitable proxy than an MBA which would provide

more content at more cost.

£0.00

Cost of setting and achieving business objective course GVE | social value lab \ £510.00 \ No year and no timescale

Minimum wage 2016 -£6.70 per hoursVolunteering is captured at a cost that could be met

by employing an individual.

Volunteers of the requisite skills to deliver the

courses could demand a higher market rate.

Volunteering GVE | wellbeing and civil society 2011 -£13,500.00 volunteer Does not recoginse limited time for volunteers

Volunteering regularly GVE | social impact of housing providers 2013 -£11,800.00 year Does not recoginse limited time for volunteers

Volunteering once a week, minimum wage GVE | improving service for young people 2010 -£250.64 year one hour per week is likely to be the norm



Lifelong learning

Entrepenuership

Volunteering

Improved confidence

Encouraging others

Sense of community

III March 2017: SROI of Bullion Hall 2015/16

Appendix II – Impact Map

Stage 1

Stakeholders Inputs Value Outputs Outcomes Indicator

Quantity (estimated

number of cases

with outcome)

Duration

(years)Financial proxy Proxy value Source Deadweight/attribution Impact

Who does Bullion Hall have

an effect on? Who has an

effect on Bullion Hall?

What is invested? Summary of activities What changes?How did we measure the

change?

How many people

changed?

How long

does the

change

last?

What proxy was used to value the

change?

Where did the information about the proxy

come from?

What would have happened

without the activity/who else

contributed to the change?

Quantity times financial proxy times

duration less deadweight/attirbution

Crafts with the Story ladies

Dot & Zooky

Dry Ice Experience

Foam making fun

Frictionless Fun

Helen

Mr Fox String Puppet

Pop up Plate Puppets

A Box of tricks

Ballet

British Horological Institute North East Branch

Bumble Bag Puppet

Caring for pets

Cestria Folk & Acoustic

Cestria Patchwork Quilters

Childrens parties

Classification

Confidence Building for Men

Confidence building for women

Musical Minis

Craft Courses

Discos

Drum Circle

Durham County Carers Anonymous

Durham Society of Magicians

Easels Art Group

Folk and Acoustic Night

food chains

Furniture Upcycling

Guitar class

Habitats

Health action groups

How to be a child minder

How to build a computer

KLX Dance

Kumon (Math’s & English

Ladies that Lunch

Ladies Together

Level 1 and 2 in Volunteering

Luncheon Club

Lyengar Yoga

Men’s Cree

Messy Play

Monday Morning Job Club

Nail Art

Northern Karate

Palm Reading, Auras and Angel

Parent & Toddlers

Parish Toddlers

Pilates Exercise Class

Probation Service

Rainforests

Savvy Saving

School holiday activities

Senior Carpet Bowls

Bullion Hall Staff Time (salary) £48,412.00 Sewing Class

Bullion Hall Venue (running costs) £35,752.00 Spiritual and Psychic Development Group

Volunteers Time (hours volunteered x minimum wage) £8,361.60 Swing Fit

Tutors Time (attendance fees/subs) £84,864.00 Tea Dance

£177,389.60 £835,874.52

£1,353.03

Stage 2 Stage 3 Stage 4

£1,747.00http://www.tesco.com/direct/party-gifts-

flowers/bunting/cat19280016.cat0% £219,023.89

61 0.75 Adult learning course/college course £210.00 Vineborough Regenerative initiaive 86%

Referrers fee, Birtley lifestyle fitness0.7526

125 1Value of living in a good

neighbourhood

0% £41,878.29

£418.2834%

https://www.snapdda.co.uk/clients/Lifestyle/

LifestyleSnapDDASignup2.aspx?branch=b

irtley

£32.50

30

155 0.75 Self-help course/life coaching £360.00http://www.judedauntcoaching.co.uk/specifi

cations/

50% £3,010.50https://www.nuffieldhealth.com/physiotherap

y/faqs£270.00Physio sesions0.75

1 Relate counselling course £300.00 http://www.relatenortheast.org.uk/page/faq 0% £13,934.29

£2,849.280%

https://www.uswitch.com/broadband/packa

ges/ &

http://www.pcworld.co.uk/gbuk/computing/d

esktop-pcs/desktop-

£508.80Cost of equipment/training

course/monthly internet cost0.75

0.75 Gym membership £342.00http://my.northtyneside.gov.uk/category/251/

memberships50% £15,518.25

0.75 Restaurant meal £312.00http://www.workgateways.com/working-in-

the-uk/cost-of-living0% £9,569.49

0.75 Club membership/social worker £2,223.00

http://neweconomymanchester.com/our-

work/research-evaluation-cost-benefit-

analysis/cost-benefit-analysis/unit-cost-

database

0% £133,237.09

https://counselling-

newcastle.co.uk/faqs/what-are-your-fees/10% £355,191.94

0.75 Part time work, 5 hours £2,563.60Life cycle UK SROI summary report, Karen

Bell0% £39,387.88

Self reported (case study)



169

20

80

41

121

7

46

Lifelong learning










IT knowledge/digital

participation

Family/relationship

improvement


Improved confidence

Encouraging others

Sense of community

0% £502.32

Participants Time (Free) £0.00


Sense of

purpose/achievement

Socialising/tackling

isolation


Exercise

Entrepenuership Self reported (case study) 4 1 Enterpeneurship course £134.55https://www.lynda.com/promo/trial/default.as

px?lpk35=8006

1 Counselling course £2,340.00

IV March 2017: SROI of Bullion Hall 2015/16

Appendix III – Approaches for valuing outcomes The selected technique of equivalent value is used in Section 4. In this Appendix the different

valuation approaches that were explored are discussed.

The use of equivalent value inhibits the authors’ ability to bias the valuation of each outcome. This

is achieved since restricting values to only those which can be demonstrated using a credible

alternative activity prohibits unjustified valuations. It places no judgement on the ability of a

participant to afford such an alternative, adding to its authenticity as an approach.

Contingent value is the self-reported value placed on an outcome by a participant. This technique

can be effectively used where no credible equivalent value can be found. Owing to a lack of

evidence in our case study interviews, and the excessively subjective nature of the technique, we

chose not to pursue contingent values in this analysis.

By considering the explicit and implicit prices of the constituent parts of participants’ attendance at

an activity at Bullion Hall it would be possible to determine proxy values for outcomes through the

revealed preference technique. This would take into account the full cost of attendance for a

participant, including potential childcare costs is none were available, transport costs were

necessary, clothing or equipment costs and a valuation of the time spend on the activity. This

technique is more objective that contingent value, however routinely undervalues the true worth to

an individual of their outcomes. We discounted this approach owing to a lack of data about the

implicit and explicit costs for individual participants, and the risk of failing to account for value

obtained through an outcome beyond the constituent cost, such as improvements in mental health

or a sense of community.

The travel cost method determines the value an individual places on an outcome by questioning

how far the individual would travel to obtain this outcome. The value is obtained by determining

the cost of travelling that distance. For our study this could have been a valid approach in that there

is a level of objectivity to the measurement, however we rejected this approach on the basis that to

gather enough data to make a robust valuation of each outcome was beyond the resources

available, and would likely not prove a significant improvement over the equivalent value technique

we have adopted.

National surveys, such as the Family Resources Survey (FRS), can provide data about average

household spending. This can be used as a proxy for the value of certain outcomes, for example

annual weekly spend on food or entertainment. Although this is an objective measure for many

everyday activities, for example participating in sports, the outcomes identified by participants at

Bullion Hall would make finding a suitable proxy challenging, for example tackling social isolation.

Bullion Community Resource Centre The Operational...

Documents

Transcript of Bullion Community Resource Centre The Operational...