Measuring social inclusion:

How are we faring?

Hagiliassis, N., Wilson, E., Campain, R.,

McGillivray, J., Caldwell, M., Graffam, J.,

Bink, M., Koritsas, S.

‘Research to Action’ CADR Conference 2014

What Are The Outcomes Sought Under The NDIS?

• “An important component of the evidence base under the

NDIS will be the systematic collection of data on

outcomes of particular services or interventions for

people with disabilities. Data on outcomes in

employment, education, social participation, and

capacity for self-care, and on the measures that

contributed to those outcomes, would help to build an

evidence base for analysing which interventions or forms

of assistance are more effective, and why”.

p 576, Productivity Commission Report, Disability Care and Support, 2011

Developing Outcome Measures: Guide Principles

Person Centred

– Prioritises self-report by people with disability (whilst acknowledging a need for

proxy reporting in some instances);

– Vantages perspective of person with disability.

Accurate

– Areas of measurement well defined;

– Operationalised through observable, behavioural indicators;

– Reliable, valid, sensitive to change.

Meaningful

– “The point is not to collect information - it is to learn enough about how things are

going so we can keep going or change direction with confidence. We want

monitoring which leads to action to improved… [practice], and to have a positive

impact on people’s lives” (Ritchie et al, 2001: 177).

Developing Outcome Measures: Guide Principles

• Achievable

– Brief and easy to use;

– Considers time, workload, training, data analysis allocations;

– Data can be analysed at the individual, service provider, disability system and

societal level.

• Accessible

– Instrument can be used (with and without support by others) by people with a

wide range of disabilities;

– Easy language version; visual scaling; low physical administration demands;

accessible response format (e.g., ‘yes’ or ‘no’);

The 1 in 4 Social Inclusion Survey

• Developed in the context of a national ARC funded research project

(Deakin University and Scope), The 1 in 4 project.

• Accessible, self-report survey designed to capture the experiences

of people with disabilities in the area of social inclusion.

• Project establishes a data set that will provide regular and reliable

information from and about people with a disability in Australia.

• Allows for benchmarking of inclusion status; data source for

monitoring social inclusion status at the population level (e.g.,

impact of policies, resources, programs on inclusion outcomes).

• It does so from the perspective of people with disabilities (affirming

the agency and contribution of PWD as citizens)

The 1 in 4 Social Inclusion Survey

• Online survey

• Content developed following extensive literature review

of existing theoretical models, frameworks, instruments:

– e.g., Social Inclusion Framework (Saunders et. al., 2007 & 2008)

– Selected, adapt, refined, reduced items in range of published

sources (over 80 items identified in multiple studies

• Easy English and Standard English versions

• On-line survey custom built (ie bespoke) to meet highest

number of accessibility standards (Gottliebsen et al., 2010).

The 1 in 4 Social Inclusion Survey Structure

Demographic and general questions

Personal wellbeing questions (8 items)

Social inclusion questions (24 items)

Improvements to social inclusion

(select 3 from list of 17 items)

Suggestions for future surveys (1 item –

open ended)

Evaluation of accessibility of survey

(1 item)

Descriptive Data

• 761 people with disability completed the survey

• Women more represented than men (60% female)

• 39.7% of respondents were in paid employment

• Main income source was disability support pension (55.1%)

• Many respondents were actively engaged in voluntary work (39.4%)

• Most respondents found the poll easy to complete (user friendliness

rating 8.5/10)

• 67% of respondents completed the poll themselves, 16% with

assistance and 17% through a proxy.

Descriptive Data

Disability Type %

Physical 24.3

Neurological 17.5

Intellectual 15.2

Chronic illness 10.8

Sensory 8.9

Psychiatric 5.5

Autism 5.4

ABI 3.5

Other 2.8

Specific learning/ADD 2.2

Speech .8

Descriptive Data

Age %

18 - 19 3.8

20 - 24 9.7

25 - 34 20.9

35 - 44 18.9

45 - 54 21.9

55 - 59 8.5

60 - 64 7.6

65+ 6.2

Descriptive Data

State %

Vic 56.5

NSW 14.2

SA 8.7

Qld 7.9

WA 7.2

ACT 2.8

Tas 1.6

NT .3

Personal Wellbeing Index items - Easy English

(examples)

Personal Wellbeing Findings

Domains People with disabilities

(1 in 4 Poll)

N=604

General population

(Australian Unity)

N=667

Satisfaction with: … Mean Mean

Life as a whole 59.07 77.39

Standard of living 62.56 78.90

Health 47.99 73.86

Achieving in life 56.64 72.43

Personal relationships 61.44 81.09

Personal safety 67.09 81.08

Community connectedness 54.29 72.54

Future security 48.61 71.15

PWI 57.08 76.01

PWI by disability type compared with general population

n M

General population (Australian Unity)

667 76.01

People with disabilities (1 in 4 Poll)

intellectual 55 69.22

sensory 58 66.60

autism 24 61.19

physical 168 58.73

neurological 126 56.43

psychiatric 33 46.02

chronic illness 76 41.71

The 1 in 4 Social Inclusion Survey – Easy English

(example items)

Social Inclusion Domains (based on factor analysis) with

example items

Social Inclusion

Social Participation Having social contact with other people

Participating in arts & cultural activities

Going to cafes, bars or pubs

Supportive and Valuing Societal Relationships

Being accepted by others for who you are

Feeling valued by society

Getting help from family and friends when you need it

Services Access Accessing disability support services

Accessing mental health services

Getting help from services when you need it

Economic/Materials Access

Keeping up payments for water, electricity, gas or phone

Having enough money to get by on

Having decent and dependable home/housing

Social Inclusion Domains

5.1382

5.8764

5.404

6.7362

Social Participation Social Relationships Services Access Economic Materials

Social Inclusion Levels: Domains 10 - Needs Fully Met

0 - No Needs Met

Proportions of respondents who reported ‘little or no’ needs

met in specific areas of social inclusion

Item %

Participating in sport or recreational activities 41.2%

Getting held from services when needed 32.6%

Participating in arts and cultural activities 32.5%

Accessing disability support services 29.8%

Feeling valued by society 27.1%

Going to cafes, bars or pubs 26.8%

Accessing mental health services 26.7%

Having a social life 23.2%

Accessing government support services 22.8%

Participating in some kind of social activity 20.8%

Having access to support in times of crisis 19.4%

Accessing dental services 18.1%

Top 5 areas to target for improvements in social inclusion

1. Attitudes of others 39%

2. Health services 28%

3. Physical access 26%

4. Money 25%

5. Transport 21%

Overall Social Inclusion Score (by primary disability)

(groups n<25 not included)

Primary disability N Mean

Intellectual 99 6.38

Sensory 56 6.34

ABI 25 6.33

Neurological 115 5.94

Physical 170 5.83

Autism 32 5.35

Psychiatric 41 5.03

Chronic illness 72 4.30

Research into action: How are we faring?

• Key findings:

– The results of the poll show that social inclusion

needs are inadequately met for most people with a

disability.

– In comparison to a general population sample,

people with disabilities rated significantly lower on

GLS and on each of the seven PWI domains.

– Area of most concern is attitudes of others

(consistent with previous research e.g., Shut Out,

2009) as well as health, physical access, money and

transport.

Research into Action: Implications for practice

• The Social Inclusion Measure:

– Measures the experience of people with disabilities directly;

affirms the agency and contribution of PWD as citizens.

– Information to better target actions towards social inclusion

change.

– Initial validity/reliability data (statistical analysis).

– Application for benchmarking, analysis at the individual, service

provider, disability system and societal level.

– Subject to larger scale psychometric study to further establish

tool’s credentials as a valid, reliable, sensitive measure of social

inclusion status (‘Scope Outcomes Project’).

Research into Action: Implications for practice

• Development of/sector access to instruments that are:

– Sufficiently broad so as to reflect the diversity of people with

disabilities and the communities in which they live in.

– Measure outcomes in ways that are person-centred, accurate,

meaningful and achievable.

• Research to understand the factors that mediate social

inclusion outcomes for different disability groups for

more targeted efforts.

• Building organisational capacity in moving to an

outcomes measurement approach/culture.

Research into Action: Implications for policy

• Reaffirms an important policy objective for all Australian

governments: the need to promote the participation and inclusion of

people with disabilities in the community

• Adoption of a framework to measure the social inclusion of

Australians with a disability that recognises four distinct

characteristics of inclusion:

– Social Participation,

– Supportive and Valuing Societal Relationships,

– Services Access,

– Economic/Materials Access

• Need for targeted actions towards social inclusion change e.g.,

strategies that address pervasive negative attitudes towards people

with a disability, particularly given such attitudes are seen as the

biggest single barrier to social inclusion

Contact

Dr Nick Hagiliassis

Manager Research and Practice Enhancement

Scope

nhagiliassis@scopevic.org.au

References

Gottliebsen, D., Layton, N., & Wilson, E. (2010). Comparative Effectiveness Report -

online surveys. Disability and Rehabilitation: Assistive Technology, 5, 401–410

National Disability Services (2012). Measuring Outcomes for People With Disability.

Retrieved www.nds.org.au/asset/view_document/979321224

National People with Disabilities and Carer Council (2009). Shut out: The experience of

people with disabilities and their families in Australia. Canberra: Commonwealth of

Australia

Productivity Commission (2011), Disability Care and Support, Report no. 54, Canberra.

Ritchie, P., Sanderson, H., Kilbane, J. and Routledge, M. (2001) “Staying on Track’ in

People, Plans and Practicalities: Achieving Change, Scottish Human Services Trust,

Edinburgh.

Saunders, P., Naidoo, Y. & Griffiths, M. (2008). Left out and missing out: Disability and

disadvantage. Social Policy Research Centre, Sydney, Australia.

Saunders, P., Naidoo, Y. & Griffiths, M. (2007). Deprivation & social exclusion in

Australia. Social Policy Research Centre, Sydney, Australia.

.