Consumer Quality Index for Shelter Consumer Quality Index for Shelter ServicesServices

Measuring client experiences with shelter services Measuring client experiences with shelter services

for homeless adults, homeless youth and battered women for homeless adults, homeless youth and battered women

in the Netherlandsin the Netherlands

Judith WolfJudith WolfMariëlle BeijersbergenMariëlle Beijersbergen

Jolanda AsmoredjoJolanda AsmoredjoMilou ChristiansMilou Christians

Research Centre for Social Care (Omz)Research Centre for Social Care (Omz)

1. Background

2. Pilot survey

3. Psychometric analyses

4. Multilevel analyses

5. Adjusted instrument

6. Future developments

Content

What is a Consumer Quality Index?

• Standardised method to measure client experiences with health care.

• Strict guidelines for development, use of instrument, data analyses and reporting (developed and monitored by Centre for Client Experience with Care - CKZ).

• Two types of CQ-questionnaires: experience and importance

• Measurements can be used for internal quality control and external accountability purposes.

• Information for clients, shelter organisations, government, etc.

1. Background

Importance of a CQI for shelter organisations

• Little known about how clients experience shelter services

• Little known about what clients find important about care

• Sector uses variety of instruments:• developed for other target groups• no standardised instruments (reliability, validity, etc)• no comparisons possible

Necessities: • Standardised instrument

• Possibility for benchmarking

1. Background

Preparatory research pilot CQI Shelter services Sources:

• Literature search

• Expert interviews

• Focus group discussions with stakeholders (clients, workers)

• Concept mapping method (clients, workers)

Process:

• Pre-pilot & pilot survey

• Monitored by Centre of Client experiences with Care (CKZ) &

advisory council

1. Background

2. Pilot survey

• 3 target groups: battered women, homeless adults and homeless youth (20 shelter facilities per target group)

• 2 versions of CQI-SS: ambulant (67) & residential (78)

• Pilot questionnaire: 42-52 items on clients’ experience: - Contact with worker

- Appropriate care & counselling - Information - Children

- Received care - Outcomes of care - Living conditions (residential version)

• Standard CQI answer categories: 1=never - 4=always

2. Pilot survey

Response pilot questionnaire

• 848 744 questionnaires to be used (+ 116 retest)

• Battered women 34%

• Homeless adults 37%

• Homeless youth 29%

• 12 respondents per shelter

3. Psychometric analyses

Factory analyses

Table: Cronbach’s alphas per CQI scale

Scale (nr. items) Total (N)

Hom. adults

(N)

Bat. women

(N)

Hom. youth

(N)

Client-worker interaction (4)

.84 (693)

.85 (269)

.81 (246)

.86 (215)

Received care (10)

.90 (693)

.90 (255)

.89 (231)

.91 (207)

Outcomes of care (5)

.87 (709)

.87 (264)

.87 (237)

.86 (208)

Living conditions (residential) (6)

.77 (499)

.79 (207)

.72 (183)

.77 (109)

• 3-4 reliable scales (Cronbach alphas .72 - .91)

3. Psychometric analyses

Scale Total (NRange=114-115)

Homeless adults

(NRange=57-58)

Battered women

(NRange=57-58)

Client interaction .67 .62 .71

Received care .67 .69 .66

Outcomes .52 .59 .47

Living conditions (residential)

.62 .54 .69

Retest stability

Table : Pearson’s Correlations Test-retest: sufficiently reliable over time

4. Multilevel analyses

Table: Intraclass correlations (ICC’s)

Scale N clients (units)

ICC ICC controlled*

Client-worker interaction

Homeless adults 266 (18) .23** .17**

Battered women 231 (24) .03 .03

Homeless youth 216 (18) .15** .16**

Received care

Homeless adults 266 (18) .37** .31**

Battered women 231 (24) 0 0

Homeless youth 216 (18) .23** .22**

Results of care

Homeless adults 266 (18) .20** .15**

Battered women 231 (24) .05 .02

Homeless youth 216 (18) .07* .06

Living conditions (residential)

Homeless adults 213 (14) .35** .20**

Battered women 193 (20) .11* .11*

Homeless youth 117 (10) .16* .15*

• Controlled for client background characteristics influencing CQI score

Content CQI-SS

• Background questions

• (Living conditions in shelter)

• Client-worker interaction

• Received care

• Results of care

• Overall opinion of facility (grade 0-10)

• Final questions

5. Adjusted instrument

Scale Number of items

Example

Client-worker interaction

4 Does the employee take you seriously?

Received care 10 Do you receive as much assistance as you need?

Results of care 5 As a result of the assistance, are you able to make better decisions about your life?

Living conditions (residential)

6 Is it clean in the facility?

Separate items 13 - Do you receive advice about your safety?

- Have any agreements been made with you about the supervision of your children?

- How do you rate this facility? 0 means: really poor. 10 means: excellent

Table : CQI-SS: Scales & items measuring cliënt experiences

5. Adjusted instrument

• CQI-SS will be publicly available in 2011, on website CKZ

• CKZ will provide Quality mark of CQI if measurements are done with approved CQI-instrument by certified research organisations.

• Building up benchmark dataset: comparing within and between organisations (over time)

CQI-Shelter Services from 2011

6. Future developments

Conclusion

• CQI-SS measures client experiences from the perspectives of the clients.

• Widely usable instrument for homeless adults, homeless youth and battered women

• Good psychometric properties: reliable and with high discriminative power (and valid – results not presented)

Thank you for your attentionThank you for your attention

ContactContact

Judith Wolf Judith Wolf

j.wolf@elg.umcn.nl