CBR 106: Developing a Client Satisfaction Survey

39
CBR 106: CBR 106: Developing a Client Developing a Client Satisfaction Survey Satisfaction Survey

description

CBR 106: Developing a Client Satisfaction Survey. Welcome & Introductions. Name & affiliation Experience with client satisfaction surveys What you want to learn today. Workshop Objectives. Upon completion of this workshop, participants will be able to: - PowerPoint PPT Presentation

Transcript of CBR 106: Developing a Client Satisfaction Survey

Page 1: CBR 106:   Developing a Client Satisfaction Survey

CBR 106: CBR 106: Developing a Client Satisfaction SurveyDeveloping a Client Satisfaction Survey

Page 2: CBR 106:   Developing a Client Satisfaction Survey

Welcome & Introductions

Name & affiliation

Experience with client satisfaction surveys

What you want to learn today

Page 3: CBR 106:   Developing a Client Satisfaction Survey

Workshop Objectives

Upon completion of this workshop, participants will be able to:

• Describe the importance and benefits of Client Satisfaction Surveys (CSS)

• List the steps required to develop a CSS tool• Analyze and critique sample CSS questions• Begin the development of their own CSS tool

Page 4: CBR 106:   Developing a Client Satisfaction Survey

Agenda

Definition of termsPurposes and benefits of CSSSteps in the CSS development process

Lunch

Analysis & critique of CSS tools and questionsDevelopment of your own CSS toolWorkshop Evaluation

Page 5: CBR 106:   Developing a Client Satisfaction Survey

Surveys

Knowing what the client wants is the key factor to success in any type of organization or business

Organizations need to know what their members want

The best way to find this information is to conduct a survey

Page 6: CBR 106:   Developing a Client Satisfaction Survey

What is a Client Satisfaction Survey (CSS)?

A relatively simple but effective way to evaluate the delivery and quality of the services your agency provides to clients

Provides you with data that tells you what clients really think about your services and the people that deliver them (rather than what you think they think!).

Page 7: CBR 106:   Developing a Client Satisfaction Survey

What does a CSS do?

Identify trends in client satisfaction Measure results over time Identify new service areas Improve your existing services Provide indicators of agency and staff strengths and areas of improvement Improve accountability

Page 8: CBR 106:   Developing a Client Satisfaction Survey

Benefits of CSS

An opportunity to target areas for (continuous)

improvement

An opportunity to present the full range of services offered to your clients on an annual basis

Page 9: CBR 106:   Developing a Client Satisfaction Survey

Clear Objectives The Clients The Survey Questions The Results Implementing Change

What Makes a Successful CCS?

Page 10: CBR 106:   Developing a Client Satisfaction Survey

The Steps in a CSS Project

Establish the goals of the project Determine your sample Choose interview methodology Design questionnaire Pre-test the questionnaire Administer survey Collect and enter data Analyze data Present data

Page 11: CBR 106:   Developing a Client Satisfaction Survey

Step 1: Establish Goals

Establish the goals of the project ……

What do you want to learn?What will you measure (e.g. overall service, specific

services, program efficiency or responsiveness, etc.) ?

Why do you want to measure this factor?What are the possible benefits?

Try not to mix too many purposes into a single CSS. Keep it focused.

Page 12: CBR 106:   Developing a Client Satisfaction Survey

Step 2: Determine Sample

Determine your sample

Who will you interview? Who is your clientele?Will this group provide you with the necessary data

to meet the goals of the project?

Keep the sample large enough for a sufficient amount of

data but small enough so that it is manageable.

Page 13: CBR 106:   Developing a Client Satisfaction Survey

Step 3: Choose Interview Methodology

Choose an interview methodology

How will you conduct the interview?­ mail out; questionnaires; individual interviews;

case study; focus group; on-line surveys

Make up of your clientele­ Language; accessibility options; age;

education

Page 14: CBR 106:   Developing a Client Satisfaction Survey

Step 4: Design Questionnaire

Create your questionnaire

What you will ask?What do you NEED to know?Will the questions asked meet the goals of the

project?

Page 15: CBR 106:   Developing a Client Satisfaction Survey

TYPES OF QUESTIONS ……

Researchers often use a combination of QUANTITATIVE and QUALITATIVE questions to

assess client satisfaction.

Page 16: CBR 106:   Developing a Client Satisfaction Survey

Quantitative Questions

Aiming for clear answers (that can be quantified)

Closed questions ­ (yes/no, agree/disagree, etc., answers)

Often focused on the descriptors/variables and their relationship­ E.g. ---- # of days in hospital

Page 17: CBR 106:   Developing a Client Satisfaction Survey

The Checklist

Please select which in-patient services you used in the last month?

Individual therapy Group therapy Medication Twelve-step program (NA, CA, AA) Other (please describe)

________________________

Page 18: CBR 106:   Developing a Client Satisfaction Survey

Ranking

Please order which in-patient services you found the most helpful (where 1= most helpful & 5 = least helpful)?

_ Individual therapy_ Group therapy_ Medication_ Twelve-step program (NA, CA, AA)_ Other (please describe)

________________________

Page 19: CBR 106:   Developing a Client Satisfaction Survey

Rating / Likert Scale

Please rate the degree to which you agree/disagree with the following statement:

“I was completely satisfied with the in-patient services I received.”

Strongly agree Agree Disagree Strongly Disagree

Page 20: CBR 106:   Developing a Client Satisfaction Survey

Typically open-ended (e.g. “how” questions)

A way of finding out more in-depth information about what client feels or experienced

Can be time consuming for client to respond to and for CSS administrator to process

Qualitative Questions

Page 21: CBR 106:   Developing a Client Satisfaction Survey

Sample Qualitative Questions

How satisfied were you with the responsiveness of the program staff in meeting your needs?

How might the program might improve its services to better meet your needs?

Page 22: CBR 106:   Developing a Client Satisfaction Survey

Exercise 1 – Evaluating Questions

Working in small groups discuss the differences between questions 1 & 2.

In what ways have our services benefited you?

Can you tell us the ways in which our services have

affected you?

Page 23: CBR 106:   Developing a Client Satisfaction Survey

Exercise 2 – Critiquing Questions

Critique the following questions:

1. How would you rate our organization’s internal capacity

(circle one)? 1-poor 2-fair 3-good 4-great 5-excellent

2. Did the group sessions ever adversely affect you?

Page 24: CBR 106:   Developing a Client Satisfaction Survey

Structuring Your Questions: Things to AvoidThings to Avoid

Avoid technical terms and acronyms, unless the respondents know what they mean

Avoid using an odd-number rating scale, such as a 1-5 scale (e.g. poor, fair, good, great, excellent) as people will tend to choose the middle choice

Avoid biased or loaded questions

Avoid questions that are unclear or can be interpreted in more than one way

Page 25: CBR 106:   Developing a Client Satisfaction Survey

Structuring Your Questions: Do’sDo’s

Ensure the questions are valid

Check that they really measure what they are aiming to measure

Ensure the questions are reliable

Ensure they are understood the same way by different people and that a person answering similar questions will answer similarly

Make sure your questions accept all the possible answers

Depending on the goal of the project, include demographic information

Page 26: CBR 106:   Developing a Client Satisfaction Survey

More Do’sDo’s…

Keep it short. • Long surveys are a turn-off to most people.

Keep it simple. • Limit the number of open-ended questions.

Keep it real. • Use language that is easily understandable to your client.

Make it look good. • Organize your questions logically, use a simple font and

keep it looking neat and tidy.

Page 27: CBR 106:   Developing a Client Satisfaction Survey

Exercise 3 – Designing Questions

In your small group, develop a CCS (on flip chart paper) that measures peoples’ satisfaction with the workshop thus far.

Determine specifically what you want to measure what you want to measure and whywhy..

Include a combination of quantitativequantitative and qualitativequalitative questions (5-8 total).

Keep in mind the question design do’s and don’ts discussed earlier.

Page 28: CBR 106:   Developing a Client Satisfaction Survey

Exercise 3 Continued – The Gallery Walk

Post your group’s CSS on the wall. Silently walk around the room to look at the other groups’ CSS’s.

Choose two (not your group’s) to respond to. Record the questions and your answers. No one will see your responses.

Note similarities and differences between the design of CSS’s and whether and how the CSS design effects your responses.

Page 29: CBR 106:   Developing a Client Satisfaction Survey

Debrief of Exercise 3

What did you notice about the various CSS’s that were created?

What was similar? What was different? Any surprises? What worked well? What could be improved? What did you learn about CSS design from this

activity?

Page 30: CBR 106:   Developing a Client Satisfaction Survey

Step 5: Pre-test the Questionnaire

How to Pre-test?

Test questionnaire with a small number of interviews Test the survey on the same kinds of people you will

include in the main study Why Pre-test?

Pre-testing can reveal unanticipated problems with question wording, instructions, structure, etc

It can help you see if the interviewees understand your questions It helps to determine if the responses meet the objectives

Page 31: CBR 106:   Developing a Client Satisfaction Survey

Step 6: Administer CSS

Distribute the survey and include the following:

A cover letter/introduction explaining the purpose of the CSS and how the information will be used

When it needs to be returned A ‘thank you’ to the client

Page 32: CBR 106:   Developing a Client Satisfaction Survey

Step 7: Collect & Enter Data

Collect and enter data

Expect to collect data 2-3 weeks after distributing

Determine ahead of time (before or while CSS is being developed) data management program (e.g. NVIVO or NuDist for qualitative; SPSS, SAS, Stata for quantative)

Determine ahead of time who will enter data; provide them with adequate training

Page 33: CBR 106:   Developing a Client Satisfaction Survey

Step 8: Analyze Data

Identify themes & patternsFrequencies

­ counts of things, how many times things happen; get mean, median, mode

Descriptives­ percentages or raw numbers of demographics

and other factors that would be nonsensical to take an average of such as gender or ethnicity

Outliers ­ things that fall outside the expected values

Page 34: CBR 106:   Developing a Client Satisfaction Survey

Step 9: Report Results

Produce the reports

Who is the target audience?Identify specific areas for changeIdentify action outcomes of the CSSMake results available to clients

Page 35: CBR 106:   Developing a Client Satisfaction Survey

Exercise 4 – Question Revision

Review your CSS questions with a partner and gather their feedback.

Specifically focus on what extent are the questions:

• Clear and simple?• Valid and reliable?• Unbiased?• Manageable in number?

Page 36: CBR 106:   Developing a Client Satisfaction Survey

Reflection & Next Steps

Reflect on and record 1-3 things you learned today.

Record 1-3 related goals you have going forward.

Record 1-3 related next steps you plan to take upon leaving the workshop.

Page 37: CBR 106:   Developing a Client Satisfaction Survey

Workshop Objectives

Having completed this workshop you should now be able to:

• Describe the importance and benefits of Client Satisfaction Surveys (CSS)

• List the steps required to develop a CSS tool

• Analyze and critique sample CSS questions• Begin the development of their own CSS

tool

Page 38: CBR 106:   Developing a Client Satisfaction Survey

Workshop Evaluation

Your feedback is extremely important! Please complete the workshop evaluation….

Thank you!

Page 39: CBR 106:   Developing a Client Satisfaction Survey

CBR 106: CBR 106: Developing a Client Satisfaction SurveyDeveloping a Client Satisfaction Survey