Needs Assessment: Surveys
description
Transcript of Needs Assessment: Surveys
Needs Assessment:Needs Assessment:Surveys Surveys
Dale A. Moore, DVM, PhD
School of Vet Med, UC Davis
ObjectivesObjectives
Describe the needs for needs assessments
Describe one way to conduct a needs assessment
Provide some examples
Needs For What?Needs For What? Program topics Location Delivery Method Etc…
A need is a discrepancy between what the desired goal or standard is with what actually exists.
Program DevelopmentProgram Development Identification of the Main Themes*** Description of the Program Purpose Conduct Needs AssessmentConduct Needs Assessment*** ID others interested in development &
delivery of the program*** Outline learning objectives*** Select format and teaching methodologies*** Select faculty / facilities*** Evaluation tool*** Marketing and promotion strategy*** / budget
Nancy Bennett, Dev CPE Programs, 1990
Traditional CETraditional CE Lecture-dominated format Little evidence of impact on behavior or
outcomes Minimal collaboration between learners and
providers Lack of timely response to learner needs Emphasis on credit Focus on course production driven by
enrollment economy
Don Moore, An approach to needs assessment in Physician practices. Alliance for CME, 1998
New Paradigm in CENew Paradigm in CE Increased emphasis on learningData-based educational activitiesCollaborative learning systemsBlending of quality mgmt and CMEFocus on improving patient outcomesCME as integral part of healthcare
systemDon Moore, An approach to needs assessment in
Physician practices. Alliance for CME, 1998
The Need For Needs The Need For Needs AssessmentAssessment ID learning objectivesData-driven education ID the standards of practice/knowledge ID the goals for outcomes &
stakeholder expectations ID the role of the educational program
in the practice of XXXXX
Needs Assessment for Needs Assessment for Better Evaluations!Better Evaluations!
1
11 1
Needs Assessment: Which Way to Go?Needs Assessment: Which Way to Go?
Different Levels of NeedsDifferent Levels of Needs Societal needs / client /
industry needs and expectations
Community needs Profession’s needs /
professional sector’s needs Farm’s needs Individual’s needs
Needs Assessment MethodsNeeds Assessment Methods Environmental Scans
Review of literature Discussion with experts
SurveySurvey Mail surveyMail survey Telephone surveyTelephone survey Face-to-face interviewsFace-to-face interviews
Focus Groups Clinical audits / Practice audits Self-assessments
SurveysSurveysAdvantage of providing quantifiable
responses from randomly-selected study participants
Usually require that the response choices are known by the investigator in advance
Issues of response-rates
List Frame for Surveys: WHO?List Frame for Surveys: WHO?
Your own/other database of previous participants
Company mailing listsOthers?
Selection From the ListframeSelection From the Listframe How to randomly select?
Random number generator in Excel Tools/Data Analysis/Random Number
Generator/OK/Variables…… Sample size? EpiCalc 2000
http://www.brixtonhealth.com/epicalc.html
Designing Your Survey and Designing Your Survey and Survey QuestionsSurvey Questions
Fowler, FJ. (1995). Improving Survey Questions: Design and Evaluation. Thousand Oaks: Sage Publications.
Dillman, DA. (2000). Mail and Internet Surveys: The Tailored Design Method. John Wiley and Sons: New York.
Online SurveysOnline SurveysANR Site Builderhttp://groups.ucanr.org/help/index.cfmThe mechanicsmechanics of putting a survey
together on-linehttp://www.surveymonkey.com/
Improving Response RatesImproving Response RatesFollow Dillman’s MethodKeep your promises to your
constituents Incentives – CASHMultiple contactsRespondent-friendly questionnaires
Improving Response RatesImproving Response RatesThe number of contacts Advance
letters, postcards, follow-up, additional copies of questionnaires, telephone calls (5)
Salience of the topic Relevance. Common reason for non-response is that survey doesn't mean anything to the person.
Improving Response RatesImproving Response RatesGovernment sponsorship: higher
response rates than private organizations.
Special third contact Follow-up of advance letter / initial follow-up with special mailing procedures: certified mail or special delivery, personal or telephone contact.
Incentive on first contact: Incentives included with first mailing increase response rate. Incentives offered later not effective. ($5 cash)
What does good questionnaireWhat does good questionnairedesign involve?design involve? • IDENTIFY MEASUREMENT OBJECTIVES! • Draft questions based on objectives • Test questions – Expert review – Cognitive interviewing – Pilot run • Format of questionnaire
Respondent-Friendly Q’airesRespondent-Friendly Q’aires
Not too longAsk most important questions first;
demographic information lastUnambiguous questions/responsesEasy to completeGreat directions -- subheadings
Critiquing/Drafting QuestionsConsistently understood (avoid
ambiguous words, define key terms)Avoid “double-barreling” "(asking two
questions at once) “Would you like to be rich and famous?”
Avoid hidden assumptions Consistently administered and
communicated – interviewers read all response options
Some Help With Questionshttp://www.statpac.com/surveys/
surveys.dochttp://www.leeds.ac.uk/iss/
documentation/top/top2.pdfhttp://home.ubalt.edu/ntsbarsh/stat-
data/Surveys.htm#rsiMany others and books, like Dillman
9 rules for writing good 9 rules for writing good questionsquestions
1. Remember your survey's purpose
2. If in doubt, throw it out3. Keep your questions simple
Example:Example: "Imagine a situation where the
production supervisor is away from the line, a series of defective parts is being manufactured, and you just heard that a new client requires ten thousand of these parts in order to make their production schedule. How empowered do you feel by your organization to stop the line and make the repairs to the manufacturing equipment?"
4. Stay focused - avoid vague issues"When did you last see a movie?"
5. If a question can be misinterpreted, it will be"What time do you normally eat dinner?"
6. Include only one topic per question
7. Avoid leading questions "Most doctors believe that exercise is good for you. Do you agree?"
8. Consider alternate ways to ask sensitive questions Income, drug or alcohol consumption and sexual habits.
9. Make sure the respondent has enough information
5 rules for obtaining usable answers 1. Response options mutually exclusive and
exhaustive. 2. Keep open-ended questions to a
minimum they pose problems in terms of coding and analysis.
3. People interpret things differently, particularly when it comes to time Trouble-spots include responses such as "Always," "Sometimes" and "Never." Build in a temporal frame of reference. "I am going to read a list of publications. For each one, please tell me whether you read it regularly. By regularly I mean, at least three out of every four issues."
4. Consider a "Don't Know" response Allow people to say they do not have an opinion.
5. Provide a meaningful scale End points must be anchored with labels. “Rate your satisfaction where 1 means 'Very Satisfied' and 5 means 'Very Dissatisfied.‘ Odd number of points provides middle alternative. Gain nothing by having scale with > 7 points.
Survey LengthSurvey Length30-40 questionsFewer the betterBooklet form?
Survey FlowSurvey Flow
Pilot-TestingPilot-TestingWith peersWith a small group of your audienceHelps refine the tool
Surveys for Needs AssessmentSurveys for Needs Assessment
What are your outcomes?What is your audience?Sampling frame, sample size, how to
select randomlySurvey distributionQuestionnaire designData analysis