To Do, Week of 11.26• Video devotional – courtesy of Dr. Ebersole • Turn in Chapter 11 Quiz (folder up front)• Pick up abstracts & quizzes for chapters 9 & 10 (AFTER CLASS – Folders
up front)• In-class assessment quiz, up front• In-class, discuss surveys• Bb discussion folders – finish strong!• Wed – discuss sampling, wrap-up surveys, another in-class assessment• NOTE: what’s left to work on for class? GROUP WORK!• Group paper extension – Due Dec. 7th, 5 pm, my office box (was due, Dec.
5th). I hope this helps.• No other assignment dates extended• If you are interested in presenting something at NCA next year in San Diego,
come see me and I’ll help you o
PROCESS OF QUANTITATIVE RESEARCH
PROBLEM
METHODOLOGY
THEORY VARIABLESRAISE A QUESTION
WRITE A HYPOTHESIS
POSE A RELATIONSHIP
OPERATIONALIZE
LITERATURE REVIEW
STATISTICS DATE COLLECTION
CODING / INPUT
ANALYSIS OF DATA / INTERPRETATION
REPORTWRITING
SAMPLINGPOPULATION / SAMPLE
MEASUREMENT SCALES
METHODOLOGY / DESIGN
Survey Examples
• Passion of the Christhttp://www.edcomresearch.com/passion.html
• Psychological sense of Community: http://campus.arbor.edu/surveys/cccusurvey.htm.
• Ideal Professor: http://campus.arbor.edu/surveys/idealprofessor.htm
• Left Behind: http://campus.arbor.edu/surveys/com_survey3.htm
Research QuestionHypotheses
A WORKING MODEL OF COMMUNICATION RESEARCH
Narrow focus
Identify topic
STEP 1:CONCEPTUALIZATION
Review of Literature
STEP 2: PLANNING & DESIGNING
Which Method?
STEP 4: ANALYZE& INTERPRET DATA
Report / Write STEP 5:RECONCEPTUALIZATION
STEP 3: SELECTING AMETHODOLOGY
OperationalizatonMeasurementTechniques
Define Key Concepts
Uses of Survey Research Population too large
When descriptivedescriptive information from population is desired• How successful? Satisfied?• What opinions, ideas?
• Who listens/watches/reads/uses? How often?
Want to gather data to build theories; testing predictions
• What beliefs, attitudes, characteristics?
Examples
What is the nature of communication?How is communication related to other things?
Christenson & Peterson (1988) used a survey questionnaire to examine the differences between 239 male & female college students regarding their preference for 26 different types of music
Buddenbaum (1988) described the characteristics of the typical religion journalist from 141 responses received from a questionnaire mailed to all 250 religious newspapers listed in Editor and Publisher Yearbook.
Bergen & Weaver (1988) conducted a systematic random sample of 470 daily newspaper journalists from 1001 U.S. Newspapers to see whether the size of the newspaper as measured by circulation was related to employee job satisfaction.
Public Opinion Polls
Market ResearchReadership surveys
Audience ratings/analysis
Survey Research Design
Basic Questions
How will samplesample be drawn from the
population?
What strategystrategy do we use to reach respondents?
How do we wordword & orderorder the questions
on the survey?
Sampling frame
Sampling Method
Random Non-random
In-person
Phone
Computer
Open/Closed
Measure
External Validity
Length
Format
Cross-sectional
Longitudinal
Self-report Observation
Writing Survey Items
Open-ended questions
Close-ended/forced response
Eg.Why do you listen to CCM Radio? ___________
Eg.What do you like best/least about School/Marriage?________
Eg.(Statement)I listen to CCM radio because they play my favorite artists...(1) Strongly Agree, (2) Agree, (3) Undecided, (4) Disagree, (5) Strongly Disagree
Eg. (Question)Have you ever visited a country outside the Continental U.S.? (1) Yes (2) No
Types of Responses1. Yes and No
Do you watch the television show, “Survivor?” Yes No
Does your car have 4-wheel drive? Yes No Don’t Know
2. Checklist (in which respondent must choose)
Which of the following television news programs do you prefer most for gathering news about national events? (one answer only)
___ ABC ___ NBC ___ C-SPAN ___ CBS ___ CNN ___ Other: ________
Types of Responses3. Rating Scales
RANKRANK ORDER RATING ORDER RATING (put answers in rank order)
Please rank the following five newscasters according to their credibility as newscasters. The top ranked should be assigned the number 1 and the lowest ranked the number 5.
___Dan Rather ___ Tom Brokaw___ Peter Brinkley ___ Bernard Shaw___ Walter Cronkite
Item rated at some point along a continuum or in categories
Please compare Steven Curtis Chapman to other Contemporary Christian Music Artists:
He is better than most He is about the same as most He is not as good as most
Types of Responses3. Rating Scales
NOMINALNOMINAL RATING RATING (put answers into categories)
Item rated at some point along a continuum or in categories
Politically, do you consider yourself (circle one):Liberal Moderate Conservative
Types of Responses3. Rating Scales
INTERVALINTERVAL RATING RATING (put answers along continuum)
Item rated at some point along a continuum or in categories
I think Christians should not listen to secular music...
(1) Strongly Agree (2) Agree (3) Undecided (4) Disagree (5) Disagree Strongly
Likert
Types of Responses3. Rating Scales
SEMANTIC DIFFERENTIAL SEMANTIC DIFFERENTIAL
Item rated at some point along a continuum or in categories
President George W. Bush
reliable __:__:__:__:__:__:__ unreliable uninformed __:__:__:__:__:__:__ informed unqualified __:__:__:__:__:__:__ qualified intelligent __:__:__:__:__:__:__ unintelligent cold __:__:__:__:__:__:__ warm friendly __:__:__:__:__:__:__ unfriendly sinful __:__:__:__:__:__:__ virtuous
7 6 5 4 3 2 1
What do you do?
• With which of the characters on Friends do you most closely identify on a personal level?
___ Chandler ___ Joey ___ Monica___ Phoebe ___ Rachel ___ Ross
How about this?
• I identified with the character of Chandler . . . Chandler . . .
___SA ___A ___ A/D ___D ___SD
• I identified with the character of PhoebePhoebe
___SA ___A ___ A/D ___D ___SD
• I identified with the character of RossRoss
___SA ___A ___ A/D ___D ___SD
ETC
5 4 3 2 1 M=?
1 2 3 4 5
TO DO, 11.28 (and beyond . . .)• Chapter 11 assessment up front (9 questions) – see how much you retained from the
readings & podcast• Remember, Group Research Paper due Dec. 7th, not 5th. Make note of it. Due: by 5:00
pm, my box. No extension on the electronic notebook or anything else. • Extra credit is due on the last day of class (in my box by 2:40 pm).• In class: finish discussion of surveys, discuss sampling (Open up POR- if you don’t
have POR, be prepared to sit next to someone who does)• Monday, Dec. 3rd – meet in prop shop for class (listen to experimental design podcast
before class) (I’ll post it this week)• If you sent me your group project for review/feedback, I’ll return it sometime this
week (on or before Friday). • Wed., Dec. 5th, in class, exam review (instead of Dec. 10th)
– So, what we had planned for Dec. 5th we’ll do on Dec. 10th, and what we had planned for Dec. 10th we’ll do on Dec. 5th Why? I may have jury duty on Monday, Dec. 10, but won’t know until Friday, Dec. 7th. And, I don’t want to chance missing the exam review.
– Our last class, then, will be virtual, on Dec. 10 – podcast plus quiz, the way I handled it for NCA.
– More details will be included in the checklist for next week
Making Communication Knowledge Claims
Producers & Process
Messages Uses/Effects
Processes of message production, transmission, and meaning making
Contexts and LevelsIndividual, dyad, group or societal levels
e.g., News agencies, dyad, organization
Content or form of the communication messages
e.g., family talk, tv content and how it portrays Black/White relations
e.g., anti-smoking ads, why people watch RTV
Functions and effects of messages; opinions, beliefs, attitudes
Survey Research Design
Basic Questions
How will samplesample be drawn from the
population?
What strategystrategy do we use to reach respondents?
How do we wordword & orderorder the questions
on the survey?
Sampling frame
Sampling Method
Random Non-random
In-person
Phone
Computer
Open/Closed
Measure
External Validity
Length
Format
Cross-sectional
Longitudinal
Self-report Observation
Rules for Writing Closed-ended Survey Items
1. Each question should be meaningfulmeaningful to respondents
2. Use standard Englishstandard English
3. Make questions concreteconcrete
4. Avoid biasedbiased words and phrases; check your own
5. Use caution when asking about the personalpersonal
6. Each question should have just one thoughtone thought
7. Use marker variablesmarker variables
Practical Guidelines
1. LengthLength
2. Question OrderQuestion Order
• What do you need to know?
• Who are your respondents? (age, language, professionals, etc.)
• Introduction First set of questions related to Intro
• First questions ask ObjectiveObjective facts SubjectiveSubjective
• Organize (GROUP) according to logical order
• Relatively easy-to-answer & sensitive questions at the end
• Use Transitions Explain groupings as needed
• Watch for response bias
Practical Guidelines
3. Format: AestheticsFormat: Aesthetics
4. Branching Questions (“Skip Patterns”)Branching Questions (“Skip Patterns”)
• Neat, bold, adequate space
• Does your car have 4-wheel drive? _____Yes_____No_____Don’t Know
5. AdministrationAdministration
• Certain topics not relevant to everyone
• Self-administered Questionnaires
• Interviews
6. Pilot TestPilot Test
Which strategy to reach respondents?
• Don’t always “go web”– Advantages and Disadvantages
• It depends on . . . – your population– the length of your survey– the incentive– your time, $, expertise– your question!!!!!!!!
Web surveys
• Less costly, quick, fairly easy to disseminate• Utilize different sensory channels & complete at own pace• More likely to report embarrassing behaviors• Web surveys offer poor coverage of household population
(55%) (plus, digital divide)– Hispanic or Black, poor, poorly educated, elderly, or disabled
• Absence of a good sampling frame (no comprehensive lists)• Response Rate lower• Web surveys take longer than phone surveys to complete
– Score higher on knowledge tests
Survey Research Design
Basic Questions
How will samplesample be drawn from the
population?
What strategystrategy do we use to reach respondents?
How do we wordword & orderorder the questions
on the survey?
Sampling frame
Sampling Method
Random Non-random
In-person
Phone
Computer
Open/Closed
Measure
External Validity
Length
Format
Cross-sectional
Longitudinal
Self-report Observation
SAMPLING
PopulationSample
the summarydescription of a givenvariable in a survey sample.
generalizable
Meaningful?
Survey about Political views, views about the Obama vs. Rudy G? Election Stand-off
Questions tied directly to candidates, political views, process, parties, etc.
What if you introduce other questions that don’t seem to have an obvious purpose to the study?
“We are asking some personal questions so we can look for connections between people’s background and their views...”
“Lastly, I’d like to ask you a few questions about yourself so we can describe our sample and then we’ll be done...”
Standard English (1 of 3)
Conventional grammar
LENGTHPoor: The paucity of psychometric scales with high degrees of stability and construct validity is most bothersome to surveyors when measuring peoples: 1. Economic characteristics 3. Knowledge 2. Feelings 4. Health
Better: The lack of reliable and valid methods causes researchers the most problems when measuring people’s. 1. Economic characteristics 3. Knowledge 2. Feelings 4. Health
Spelling & syntax
+ - ABCSpecialized
words- - Lengthy
items= Accuracy
Standard English (2 of 3)
Conventional grammar
CLARITYPoor: What remedy do you usually use for stomachaches?Better: What brand of medicine do you usually use for stomachaches?
Spelling & syntax
+ - ABCSpecialized
words- - Lengthy
items= Accuracy
Better: Do you usually use tablets or powder for stomachaches?
CLARITYPoor: What are the reasons and motivations that drive you to consume CCM?Poor: For what purposes do you consume or otherwise use CCM?
Better: Why do you listen to 89.3 VIBE FM radio?
Standard English (3 of 3)
Conventional grammar
ABBREVIATIONSPoor: What political party is responsible for the expanding size of the GDP?: 1. Republican 2. Democrat
Spelling & syntax
+ - ABCSpecialized
words- - Lengthy
items= Accuracy
Better: What political party is responsible for the expanding size of the gross domestic product?: 1. Republican 2. Democrat
Concrete Questions
Less concrete: How much do you listen to the radio?More concrete: During a normal (average) day, how many hours do you listen to the radio?
Close to the respondent’s personal experience
Less concrete: Did you enjoy the book?More concrete: Have you recommended the book to anyone else?More concrete: Have you read any other books by the same author?
Biased Words and Phrases
Poor: Do you think the leftist liberals and right-wing conservatives will soon reach a greater degree of understanding?
Better: In your opinion, in the next four years, how is the relationship between the liberals and the conservatives likely to change? (1) Much improvement (4) Much Worsening (2) Some improvement (5) Impossible to Predict (3) Some worsening
Emotionally charged, indicates personal biases, unduly influences
Poor: Are you pro-choice or anti-choice on the abortion issue?
Poor: Would you vote for Dr. Roger Fields, a liberal.
Thin or slender; portly, heavy or fat....
1) What’s your view on the Abortion issue? A. Pro-choice B. Pro-life
2) Many labels are used to describe how someone views the abortion issue (for example, pro-life, anti-choice, pro-choice). Which label best describes your views? ______________________________________________________________.
3) What’s your view on the abortion issue?a. I believe that the mother has the right to abort any time before the first tri-mesterb. I believe that the mother has the right to abort any time after the first tri-mester
but not after the birthing process beginsc. I believe that the mother does not have the right to abort the baby after
conception for any reasond. I believe that the mother does not have the right to abort after conception unless
it has been a case of rape or incest
Asking the personal
Poor: How much do you earn each year? ______________
Better: With which ethnic group do you most closely associate? (1) White/Caucasian (3) Hispanic (5) Pacific Islander
(2) Black/African American (4) Asian (6) Other: _____
Potential source of bias
Poor: Are you single or divorced? ____________
Poor: Are you Black, White, or some other ethnicity? _________
Better: In which category does your annual income last year best fit? (1) Below $10,000 (4) Between $41,000 - $75,000 (2) Between $10,000 - $20,000 (5) Over $75,000 (3) Between $21,000 - $40,000
Better: What is you marital status? (1) Single/never married (2) married (3) divorced (4) widow/er
Just One Thought
Poor: Should the United States cut its military and domestic spending? (1) Yes (2) No
Better: Should the United States substantially reduce its military spending? (1) Yes (2) No
Better: The United States should substantially reduce its military spending? (1) Agree Strongly, (2) Agree (3) Undecided, (4) Disagree, (5) Disagree Strongly
Better: The United States should reduce its military spending? (1) Substantially, (2) Somewhat, (3) Not at all, (4) Undecided
Sample Intro + First Set of Q’s
Hello. I am calling from Spring Arbor University. We are surveying people who live in student housing to find out whether it is a satisfactory place to live. Your name was selected at random from the housing registry, a directory of students who have voluntarily listed their telephone numbers. Our questionnaire will take no more than 4 minutes. You can interrupt me or quit at any time. There are no right or wrong answers. All responses are confidential. May I ask you the questions?
[If yes, continue, If no, say “Thanks” and hang up]The first question asks about your overall satisfaction with your apartment. Do you consider it: (1) Definitely Satisfactory, (2) Probably Satisfactory.....ETC.
Objective to Subjective
Q. You want to know about the success of a communication skills training seminar offered on campus last year...
FIRST How did you first hear about the program?When did you take it?Did you complete the entire seminar?
THENWhat did you like best/least about it?Were you satisfied with the instructor?
Groupings/Transitions
“Now that I’ve asked you to agree or disagree with different planks of the Democratic Party Platform, I’ll ask you 10 questions that ask to agree/disagree with different planks of the Republican Platform”
“We’re over half way through this list of statements now...”
Branching Questions (“Skip”)
#3 Do you watch reality TV shows?
a. No (GO TO QUESTION 4)b. Yes
IF YES, which reality TV shows do you watch? (circle)Spy TV Yes No Fear Factor Yes NoSurvivor Yes No
#3 Do you watch reality TV show?
a. Yes (COMPLETE SECTION A)
b. No (GO TO SECTION B)
DISADVANTAGES? COMPUTERS?
Self-administered Survey Tips
• Pre-notification letter (increases by 50%)• Purpose and Sponsorship• Keep it short, simple (Instructions VERY Clear)• Describe incentives• Offer to send an executive summary of findings• Self-addressed, pre-paid envelope
– First class– Commemorative versus plain stamps
• Color matters (warm colors)
Checklist: Self-Administered Questionnaires
• Pre-letter• short, formal letter to go with Q’aire• offer to send summary of findings• explain why you ask personal questions• keep procedures simple and survey short
– Use bulleted, ordered lists, pre-test on the target population
• consider incentives• follow-up or send reminder
Checklist: Interviews
• Finding interviewers--fit with respondents?
• Training interviewers--FtF, space, manual
• Conducting interviews– introductory statement (who and why, time)– impress with importance of script (e.g.,CCM)– include detailed instructions (e.g., CCM)
• Monitoring quality
Pilot Test/Pre-test
• Will the survey provide the needed info?
• Are certain words/questions redundant? Vague?
• Will data collectors be able to use the forms?
• Are the procedures standardized?
• How consistent is the information obtained?
Guidelines
1. Anticipate actual circumstances2. Choose similar respondents3. Get as many people as possible
Sample Letter of Introduction with Incentive
Greetings, [Wheaton/Taylor/Westmont/Gordon College] students:
Our institution has been selected from a list of all Christian colleges in the country to take part in a research project involving student life on Christian college campuses.
If you click on http://campus.arbor.edu/surveys/cccusurvey.htm it will take you to a web survey that should take less than 10 minutes to complete. After you answer each question, click “submit” at the end of the survey. Be sure to enter your email if you want to receive the results. Once all the surveys have been returned, the results will be sent to you at the address indicated.
At the very end of the survey you can enter your email address in the box provided and be entered into a drawing for one of 5 $50 cash awards. You will be notified by email if your name is drawn.
Thanks for taking a couple of minutes to help with this important project. Thanks for your valuable time!