Measurement Measurement and and

ObservationObservation

Choices During Choices During OperationalizationOperationalization

Researchers make a number of key Researchers make a number of key decisions when deciding how to measure a decisions when deciding how to measure a conceptconcept Dimensions and sub-dimensionsDimensions and sub-dimensions Range of variation within dimensionsRange of variation within dimensions

Categories to represent rangeCategories to represent range Levels of measurementLevels of measurement

Nominal, ordinal, interval, ratioNominal, ordinal, interval, ratio

Operationalization: Operationalization: A deliberative processA deliberative process

Not a simple, linear processNot a simple, linear process Complicated and fraught with trade-offsComplicated and fraught with trade-offs Iterative process with cycles of considerationIterative process with cycles of consideration

Debate over proper measurement is keyDebate over proper measurement is key

Dimensions of the Dimensions of the ConceptConcept

Creating operational measures forces Creating operational measures forces realization about lack of conceptual clarityrealization about lack of conceptual clarity

List of possible dimensions may be longList of possible dimensions may be long Need to decide which ones are most relevantNeed to decide which ones are most relevant Ask which ones are central to the inquiryAsk which ones are central to the inquiry Reflect on research hypotheses or theoriesReflect on research hypotheses or theories

Range of VariationRange of Variation

Sense of the upper and lower limitsSense of the upper and lower limits How much are you willing to combine How much are you willing to combine

different people into the same category?different people into the same category? Extremely high and Extremely low may be Extremely high and Extremely low may be

collapsedcollapsed Eg. Income, age, height, etc.Eg. Income, age, height, etc.

Opposition and support for attitudesOpposition and support for attitudes Agreement and disagreementAgreement and disagreement

Variation Between Variation Between ExtremesExtremes

Degree of precisionDegree of precision How detailed you need to be in measurementHow detailed you need to be in measurement Eg. Age breaks or Exact age?Eg. Age breaks or Exact age?

Related to purpose of studyRelated to purpose of study Eg. Political Party ID: Eg. Political Party ID:

Dichotomy: Democrat or RepublicanDichotomy: Democrat or Republican Continuum: 7-point scale w/ “independent-Continuum: 7-point scale w/ “independent-

leaner”leaner”

Levels of MeasurementLevels of Measurement

Nominal MeasuresNominal Measures

Ordinal MeasuresOrdinal Measures

Interval MeasuresInterval Measures

Ratio MeasuresRatio Measures

Nominal MeasuresNominal Measures

Names for characteristicsNames for characteristics

Do not Exist along an Explicit continuumDo not Exist along an Explicit continuum

ExhaustiveExhaustive

Mutually ExclusiveMutually Exclusive Eg. Religious AffiliationEg. Religious Affiliation Eg. Place of BirthEg. Place of Birth

Ordinal MeasuresOrdinal Measures

Can be logically rank-orderedCan be logically rank-ordered

Represent relatively more of less of Represent relatively more of less of variablevariable No consistent distance between points of No consistent distance between points of

measurementmeasurement

Not just different from one anotherNot just different from one another More of less of some attributeMore of less of some attribute Eg. “Not very important,” “fairly important,” Eg. “Not very important,” “fairly important,”

“very important” “Extremely important”“very important” “Extremely important”

Interval MeasuresInterval Measures

Consistent distance separating attributeConsistent distance separating attribute We can say how much more of an attributeWe can say how much more of an attribute

Logical distance between attributes can be Logical distance between attributes can be Expressed in meaningful standard intervalsExpressed in meaningful standard intervals Eg. TemperatureEg. Temperature

90 degrees vs. 80 degrees = 10 degree 90 degrees vs. 80 degrees = 10 degree differencedifference

50 degrees vs. 40 degrees = 10 degree 50 degrees vs. 40 degrees = 10 degree differencedifference

Zero-point is arbitraryZero-point is arbitrary

Ratio MeasuresRatio Measures

In addition to all the properties of nominal, In addition to all the properties of nominal, ordinal, and interval measures, ratio ordinal, and interval measures, ratio measures have a true zero pointmeasures have a true zero point Eg. Length of timeEg. Length of time Eg. Number of timesEg. Number of times Eg. Number of affiliationsEg. Number of affiliations

Can actually state ratio of one to anotherCan actually state ratio of one to another X has twice as many affiliations as YX has twice as many affiliations as Y

What’s that scale?What’s that scale?

Style of music in a music videoStyle of music in a music video

Number of violent acts in a music videoNumber of violent acts in a music video

Whether a music video has violence or not?Whether a music video has violence or not?

High, Medium or Low violence in a music videoHigh, Medium or Low violence in a music video

Hair colorHair color

Number of hairs on your headNumber of hairs on your head

Sat scoresSat scores

Social Security NumberSocial Security Number

What’s that scale?What’s that scale?

A baseball player's batting averageA baseball player's batting average

A baseball player's field positionA baseball player's field position

A baseball player's position in the batting A baseball player's position in the batting orderorder

A baseball player's uniform numberA baseball player's uniform number

College football rankingsCollege football rankings

IQIQ

Types of questionsTypes of questions

Multiple choice questionsMultiple choice questions Agree/disagree questionsAgree/disagree questions Likert questionsLikert questions Frequency scalesFrequency scales Semantic differential scalesSemantic differential scales Forced-choice statement pairsForced-choice statement pairs Thermometer feeling scalesThermometer feeling scales Nominal checklistsNominal checklists Ordinal categoriesOrdinal categories Rank-order questionsRank-order questions Filter questionsFilter questions Open-endedOpen-ended

Multiple Choice Multiple Choice QuestionQuestion

Multiple Choice with Multiple Choice with Range OptionsRange Options

Agree/Disagree Agree/Disagree QuestionsQuestions

Likert ScaleLikert Scale

Frequency ScaleFrequency Scale

Semantic Differential Semantic Differential ScalesScales

Forced-choice Statement Forced-choice Statement PairsPairs

Thermometer Feeling Thermometer Feeling ScalesScales

Nominal ChecklistNominal Checklist

Ordinal CategoriesOrdinal Categories

Rank-order Preference Rank-order Preference QuestionsQuestions

Rank-order Evaluation Rank-order Evaluation QuestionsQuestions

Filter QuestionsFilter Questions

Open-ended Open-ended QuestionsQuestions

Tips on Question Tips on Question ConstructionConstruction

1. Make questions clear using simple 1. Make questions clear using simple languagelanguage

2. Keep questions concise2. Keep questions concise 3. Provide instructions for answering 3. Provide instructions for answering

questionsquestions Don’t assume respondent knows question styleDon’t assume respondent knows question style

4. Keep research purpose in mind4. Keep research purpose in mind Make sure items can answer research questionMake sure items can answer research question

5. Don’t ask double-barreled questions5. Don’t ask double-barreled questions E.g., “How well do you think the current E.g., “How well do you think the current

Presidential Administration is handling foreign Presidential Administration is handling foreign policy and the war on terrorism?”policy and the war on terrorism?”

More TipsMore Tips

6. Avoid leading questions6. Avoid leading questions E.g., “Like most Americans, do you read a E.g., “Like most Americans, do you read a

newspaper every day?”newspaper every day?”

7. Avoid negative questions7. Avoid negative questions E.g., “The U.S. should not invade Iraq” Agree E.g., “The U.S. should not invade Iraq” Agree

or disagree?or disagree?

8. Do not ask questions that require 8. Do not ask questions that require complicated mental calculuscomplicated mental calculus E.g., “In the past 30 days, how many hours E.g., “In the past 30 days, how many hours

have you spent watching television with your have you spent watching television with your family?”family?”

9. Keep ordering of questions in mind9. Keep ordering of questions in mind

Using Pre-Existing Using Pre-Existing MeasuresMeasures

It is okay to borrow measuresIt is okay to borrow measures Cite source of questions to give creditCite source of questions to give credit

Benefits of using Existing measures:Benefits of using Existing measures: Saves workSaves work Pre-tested for reliability/validityPre-tested for reliability/validity Research becomes cumulativeResearch becomes cumulative

PretestingPretesting

Clarity in question wordingClarity in question wording

Are categories:Are categories: Exhaustive?Exhaustive? Mutually Exclusive?Mutually Exclusive?

Realistic time estimateRealistic time estimate

Preliminary empirical analysisPreliminary empirical analysis