1

Class 2Measurement Terminology and

Locating Measures

October 6, 2005

Anita L. StewartInstitute for Health & Aging

University of California, San Francisco

2

From Last Week

Some information based on last week’s discussion – On weights and values

– Does cognitive functioning reflect physical health?

3

Implicit and Explicit “Weighting”

Description (describe health states) Valuation (mapping those health states

onto a set of values) Most measures use “implicit” values Utility or preference measures represent

“explicit” values

Kind P. Values and valuation in the measurement of HRQoL, in P Fayerset al (eds) Assessing quality of life in clinical trials, 2nd ed, Oxford, 2005

4

Example of “Implicit” Weighting: Number of Items per Domain in 3 MOS Indexes

MHI32 MHI17 MHI5Anxiety 7 4 1

Depression 13 8 2Positive affect 7 4 2Belonging/loneliness 5 1 02 summary scores:

Distress 22 12 3Well-being 10 5 2

5

Example of “Implicit” Weighting: Percent of Items per Domain in 3 MOS Indexes

MHI32 MHI17 MHI5Anxiety 22 24 20

Depression 41 47 40Positive affect 22 24 40Belonging/loneliness 16 6 02 summary scores:

Distress 69 71 60Well-being 31 30 40

6

Correlations of MOS Long Form Measures with Physical and Mental Health

Physical Mental

Pain intensity -.71 -.35

Role limitations – emotional -.48 -.66

Cognitive functioning .44 .80

Symptoms -.69 -.51

Energy/fatigue .79 .61

Sleep problems -.54 -.63

Current health perceptions .77 .50

Health distress -.70 -.63

Hays RD and Stewart AL, The structure of self-reported health inchronic disease patients, Psychol Assessment, 1990;2:22-30.

7

Overview of Class 2

Measurement terminology– Confusion, variation, and overlap

Types of response scales, contribution to concept definition

Measurement models– Depicting structure of measures

Locating measures

8

Measurement

Assignment of numbers to aspects of objects or events according to a rule

Assignment of numbers to an unobservable characteristic of individuals (construct)

9

Construct, Concept, Variable, Latent Trait, Latent Variable

Terms defined differently by different investigators/authors

Meaning is very similar– considerable overlap among them,

Often used interchangeably

10

Construct/Concept

A variable that is relatively abstract as opposed to concrete and is defined or operationalized in terms of observed indicators

An idea developed or constructed through informed scientific theory. Concepts that are directly or indirectly observable.

An abstraction based on observations of certain behaviors or characteristics (e.g. pain, stress)

11

Latent Trait, Factor

Latent - present but not visible, unobservable Latent trait - unobservable set of characteristics

that can be empirically inferred and estimated through answers to a set of questions

(Medical Care glossary) Factor - a latent variable or theoretical construct

operationalized in terms of associations among indicators in a factor analysis

(Stewart and Ware, 1992 glossary)

12

Latent Variable, Manifest Variable

Latent variable - unobservable latent trait given a label

Manifest variable is the “observed” information, i.e., the answers to items used to infer the latent trait

13

Domain, Dimension, Sub-domain, Component

Domain: a state, attitude, perception, behavior, or other sphere of action or thought related to health …. All entities in a single domain have some property in common ….

(Patrick & Erickson, 1993) Dimension: a distinct component of a

multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)

(Stewart & Ware, 1992, glossary)

14

Example: Dimension, Component, Sub-domain

Mental health (dimension)– Psychological distress (component)

» Depression (sub-domain)» Anxiety

– Psychological well-being» Positive affect» Sense of belonging

15

Components of Domains of MOS Long-form Measures

Role Role limitations due to:Functioning -- physical health

-- emotional problems

Social Social limitations due to healthFunctioning Sexual problems

Family functioningMarital functioning

Pain Pain severityEffects of painDays with pain

16

Multidimensional and Unidimensional

A multidimensional “measure” or “instrument” provides several scores– A “profile” of scores

Unidimensional measure - all items measure only one concept– Only one score is obtained

Dimensionality must be empirically tested– e.g., factor analysis identifies one factor,

or more than one factor

17

Example of Multidimensional Instrument

Patient Satisfaction Questionnaire (PSQ) 55 items, 18 subscales, e.g.

– Access to care– Technical quality– Interpersonal manner– Explanations– Continuity of care

Marshall, GN Psychological Assessment, 5:477-483, 1993

18

Example of Unidimensional Measure

Perceived Stress Scale (PSS) 14 items, subjective experiences of stress

– felt confident could handle life’s problems

– able to control irritations in your life

– difficulties piling up so high, could not overcome them

Single score from all itemsCohen, S, J Health Soc Behav 24:385-396, 1983

19

Instrument, Profile, Battery, Measure, Questionnaire, Survey, Tool, Inventory Instrument - typically refers to a published,

named measure or set of measures Profile - same, but all scores on the same

metric and can be compared (0-100 scales) Battery (of independent measures)

– collection of measures from diverse sources Measure - single- or multi-item scale or

index (umbrella term)

20

Example of a Multidimensional Profile Instrument: SF-36Measures 8 health concepts (36 items): Physical functioning Role limitations due to physical health Role limitations due to emotional problems Social functioning Pain Vitality (energy/fatigue) Mental health Health perceptions

21

Generic vs. Disease-Specific Approaches

Generic– Applicable across many conditions and

diseases (e.g., sleep, health perceptions, fatigue, pain)

Disease-specific– Specific to particular disease or condition

(e.g., arthritis, cancer, depression)» Generic approach (e.g., general dexterity scale for arthritis

study)» Specific (nausea due to chemotherapy, back pain)

22

Generic or Disease Specific bySingle Domain, Profile, or Index

Single domain

Profile Summary index

Generic Pain

Fatigue

SF-36

SIP

PCS

MCS

QWB

Disease specific

Nausea

Back pain

Diabetes impact

23

Questionnaire, Survey, Tool, Inventory

Questionnaire - any of the above formatted into a “questionnaire” or set of measures

Survey - same as questionnaire Tool = Questionnaire = Survey? Inventory?

24

Examples of Names of “Instruments”

SF-36 Short Form Health Survey Beck Depression Inventory Health Assessment Questionnaire Sickness Impact Profile Center for Epidemiological Studies

Depression Scale

25

Example of a Battery

A survey for a particular study containing the following measures/instruments:– SF-36 (8 domains)– Perceived stress scale (1 domain)– Social support survey (4 domains)– Pain measure

Typical in clinical trials, epidemiological studies– Each measure serves particular purpose (outcome,

predictor, mediator, covariate)

26

Measure, Scale, Index, Item

Measure - single- or multi-item scale or index (umbrella term)

Scale - aggregation of items from one concept or domain, scored according to an accepted scaling method

Index - aggregation of 2 or more scales into a summary score

Item - a single question or statement including its response scale

27

Item

Consists of an item stem and a response scale or response choices

Item stem – – The question or statement

Response scale or choices –– Answers presented to a respondent with which to

respond to a (closed-ended) question

28

Example of a Response Scale

How often have you felt tired inthe past week? 1 Never 2 Once or twice 3 A few times 4 Fairly often 5 Very often

Responsescale

Itemstem

29

Response Choice Formats

Verbal descriptors of all levels Verbal descriptors of endpoints only Verbal descriptors of endpoints plus

midpoint Visual analogue scales:

10cm line with endpoints labeled

30

Format: Verbal Descriptors of All Levels

1 - All of the time2 - Most of the time3 - Some of the time4 - A little of the time5 - None of the time

31

Vague, Imprecise Quantifiers

How often?– very often, pretty often, not too often

– Sometimes, often, never How much?

– Too little, about right, too much

– Below average, average, above average

Bradburn NM, Vague quantifiers. Public Opinion Quart 1979, 92-101.

32

Intensity Scales – “How distressed have you been?”

Not at all, slightly, moderately, quite a bit, extremely

Not at all, a little, a fair amount, much, very much

Not at all, a little, fairly, quite, very, extremely

Not at all, a little, somewhat, very much

33

“How Often” Response Scales

1 - Never

2 - Once or twice

3 - A few times

4 - Often

1 – Never

2 - Hardly ever

3 - Some days

4 - Most days

5 - Almost every day

6 - Always, every day

34

“How Often” Response Scales

1 - Never

2 - Almost never

3 - Sometimes

4 - Fairly often

5 - Very often

6 - Always

1 - Once or twice

2 - A few times

3 - Fairly often

4 - Very often

5 - Almost every day

6 - Every day

35

Format: Verbal Descriptors of Endpoints Only

0 1 2 3 4 5 6 7 8 9 10 No Pain pain as bad

as you can imagine

36

Format: Verbal Descriptors of Endpoints with Midpoint Labeled

Very Very

much 1 2 3 4 5 6 7 muchworse No better

change

37

Format: Verbal Descriptors of Endpoints with Midpoint Labeled

-3 -2 -1 0 1 2 3Very No Very much change muchworse better

38

Format: Visual Analogue Scale, 10 cm Line

No Pain pain as bad

as you can imagine

39

Optimal Number of Response Choices

Are 100 levels better than 5 or 10? 5-points is probably optimal for respondents

– Especially if items will be combined into multi-item scales

For single-item measures– 7 points may be preferred– Allows distribution over 5 points since people

seldom use endpoints Hard to come up with labels for 7-point scales

40

Format: Verbal Descriptors of Endpoints Only

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

No Pain pain as bad

as you can imagine

41

Magic Number 7 + 2

George Miller determined that 7 “chunks” of information is the most that short-term memory can retain– Plus or minus two (5-9 possible)

Seven (+ 2) is also the most points that people can discriminate along a continuum

Principle has held up well in measurement

Miller, GM Psychological Bulletin, 63:81-97, 1956

42

Can You Make a 7-point Intensity Scale? (How physically active were you this week?)

1 - Not at all active

2 - A little bit active

3 - Moderately active

4 - Quite active

5 - Extremely active

1 - Not at all active

2 - A little active

3 - Fairly active

4 - Quite active

5 - Very active

6 - Extremely active

43

Measurement Models

The dimensional structure of a measure in terms of how the items related to the constructs (latent traits, factors, concepts)

Can be depicted as a list or visually Standard visual format for structural

equation modeling (SEM)

44

Measurement Models

Physical Functioning (4 items) Psychological Distress (7 items)

45

Measurement Model (List format)

Physical Functioning defined in terms of:– Walking

– Climbing stairs

– Bending

– Reaching

46

Measurement Model (Visual format)

Physical Functioning

WalkingClimbing

StairsBending Reaching

47

Measurement Model (List format)

Psychological distress– Depression

» Sad» Lost interest» Can’t get going

– Anxiety» Restless» Nervous

48

Measurement Model (Visual format)

Psychological Distress

Depression Anxiety

SadLost

interestCan’t

get goingRestless Nervous

49

What’s in a Label?

Don’t be fooled

50

Content Areas: Four Depression Measures

CES-D HAM-D Beck GDS

Sleep problems X X X X

Concentration X X

Fatigue, energy X X

Appetite loss X X

Depressed mood X X X

Anxiety, restlessness X X

Irritability X X

Sense of failure X X

Social withdrawal X X

Guilt X

Self-criticism X X

51

Content Areas: Four Standard Measures of Physical Functioning

AIMS MOS HAQ SIP

Walking X X X X

Climbing stairs or steps X X X X

Bending, kneeling X X X X

Lifting, carrying X

Getting out of bed X

Bathing X X X

Running errands X

Opening jars X

Vigorous activities X X X

52

MOS SF-36 versus SF-12:Physical Functioning

SF-36 SF-12 Vigorous activities X Moderate activities X X Lift/carry X Climb several flights stairs X X Climb one flight stairs X Bend/kneel/stoop X Walk > 1 mile X Walk several blocks X Walk 1 block X Bathe/dress X

53

MOS SF-36 versus SF-12:Mental Health Items

SF-36 SF-12 Nervous X Down in the dumps X Calm and peaceful X X Downhearted and blue X X Happy person X

54

Sequence of Steps in Developing New Self-Report Measures

Develop concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

55

Locating Specific Measures

Compendia Web Organizations National surveys Large research studies Many other sources

56

Locating Specific Measures

Compendia of measures (reviewed)– Books that compile various measures and

review their characteristics (e.g., McDowell and Newell)

Web sources of Compendia– HaPI-Health and Psychosocial Instruments– Measurement Excellence and Training Resource

Information Center (METRIC)

57

Locating Specific Measures

Organizations– RAND publishes measures, scoring manuals, and

lists citations– University of Michigan, Survey Research Center

National and State Surveys– NHIS, NHANES, HHANES, BRFSS, CHIS, MEPS– Entire surveys can be downloaded– Seldom includes measurement information

58

Locating Specific Measures

Large research studies on similar topic– Health ABC, CARDIA, WHI– EPESE

Quality tools (AHRQ)– Consumer Assessment of Health Plans Survey

(CAHPS)– National Quality Measures Clearinghouse– National Healthcare Quality Report– National Healthcare Disparities Report

59

Locating Specific Measures

National Institutes of Health – NCI has a measurement initiative

Health Information National Trends Survey (HINTS)– Working group compiled cancer screening

questions, identified best ones, conducted extensive pretesting, cognitive interviewing

– Measures are on the NCI web site

60

Locating Specific Measures

Projects/Centers– Toolkit to measure end of life care (TIME)– Stanford Patient Education Research Center– Michigan Diabetes Research and Training Center– Resource Centers for Minority Aging Research

Proprietary measures– SF-36, SF-12 (have to pay for instruments, scoring

manuals)

61

Locating Specific Measures: Bibliographic Searches

Published measurement articles– Medline Searches

– MESH headings or keywords:» health status indicators

» outcome assessment (health care)

» psychometrics (methods)

» questionnaires

62

Locating Specific Measures: Finding Authors of Measures

Published research using measure you are interested in– Unpublished measures often described in

methods– Authors may provide measures

63

Summary

Don’t let the terminology get you down– Be aware of alternative uses of words/labels

Measurement models – SEM depiction is useful

Locating measures– Maintain a file of measures of concepts you

are interested in