1 EPI 225 Measurement in Clinical Research Fall 2009 Anita L. Stewart, Ph.D. Institute for Health &...

1

EPI 225

Measurement in Clinical Research

Fall 2009

Anita L. Stewart, Ph.D.Institute for Health & Aging

University of California, San Francisco

2

Overview of Course How to find, critique and select good self-

report measures for your research Weekly homework with weekly feedback I am available for questions/discussion:

– after class: 4:30-5:30– by email

Focus on one concept of interest to you– Health outcome or a determinant of health– Measurable (need to find 2-3 measures to review)

3

Importance of Appropriate Measures

Inadequate measures can result in: Conceptual inadequacy Poor data quality (e.g. missing data) Poor variability Poor reliability and validity Inability to detect associations

– correlations, mean differences

4

Class 1

Concept Development and Concept Definitions

September 24, 2009

5

Overview of Class 1

Role of concepts in measurement Terminology, depicting concepts Complexity of most concepts Concept development methods Importance of defining concepts prior to

selecting existing measures

6

Typical Sequence of Developing New Self-Report Measures

Develop/define concept

Create item pool

Pretest/revise

Field survey

Psychometric analyses

Final measures

7

Typical Sequence of Developing New Self-Report Measures


Create item pool

Pretest/revise

Field survey


Final measures

8

Process of Selecting Good Measures for Your Studies

Define concept (variable)

Identify potential measures

Review measures’ properties--conceptual adequacy

--psychometric adequacy

Pretest best 1-2 measures

Select final measure

9








10

Overview of Class 1



11

Concept/Construct

A variable that is relatively abstract as opposed to concrete– e.g. health status, stress, acculturation as

opposed to height, body temperature An abstraction based on observations of

certain behaviors or characteristics Cannot be assessed directly

12

Concept/Construct: Other Terms

Variable, latent trait, latent variable Terms defined differently by different

investigators/authors Similar meaning

– often used interchangeably

13

Latent Variable or Trait

Latent - present but not visible, unobservable

Latent trait - unobservable set of characteristics that can be empirically inferred and estimated through answers to a set of questions

14

Measures of Concepts

Concepts are defined and operationalized in terms of observed indicators

Measures are “proxies” for the latent variables we cannot directly observe

15

Depicting Latent Variables and Measures

CONCEPTVariable A

Measure A

CONCEPTVariable B

Measure B

16

Depicting Latent Variables and Measures

Stress

Perceived Stress Scale

Health status

SF-36

17

Overview of Class 1



18

Complexity of Concepts

By definition, due to abstract nature of concepts, most are complex – Hard to define – Multidimensional

» Concepts within concepts

Some examples– Health– Quality of life– Interpersonal processes of care – Shared decision making

19

“Health” as a Concept or Latent Variable

What comes to mind when you think of the word “health”?

Health

20

Some Common Terminology of “Health”

Health status Health outcomes Clinical status Functional status Functioning and well-being Quality of life Health-related quality of life

21

Definition Varies by Perspective

Health

Bench scientist

Individual

Populationscientist Clinician

22

Perspectives on Health Bench

Scientist Clinician Individual

Molecular, genetic, cellular X ??Anatomic, physiologic signs XSymptoms X XDiagnosed conditions X XSeverity of conditions X XFunctioning in daily life XWell-being X

23

Contradictory or Different Perspectives?

Patient 1 MD - signs of illness

Pt – perceives very good health

Patient 2 MD - no signs of illness

Pt – perceives poor health

Patient 3 Lab: cellular abnormality

Pt - no symptoms

24

Dimension, Domain, Sub-domain, Component

Terms somewhat interchangeable Dimension: a distinct component of a

multidimensional construct that can be theoretically or empirically specified (e.g., physical and mental health)

25

Medical Outcomes Study (MOS)

Large study of patients with chronic disease– 1989 forward

Extensive measurement development part of MOS– Health, functioning, and well-being

26

MOS Framework: Physical and Mental Health are “Latent Variables”

Physicalhealth

Mentalhealth

27

Physical Health

MOS Physical Health: Dimensions

Physicalfunctioning

Physicalfunctioning

Health perceptions

Health perceptions

PainPainEnergy &

fatigueEnergy &

fatigue

Role limitationsdue to physical

health


health

28

Physical Health

MOS Physical Health: Dimensions, Sub-domains

Physicalfunctioning

Physicalfunctioning

Health perceptions

Health perceptions

PainPainEnergy &

fatigueEnergy &

fatigue


health


health

Painfrequency

Painfrequency

Painseverity

Painseverity

29

MOS Framework: Indicators of Physical and Mental Health

Physical Mental

Indicators Health Health Physical functioning X

Pain X Energy/fatigue X X Sleep problems X X Cognitive functioning X Psychological distress/well-being X Role limitations due to physical health X Role limitations due to emotional problems X Health perceptions X X RD Hays and AL Stewart, Psychological Assessment:

A Journal of Consulting and Clinical Psychology, 2:22-30, 1990.

30

What is Perfect Health?

An ideal toward which people are oriented – not a state they expect to attain

“A healthy person is someone who has been inadequately studied” (p. 31)

AC Twaddle, The concept of health status, Soc Sci Med 1974;8

31

Health as a continuum?

Illness Perfecthealth

32

Health as a continuum?

Illness Perfecthealth

Less thanperfecthealth?

33

What are normal health states?

Illness

(abnormal)

Perfecthealth

(abnormal)

Less thanperfecthealth?

(normal)

34

Concept of Quality of Life

What comes to mind when you think of the word “quality of life”?

Quality of Life

35

Quality of Life Terminology

Satisfaction with life “as a whole” Well-being Global well-being Subjective well-being Sense of well-being Global happiness

36

Dimensions/Domains of Quality of Life

Marriage Family life Health Neighborhood Friendships Job

City Housing Standard of living National government Local government

…. and many more

37

Health-Related Quality of Life?

Domains of life relevant to health, disease, and medical care

Well-being– Physical (malaise, symptoms)– Mental (psychological well-being)

Functioning– Limited in activities because of health problems

or treatments

38

Shared Decision Making

How would you define “shared decision making”?

39

Shared Decision Making (DM):Some “Definitions”

MD propensity to encourage shared DM Pt desire for active role in DM MD encourages pts to take active role Pt plays active role in deciding Pt and MD share process of deciding on

treatment

40

Varying Definitions Suggest Complex Concept

Process of shared decision making

MD propensityto encourage shared DM

Patient desire for active role in DM

41

Concepts within Concepts




42

MD Propensity to Encourage Shared DM

“Participatory Decision-Making Style”: Propensity (of physicians) to

– offer pts choices among treatment options

– to give pts a sense of control of care

– to give pts a sense of responsibility for care

Kaplan SH et al. Ann Int Med, 1996;124:497

43

Varying Definitions Suggest Complex Concept




44

Process of Shared Decision Making in ICU: Multidimensional

Information exchange – MD provides relevant medical information – Family member provides info on pt’s values/preferences

Deliberation– MD + family member discuss appropriate treatment options

Treatment decision– MD + family member clarify best treatment choice

Doug White, MD, Dept Med, Div Pulmonary and Critical Care

45

Overview of Class 1



46

Concept Development Methods

Primarily qualitative methods (formative research)– ethnography

– direct observation

– in-depth interviews

– focus groups

47

Qualitative Methods to Develop Concepts and Measures

Focus groups, in-depth interviews Data consist of words (text) Useful for

– Concept development

– Item construction

48

Focus Groups

Open-ended guided group discussion with probing of responses

Listen to perspective of likely “subjects”– How they discuss “concept” in their own

words Participants stimulate comments of others

49

Who Provides Input?

Patients or representatives of the population of interest

“Experts” with extensive clinical or personal experience with population of interest

50

Focus Group Methods

Led by experienced moderator– usually 1 - 2 hours

Purposeful sampling of 6-10 homogenous participants per group

Interview guide – set of open ended questions

Audio-record and transcribe discussion

51

Focus Group Methods Resources

Good overview of focus group methods and application - Vogt 2004 reading– They refer to “enhancing content validity” which is

another way of discussing “concept development” Handout: Using focus groups to develop

structured surveys– Annotated bibliography

52

Focus Groups to Develop Concept of Expectations Regarding Aging (ERA)

How older adults think about aging– Do they expect to decline on various

dimensions?

– Would their expectations affect decisions about health and health care?

53

ERA Concept Development : Qualitative Methods

Conducted 5 focus groups of older adults 10 vignettes describing older adults

experiencing common age-associated changes– Probed “does this describe changes you would

expect to occur with aging?” Analyzed transcripts using qualitative methods Results: 26 “domains”

CA Sarkisian et al. Med Care. 2003;39:1025-36.

54

Domains Resulting from ERA Focus Groups

Physical function Cognitive function Social function Pain Sexual function Personal autonomy Loneliness

Appearance Energy Urinary incontinence Falling Happiness Spirituality Being treated differently

55

Example: Interview Guide Exploring Depression Concept in Diverse Groups

Sample: White, South Asian, and Black Caribbean individuals in U.K.

Questions: – What does the word depression mean to you?

– Do you consider depression to be an illness?

– What do you think might cause depression?

V Lawrence et al., Gerontologist, 2006;46:23-32

56

More Advanced Concept Development Methods

Iterative process Integrates qualitative and quantitative

methods (“mixed methods”)

57

Mixed Methods: Iterative Sequence to Develop Structured Measures

Focus groups In-depth interviews Input from quantitative studies Develop preliminary items Review by expert panel Cognitive interview pretest Pilot study Nationwide survey Psychometric testing

N Krause, J Geront: Soc Sci, 2002;57B:S263-274

58

Mixed Methods: Iteratively Refine Concept Based on Quantitative Results


Create item pool

Pretest/revise

Field survey


Final measures

59

Iterative Concept Development: Interpersonal Processes of Care (IPC)

Interpersonal processes between physicians and patients, emphasis on vulnerable patients

Two “rounds” of concept development Round one – initial concept based on:

– Review of literature– Content of various satisfaction with care measures– Clinical experience of investigators

IN READINGS: AL Stewart et al: Milbank Quarterly, 1999

60

Initial Hypothesized “Multidimensional” IPC Conceptual Framework

I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Friendly, courteous Elicitation of Respectful patient issues Perceived discrimination Explanations of Cultural sensitivity -condition Emotional support -self care -process of care II. DECISION MAKING Responsiveness to patient preferences Consideration of ability to comply

Stewart et al., Milbank Mem Q,1999, Table 1

61

IPC Concept Development (cont)

Initial concept modified after measurement analyses

Differences from hypothesized concept:– No cultural sensitivity scale

– Content of many dimensions changed

Stewart et al., Milbank Mem Q, 1999, Tables 2 and 3

62

“Round Two” IPC Concept Development

Draft IPC II

conceptualframework

IPC framework (Milbank Quarterly)

19 new focus groups -African American, Latino,and White adults

Updated literature review of quality of care in diverse groups

63

IPC “Round Two”Hypothesized Conceptual Framework

I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Friendly, courteous Elicitation Respectful Explanations of Perceived discrimination --condition Cultural sensitivity --process of care Emotional support --self-care Empowerment Office staff: friendly II. DECISION MAKING courteous, respectful Responsive to pt preferences Consider ability to comply

Stewart et al., Health Serv Res, 2007;42:3(Part I):1235-1255, Table 1

64

IPC “Round Two”Measures Developed and Tested


Create item pool

Pretest/revise

Field survey N=1,664


Final measures

65

IPC “Round Two”Final Conceptual Framework

I. COMMUNICATION III. INTERPERSONAL STYLE General clarity Emotional support Hurried, distracted Respectful Elicited concerns Assumptions re SES Explained results, Discriminated due to R/E medications Disrespectful office staff II. PATIENT-CENTERED DECISION MAKING Considered ability to comply Decided together Stewart et al., Health Serv Res, 2007 - Fig 1

66

IPC Concept Development: Cultural Sensitivity of Physicians

Hypothesized scale did not work in either of our IPC studies

For minority patients and those with limited English proficiency– Sensitivity of providers to their cultural

perspective

67

New Qualitative Analyses of Prior Focus Groups

Two open-ended focus group questions:– What does the word culture mean to you?– What do your doctors understand about your

culture that might affect your visits? Transcripts analyzed using qualitative

methods

Nápoles-Springer et al. Health Expectations. 2005;8:4-17.

68

Results: Multiple Dimensions of Cultural Factors Affecting the Medical Encounter

Examples … Values, beliefs, and

attitudes Communication

including sensitivity to language access

Complementary and alternative medicine

Privacy and modesty Ethnic concordance

of physician and patient

Empowerment and submissiveness

69

Another Concept Development Method:Review Existing Measures

Work “backwards” by organizing items in published measures

Can identify underlying “concept” implied by measures

70

Physical Functioning Concept “Derived” from Four Measures

AIMS MOS HAQ SIP

Walking X X X X

Climbing stairs or steps X X X X

Bending, kneeling X X X X

Lifting, carrying X

Getting out of bed X

Bathing X X X

Running errands X

Opening jars X

Vigorous activities X X X

71

Depression Concept “Derived” from Four Measures

CES-D20

HRSD21

Beck21

GDS15

Personal functioning, behavior

-sleep problems, substance abuse

3 3 4 3

Social functioning, behavior -cut down work, withdraw

1 1 0 0

Physical symptoms

-fatigue, low libido

2 8 5 1

Emotional status

-depressed mood, lonely, happy

9 3 4 5

Cognitions/perceptions

-sense of failure, suicidal thoughts

5 6 8 6

72

Depression Concept “Derived” from Four Measures (cont)

CES-D20

HRSD21

Beck21

GDS15

Personal functioning, behavior

-sleep problems, substance abuse

3 3 4 3

Social functioning, behavior -cut down work, withdraw

1 1 0 0

Physical symptoms

-fatigue, low libido

2 8 5 1

Emotional status

-depressed mood, lonely, happy9 3 4 5

Cognitions/perceptions

-sense of failure, suicidal thoughts

5 6 8 6

73

Concept of Body Awareness

Interested in measuring body awareness as an outcome of integrated medicine treatments/therapies

Reviewed 12 measures of concepts related to body awareness– e.g., Body Intelligence Scale, Body Awareness

Measure, Scale of Body Connection

IN READINGS: W Mehling et al., PLosOne, 2009;4(5):1-18.

74

Adapted From Mehling’s Table 6: Concept Derived from Measures?

BIS BRQ BAM TQ SBA SBC

Body sensations

Stress, worry X X X

Well-being X X X

Pain, symptoms X X

Attention quality

Amount of attention

X X X X X X

Thinking vs experiencing

X

Emotional awareness

X X X

75

Multidimensional Concept: Result of Measures Review (Mehling’s Table 5)

Dimension Sub-Domain

Perceived body sensation

Sensations of distress, worry, pain, tension

Sensations of well-being

Neutral sensations

Affective aspect of sensation

Attention quality Intensity of attention/ignoring sensation

Self-efficacy in attention control

Thinking/labeling vs. present sensation

Attitude Trusting

Catastrophizing

Mind-body integration

Emotional awareness

Sense of embodied self vs disconnected

76

Overview of Class 1



77








78








79

State Role of Concept in Your Research

Evaluate intervention (outcome) Determinant (e.g., social support) of an

outcome Predict future event (e.g. functioning to

predict hospitalization) Describe population Covariate

80

Define Concept for Your Research

Define it first from your point of view – How you would define it based on your experience

and understanding of your research question For dependent variables

– Describe how intervention or independent variables might affect it - types of changes you expect

For independent (predictor) variables– Describe how it might be predictive

81

Measure of Concept Needs to Match Your Definition

Concept definition is the basis for selecting appropriate measure

Measure must reflect all key dimensions of your concept

82

Measure of Concept Needs to Match Your Definition (cont)

Label or name of a measure may not reflect what it actually contains

May find measure of your concept with the right label– But mismatched

» Measures something different than you need» Measures only part of what you need (missing

elements)

83

Homework: Define Concept

Summarize briefly a key research question of interest to you

Choose one key concept from this question and describe its role (e.g. dependent variable)

Define key concept from your point of view including its dimensions

84

Send Homework by Email

Include your last name in the file name of your homework

Send to [email protected]

85

Use Syllabus!

Print out syllabus– A pdf file of the syllabus is posted along with

class 1 readings Syllabus contains all details (readings,

handouts, homework)

86

Next Week: Class 2

Measurement terminology and locating specific measures

1 EPI 225 Measurement in Clinical Research Fall 2009 Anita L. Stewart, Ph.D. Institute for Health &...

Documents

Transcript of 1 EPI 225 Measurement in Clinical Research Fall 2009 Anita L. Stewart, Ph.D. Institute for Health &...