USING OBSERVATION-ORIENTED MODELING TO EXAMINE DAILY PATTERNS AND

PREDICTORS OF POST-TRAUMATIC STRESS SYMPTOMATOLOGY

IN A SAMPLE OF FEMALE RAPE VICTIMS

Amy M. Cohn, James Grice, Brett Hagman, and Liz Schlimgen

November 17, 2012ABCT

Aim of Presentation

Test a mediation model with daily diary data

Compare multi-level modeling (MM) to Observation Orientated Modeling (OOM)

Daily post-traumatic

stress symptoms

Daily negative

affect

Daily alcohol involvement

Assumptions of MM have limitations

Homogeneity between individuals Within-person fluctuations in behavior represented

as aggregated, over-time association (slope) Linear monotonic changes in behavior over time

Associations between variables are additive, not (necessarily) dynamic

Random sampling Normal population distribution (for differences)

Abstract population parameters that have little (or no) empirical basis

The population is completely theoretical

What’s wrong with the p-value? Not a new argument Relies on population statistics that may

not represent the data The sample is different, the way you collect

the data is different, the questions you ask are different….

Creates a “false belief” in the validity and generalizability of findings Many study results cannot be replicated

Observation Orientated Modeling

“Why is it that the patterns of phenomena are the way they are?” (Harre, 1986)

“Fundamentally incompatible with prevailing research tradition in Psychology”(Grice, 2012)

OOM Incompatible with MM

Variable-based approach (such as MM) is linear, causal, and based on aggregate statistics such as betas and variances

OOM approach is integrative and focused at the level of the individual

Independent variable

Dependent variable

OOM

Non-parametric, idiographic Examines qualitative pattern in the data Rooted in Aristotle’s notion that most things in

nature are not produced by people The researcher does not control everything in a

study Eschews null hypothesis significance testing

(NHST) Results based on probabilities found within the

data, not comparison to population distribution Variables not described in a cause-effect format

OOM describes how the effect conforms to the cause

OOM

To Repeat……. EFFECTS SHOULD CONFORM TO THEIR

CAUSES What the @#*&$?

We do not always know why participants do what they do

Effects are never truly “causal” Unmeasured pieces of “error” or “garbage” in

the data collection process With OOM, patterns of observations reveal

what are in the data – The EFFECTS

Study 1 Hypotheses

NA will be greater on days characterized by greater PTSD

Craving and consumption will be greater on days characterized by more intense PTSD and NA

Daily PTSD

symptoms

Daily NAa

Alcohol Involvem

ent

b

c (c’)

Sample characteristics (n = 54)

Characteristic Statistic

Age 26 (SD = 9.08)

Some college or post high school education

70% (n = 38)

Employed (full or part-time) 25% ( n = 15)

Single 60% (n = 34)

Caucasian 70% ( n = 38)

Income (Median) $9,000 (SD = $20,032)

54 untreated female rape victims who completed at least one day of daily interactive voice response (IVR) monitoring

IVR Assessment

1x a day in the evening (6pm to 12am) Alcohol use, negative affect intensity,

craving intensity, and PTSD symptoms (presence/absence)

Since previous phone call 93% compliance rate

13/14 calls were completed

HLM Analysis DV’s = Number of drinks consumed and

intensity of craving (850 observations) Controlled for day of week Poisson distribution with log link function

for drinking Examined relationship of one variable

EACH DAY to the outcome variable ON THAT SAME DAY

Figure 1. Mediation of NA on the PTSD-alcohol link.

Daily NA

Daily PTSD symptoms

Number of standard

drinks/day

0.13*** 0.42***

-0.14

*** p < .001

(-0.02)

In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)

Figure 2. Mediation of NA of the PTSD-craving link.

Daily NA

Daily PTSD symptoms

Daily craving intensity

0.13*** 0.39***

-0.10

Note. Covariates included day of the week, baseline PTSD symptom severity, baseline alcohol use. *** p < .001

(-0.12)

In Cohn, Hagman, Moore, Mitchell, Ehlke, and Bramm (under review)

OOM Analysis: Mediation Steps

Daily PTSD symptoms

Daily NA

Number of standard

drinks/day

Step 1: Because the effect conforms to the cause, we first examine the probability that number of standard drinks consumed each day conforms to daily ratings of NA intensity

OOM Results

Accuracy rate: % observations correctly classified out of total number of observations Missing data is not a problem

Randomization test Out of1000 trials of randomized versions of the

same observations, what number of instances do we obtain a result high or higher than percent correct classification?

Binomial p-value or chance value should be small (less than .01) Indicates pattern is unique

Results for individual and group-level patterns

Perfect Ordinal Matches for 14 Occasions

Proportion of Matches = 1.00; Binomial p-value = .00012

Weak Ordinal Matches for 14 Occasions

Proportion of Matches = .15; p-value = .99

Overall Results (n = 54 women) :

Number of Matches : 123 Number of Observations : 399 Proportion of Matches : 0.31

Randomization Results :

Observed Proportion of Matches : 0.31

Number of Randomized Trials : 5000.00 Minimum Random Proportion of Matches : 0.24 Maximum Random Proportion of Matches : 0.36 Values >= Observed Proportion : 1758.00 Matching c-value : 0.35

Proportion of matches is unimpressive at .31

C-value of the Randomization Test indicates that .31 is not an unusual aggregate outcome compared to randomized versions of the same observations

Aggregate Results for all 54 Women

Aggregate Results for all 54 Women 1.Proportion of Matches > .50 for only 9

women(5 of these women had 7 or fewer data points)

2.Fourteen women (26%) showed no variability in their drinking across the 14 days

3.An additional 6 women drank on only one day

Conclusions

Women showing no variability in drinking and those who did not drink across 14 days are “swept” into HLM aggregates Should this disturb us?

OOM recognizes women with no variability in their drinking Since OOM not based on means and

variances, impact of these women does not adversely effect the overall percent matches

Conclusions

OOM “effect sizes” are proportions of matches that are readily interpretable and linkable to individual women No need for interpretations- such as Cohen’s effect

sizes Idealized p-values are primary in HLM, even over

effect sizes Even if effect is small, if p < .05 we say “YES”!

Proportions of matches consistent with causal hypotheses are primary in OOM Distribution free p-values (from binomial and

randomization tests) are secondary

Erroneously enticed to posit a mediation mechanism that operates successfully for every woman with HLM OOM treats the women and their individual observations as primary Does not rely on p-values, means, or

variances estimated from a theoretical population

OOM develops integrated models More accurately explains patterns of

observations

Summary

Acknowledgments

Participants who dedicated their time and effort

Research assistants: Jessica Mitchell, Stephanie Bramm, Sarah Ehlke, Ruschelle Leone, Joanne Wang

Grants: NIDA P30DA028807; USF 582000 / MHBCSG

Thank you!

Questions?

Dr. James GriceDepartment of PsychologyOklahoma State University

Stillwater, OK 74078James.grice@okstate.edu

Deep Structure Transformation

M F

0 10 11 01 0

0 1 2 3 4 5 6 7 8 9 10

0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 1 00 0 1 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0

0 10 11 01 0

0 10 11 01 0

ConformedEffectObservations

EffectObservations

CauseObservations

EffectObservations

3. Accuracy is our central judgment (not statistical significance) and shows how many observations were correctly classified by the algorithm, or how many observations match the pattern.

1. Observations are transformed into their “deep structure”

2. Rotate deep structure effect observations into “conformity” with deep structure observations