NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI Jill Pentimonti...
-
Upload
diamond-gentleman -
Category
Documents
-
view
217 -
download
0
Transcript of NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI Jill Pentimonti...
NOT YOUR GRANDPA'S STATISTICS: NEW MODELING APPROACHES TO STUDENT ACHIEVEMENT & RTI
Jill PentimontiAdrea Truckenmiller
Jessica LoganSara Hart
Discussion: Grandpa Schatschneider Presented Feb 6, 2014
Pacific Coast Research Conference, San Diego
Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS
Sara A. Hart&
GrandpaFlorida State University
Expanding our search for moderators of intervention
• A little about me– Behavioral genetics background– PCRC participant
• Even with modest effect sizes, individual differences in intervention response
• Bioecological model (Bronfenbrenner & Ceci, 1994)
– Provides framework for differentiating students based on non-intervention related traits
Integrative Data Analysis (IDA)
• Item-level pooled data (Curran & Hussong, 2009)
• Capitalizes on cumulative knowledge – Longer developmental time span– Increased statistical power– Increased absolute numbers in tails
• Controls for heterogeneity – Sampling, age/grade, cohort, geographical, design,
measurement
Project KIDS
• Expanded definition of moderators of response to intervention– Cognitive, psychosocial, environmental, genetic risk
• IDA across 9 completed intervention projects – Approximately 5600 kids
• Data entry of item level data common across at least 2 projects– ~30 different assessments
• Questionnaire data collection
Proof of Concept
• Behavior problems and achievement are associated
• More behavior problems are typically seen in LD populations
• Is adequate vs inadequate response status differentiated by behavior problems?
Method
• Participants– 2005-2006 ISI intervention project (Connor et al., 2007)
• RCTish : 22 treatment, 25 contrast teachers, 3 pilot• 821 first graders• A2i recommendations vs standard practice
– 2007-2008 ISI intervention through FL LDRC (Al Otaiba et al., 2011)
• RCT: 23 treatment, 21 contrast teachers• 556 kindergarteners • A2i recommendations vs enhanced standard practice
Method• Measures– WJ Tests of Achievement Letter-Word
Identification (LWID)• Pre- and post-intervention testing periods
– Social Skills Rating Scale: Behavioral Problems subscale • Teacher completed during intervention year
05/06 1 ISIMean (SD)
07/08 K ISI LDRCMean (SD)
WJ LWID Fall 24.28 (7.97) 11.96 (5.53)
WJ LWID Spring 36.54 (7.39) 21.64 (7.10)
SSRS .53 (.44) .48 (.44)
Results: Calibration LWID
• Randomly selected 1 time point/child/project to form “calibration sample” for LWID
• IRT with decision to include only items > 5% endorsement rate
• Reduced item sample from 75 36 – Items 8 to 44
Results: Calibration LWID
• Generalized linear factor analysis (GLFA)– Combines latent factor
analysis and 2-PL IRT model
• Here, equivalent of 2-PL IRT model with DIF
• No significant DIF was found
Results: Second data sample LWID
• Using remaining data, GLFA model run again, setting parameters based on calibration sample
• Separately by project– If significant, add DIF estimates to parameters
Results: SSRS
• IRT to GLFA model with Project DIF on full data
Results
Results: Response
• Proc mixed: covariance adjusted LWID score– 1169 children
Results: Response
• 648 treatment children
Results: Response
• 648 treatment children
UnresponsiveCutoff < 20%
N=110!
Results: Response
• 648 treatment children
UnresponsiveCut offFall SS = 95Spring SS= 104
MeanFall SS = 86Spring SS = 96
Results: Response
• 648 treatment children
UnresponsiveCut offFall SS = 95Spring SS= 104
MeanFall SS = 86Spring SS = 96
Responsive MeanFall SS = 99Spring SS = 111
Results
• Logistic regression– SSRS behavior problems significant predictor of
response status (OR = 1.45, CI = 1.12-1.88)• average behavior problems = 19% probability of being
“unresponsive”• greater than average behavior problems(+ 1SD) = 29%
probability of being “unresponsive”• Less than average behavior problems (-1SD) = 12%
probability of being “unresponsive”
Conclusions
• Response status is differentiated by behavior problems– Mo’ behavior problems, mo’ (reading) problems!
• The questionnaire data we will be adding will be real test of bioecological model on response to intervention
Overall IDA conclusions
• IDA is a “cheap” way to get more power, more n at tails, and show more generalizable effects
• Given how similar many of our projects are, consider doing item-level data entry – Easy potential to combine data– Can you do factor analysis and IRT? You can do IDA!
• These data are more useful together than apart– IRT within and between samples?– Treatment effectiveness across samples?– Characteristics of lowest responders?
Acknowledgements • Stephanie Al Otaiba • Carol Connor• Chris Schatschneider• Great staff & grad students, and a small army of data
enterers
NICHD grant HD072286