Investigating Educators’ Statistical Literacy and Score Report Interpretation

Saad Chahine, PhD April 30, 2013 NCME | San Francisco

13

-04

-30

Direction

• Need for research on how educators read and interpret score reports

• Current research focuses on the culture of data use in schools

• This research study asks the question:

How can the design score reports support the utility of achievement data by personnel in the school system?

Research Questions

1. How are educators proficient in statistical literacy?

2. How do educators interpret and propose to use test score reports?

ππ

33

Statistical Literacy

Score Report Interpretation

Conceptual Framework

“habit of mind” as a metaphor

Methods

• Cognitive interviews with 20 educators

• Statistical literacy tasks

• Two mock score reports

• Six stage protocol analysis

ππ

77

ππ

88

Example Statistical Literacy Task

Coopers Score Report ππ

99

Jabberwocky Score Report ππ

1010

13-04-30 Mount Saint Vincent University 11

Skill Minimum Maximum Mean Std. D.

Statistical Literacy

       

Describing 3.00 4.00 3.77 0.24

Organizing and Reducing 2.00 4.00 2.89 0.65

Representing 1.67 4.00 3.33 0.65

Analyzing and Interpreting 2.00 4.00 3.38 0.53

Score Report Interpretation: Cooper’s Fitness Test Score Report

Describing 4.00 4.00 4.00 0.00

Summarizing 3.00 4.00 3.75 0.44

Questioning 1.00 4.00 2.10 0.09

Proposed Application 1.00 4.00 2.45 1.10

Score Report Interpretation: Jabberwocky Test Score Report

Describing 4.00 4.00 4.00 0.00

Summarizing 3.00 4.00 3.75 0.44

Questioning 1.00 4.00 1.95 0.89

Proposed Application 1.00 4.00 2.00 0.97

ππ

1212

Discussion P1

• Data Habit of Mind represents a starting point

• Educators on average appear to have well developed statistical literacy skills

• Educators on average appear to struggle on score report interpretation skills: Questioning & Proposing an Application

ππ

1313

Discussion P2

• We can begin to identify clusters and patterns

• Potentially there are four ways in which we can cluster the patterns that we see when it comes to data habit of mind

ππ

1414

4-Way Data Habit of Mind

ππ

1515Mount

Sain

t V

ince

nt

Univ

ers

ity

Mount

Sain

t V

ince

nt

Univ

ers

ity

Thank You

saad.chahine@msvu.ca

in@saadchahine

@saadchahine

ππ

1616