Using accurate data to drive decision making

Paula BrownDirector of Instructional Accountability

Hampton City SchoolsOctober 2010

1

CongratulationsYour schools have worked very hard

Most of you represent localities where all schools are

fully accreditedMost are making AYP (even with

changing achievement goals)

Rising Goals2009-2010

English: 81% passingMath: 79% passing

With most schools reaching the 90’s in core subjects, the question then becomes how do we reach the last 10 percent?

The Last 10 PercentToday, we are going to look closely at a process

for getting the last 10% to be successful on SOL assessments

100% passing is an attainable goaland many great instructional practices will

workIF…

they are being driven by accurate data

Data SourceAfter SOL testing, we can retrieve our SOL

assessment data in a CSV file from Pearson Access

We can retrieve that file by division and by each individual school

From that file, we make pivot tables and report outcomes

Today, I want to show you another way to use that file to give your teachers and administrators very clear direction about where to work

5

6

Data SourceAfter the file is downloaded, we often cut the

file down by deleting columns that make the file a little cumbersome to work with; especially at the school level

We also usually change it to an Excel file for ease of use

Most of the time we keep the following labels:School name, group (teacher)name, student

information, test name, reporting categories, overall scaled score and proficiency rating

7

EIMSEIMS is another source for data retrieval that

is often in a user friendly formatOver the years, it has become easier to

manipulate and gives many schools and school divisions the opportunity to view their data more specifically

It also allows the viewer to compare data over several years in a variety of ways

8

9

Very Helpful Data

This report is very helpful for getting pass rates for a variety of students and individual teacher pass rates

EIMSOne of those reports includes scaled category

data for each SOL assessmentThis data was useful initially, but as pass

rates have increased, it tends to send educators in the wrong direction

The scaled score compares categories as if they are equal in number of questions and it uses a different pass bar for each reporting category

These two factors diminish its usefulness

10

11

VERY Unhelpful Data

This reports that for this school in grade 3 science the category of weakness is Scientific Investigation with 76% passing the category scaled mark of 30. The strength for this school is identified as Earth/Space Systems.

Accurate DataBy using the method that I will show you today,

you will see that the actual weakness for this school was

Life Processes and Living Systems and that Scientific Investigation was the strongest area of performance

12

Why Do You Need This ProcessALL instructional decisions and practices must be based on accurate data

Misleading DataEveryone data

This data includes all three performance groups

FailingPassing proficientPassing advanced

Operative DefinitionsFailing students are those who received a

scaled score from 0 to 399Passing students are those who received a

scaled score from 400 to 499Advanced students are those who received

a scaled score from 500 to 600

15

Data FactsLook atFailing data to change pass ratesPassing data to change advanced ratesLook at category data by raw numbers to

identify accurate areas of weakness in both performance groups

Look at skill data (SPBQ) to narrow down another step Failing students Passing students

Subtract the Average Raw Score in each category from the total number of questions. This identifies the size of the gap. Reducing the largest gap is the fastest path to student success on state tests

This figure shows the largest gap (by a tiny margin) to be in the area of Computation and Estimation

GAP

2.61

Everyone Data

2.912.93

1.701.68

This is the same grade level at the same school, but this time the passing students have been removed.

This time the gap is much wider between the categories. This narrows the focus to

Computation and Estimation.

GAP

5.865.00

3.142.714.64

Failing Data

SPBQ Report: Everyone data

It looks like there are four skills that are the weakest in this category…

SPBQ Report: Failing students

BUT, when we take out the passing kids we see that “finding the product” ( or multiplication) has the lowest three scores(44%, 48%, 56%) followed by subtraction

How Do You Do It?Providing this information to teachers and

administrators will change how they look at data and how they use it to make decisions

In Hampton City Schools, we use SOLAR to provide this data to our schools

Today, I want to show you how to do this with your Pearson data file and how to use it once you have created it

21

How Do You Do It?1. Take your raw data file (CSV) and convert it to an

Excel file for easier use2. Eliminate all of the columns except test name,

reporting categories 1-7 and the total scaled score

3. Using the sort feature, sort the edited file by test name

4. Select, cut and paste each test name (with its adjoining data) into its own data sheet

5. Label the tabs with the test names as you create each sheet (math, science and social studies)

6. You can delete the reading scores for this activity

22

How Do You Do It?7. Make sure to retain the labels at the top for each

new page that you create ( see step 2)8. Using the sort feature, sort the edited file by

scaled score (lowest to highest or highest to lowest)

9. Add rows in the data to visually separate the failing scores from the passing scores and the passing scores from the advanced scores

10.Using the test blueprint, rename the categories by their true name (i.e. number and number sense, computation and estimation, etc.) and delete extra columns

23

How Do You Do It?11.Add the number of questions in the category next

to the name (i.e. NNS 8q, C&E 12q, M&G 12q, etc.)12.Starting with the failing scores, use the averaging

function provided in Excel to average each column and type in that number at the end of each row

13.Then subtract the averaged number from the total number of questions (i.e 8-6.53=1.47)

14.That creates a question gap (1.47 questions is the gap)

15.Do this for each category…the larger the gap the bigger the problem

24

How Do You Do It?12.Once all of the gaps have been identified for

the failing data, then do the same process for the passing data.

13.Enter all of the gap data onto a collection chart

14.This will show the schools and teachers exactly where the areas of weakness are and how to structure their review plans

15.Doing this process over a few years, really shows entrenched practices and cultures that haven’t changed

25

26

This is a copy of a collection chart developed in Hampton City Schools

The category of weakness for both achievement groups is Patterns, Functions and Algebra

What is the Next Step?Using the data to plan

Yearlong changes to the pacing guide (usually in the form of adding review time or pre-teaching some SOLs) Example: Science review booklet

All remediation programs focus on category of weakness After and before school programs Tutoring and volunteers Summer School

27

Things to WatchBe careful of review plans that review

everythingThis leads to a repeat of the scores earned

last yearIt also causes the large categories to be short

changed and therefore leads to the failure of many students who could have passed

It doesn’t matter what you believe, if you ignore the size of the large categories, they will be the root of student failure

28

29

Reviewing Evenly

The Next Step: Skill DataIn order to divide skill data into passing and failing

groups, you will need some very advanced Excel skills or you can use another manual process to do this: 1. Print out a copy of the SPBQ report for each student

(School level or classroom level)

2. Divide the copies into three stacks: failing, passing and advanced(set the advanced aside, you will not need them for this activity)

3. Number a sheet of paper to match the number of questions on the test

30

The Next Step: Skill Data4. Take the failing stack and make a mark every

time a student misses a question5. When you are done with the stack, look at the

category you identified earlier as the weakness6. Identify the questions in that category where

there was the most failure7. This process will narrow your focus even deeper

31

32

Highly missed skills

The First Step: Skill DataEnglish focus skills

Seven skills per grade level Context clues Main idea, summarizing Drawing conclusions Author’s purpose Ask and Answer questions Characterization Cause and Effect

Must be reviewed to mastery

English Skill Data1. Using the individual SPBQ sheets, do a tally

of missed skills (see slide 32)2.Combined questions that are for the same

skill3.Enter the data onto the data collections sheet

(see slide 35)4.Identify the focus skills (they are shaded)5.Order them from lowest to highest

performing

34

35

100% Passing

We can ALL

reach this goal