EVAAS Concepts: NCEs and Standard Errors Sandy Horn January 2013

Copyright © 2010, SAS Institute Inc. All rights reserved.

EVAAS Concepts:NCEs and Standard Errors

Sandy HornJanuary 2013

SAS® EVAAS® for K-12

2


Similarities between the EVAAS Growth Standard Model and the Predictive ModelFor both models:• Progress is measured relative to the

average progress for a grade/subject in North Carolina in the year tested.

• Average Progress/Growth Standard is 0.0.

• Standard errors determine whether a district, school, or teacher’s growth measure is different from average. (More on this, shortly)

3


Differences between the EVAAS Growth Standard Model and the Predictive Model

Growth Standard Model Only used with tests

administered in sequential years.

Does not predict where students will score, so doesn’t need three prior test scores.

Predictive Model Can be used with tests,

whether administered sequentially or not.

Requires at least three prior test scores to predict where students will score, relative to other NC students who take the same test. (Prior scores don’t have to be in the subject being analyzed.)

4


Differences between the EVAAS Growth Standard Model and the Predictive Model

Growth Standard Model Measures the difference

between a cohort’s position in the state distribution in a grade/subject at the end of one year and their position in that distribution at the end of the next year.

Reported in NCEs.

Predictive Model Measures the difference

between where students would be expected to score, assuming the average progress, statewide, and where they did score, when tested.

Reported in Scale Score Points.

5


Growth Standard Value Added Report

6


Normal Curve Equivalent Units - NCEs

7


NCEs and the Growth Standard

30 40 50 60 70

5th Grade:NCE 37

6th Grade:NCE 37

The State Growth Standard (0.0) is achieved when students do not lose ground from year to year, relative to other students, across the

state, who take the same test. It signifies one year’s growth.

6th grade NCE 37 ― 5th grade NCE 37 = 0 = State Growth Standard30 40 50 60 70

8



9


What are Standard Errors, and Why aren’t they all the same?Standard Errors let us know how confident we can be that a sample approximates a population. In other words, how sure can we be that our students (a sample) made progress similar to or different from the progress that is average for all the students in North Carolina (the population)?

If we have a lot of students and/or a lot of data points (test scores) and/or somewhat complete data for our sample and so on, we can be pretty precise in estimating the progress of that group of students. The standard error would be small.

If we have few students and/or few data points and/or very spotty data, we can’t be as certain of the estimated progress of that group of students. The Standard Error would be large.

10


Standard Errors are Protection against Misclassification

Small negatives and small positives may not be significantly different from Average. The standard error lets you know whether the difference is significant or not.

Unless there is strong evidence to the contrary, standard errors ensure that an estimate is considered “Average.” They protect districts, schools, and teachers from being misclassified as different from Average unless there is a significant difference.

11


About those Standard Errors…How much confidence do we have that the estimate is different from 0.0?

0.0Growth Standard

State Average

-3se -2se -1se +1se +2se +3se

80% 80%90% 90%

95%+ 95%+

99% 99%

Meets Expected Growth ExceedsExpected Growth

Does Not Meet Expected Growth

12



6th grade NCE 38.8 ― 5th grade NCE 37 = 1.8 NCEs Is it different from the State Growth Standard?

30 40 50 60 70

5th Grade:NCE 37

6th Grade:NCE 38.8

30 40 50 60 70

+1.8 NCEs above 0.0

Growth Measure: 1.8 NCEs/points

0.0

13



+1.8 NCEs above 0.0

School 1:Growth Measure: 1.8 NCEsSE: 1.1 NCEs+/- 2 SEs = 2.2 NCEs

+2.2 NCEs

-2.2 NCEsGrowth Std/State Avg (0.0)

School 1 Meets Expected Growth because its Growth Measure is less than 2 SEs above or below the Growth Standard.

School 2:Growth Measure: 1.8 NCEsSE: 0.8 NCEs+/- 2 SEs = 1.6 NCEs

+1.6 NCEs

-1.6 NCEs

School 2 Exceeds Expected Growth becauseits Growth Measure is 2 SEs or more abovethe Growth Standard.

It works just the same for the Predictive Model.Just change NCE to Scale Score and Growth Standard to State Average.

14



EVAAS Concepts: NCEs and Standard Errors Sandy Horn January 2013

Documents

Transcript of EVAAS Concepts: NCEs and Standard Errors Sandy Horn January 2013