The interaction of measurement, models and accountability: How values affect our growth model...
12
The interaction of measurement, models and accountability: How values affect our growth model choices. Jennifer L. Dunn Center for Assessment
-
Upload
julian-bridges -
Category
Documents
-
view
217 -
download
2
Transcript of The interaction of measurement, models and accountability: How values affect our growth model...
The interaction of measurement, models and accountability: How values affect our growth model choices.
Jennifer L. DunnCenter for Assessment
Center for Assessment2
Introduction Can NCLB accountability be improved by
growth?What do we mean by growth?
Learning that occurs beyond what the student knew when they entered the system
Movement towards proficiency
Is the construct being measured? Measurement models Accountability system One should not be considered without the other.
Center for Assessment3
Purpose Explore the properties of two NCLB growth
pilot modelsRelationship scores and classifications to
Status and Scaled score changes
Student performance in schools that are ‘helped’ by the growth model
Examine the similarities and differences of the models
Growth of proficient students, amount of required growth, and how the models value growth
Apply multiple models to the same data
Center for Assessment4
Year 1 Performance
Year 2 Performance Level
One Two Proficient Four Five
Level 1 0 200 300 300 300
Level 2 0 150 300 300 300
Proficient 0 0 300 300 300
Level 4 0 0 300 300 300
Level 5 0 0 300 300 300
NCLB Value Table
School Score = Average of value table points
Center for Assessment5
Initial Performance Growth Target
< Proficient - 1SD 1/3 of the Distance
> Proficient - 1SD &< Proficient - ½ SD
½ of the Distance
> Proficient - ½ SD &< Proficient + ¼ SD
Proficient
> Proficient + ¼ SD &< Proficient + 1¼ SD
Score – ¼ SD
> Proficient+ 1¼ SD Proficient + 1 SD
NCLB Change Model
School Score = % of students who meet or exceed target
6
Year 1 Status
0 20 40 60 80 100
05
01
50
25
0
Year 2 Status
0 20 40 60 80 100
05
01
50
25
0
Scaled Score Change
-0.4 -0.2 0.0 0.2 0.4
05
01
50
25
0
Year 1 Status
0 20 40 60 80 100
0.2
0.4
0.6
0.8
Year 2 Status
0 20 40 60 80 100
0.2
0.4
0.6
0.8
Scaled Score Change
-0.4 -0.2 0.0 0.2 0.4
0.2
0.4
0.6
0.8
NCLB Value Table NCLB Change
7
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
Sca
led
Sco
re C
ha
ng
e
Status Score
NCLB Value Table NCLB Change
PassFail
AMO = 20%
AMO = 50%
AMO = 80%
8
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
0 20 40 60 80 100
-0.4
-0.2
0.0
0.2
0.4
Sca
led
Sco
re C
ha
ng
e
Status Score
NCLB Value Table NCLB Change
PassFail
AMO = 20%
AMO = 50%
AMO = 80%
Center for Assessment9
Year 1 Performance
Year 2 Performance Level
One Two Proficient Four Five N Student
Level 1 56.7 25.6 7.3 0.4 277
Level 2 19.3 34.8 44.4 1.5 342
Proficient 2.8 8.9 70.5 14.7 3.1 1,758
Level 4 0.5 35.9 37.3 26.1 568
Level 5 7.9 24.8 67.3 355
Center for Assessment10
Initial Performance Growth TargetN
Students
% Meet
< Proficient - 1SD 1/3 of the Distance 111 46.0
> Proficient - 1SD &< Proficient - ½ SD
½ of the Distance 160 43.1
> Proficient - ½ SD &< Proficient + ¼ SD
Proficient 535 61.7
> Proficient + ¼ SD &< Proficient + 1¼ SD
Score – ¼ SD 884 87.7
> Proficient+ 1¼ SD Proficient + 1 SD 514 92.8
Total 2204 77.2
Center for Assessment11
Summary of Results
NCLB growth models:Are more related to status than scaled score
changes Relationship between scaled score change and the
NCLB change model is stronger than the VT model
“Help” more schools as the AMO increasesValue moving students toward proficient while
preventing students from falling below proficient
Center for Assessment12
Conclusion Potential to Improve NCLB accountability? It depends:
We must carefully consider: The construct, the model, the system, & the
interactions The use growth in accountability
Replace Status Multi-tier system
Adjusting the NCLB Core Principles Inclusion Different growth standards for different students Moving beyond counts of students
