Evaluating the State Systemic Improvement Plan (SSIP) ECTA Center National Center for Systemic...
-
Upload
victoria-quinn -
Category
Documents
-
view
215 -
download
0
Transcript of Evaluating the State Systemic Improvement Plan (SSIP) ECTA Center National Center for Systemic...
Evaluating the State Systemic Improvement Plan (SSIP)
ECTA CenterNational Center for Systemic Improvement
DaSy
Evaluating the Implementation
• Built into the plan from the beginning• Based on your theory of action• Based on data that informed the plan
development• Formative data and summative data• Evidence to show progress
Evidence of Progress
• Activities occurred and intended outcomes of each activity accomplished
• Changes are occurring at the system, practice and individual level
For Each Activity…
• Did the activity occur?– If yes, what evidence exists that it occurred?– If not, why not?– What needs to be done next?
• Did the activity accomplish the intended outcomes?– If yes, what evidence exists that the intended
outcomes were accomplished?– If not, why not?– What needs to be done next?
If the Activity is an Output/Product e.g. Training Resources
• How will we know if the activity occurs?– Evidences:
• Training materials developed• Training materials used as intended
• How will we know if the activity accomplishes the intended outcomes?– Evidences:
• Data indicating number frequency of training• Data indicating trainer fidelity and competency
If the Activity is a Process e.g. Training for Practitioners
• How will we know if the activity occurs?– Evidences
• Data indicating number of trainings• Participation records
• How will we know if the activity accomplishes the intended outcomes?– Evidences
• Data indicating frequency of use by participants• Measure of competencies
For Each Level of the System of Support…
• What are the guiding evaluation questions?• What are our performance measures to know
we’ve been successful?• What are the data sources for the
performance measures?
Measuring Results
• Results questions(s):– Have student outcomes improved?– Are school climates more collaborative?
• Performance measure:– % of students proficient and advanced– Improved collaboration among building educators
• Data sources:– MAP building outcome data– Self assessments and observations
Measuring Practices
• Practice-level questions:– Are more teachers implementing effective practices?– Is the use of effective practices increasing
• Performance measures:– % of practitioners implementing effective practices– % of buildings with 80% or more practitioners
implementing at a high level– Frequency of effective practices use
• Data sources:– Self-assessments, observations – Teacher evaluation summary data
Measuring Local Capacity
• Local system-level questions:– How has the local infrastructure been improved to
support implementation of the effective practices?• Performance measure:
– Changes to local system (governance, finance, personnel—TA and PD, data systems, expectations, teachers feel supported
• Data sources:– Collaborative climate, effective practices and data use
are part of the educator evaluation process
Measuring State Capacity
• State system-level questions:– How has the infrastructure been improved to support
implementation of the SSIP?• Performance measure:
– Changes to state systems (governance, finance, personnel—TA and PD, data systems, quality improvement, focus
• Data sources:– Plan process data sources– Plan outcome data sources– Observations related to time and focus
CW Data Update
March 2015Year 3
CW Events Associated w/ CW Buildings 7/1/2014 - 2/24/2015 by Type
RPDC
1 RPDC
2 RPDC
3 RPDC
4 RPDC
5 RPDC
6 RPDC
7 RPDC
8 RPDC
9
Professional Development
129 78 33 37 34 21 55 36 29
Coaching 48 30 48 88 39 65 45 202 14
Planning/ Communication/ Collaboration
50 19 59 19 31 11 80 70 31
State Meeting/ Conferences
0 0 0 0 1 0 0 0 0
Technical Assistance
12 8 2 38 0 7 0 119 14
Training 92 32 65 22 33 74 25 53 69
Other 0 0 2 1 0 0 0 1 0
Total 331 167 209 205 138 178 205 481 157
CW Events Associated w/ CW Buildings 7/1/2014 - 2/24/2015 by Type
RPDC
1 RPDC
2 RPDC
3 RPDC
4 RPDC
5 RPDC
6 RPDC
7 RPDC
8 RPDC
9
Professional Development
39.0% 46.7% 15.8% 18.0% 24.6% 11.8% 26.8% 7.5% 18.5%
Coaching 14.5% 18.0% 23.0% 42.9% 28.3% 36.5% 22.0% 42.0% 8.9%
Planning/ Communication/ Collaboration
15.1% 11.4% 28.2% 9.3% 22.5% 6.2% 39.0% 14.6% 19.7%
State Meeting/ Conferences
0.0% 0.0% 0.0% 0.0% 0.7% 0.0% 0.0% 0.0% 0.0%
Technical Assistance
3.6% 4.8% 1.0% 18.5% 0.0% 3.9% 0.0% 24.7% 8.9%
Training 27.8% 19.2% 31.1% 10.7% 23.9% 41.6% 12.2% 11.0% 43.9%
Other 0.0% 0.0% 1.0% 0.5% 0.0% 0.0% 0.0% 0.2% 0.0%
Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%
CW Hours Associated w/ CW Buildings 7/1/2014 - 2/24/2015 by Type
RPDC
1 RPDC
2 RPDC
3 RPDC
4 RPDC
5 RPDC
6 RPDC
7 RPDC
8 RPDC
9 Planning/ Communication/ Collaboration
474 439 519 449 394 326 757 1732 725
Training 1012 382 648 585 512 533 292 602 247
Meetings 0 0 0 0 12 0 44 0 0
Travel 205 131 218 338 155 228 147 400 116
CW Events Associated w/ CW Buildings 7/1/2014 - 2/24/2015 by Topic
RPDC
1 RPDC
2 RPDC
3 RPDC
4 RPDC
5 RPDC
6 RPDC
7 RPDC
8 RPDC
9
Assessment-Capable Learners
28 39 34 15 14 39 35 28 2
Collaborative Data Teams 48 3 95 8 64 8 1 109 23
Common Formative Assessment
52 27 1 28 20 40 21 45 60
Collaborative Work Overview
91 14 16 28 6 21 25 111 22
Data-Based Decision Making 23 39 7 57 11 33 0 95 9
Effective Teaching/Learning Practices
19 0 1 16 1 6 43 1 2
Feedback 18 10 12 33 4 5 30 2 5 Metacognition 0 0 3 0 0 2 5 1 8 Reciprocal Teaching 27 23 33 20 15 24 36 78 13 Spaced vs Massed Practice
14 12 7 0 0 0 9 11 8
Other 11 0 0 0 3 0 0 0 5
CW Events Associated w/ CW Buildings 7/1/2014 - 2/24/2015 by Topic
RPDC
1 RPDC
2 RPDC
3 RPDC
4 RPDC
5 RPDC
6 RPDC
7 RPDC
8 RPDC
9
Assessment-Capable Learners
8.8% 23.4% 16.3% 7.3% 10.4% 21.9% 17.1% 5.8% 1.3%
Collaborative Data Teams 15.0% 1.8% 45.5% 3.9% 47.4% 4.5% 0.5% 22.7% 15.1%
Common Formative Assessment
16.3% 16.2% 0.5% 13.7% 14.8% 22.5% 10.2% 9.4% 39.5%
Collaborative Work Overview
28.4% 8.4% 7.7% 13.7% 4.4% 11.8% 12.2% 23.1% 14.5%
Data-Based Decision Making 7.2% 23.4% 3.3% 27.8% 8.1% 18.5% 0.0% 19.8% 5.9%
Effective Teaching/Learning Practices
5.9% 0.0% 0.5% 7.8% 0.7% 3.4% 21.0% 0.2% 1.3%
Feedback 5.6% 6.0% 5.7% 16.1% 3.0% 2.8% 14.6% 0.4% 3.3% Metacognition 0.0% 0.0% 1.4% 0.0% 0.0% 1.1% 2.4% 0.2% 5.3% Reciprocal Teaching 4.4% 7.2% 3.3% 0.0% 0.0% 0.0% 4.4% 2.3% 5.3% Spaced vs. Massed Practice
8.4% 13.8% 15.8% 9.8% 11.1% 13.5% 17.6% 16.2% 8.6%
Other 3.4% 0.0% 0.0% 0.0% 2.2% 0.0% 0.0% 0.0% 3.3%
Effective Teaching Practice of CFA by RPDC
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9 Unselected 10 2 2 0 0 0 3 15 9
Assessment Capable Learners 173 72 223 35 217 252 52 228 0 Classroom Discussion 0 10 1 0 0 0 0 2 0 Cooperative Learning 0 0 1 0 0 0 0 75 0
Differentiated Instruction 1 2 13 3 0 0 1 233 0 Direct Instruction 0 54 19 5 7 6 2 201 0
Feedback 94 43 152 196 0 110 152 380 237 Increasing Student Engagement 0 0 1 0 12 0 3 25 0
Metacognition 0 0 0 0 0 0 0 2 0 Reciprocal Teaching 15 8 1 0 0 31 2 48 2
Spaced vs. Massed Practice 30 3 13 0 16 122 0 13 0 Teacher-Student Relationship 0 0 0 0 0 0 0 4 0
Total 323 194 426 239 252 521 215 1226 248
Effective Teaching Practice of CFA by RPDC
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9 Unselected 3.1% 1.0% 0.5% 0.0% 0.0% 0.0% 1.4% 1.2% 3.6%
Assessment Capable Learners 53.6% 37.1% 52.3% 14.6% 86.1% 48.4% 24.2% 18.6% 0.0% Classroom Discussion 0.0% 5.2% 0.2% 0.0% 0.0% 0.0% 0.0% 0.2% 0.0% Cooperative Learning 0.0% 0.0% 0.2% 0.0% 0.0% 0.0% 0.0% 6.1% 0.0%
Differentiated Instruction 0.3% 1.0% 3.1% 1.3% 0.0% 0.0% 0.5% 19.0% 0.0% Direct Instruction 0.0% 27.8% 4.5% 2.1% 2.8% 1.2% 0.9% 16.4% 0.0%
Feedback 29.1% 22.2% 35.7% 82.0% 0.0% 21.1% 70.7% 31.0% 95.6% Increasing Student Engagement 0.0% 0.0% 0.2% 0.0% 4.8% 0.0% 1.4% 2.0% 0.0%
Metacognition 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.2% 0.0% Reciprocal Teaching 4.6% 4.1% 0.2% 0.0% 0.0% 6.0% 0.9% 3.9% 0.8%
Spaced vs. Massed Practice 9.3% 1.5% 3.1% 0.0% 6.3% 23.4% 0.0% 1.1% 0.0% Teacher-Student Relationship 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.3% 0.0%
Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%
CFA Results for All Students by RPDC
All St
uden
ts
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9
Mid
-Instr
uction
Proficient and Higher 15767 5258 12490 2879 3508 18768 3453 22623 6134 Close to Proficient 8114 3790 9840 2056 1198 9908 1814 20931 4849
Far to Go 6455 2946 10294 1755 1984 7625 2561 26468 3893 Intervention Students 8188 2950 14159 1961 1746 6262 3565 30978 5579
Total 38524 14944 46783 8651 8436 42585 11393 101003 19092
After
Retea
ch Proficient and Higher 23732 7757 24907 5309 4723 26920 6050 50235 10128
Close to Proficient 7198 3781 10505 1671 1111 8691 1863 23361 4897 Far to Go 3862 1891 6387 766 1503 3939 1780 15847 2810
Intervention Students 3449 1525 5138 656 1096 2323 1494 10932 2856 Total 38241 14954 46937 8402 8433 41746 11187 100375 20691
Differe
nce Proficient and Higher 7965 2499 12417 2430 1215 8152 2597 27612 3994
Close to Proficient -916 -9 665 -385 -87 -1217 49 2430 48 Far to Go -2593 -1055 -3907 -989 -481 -3686 -781 -10621 -1083
Intervention Students -4739 -1425 -9021 -1305 -650 -3939 -2071 -20046 -2723 Total -283 10 154 -249 -3 -839 -206 -628 1599
CFA Results for All Students by RPDC
All St
uden
ts
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9
Mid
-Instr
uction
Proficient and Higher 40.9% 35.2% 26.7% 33.3% 41.6% 44.1% 30.3% 22.4% 30.0% Close to Proficient 21.1% 25.4% 21.0% 23.8% 14.2% 23.3% 15.9% 20.7% 23.7%
Far to Go 16.8% 19.7% 22.0% 20.3% 23.5% 17.9% 22.5% 26.2% 19.0% Intervention Students 21.3% 19.7% 30.3% 22.7% 20.7% 14.7% 31.3% 30.7% 27.3%
Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%
After
Retea
ch Proficient and Higher 62.1% 51.9% 53.1% 63.2% 56.0% 64.3% 54.1% 50.0% 48.9%
Close to Proficient 18.8% 25.3% 22.4% 19.9% 13.2% 20.8% 16.7% 23.3% 23.7% Far to Go 10.1% 12.6% 13.6% 9.1% 17.8% 9.4% 15.9% 15.8% 13.6%
Intervention Students 9.0% 10.2% 10.9% 7.8% 13.0% 5.5% 13.4% 10.9% 13.8% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%
Differe
nce Proficient and Higher 21.1% 16.7% 26.4% 29.9% 14.4% 20.2% 23.8% 27.6% 19.0%
Close to Proficient -2.2% -0.1% 1.3% -3.9% -1.0% -2.5% 0.7% 2.5% 0.0% Far to Go -6.7% -7.1% -8.4% -11.2% -5.7% -8.5% -6.6% -10.4% -5.5%
Intervention Students -12.2% -9.5% -19.3% -14.9% -7.7% -9.2% -17.9% -19.8% -13.5%
CFA Results for Students with Disabilities by RPDC
SWDs
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9
Mid
-Instr
uctio
n Proficient and Higher 959 261 469 95 271 1322 116 1361 208
Close to Proficient 707 239 505 130 135 1298 71 1516 272 Far to Go 813 302 758 182 216 1603 210 2772 346
Intervention Students 1539 505 1723 335 381 1483 596 5023 1163 Total 4018 1321 3455 742 1003 5433 993 10672 2006
Not Assessed 367 24 124 49 47 492 106 917 111
After
Rete
ach
Proficient and Higher 1679 414 1071 281 370 2348 314 3359 430 Close to Proficient 810 295 795 195 124 1259 143 2340 387
Far to Go 609 288 765 125 211 903 197 2388 363 Intervention Students 857 278 872 130 299 815 330 2548 792
Total 3955 1305 3503 732 1004 5293 984 10604 2028 Not Assessed 416 57 120 43 44 531 106 884 89
Diffe
renc
e
Proficient and Higher 720 153 602 186 99 1026 198 1998 222 Close to Proficient 103 56 290 65 -11 -39 72 824 115
Far to Go -204 -14 7 -57 -5 -700 -13 -384 17 Intervention Students -682 -227 -851 -205 -82 -668 -266 -2475 -371
Total -63 -16 48 -10 1 -140 -9 -68 22 Not Assessed 49 33 -4 -6 -3 39 0 -33 -22
CFA Results for Students with Disabilities by RPDC
SWDs
RPDC 1 RPDC 2 RPDC 3 RPDC 4 RPDC 5 RPDC 6 RPDC 7 RPDC 8 RPDC 9
Mid
-Instr
uctio
n Proficient and Higher 23.9% 20.0% 13.6% 12.8% 27.0% 23.2% 11.7% 12.8% 10.5%
Close to Proficient 17.6% 18.3% 14.6% 17.5% 13.5% 22.7% 7.2% 14.2% 13.7% Far to Go 20.2% 23.1% 21.9% 24.5% 21.5% 28.1% 21.1% 26.0% 17.4%
Intervention Students 38.3% 38.6% 49.9% 45.1% 38.0% 26.0% 60.0% 47.1% 58.5% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%
Not Assessed N/A N/A N/A N/A N/A N/A N/A N/A N/A
After
Ret
each
Proficient and Higher 42.5% 32.5% 30.6% 38.4% 36.9% 44.1% 31.9% 31.6% 21.8% Close to Proficient 20.5% 23.1% 22.7% 26.7% 12.4% 23.6% 14.5% 22.0% 19.6%
Far to Go 15.4% 22.6% 21.8% 17.1% 21.0% 17.0% 20.0% 22.5% 18.4% Intervention Students 21.7% 21.8% 24.9% 17.8% 29.8% 15.3% 33.5% 24.0% 40.2%
Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Not Assessed N/A N/A N/A N/A N/A N/A N/A N/A N/A
Diffe
renc
e
Proficient and Higher 18.6% 12.5% 17.0% 25.6% 9.8% 20.9% 20.2% 18.8% 11.3% Close to Proficient 2.9% 4.9% 8.1% 9.2% -1.1% 0.9% 7.4% 7.8% 5.9%
Far to Go -4.8% -0.5% -0.1% -7.4% -0.5% -11.1% -1.1% -3.5% 1.0% Intervention Students -16.6% -16.8% -25.0% -27.4% -8.2% -10.7% -26.5% -23.1% -18.3%
Total N/A N/A N/A N/A N/A N/A N/A N/A N/A Not Assessed N/A N/A N/A N/A N/A N/A N/A N/A N/A