Power Standards and SMART Goals; Expectations for Assessment and Data-based Decision-Making....
-
Upload
mark-norris -
Category
Documents
-
view
215 -
download
0
Transcript of Power Standards and SMART Goals; Expectations for Assessment and Data-based Decision-Making....
Power Standards and SMART Goals; Expectations for Assessment and
Data-based Decision-Making.Presented by:
Ben Ditkowsky, Ph.D.
Waukegan Unit School District 60
Power Standards
• A power standard refers to those few things that are really important to teach (Reeves). – BIG IDEAS (Kame’enui & Carnine)– Enduring understandings (Wiggins & McTigue)
• The key question here seems simple:• What is it that we want students to know
and be able to do?
The practical reality of NCLB
If we are going to be able to “leave no child behind” the state standards, then we are going to have to have a way of determining how far behind they are, from where we expect them to be, so that we can do something to accelerate their academic growth.
…a way of determining how far behind they are, from
where we expect them to be
That means we need to use data to make decisions
Uses of data? + -Good Use
Plan and group for instruction
Monitor student progress for a goal
Identify a student as “at –risk”
Identify new goals for students who have exceeded
Promote student reflection and goal setting
To adjust teaching practices
Poor Use
Grade a pretest and send home
To ensure file folders are not empty
Meet district compliance
Lock students into a group
Make high – stakes decision based on one test
To decide who has failed to meet standards (grade retention)
Increase teacher anxiety
Plan and group for instructionPlan and group for instructionGrade a pretest and send homeGrade a pretest and send homeTo ensure file folders are not emptyTo ensure file folders are not emptyMeet district complianceMeet district complianceMonitor student progress for a goalMonitor student progress for a goalIdentify a student as “at –risk”Identify a student as “at –risk”Identify new goals for students who have Identify new goals for students who have exceeded expectationsexceeded expectations
Lock students into a groupLock students into a groupMake high – stakes decision based Make high – stakes decision based on one teston one test
To decide who has failed to meet To decide who has failed to meet standards (grade retention)standards (grade retention)Promote student reflection and goal settingPromote student reflection and goal settingTo adjust teaching practicesTo adjust teaching practicesIncrease teacher anxietyIncrease teacher anxiety
When we use data we need to FOCUS: Things to consider
• Entry level skills– If students come in with less skill than we expect
then we need to get them to learn more with the same amount of time.
• Efficiency and Effectiveness of Instruction– Time on task– Intensity of instruction
• Data-based decision-making– Frequency of Assessment– Use of data to drive instructional decision-making
Entry Level Skills
• Typical students have a benefit over low performing students.• They come in higher, so they don’t need to gain as much• Keep in mind that we need to make sure students have
prerequisite skills
MoreSkill
LessSkill
Instruction
• To make up the difference, instruction should:– Be focused on the goal– Maintain student time on-task– Provide sufficient practice to ensure students have the opportunity to learn
MoreSkill
LessSkill
Assessment
• The more dire the student needs, the more carefully we should watch their progress.
• When students are not progressing, we need to change what they receive in terms of instruction.
• The more dire the student needs, the more carefully we should watch their progress.
• When students are not progressing, we need to change what they receive in terms of instruction.
Monitoring progress requires us to pay attention
• Is the instruction working?• Is it intense enough?• Do we need to change what we are
doing?MoreSkill
LessSkill
Use Assessments from the Curriculum
• To determine when to change instruction
• Rule: 3 data points below the aim-line
MoreSkill
LessSkill
Set Goals, and make them smart!• Goals should be
Specific and Strategic
• Goals should be Observable and
Measurable
• Goals should be Acceptable
Action oriented and Attainable
• Goals should be Realistic and
Relevant
• Goals should be Trackable and
Time-bound
•
S•
M
•
A•
R
•
T
How do we know what goals to set?
Use Power StandardsPower Standards:
What is it that we want students
to be able to DO?
Curriculum-Based Measurement
• Does how quickly a student can orally read relate to how well they do on ISAT?
• Will a 5th grade student who…– … reads 120 cwpm meet standards? – What about a student who can read 150
cwpm, will they meet?
Think – Pair -Share
1. Does fluency matter? Yes/No
2. How many words should a student in your class be able to read in a minute?
3. Why do you think so?
Does Fluency really matter?
1. Does it matter? Yes/No
2. How many words should a 5th grade student read in a minute?
Let’s consider some data…
• How many correct words did 5th grade students read in one minute?
• Does the number of words that students can read in one minute relate to how well they do on the ISAT?
Kelsey read 120 words
Dorothy read 225 words
Sarah read 147 words
Mark read 175 words
How many words can they read in a minute?
Let’s reorganize things
• Some times we need to change the way we look at things before we can FOCUS.
Setting up a chart to compare scores
Exceeds
Meets
Below
Warning
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
Before ISAT testing what we knew was how Before ISAT testing what we knew was how fast students read a grade level passage.fast students read a grade level passage.
Kelsey read 120 words
Dorothy read 225 words
Sarah read 147 words
Mark read 175 words
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
Setting up a chart to compare scores
A note about “Scatter – Plots”
Scatter-Plot can help us to see– How scores from two different tests
relate to each other. – Where we should set cut-points for
students meeting expectations – How we can increase student’s
chances to meet ISAT Standards.
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
Looking at relations
Let’s see how Dorothy did on ISAT…
Dorothy read 225 Words in a minuteAnd Dorothy obtained a 169 on ISAT
How did she do?
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
Let’s consider…
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
19
0
20
0
21
0
22
0
23
0
24
0
25
0
26
0
27
0
28
0
29
0
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
100% of (62, 5th grade) students who read less than 120 wordsper minute were below state standards on ISAT
Remember Kelsey?
Students who read less than 120 correct words per minute
She read 120 words correctly in a minuteShe obtained a score of 150 on ISAT
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
19
0
20
0
21
0
22
0
23
0
24
0
25
0
26
0
27
0
28
0
29
0
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
100% of (62, 5th grade) students who read less than 130 wordsper minute were below state standards on ISAT
Students who read less than 130 correct words per minute
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
19
0
20
0
21
0
22
0
23
0
24
0
25
0
26
0
27
0
28
0
29
0
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
Students who read less than 140 correct words per minute
77% of (62, 5th grade) students who read less than 140 wordsper minute were below state standards on ISAT
What can we infer about students who are not fluent readers and ISAT?
(<150 cwpm)
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10
20
30
40
50
60
70
80
90
10
0
11
0
12
0
13
0
14
0
15
0
16
0
17
0
18
0
19
0
20
0
21
0
22
0
23
0
24
0
25
0
26
0
27
0
28
0
29
0
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
75% of (62, 5th grade) students who read less than 150 wordsper minute were below state standards on ISAT
Think – Pair - Share
What are two reasons why you think the speed at which our students read influences their scores on high – stakes tests?
1.There is time to complete the test.
2. They don’t have to spend time decoding, instead they can spend time comprehending.
Let’s consider just the students who read more than 150 correct words per minute
Grade 5
0
20
40
60
80
100
120
140
160
180
200
220
240
0 10 20 30 40 50 60 70 80 90 100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
90% of (62, 5th grade) students who read more than 150 wordsper minute were above state standards on ISAT.
Remember Mark?
And Dorothy?
Place your bets
• If you were going to bet that a student was going to meet expectations on the ISAT who would you choose?– A student who reads 120 cwpm?
• 100% of our sample did not
– A student who reads 130 cwpm?• 100% of our sample did not
– … 140 cwpm? Or 150 cwpm?• 77% and 75% of our sample did not
– What about a student who reads more than 150 cwpm?• 90% of our sample of students reading more than 150
cwpm met State Standards.
– Pick your student.
For 5th grade students• By April 2, 200x• When given a randomly selected passage from the
5th grade Reading Series and one minute to read• 90% of students will orally read• 150 words correctly with
95 to 98% accuracy
Reading CBM: An Instructional Focus by grade level
• Specific• Measurable• Action-oriented• Relevant• Time-bound
Consider the data:90% of 5th grade students who read 150 words correctly met ISAT standards
Think – Pair - Share
What are two factors that WE CAN CONTROL, other than the speed at which children read, that might influence their scores on high – stakes tests?
1. Vocabulary
2. Comprehension strategies
3. Other Basic skills
4. Language skill
5. Test – taking strategies, etc…
Grade 3
100
120
140
160
180
200
220
R-CBM
ISA
T
Exceeds
Meets
Below
Warning
92% of students who read 94 CWPM or more met ISAT Standards
What about 3rd Grade?
Can you see a pattern?
93% of students who read 115 CWPM or more met ISAT Standards
100% of students who read 140 CWPM met ISAT Standards
9/14 : 64% of students who read less than 94 did not meet ISAT Standards
Should Oral Reading Fluency be a FOCUS for third grade?
Consider this: – 92% of 3rd students in the (2 school sample) who read more
than 115 words correctly in one minute met ISAT standards.
– Fewer than 10% of students who failed to read at a target rate of 85 cwpm met ISAT standards.
• Make goals S, M, A, R, T
By April 2, 200x, when given a randomly selected passage from the 3rd grade Harcourt Reading Series and one minute to read, 90% of 3rd grade students will orally read 94 words correctly with 95 to 98% accuracy
Re-confirmation of data (Grade 3 N > 500)
Curriculum-Based Measurement and High Stakes Testing
120
130
140
150
160
170
180
190
200
210
220
0 20 40 60 80 100
120
140
160
180
200
220
240
260
Reading (CWPM)
Rea
ding
IS
AT
Sca
le S
core
27%
73%
66%
34%
90%
10%
Grade 3 (N =537)
Re-confirmation with data (Grade 5 N > 500)
Curriculum-Based Measurement and High Stakes Testing
120
130
140
150
160
170
180
190
200
210
220
0 20 40 60 80 100
120
140
160
180
200
220
240
260
Reading (CWPM)
Rea
ding
IS
AT
Sca
le S
core
36%
64%
61%
39%
88%
12%
Grade 5 (N =523)
What should we expect?• Some variation is expected in oral reading fluency
depending on passage difficulty, but clear targets are supported:
• Grade 3– Fall - 60 is cause for concern 80 is a target– Winter - 75 is cause for concern 100 is a target– Spring - 90 is cause for concern 115 is a target
• Grade 5– Fall - 95 is cause for concern 130 is a target– Winter - 125 is cause for concern 145 is a target– Spring - 140 is cause for concern 155 is a target
What about Vocabulary?
• A pilot for middle school
Grade level Teams
Identify critical vocabulary
Student friendly definintions
Vocabulary Matching - CBM1 Science
Vocabulary Matching
Directions:
You will have 5 minutes to work on this activity. Read through the definitions below. Beside each definition, write the number of the word that best matches that definition. Remember, not all of the words will be used.
Name: Date: Definitions Word Bank
The planet you live on (sample) 1 density
Anything that has mass and volume 2 respiratory system
Force of attraction between two objects 3 Momentum
In an experiment, the part that is changed on purpose 4 matter
Force needed to change movement 5 manipulated variable
The maintenance of a stable internal environment 6 scientific
The amount of matter in a given space; mass per unit volume 7 accurate
Correct 8 scientific method
All of space 9 immune system
In an experiment, the part that is measured 10 Universe
Force of a moving object 11 responding variable
Way to compare things 12 ecosystem
Study of genetics 13 chemical reaction The process by which one or more substances undergo change to produce one or more different substances 14 Inertia A series of steps that scientists use to answer questions and solve problems 15 atom
Relating to science 16 Gravity 17 Measurement 18 Earth
A collection of organs that gather and interpret information about the body's internal and external environment and respond to that information. 19 adaptation
Living and nonliving organisms interacting 20 homeostasis
Relationship among organisms 21 Interaction
A prediction based on available information. 22 Heredity
23 reproduction
24 Volcano
25 hypothesis
26 nervous system
18
5142017101131722
13
8
6
26
1221
416
25
Administration and Scoring Vocabulary Matching
• VM Curriculum-Based Measurement can be individually or group administered.
Teacher StudentReads standardized directions Reads for two to five minutes.
Monitors student performance on practice items
Monitors students while they take the test.
Writes the number of words matching definitions
Counts the total number of correctly circled items
Does Vocabulary relate to how well students do on the state test?
• How many students who obtained a score of fewer than 5 on a VM (5 minutes) measure met standards for grade 7 science?
• How about 10?
• What about more than 10 correct?
34%
65%
97%
Does Vocabulary relate to how well students do on the state test?
Vocabulary and Science ISAT
0
10
20
30
40
50
60
0 2 4 6 8 10 12 14 16 18 20VM Score
No
. S
tud
ents
Meet
DNM
Likely Likely NOT to MeetNOT to Meet
Likely Likely to Meet Standardsto Meet Standards
The Measurement Pool
• What words will a successful student know by the end of the school year?
• The student is likely to know some of the key words before you begin.
• As the year progresses, more words are taught, and more words are learned
How can writing be assessed at the classroom level
A first grade classroom example, goals and data:
By March 19, 200X, when given a writing prompt, 90% first grade students will write a complete sentence in the language of instruction that directly includes the prompt in the response with an explanation (quality).
7
6
5 38%
4 50%
3 10%
2 1%
1
0 1%5 10 15 20 25 30 35 40 45 50 55 60 65
Time 1
By December 19, 200X, when given a writing prompt, 90% of first grade students will write a complete sentence in the language of instruction.
A complete sentence includes (1) a noun and a verb (2) begins with a capital letter, (3) ends with punctuation, (4) has a space between
words. 5) To exceed means more than one sentence with 4 elements.
88% met the goal
7
6 57%
5 36%
4 7%
3
2 High Quality IndicatorBeginning Quality Indicator
1 Quality absentInsufficient quantity indicators
05 10 15 20 25 30 35 40 45 50 55 60 65
Time 2
By January 19, 200X, when given a writing prompt, 90% of first grade students will write a [part 1] complete sentence in the language of instruction that [part 2]
directly includes the prompt in the response (quality).
A complete sentence includes (1) a noun and a verb (2) begins with a capital letter, (3) ends with punctuation, (4) has a space between words. (5) more
than one sentence with 4 elements,
93% met part 1 of the goal
77% met part 2 of the goal
What do we need to do when not everyone meets?
• Meet with peers / peer mentors (Grade 1 Team meetings)
• Instructional support• Increasing focus and intensity of instruction
By January 19, 200X, when given a writing prompt, 90% of first grade students will write a [part 1] complete sentence in the language of instruction that [part 2] directly includes
the prompt in the response (quality).
93% met part 1 of the goal
77% met part 2 of the goal
By March 19, 200X, when given a writing prompt, 90% of 1st grade students will write a complete sentence in the language of instruction that directly includes the prompt in the response with an explanation (quality).
A complete sentence includes (1) a noun and a verb (2) begins with a capital letter, (3) ends with punctuation, (4) has a space between
words. (5) more than one sentence with 4 elements,
7
6 68%
5 22%
4
3
2 High Quality IndicatorBeginning Quality Indicator
1 Quality absentInsufficient quantity indicators
05 10 20 15 25 30 35 40 45 50 60 65 70
Time 3
78 %
Making Instructional Decisions based on Data
1. Identification of Needs
2. ValidateNeeds
3. Plan Development
4. PlanImplementation
5. Plan Evaluation and
Modification
Is the difference between what is expected and what is occurring?
What does the student need?Do we have enough data to confirm or refute a hypothesis?
What is the goal of instruction?What materials will be used?How will progress be monitored?
How will implementation integrity be ensured?Was the intervention plan successful?
The Plan:Three Tiers of Intervention
• Defined: A data – driven model for differentiated instruction for all students
• Focus on what works– Scientifically Based Reading Instruction– In the context of a self – correcting model of
decision-making
• Identifies / verifies need then organizes support that students need
Three Tiers, with an Emphasis on Prevention at Each Level
• Tier III (FEW students) Tertiary Intervention– Reduce complications, intensity,
severity of current problems
• Tier II (SOME students) Secondary Intervention– Reduce current cases of
academic and behavior problems
• Tier I (ALL students) Primary Intervention– Reduce new cases of academic
and behavior problems
Intensity of Need
Inte
nsi
ty o
f se
rvic
e
Tier 1:Benchmark Instruction(Typical)
Tier 2:Strategic Instruction
Tier 3:Intensive Instruction
Students who need more… receive more
Intensity of Need
Inte
nsi
ty o
f se
rvic
e
Tier 1:Typical Instruction
Tier 2:Strategic Instruction
Tier 3:Intensive Instruction
2004-2005 Meeting Expectations: In Phonological Awareness
By June 1st 200X, when given a DIBELS Phoneme Segmentation Fluency Probe, 95% of students will orally identify more than 10 phonemes per minute (Emerging phonological awareness or better). Sixty-percent (60%) of Kindergarten students will identify 35 or more phonemes per minute (Established Phonological Awareness).
60
50
40
30
20
10
0
Co
rrec
ts
FallBenchmark
Oct Nov Dec Jan WinterBenchmark
Feb Mar
ISF
LNF
PSF
NWF
Intensity of Need
Inte
nsi
ty o
f se
rvic
e
Tier 1:Typical Instruction
Tier 2:Strategic Instruction
Tier 3:Intensive Instruction
Student: Jen• September: Tier 3• October: Strategic Monitoring• Successful intervention confirmed
with progress monitoring• December: Tier 1
Three Tiers in action
Monitoring progressThink graphically
How
muc
h or
how
man
y?
Time
PLEP
Collect enough data to bereasonably confident that you know baseline (e.g., the median of 3)
What is your goal?
What should progress look like?
Collect and graph data!
If data don’t show growth change something!
If data do not match the expected course, change something!
Keep changing until the data look the way you expect them to look!
Leaving No Child Behind• We agree with the goal of No Child Left Behind
– Safe schools– Qualified teachers, etc.– All children reading at grade level
• The practical reality is that while we may not be able to demonstrate on level results on State tests, SMART goals will show us which students need more/different instruction to improve their rate of progress.
• SMART goals will empower teachers and allow them to celebrate their success with students.
BIG IDEAS• Determine your power standards • set smart goals• Monitor progress• If it’s not working, or it’s not working well
enough change it
BIG IDEAS• There are things that we can monitor that can help us see how
we are going to do.– Whether we are talking about student academic achievement,
appropriate behavior, or diffusion of innovations – the first step is identifying specifically what we are going to measure (Power Standards).
• We know that once we have the Basics,– Some need more to “meet” expectations.– And it is always nice to exceed.
• We can be reasonably confident when we answer the question, “How are we doing?” before answering “How did we do?”
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
A C T Math Score (2004)
Cum
ula
tive
(es
timat
ed)
prob
abili
ty o
f m
eetin
g
sta
ndar
ds o
n P
S A
EThe relation between A C T and meeting standards on P S A E
19
59%
The odds of meeting
standards on PSAE for a student who scored 19 on ACT is 58.7%
The odds of meeting
standards on PSAE for a student who scored 19 on ACT is 58.7%
A C T Cut scores for Reading and Science
Reading ACT SCORE
Predicted probability
2 0.0%5 0.0%6 0.0%8 0.1%9 0.2%
10 0.4%11 0.8%12 1.7%13 3.4%14 6.7%15 12.9%16 23.3%17 38.4%18 56.2%19 72.5%20 84.4%21 91.7%22 95.8%23 97.9%24 99.0%25 99.5%26 99.8%27 99.9%28 99.9%29 100.0%30 100.0%31 100.0%32 100.0%33 100.0%36 100.0%
Science ACT SCORE
Predicted probability
Mean7 0.0%8 0.0%9 0.0%
10 0.0%11 0.0%12 0.0%13 0.0%14 0.0%15 0.1%16 0.6%17 2.6%18 10.0%19 31.8%20 66.2%21 89.2%22 97.2%23 99.3%24 99.8%25 100.0%26 100.0%27 100.0%28 100.0%30 100.0%32 100.0%33 100.0%
Substantial jumps in probability of meeting standards for Reading and Science (PSAE) were obtained based on ACT as well
Approximately 90% of students who score 21 on any given sub-test meet standards on PSAE
So what does this tell us?FINDINGSFINDINGS
The probability of meeting PSAE standards increase by nearly 30% by increasing from an ACT score of 17 to 18 in Reading; 18 to 19 in Math; and 19 to 20 in Science. Approximately 90% of students obtaining a score of 21 met standards regardless of the subject area.
These findings are reliable - Analysis of a data set representing more than 10,000 11th grade students resulted in nearly identical findings
IMPLICATIONIMPLICATION
Although this is not causal (i.e., 19 does not cause PASE = Meet), the dramatic increase in probability does compel us to investigate this relation.
ACTION QuestionACTION Question
Is there something that teachers can do to increase the chances of students obtaining a score of 18,19,20 (for Reading, Math, Science) on ACT?
FOCUSFOCUS
We can use the practice ACT data to drive the focus of instruction!
On which type of item ought we focus?
• All items were examined and sorted according to difficulty.
• The raw score equating to 21 was determined.• Items were coded as Most difficult, Difficult,
FOCUS and Easiest
SuggestionSuggestion: Use the actual practice test, examine the items as a team and with students to determine why those items were missed. Don’t teach the test, but do teach the concepts.
With which students ought we focus and on what?• All scale scores were computed.
• Scores were compared with probabilities to designate the risk status of students – regarding the likelihood of meeting standards on PSAE – based on data and a little guessing.
• Scores were sorted by class – English, Science, Math, unidentified class and – color-coded.
HR
IR
LR
NR
High Risk
Indeterminate Risk
Low Risk
Negligible Risk
Likely to be on academic warning
May need to assess further, strategize instruction
Maintain current program
Likely to meet PSAE if it were given today