Progress Monitoring in Reading: How to Use the Data

MN RtI CenterMN RtI Center1

Progress Monitoring in Reading:How to Use the Data

A module for pre-service and in-service professional development

MN RTI CenterAuthors: Lisa Habedank Stewart, PhD & Adam Christ, graduate

student Minnesota State University Moorhead

www.scred.k12.mn.us click on RTI Center


MN RTI Center Training Modules

This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center

for use in preservice and inservice training:

Module Title Authors

1. RTI Overview Kim Gibbons & Lisa Stewart

2. Measurement and RTI Overview Lisa Stewart

3. Curriculum Based Measurement and RTI Lisa Stewart

4. Universal Screening (Benchmarking): (Two parts)

What, Why and How

Using Screening Data

Lisa Stewart

5. Progress Monitoring: (Two parts)

What, Why and How

Using Progress Monitoring Data

Lisa Stewart & Adam Christ

6. Evidence-Based Practices Ann Casey

7. Problem Solving in RTI Kerry Bollman

8. Differentiated Instruction Peggy Ballard

9. Tiered Service Delivery and Instruction Wendy Robinson

10. Leadership and RTI Jane Thompson & Ann Casey

11. Family involvement and RTI Amy Reschly

12. Five Areas of Reading Kerry Bollman

13. Schoolwide Organization Kim Gibbons


Overview

This module is Part 2 of 2 Part 1: Why, What, How to Progress Monitor

Part 2: Using Progress Monitoring Data


Why Progress Monitor? When teachers USE progress monitoring

Students learn more! Teachers design better instructional

programs Teacher decision making

improves Students become more aware of

their performance Safer & Fleishman, 2005

MN RtI Center5

Is this student making progress? Adam Gr 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

week

wo

rds

corr

ect

pe

r m

in (

gra

de

leve

l pa

ssa

ge

)

wrc

errors

Repeated Reading 1:1, 20 min day Repeated Reading 1:1, 10 min 2xday

MN RtI Center6

Graphing/Displaying the Data

“A picture is worth a thousand words”

MN RtI Center7

Making a Graph

Label your axes

Have an “aimline” that shows what the end goal is

Show changes in instruction with “lines” and labels

MN RtI Center8

Use Graphs!


Aimline

Shows general trajectory needed for student to reach his/her goal Typically set so student gets back “on target” or “on grade level”

within a set amount of time (e.g., by the end of the year) if possible

Simply draw a straight line from the student’s first data point on the graph to the date and score representing his target or goal

Use SMART goals: specific, measurable, attainable, realistic, with a clear timeframe


Aimline and Setting Goals (Cont’d)

For setting CBM goals

Can use local norms or benchmark targets set by your district or based on national datasets and research (e.g., DIBELS targets, AIMSweb targets)

Can use information on the amount of progress students who were successful have made in the past in this intervention or curriculum (e.g., what was the slope of progress in the research?)

MN RtI Center11

Example Gr 1-5 “Targets” for Aimline

Based on the St. Croix River Education District 08-09 Targets

linked to success on Minnesota Comprehensive Assessment – II

Grade Measure Target

1 Nonsense Word Fluency January = 52 letter sounds correct/min

1 CBM Grade Level Oral Reading Fluency (ORF)

Spring = 52 words correct/min

2 CBM ORF Spring = 90 words correct/min





MN RtI Center

Early CBM national “norms” and growth rates in oral reading (words correct per min.)Grade Percentile Fall ORF Winter ORF Spring ORF Weekly

Progress

1 50 27 54 1.80

2 75

50

25

82

53

23

106

78

46

124

94

65

1.66

3 75

50

25

107

79

65

123

93

70

142

114

87

1.18

4 75

50

25

125

99

72

133

112

89

143

118

92

1.01

5 75

50

25

126

105

77

143

118

93

151

128

100

0.58

6+ 50 125-150 125-150 125-150 0.66

Hasbruck & Tindal 1992, Teaching Exc. Children, Deno et al, 2001 School Psych Review

MN RtI Center13

Draw an Aimline for Adam (Gr 4)Adam Gr 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

1/13

/200

9

1/20

/200

9

1/27

/200

9

2/3/

2009

2/10

/200

9

2/17

/200

9

2/24

/200

9

3/3/

2009

3/10

/200

9

3/17

/200

9

3/24

/200

9

3/31

/200

9

4/7/

2009

4/14

/200

9

4/21

/200

9

4/28

/200

9

5/5/

2009

5/12

/200

9

5/19

/200

9

5/26

/200

9

week

wor

ds

corr

ect

per

min

(g

rad

e le

vel p

ass

age

)

wrc

errors


MN RtI Center14

Adam’s Aimline Example Using SCRED TargetsAdam Gr 4

0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

week

wor

ds c

orre

ct p

er m

in (

grad

e le

vel p

assa

ge)



Looking at the Graphs

Is there “go upness”????

Is there ENOUGH “go upness”????

MN RtI Center16

Basic Visual Analysis: “Go Upness”?Moira, Grade 3

0

10

20

30

40

50

60

70

80

90

100

110

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Weeks

wor

ds c

orre

ct p

er m

in.

(gra

de le

vel p

assa

ges)

MN RtI Center17

Using an AimlineMoira, Grade 3

0

10

20

30

40

50

60

70

80

90

100

110

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Weeks

wor

ds c

orre

ct p

er m

in.

(gra

de le

vel p

assa

ges) AIMLINE


Data Decision Guidelines If the student has some data points above and

some below the aimline (doing the “aimline hug”), keep doing what you are doing!

If the student has 4 consecutive data points above the aimline, consider moving the student to less intervention (e.g., decreasing minutes, or moving from Tier 2 to Tier 1 or Tier 3 to Tier 2) Also use other pieces of information

Continue to progress monitor


Data Decision Guidelines (Cont’d) If the student has 4 consecutive data points below the

aimline, ASK THE FOLLOWING QUESTIONS (and continue to progress monitor):

What does the “other” evidence available suggest about the student’s progress? Error rates? Behavior during the intervention?

What is the general “trend” of the data? Is the student likely to get where we want if this continues? Use visual analysis and other evidence Use “trendlines” and “aimlines”


Trendline Shows the general “trend” or trajectory of the student’s

data so far Web-based programs typically use an Ordinary Least Squares

regression line AIMSweb, DIBELS data system, Excel

Need approx. 7 to 9 data points Trendlines on few data points or on highly variable data are NOT

reliable!!! Christ, T. (2006). Short term estimates of growth using CBM

ORF: Estimating Standard Error of Slope to construct confidence intervals. School Psychology Review, 35(1) 128-133.

MN RtI Center21

Moira, Grade 3

y = -0.8x + 64.6

R2 = 0.0623

0

10

20

30

40

50

60

70

80

90

100

110

120

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Weeks

wor

ds c

orre

ct p

er

min

. (g

rad

e le

vel p

assa

ges) AIMLINE

TRENDLINE

Trendline and Aimline


How Much Progress is “Enough”? What is “adequate” progress?

Criterion referenced Will student meet goal? In reasonable amount of time? Growth is at or above “target” growth rate

Norm referenced Growth is at or above growth of grade level peers

Individually referenced Growth is better than before

“Intervention”/research referenced Growth is similar to what was seen in research on this

intervention (with similar population)


Remember to Use your Brain! (And Eyes and Ears)

If overall trend of progress is good

but s/he happens to have 4 data points just barely below the aimline, you may decide to continue your intervention for a week and see what happens.

Use convergence of data (teacher report, mastery monitoring, behavioral indicators)

These are guidelines, THINKING is REQUIRED…


Practice Exercises:

1. Is there go upness?

2. Is there enough go upness?

3. What else would you like to know?

4. What would you do?• Exit to less intense service • Keep going and collect more data• Problem solve and change something

MN RtI Center25

Finn Gr 2 CBM-ORFFinnegan Grade 2

0

10

20

30

40

50

60

70

80

90

100

110

120

9/9/

2009

9/23

/200

9

10/7

/200

9

10/2

1/20

09

11/4

/200

9

11/1

8/20

09

12/2

/200

9

12/1

6/20

09

12/3

0/20

09

1/13

/201

0

1/27

/201

0

2/10

/201

0

2/24

/201

0

3/10

/201

0

3/24

/201

0

4/7/

2010

4/21

/201

0

5/5/

2010

5/19

/201

0

week

wor

ds c

orre

ct p

er m

in (

grad

e le

vel p

assa

ge)

Aimline

Reading Links 1:5 for 15 min.

MN RtI Center26

Finn Gr 2 CBM-ORF (Cont’d)Finnegan Grade 2

0

10

20

30

40

50

60

70

80

90

100

110

120

9/9/

2009

9/23

/200

9

10/7

/200

9

10/2

1/20

09

11/4

/200

9

11/1

8/20

09

12/2

/200

9

12/1

6/20

09

12/3

0/20

09

1/13

/201

0

1/27

/201

0

2/10

/201

0

2/24

/201

0

3/10

/201

0

3/24

/201

0

4/7/

2010

4/21

/201

0

5/5/

2010

5/19

/201

0

week

wor

ds c

orre

ct p

er m

in (

grad

e le

vel p

assa

ge)

Aimline

Added distributed practice and preteachingReading Links 1:5 for 15 min.

MN RtI Center27

Justan Gr 1 NWFIs Justan Making Progress?

MN RtI Center28

And Now?

MN RtI Center29

And Now????

MN RtI Center30

On Track…

MN RtI Center31

What Decision Would You Make?

MN RtI Center32

And Now?

MN RtI Center33

Is There “Enough” Go Upness?

MN RtI Center34

Can Also Make Decisions About Exiting to Less Intensive Service!

MN RtI Center35

What Can You Do About “Bounce” in the Data?


Dealing With Bounce

Is there a “measurement” problem?

Fidelity of administration and scoring

Materials aren’t well designed or are too difficult

Who, where, and when measurement takes place can matter (esp. for some kids)

Motivation issues (can’t do vs. won’t do)


Dealing with Bounce (Cont’d)

Other ways to minimize bounce or make decisions despite bounce Do more probes at one time and take median or

average score Do more frequent measurement (e.g., weekly or 2x

week) Look at trend over time with many data points

Look at ALL data together (errors, mastery data, etc.) Use the least dangerous assumption…


What if There isn’t Adequate Progress?

If you keep doing what you’ve been doing then you will keep getting

what you’ve got.


Back to Problem Solving


What if There isn’t Adequate Progress?

Is the intervention being done with fidelity?

Has fidelity checks been done?

Is the student in the right level of materials?

Has the student been in school? Are they getting enough minutes of intervention per week?

MN RtI CenterMN RtI CenterDRAFT May 27, 2009 41

What if There isn’t Adequate Progress? (Cont’d)

Should the intervention be “tweaked”? Changed? Is there an intervention better “matched” to this student’s needs?

Changes could include trying a different intervention or just “tweaking” the current intervention such as adding a 5th repeat to a repeated reading or a sticker incentive for accurate reading.

Grade level or problem solving team members work together to discuss the data, the student, and what intervention changes would have the best chance of success.


What Could We Change? Focus or skill

Teaching strategies: More explicit, more modeling, more practice, more previewing, better matched with core

Materials: Easier, better matched (cultural, interests, etc.)

Arrangements: Size group, location, who is teaching?

Time: Amount of time, days per week, time of day

Motivation: Interests, goals, rewards, home/school


Adam, Gr 4 Benchmark data

Winter: 85 wrc (target= 114) Fall: 89 wrc (target= 93) Error rate moderate (4, 4, & 6)

Very inconsistent academically; good attendance but attention, accuracy and work completion issues; basic decoding skills ok; can correct errors; can read better (with expression, meaning) in high interest material?

Grade Level Team put Adam in Tier 2 intervention- working with MRC 1:1 on repeated reading intervention 20 min per day

MN RtI Center44

Is the Intervention Working?Adam Gr 4

0102030405060708090

100110120130140

w eek

wor

ds c

orre

ct p

er m

in (g

rade

leve

l pas

sage

)

w rc

errors


If We Do Change, What Should We Change?

What else would you want to know about Adam and his intervention, curriculum and class?

What are at least 5 different ideas for changes that could be made?

Is this likely to be a tweak or a major shift?

How would you know if you made a good decision?

MN RtI Center46

And Now?Adam Gr 4

0102030405060708090

100110120130140

w eek

wor

ds c

orre

ct p

er m

in (

grad

e le

vel p

assa

ge)

w rc

errors



Sharing the Data

Just having progress monitoring data is not enough. You need to USE it.

Scheduled graph review dates

Grade level meetings

Problem solving meetings


Remember: Garbage IN…. Garbage OUT….

Make sure your data have integrity or they won’t be good fer nuthin… Training Integrity checks/refreshers Well chosen measures and

materials


Avoid Common Mistakes

Don’t use the same passage/probe every week!

Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data


Remember… When teachers USE progress monitoring

Students learn more! Teachers design better instructional

programs Teacher decision making

improves Students become more aware of

their performance Safer & Fleishman, 2005


Web Resources Research Institute on Progress Monitoring

http://progressmonitoring.net/ Includes…

A Study Group Content Module with 15 sections on CBM including activities http://progressmonitoring.org/pdf/cbmMOD1.pdf

Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities http://progressmonitoring.org/pdf/cbmMODldrshp.pdf

Handouts, videos, and power point presentations Technical reports of CBM measures


Web Resources, Cont’d www.studentprogress.org

Growth rates, use in RTI model, etc

http://www.rti4success.org/ click on Progress monitoring on right side

www.interventioncentral.org look for information on CBM, graphing, etc.

www.aimsweb.com, www.edcheckup.com, dibels.uoregon.edu Look for information about progress monitoring as well as

access to materials and graphing for progress monitoring


Print Resources available with this module

Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83.

Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MultilevelPrevention

Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009,

Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/Using CBM/1


Other Recommended Articles & Texts

Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press.

Stecker, Lembke, & Fogen (2008). Using progress-monitoring data to improve instructional decision making. Preventing School Failure, 52(2), 48-58. Case study included


Activity for Teachers or Practicum Students Obtain progress monitoring probes and graphs

Passages and graphing materials self-created or downloaded www.interventioncentral.org dibels.uoregon.edu

Sign up for an account with AIMSweb (instructor accounts and student accounts available) www.aimsweb.org

Practice administration and scoring Progress Monitor a “real” kid (ideally 2-4 kids of varying risk

levels monitored for at least 7-10 weeks) Graph, analyze, and use data

MN RtI CenterMN RtI Center

Articles

Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83.

Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009, from AIMSweb, Web site: ???

DRAFT May 27, 2009 56


Quiz

1.) What shows the general trajectory of the student’s data so far? A.) Axis B.) Trendline C.) Aimline D.) Target

2.) What shows the general trajectory needed to reach the end goal? A.) Axis B.) Trendline C.) Aimline D.) Target


Quiz (Cont’d)

3.) When should you keep doing what you are doing? A.) If the student has 4 consecutive data points

above the aimline B.) If the student has 4 consecutive data points

below the aimline C.) If the student is doing the “aimline hug” D.) None of the above


Quiz (Cont’d)

4.) Describe “go upness.”

5.) If the student has 4 consecutive data points below the aimline, what would you do?

MN RtI CenterMN RtI Center

Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only.

Special Thanks: Thank you to Dr. Ann Casey, director of the MN RTI Center, for

her leadership Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan

Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials

Progress Monitoring in Reading: How to Use the Data

Documents

Transcript of Progress Monitoring in Reading: How to Use the Data