How Do You Know Your How Do You Know Your Instruction is Working:Instruction is Working: An Introduction to An Introduction to Progress MonitoringProgress Monitoring

Summer ManosSummer ManosFCCPS RTI SpecialistFCCPS RTI Specialist3/21/123/21/12

Our Goals for TodayOur Goals for Today

1. What is Progress Monitoring?2. Why is it important?3. How to do I do it?4. What resources are available to me?

What is Progress Monitoring?What is Progress Monitoring?

Frequent assessments that occur during learning-NOT after (CBM’s and CBA’s)

A way to quantify a student’s rate of growth toward defined instructional goals

A method that evaluates the effects of instruction/interventions on student performance and allows for changes

Who do we Progress Monitor?Who do we Progress Monitor?A closer look at a MTSS (RTI framework)A closer look at a MTSS (RTI framework)

Addl.Diagnostic

Assessment

InstructionResults

Monitoring

IndividualDiagnostic

Individual Intensive

weekly

All Students at a grade level

Fall Winter Spring

UniversalScreening

None ContinueWithCore

Instruction

GradesClassroom

AssessmentsBenchmarks

GroupDiagnostic

SmallGroupDifferen-tiatedBy Skill

2 times/month

Step 1Step 2 Step 3 Step 4

Supplemental

1-5%

5-10%

80-90%

Core

Intensive

How Does Progress Monitoring Inform our How Does Progress Monitoring Inform our Intervention?Intervention?

Match intervention to problem Humans tend to employ interventions with which

they are comfortable instead of intervention that the student needs

Intervention should be developed with the expectation that it will be altered in some way as a result of the progress monitoring data

No intervention works all of the time for every student

Benefits for the TeacherBenefits for the Teacher

Results can guide individual instruction

Teachers are able to make objective decisions based on data.

Results may be easily communicated to parents and staff using charts and graphs.

Additional information for SBT

Curriculum-Based MeasurementCurriculum-Based Measurement

Specific type of Progress Monitoring A measure of student growth in basic skill areas

(reading, math, and writing) Reliable and valid tests sensitive to measuring student

improvement Easy and brief to administer (e.g., 1-8 minutes) Standardized administration Student data can be compared to national or local

norms or criterion standards

Curriculum-Based MeasurementCurriculum-Based Measurement

Used for screening all students and/or monitoring individual

Multiple probes of equal difficulty within each basic skill area

CBMs sample basic skills across the entire grade-level curriculum

Results are graphed to make instructional decisions

Why Graph?

Provides a visual representation of a large amount of data

A visual representation of student’s acquisition of skills and allows for easier analysis of progress

Paper and pencil or electronic based graphing

Research tells us…Research tells us…

Progress monitoring and charting are components of formative evaluation Allows you to “determine the effectiveness of an

intervention during implementation so that it can be modified or changed to increase the likelihood that intended results will be achieved.” (Deno, 2002)

If we use research validated reading practices, monitor student progress and make changes to instruction based on what we find, between 95-100% of children can become proficient readers. (Torgensen, 2000)

Data Based Instructional Decisions

Essential for four reasons There is no guarantee that interventions will be successful,

thus the intervention must be “tested” to evaluate effectiveness

Increased emphasis of specific outcomes for students, data base must be generated to guide intervention decision making

Pre/post testing has be shown to be unreliable (small amount of data) and provides too little data to allow for instructional decision making – progress monitoring allows for evaluation of level of performance and rate of learning

Research has shown that progress monitoring is associated with improved educational outcomes

Research has shown that it works!

Treatment/Intervention Effect Size

Special Education Placement -.14 to .29

Modality Matched Instruction (Auditory) +.03

Modality Matched Instruction (Visual) +.04

Curriculum-Based Instruction/ Graphing and Formative Evaluation

+.70

Curriculum-Based Instruction, Graphing, Formative Evaluation and Systematic use of Reinforcement

+1.00

CBM Research tells us…CBM Research tells us…

“… “… that the best method of progress that the best method of progress monitoring is Curriculum-Based monitoring is Curriculum-Based Measurement .”Measurement .”

““CBM data correlates well with high-CBM data correlates well with high-stakes tests”stakes tests”

McGlinchy & Hixton, (2004). Using curricular-based measurement McGlinchy & Hixton, (2004). Using curricular-based measurement to predict performance on state assessments in reading. to predict performance on state assessments in reading. School School

Psychology Review,33Psychology Review,33

Based on 2008-2009 data

Virginia (predicted)

Grade Fall Winter Spring

1 28 56

2 61 82 96

3 85 105 124

4 69 86 100

5 99 112 127

6 113 126 140

7 105 122 135

8 123 130 138

R-CBM, 80% Probability of Success on State Test

Progress Monitoring

Essential components that must be in place for successful progress monitoring within each tier A well-defined behavior Identification of student’s current level of

performance (baseline) Instruction/Intervention Goal A measurement strategy Graph Decision-making plan

Well Defined Behavior

Target behavior, observable, measurable, and specific

Focus on enabling skills Skills that are prerequisite skills for more complex

skills Deficiencies in enabling skills often adversely

affects performance on global assessments

Skills we can measure… Reading

Phonemic AwarenessAlphabetic understandingFluencySight WordsComprehension

MathNumber SenseFactsComputationApplicationsProblem Solving

Skills Continued

Written ExpressionMechanicsExpression

BehaviorSocial SkillsWork CompletionComplianceProblem Solving Skills

CBM Procedures

Scoring Reading assessments are scored number or

corrects per minute Math computations are scored digits correct per

minute Written expression is scored according to correct

sequences

Samples of CBM Procedures

Reading probes scored corrects per minute

MAZE scored words correctly restored (WCR) in 3 minutes

CBM Procedures

Math computations are scored by correct digits per minute

(example provided by Dan Reschly)

CBM Procedures

Correct Sequences for written expression

Two words form a sequence, word and punctuation form a sequence.

Most words and punctuation are used twice

Three minutes to brainstorm, write, and edit

(example provided by Tracy Hall)

How do we Progress Monitor?How do we Progress Monitor?

Step 1: Determine the area/s that needs remediation (review past data or screen)

Step 2: Collect baseline data using CBM probes and set the goal line

Step 3: Implement the appropriate intervention Step 4: Administer probes (4 x 4) and chart the

results Step 5: Interpret results. Is the instruction

working?

Goal

Standard against which progress can be compared Allows for aimline to be established Possible goals

Level of behavior that is expected – several ways to establish this

Norms/percentile cutoffs and benchmarks Behavioral expectations Calculated growth rates – norms and benchmarks

Tier II & Tier III Progress Monitoring

Charting Activity Using the data below to decide Nicole’s area of need for a

reading intervention Let’s say our intervention is ‘Timed Repeated Readings’ Let’s calculate a goal for Nicole

Skill Nicole’s Baseline MOYBM

Norms for 3rd grade 50%

EOYBM

Norms for 3rd grade 50%

Oral Reading Fluency

83 92 107

MAZE 20 15 16

Calculating Nicole’s Goal

Step 1EOYBM-MYOBM /weeks left in school year107 – 92 ÷ 18 weeks= .83 Typical Growth

.83Average weekly improvement

Step 2.83 (avg weekly improvement) x 2.0( ambitious growth rate for words per week)

Most ambitious GR is 2.0 words per week.83 x 2.0 = 1.7 expected per week

Calculating Nicole’s Goal

Step 31.7 (targeted growth rate) x 8 (weeks of intervention)

14 total words of improvement

ORF(initial oral reading fluency rate) + 14 (total words improvement) = Growth Goal

83 wpm + 14 = 97 wpm (Growth Goal)

Charting Activity

Plot the baseline data for Nicole 83 WCPM

Plot the goal at the end of eight weeks 97 WCPM

Indicate the aimline by connecting the 2 points

Charting Activity

Chart the following scores: Week 1

Thursday - 80 Week 2

Tuesday - 81Thursday - 85

Week 3Tuesday - 80

Make an informed decision regarding the effectiveness of the intervention “Timed Repeated Readings”

Charting Activity Decision Rules

Performance trends should be analyzed periodically Trend of the data above aimline – raise the goal Trend of the data below aimline – adjust intervention

If changes are made to the intervention, indicate change on the graph with a squiggle line

Describe the changes on the back of the chart This allows for understanding of specific instructional

adjustments that were successful/unsuccessful 4 x 4 approach

Charting Activity: Swiggle and Tweak

Draw a “swiggle” to indicate a new intervention. Lets go with ‘Read Naturally’.

Plot progress monitoring data for next three weeks Week 3

Thursday - 85 Week 4

Tuesday - absentThursday - 85

Week 5Tuesday - 90

Make an informed decision regarding the effectiveness of the intervention

Decision Rules: What is a “Good” Response to Intervention?

Positive Response

Gap is closing

Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range

Level of “risk” lowers over time

Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

Poor Response

Gap continues to widen with no change in rate.

Performance

Time

Positive Response to Intervention

Expected Trajectory

Observed Trajectory

Decision Rules: What is a “Questionable” Response to Intervention?

Positive Response

Gap is closing

Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range

Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

Level of “risk” remains the same over time

Poor Response

Gap continues to widen with no change in rate.

Performance

Time

Questionable Response to Intervention

Expected Trajectory

Observed Trajectory

Decision Rules: What is a “Poor” Response to Intervention?

Positive Response

Gap is closing

Can extrapolate point at which target student(s) will “come in range” of target--even if this is long range

Questionable Response

Rate at which gap is widening slows considerably, but gap is still widening

Gap stops widening but closure does not occur

Poor Response

Gap continues to widen with no change in rate.

Level of “risk” worsens over time

Performance

Time

Poor Response to Intervention

Expected Trajectory

Observed Trajectory

Performance

Time

Response to Intervention

Expected Trajectory

Observed Trajectory

Positive

Questionable

Poor

Decision Rules: Linking RTI to Intervention Decisions

Positive Continue intervention with current goal Continue intervention with goal increased Fade intervention to determine if student(s)

have acquired functional independence.

Decision Rules: Linking RTI to Intervention Decisions

Questionable Was intervention implemented as intended?

If no - employ strategies to increase implementation integrity

If yes - Increase intensity of current intervention for a short

period of time and assess impact. If rate improves, continue. If rate does not improve, return to problem solving.

Decision Rules: Linking RTI to Intervention Decisions

Poor Was intervention implemented as intended?

If no - employ strategies in increase implementation integrity If yes -

Is intervention aligned with the verified hypothesis? (Intervention Design)

Are there other hypotheses to consider? (Problem Analysis)

Was the problem identified correctly? (Problem Identification)

Decision Rules: Linking RTI to Intervention Decisions

What is the number 1 reason students don’t make progress?

They are not actually receiving the intervention!

We need to continue to look at how we document the time and frequency of our interventions

What Resources are available for Progress Monitoring?

PALS Quick Checks www.easycmb.com www.edcheckup.com www.interventioncentral.com www.xtramath.org STAR Math STAR Reading Please consult with Reading Specialists, Math Specialist,

Special Education teachers, CIRTS, RTI Specialist “The ABCs of CBM” by Hosp, Hosp & Howell