1 FUNCTIONAL ASSESSMENT: Curriculum-Based Measurement Progress Monitoring NC Department of Public...

11

FUNCTIONAL ASSESSMENT: FUNCTIONAL ASSESSMENT: Curriculum-Based Curriculum-Based

MeasurementMeasurementProgress MonitoringProgress Monitoring

NC Department of Public InstructionNC Department of Public InstructionExceptional Children DivisionExceptional Children Division

Program Improvement & Professional Program Improvement & Professional Development-2008Development-2008

22

Today’s AgendaToday’s Agenda

• What is CBM?What is CBM?

• State norming project reviewState norming project review

• Hands on CBMsHands on CBMs

• Measuring Behavior in an RtI Measuring Behavior in an RtI modelmodel

• Progress MonitoringProgress Monitoring– Charting ActivityCharting Activity– Setting GoalsSetting Goals– Generalizing ChartingGeneralizing Charting

33

What is the Responsiveness to What is the Responsiveness to Instruction Model?Instruction Model?

• A student with academic delays is given one A student with academic delays is given one or more research-validated interventions.or more research-validated interventions.

• The student's academic progress is The student's academic progress is monitored frequently to see if those monitored frequently to see if those interventions are sufficient to help the interventions are sufficient to help the student to catch up with his or her peers. student to catch up with his or her peers.

• If the student fails to show significantly If the student fails to show significantly improved academic skills despite several improved academic skills despite several well-designed and implemented well-designed and implemented interventions, this failure to 'respond to interventions, this failure to 'respond to intervention' can be viewed as evidence of intervention' can be viewed as evidence of an underlying Learning Disability.an underlying Learning Disability.

www.interventioncentral.com

Target Student

Discrepancy 1: Skill Gap (Current Performance Level)

Avg Classroom Academic Performance Level

‘Dual-Discrepancy’: RTI Model of Learning Disability (Fuchs 2003)

Discrepancy 2:Gap in Rate of Learning (‘Slope of Improvement’)

55

Why RtI?Why RtI?

• One advantage of RtI in the diagnosis One advantage of RtI in the diagnosis of educational disabilities is that it of educational disabilities is that it allows schools to intervene early to allows schools to intervene early to meet the needs of struggling meet the needs of struggling learners. learners.

• Another advantage is that RtI maps Another advantage is that RtI maps those specific instructional strategies those specific instructional strategies found to benefit a particular student. found to benefit a particular student. This information can be very helpful This information can be very helpful to both teachers and parents.to both teachers and parents.

www.interventioncentral.com

What are Summative and What are Summative and Formative Evaluation?Formative Evaluation?

SumSummativemative Assessment: Culmination measure. Assessment: Culmination measure. Mastery assessment. Pass/fail type assessments Mastery assessment. Pass/fail type assessments which summarize the knowledge students learn. which summarize the knowledge students learn.

• High-stakes tests (EOGs)High-stakes tests (EOGs)• GRE, ACT, SAT, GMAT, etc. testsGRE, ACT, SAT, GMAT, etc. tests• Driver’s license testDriver’s license test

Adapted from www.aimsweb.com

What are Summative and What are Summative and Formative Evaluation?Formative Evaluation?

Formative Evaluation: Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students.

• When students are progressing, continue using your instructional programs. • When tests show that students are not progressing, you can change your instructional programs in meaningful ways.


Types of AssessmentTypes of Assessment

PurposePurpose MedicalMedical EducationalEducationalScreeningScreening Temperature,Temperature,

Blood Pressure, Blood Pressure, Urine samples, Urine samples,

etc.etc.

CBMsCBMs

DiagnosticDiagnostic Exam, history, Exam, history, more invasive more invasive

teststests

R.I.O.T.R.I.O.T.

Progress Progress MonitorinMonitorin

gg

Temperature,Temperature,

Blood Pressure, Blood Pressure, Urine samples, Urine samples,

etc.etc.

CBMsCBMs

Summative Assessment Summative Assessment Example: WeightExample: Weight

High standard: All children will have a healthy weight High standard: All children will have a healthy weight by the end of third grade.by the end of third grade.

High Stakes Assessment: Assessing body density.High Stakes Assessment: Assessing body density.• Weighing each student.Weighing each student.• Immersing each student in a large tub filled with water Immersing each student in a large tub filled with water

and measuring the amount of water displaced. and measuring the amount of water displaced. • Divide weight by displacement and get density, a very Divide weight by displacement and get density, a very

accurate picture of physical status. accurate picture of physical status.

After 8-9 YEARS of growth, we would:After 8-9 YEARS of growth, we would:1.1. Place students who are “unhealthy” in remedial Place students who are “unhealthy” in remedial

programs. programs.

2.2. Create new health programs. Create new health programs.

3.3. Blame the effectiveness of old health programs. Blame the effectiveness of old health programs.

4.4. Blame the students (or their families) for over or under Blame the students (or their families) for over or under eating.eating.


Formative Evaluation Example: Formative Evaluation Example: Same Standard, Different Same Standard, Different AssessmentAssessmentHigh Standard: High Standard: All children will have a healthy All children will have a healthy

weight by the end of third grade.weight by the end of third grade.

Universal Screening Assessment: Universal Screening Assessment:

Monitor weight directly, Monitor weight directly, frequently, and continuously.frequently, and continuously.

From birth, measure weight From birth, measure weight frequently and continuously with frequently and continuously with a simple, albeit less precise, a simple, albeit less precise, general outcome measure, general outcome measure, weight in pounds, using a scale. weight in pounds, using a scale.

(General Outcome Measure)(General Outcome Measure)Adapted from www.aimsweb.com

At any point in development…At any point in development…

The child could be weighed and a decision made The child could be weighed and a decision made about healthy weight.about healthy weight.

This process is:This process is: • Efficient. Efficient. • Sufficiently accurate.Sufficiently accurate.• Proactive.Proactive.• Cost effective.Cost effective.

We would know their health status before they We would know their health status before they reached the high stakes point!reached the high stakes point!

Formative Evaluation Example: Formative Evaluation Example: Same Standard, Different Same Standard, Different AssessmentAssessment


Characteristics of General Characteristics of General Outcome Measures (GOMs)Outcome Measures (GOMs)

PowerfulPowerful measures that are: measures that are:• SimpleSimple • Easier to obtain data (less time and good Easier to obtain data (less time and good

data)data)

• AccurateAccurate • Very specific dataVery specific data

• EfficientEfficient • Only a few minutes to administerOnly a few minutes to administer

• GeneralizableGeneralizable • Reliable Reliable • Can compare and contrast student Can compare and contrast student

performance across school, district, countryperformance across school, district, countryAdapted from www.aimsweb.com

General Outcome Measures General Outcome Measures (GOMs) from Other Fields(GOMs) from Other Fields

Medicine measures height, weight, temperature, Medicine measures height, weight, temperature, and/or blood pressure.and/or blood pressure.

Federal Reserve Board measures the Consumer Federal Reserve Board measures the Consumer Price Index.Price Index.

Wall Street measures the Dow-Jones Industrial Wall Street measures the Dow-Jones Industrial Average.Average.

Companies report earnings per share. Companies report earnings per share.

McDonald’s measures how many hamburgers they McDonald’s measures how many hamburgers they sell.sell.

In Education, Curriculum Based Measurement is a In Education, Curriculum Based Measurement is a General Outcome MeasureGeneral Outcome Measure


What is Curriculum Based What is Curriculum Based Measurement?Measurement?

• Curriculum Based Measurement is a form of Curriculum Based Measurement is a form of Curriculum-Based Assessment (CBA). Curriculum-Based Assessment (CBA).

• CBM is a method of monitoring student educational CBM is a method of monitoring student educational progress through progress through directdirect assessment of academic skills assessment of academic skills

• CBMs measure basic skills in reading, mathematics, CBMs measure basic skills in reading, mathematics, spelling, written expression and readiness skillsspelling, written expression and readiness skills– Teacher gives the student brief, timed samples, or Teacher gives the student brief, timed samples, or

“probes,” made up of academic material that the “probes,” made up of academic material that the student is expected to learnstudent is expected to learn

• Early literacy skills (phonics and phonological Early literacy skills (phonics and phonological awareness) are downward extensions of CBM.awareness) are downward extensions of CBM.

• Because CBM probes are quick to administer and Because CBM probes are quick to administer and simple to score, they can be given frequently to provide simple to score, they can be given frequently to provide continuous progress data. The results are charted and continuous progress data. The results are charted and provide for timely evaluation based on hard data.provide for timely evaluation based on hard data.


Using Curriculum Based Using Curriculum Based Measures as General Outcome Measures as General Outcome MeasuresMeasuresIt’s about using General Outcome Measures It’s about using General Outcome Measures (GOMs) for formative assessment/evaluation (GOMs) for formative assessment/evaluation to:to:

•• Inform teaching Inform teaching

ANDAND

• • ensure accountability.ensure accountability.

It’s different from, but related to, summative It’s different from, but related to, summative high-stakes testing/evaluation, which:high-stakes testing/evaluation, which:

• • Doesn’t inform teaching.Doesn’t inform teaching.

• • Mostly used for accountability/motivation.Mostly used for accountability/motivation.


Universal (school-wide) screening using CBMs allows Universal (school-wide) screening using CBMs allows us to add systematic Formative Evaluation to current us to add systematic Formative Evaluation to current practice.practice.

• For Teachers (and Students)For Teachers (and Students)• Early Identification of At Risk StudentsEarly Identification of At Risk Students• Instructional PlanningInstructional Planning• Monitoring Student ProgressMonitoring Student Progress

• For ParentsFor Parents• Opportunities for Communication/InvolvementOpportunities for Communication/Involvement• AccountabilityAccountability

• For AdministratorsFor Administrators• Resource Allocation/Planning and SupportResource Allocation/Planning and Support• AccountabilityAccountability

Using Curriculum Based Using Curriculum Based Measurement as a General Outcome Measurement as a General Outcome MeasureMeasure


Using Curriculum Based Using Curriculum Based Measurement as a General Measurement as a General Outcome Measure: ResearchOutcome Measure: Research• Curriculum-Based MeasurementCurriculum-Based Measurement (CBM) was (CBM) was developed more than 20 years ago by Stanley Deno developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal at the University of Minnesota through a federal contract to develop a reliable and valid measurement contract to develop a reliable and valid measurement system for evaluating basic skills growth.system for evaluating basic skills growth.

• CBM is supported by more than CBM is supported by more than 25 years of school-25 years of school-based researchbased research by the US Department of Education. by the US Department of Education.

• Supporting documentation can be found in Supporting documentation can be found in 100s of 100s of articles, book chapters, and books in the professional articles, book chapters, and books in the professional literatureliterature describing the use of CBM to make a describing the use of CBM to make a variety of important educational decisions. variety of important educational decisions.


Summary of Research Validating Summary of Research Validating Curriculum Based MeasurementCurriculum Based Measurement

ReliableReliable and and validvalid indicator of student indicator of student

achievementachievement

Simple, efficientSimple, efficient, and of , and of shortshort duration to facilitate duration to facilitate

frequent administration by teachersfrequent administration by teachers

Provides assessment information that Provides assessment information that helps helps

teachers plan better instructionteachers plan better instruction

Sensitive to the improvementSensitive to the improvement of students’ of students’

achievement over timeachievement over time

Easily understoodEasily understood by teachers and parents by teachers and parents

Improves achievementImproves achievement when used to monitor when used to monitor

progressprogress


Curriculum Based Measurement: Curriculum Based Measurement: AdvantagesAdvantages• DirectDirect measure of student performance measure of student performance

• Helps Helps targettarget specific areas of instructional need for specific areas of instructional need for studentsstudents

• QuickQuick to administer to administer

• Provides Provides visual representation visual representation (reports) of (reports) of individual student progress and how classes are individual student progress and how classes are acquiring essential reading skillsacquiring essential reading skills

• Sensitive to even Sensitive to even smallsmall improvements in improvements in performance performance

• Capable of having Capable of having many formsmany forms

• Monitoring frequentlyMonitoring frequently enables staff to see trends in enables staff to see trends in individual and group performance—and compare individual and group performance—and compare those trends with targets set for their students. those trends with targets set for their students.

• Correlates strongly Correlates strongly with “best practices” for with “best practices” for instruction and assessment, and research-instruction and assessment, and research-supported methods for assessment and supported methods for assessment and intervention.intervention. Adapted from

www.aimsweb.com

Curriculum Based Measurement: Curriculum Based Measurement: Things to Remember Things to Remember

• Designed to serve as “indicators” of general reading Designed to serve as “indicators” of general reading achievement: CBM probes don’t measure everything, achievement: CBM probes don’t measure everything, but measure the important things.but measure the important things.

• Standardized tests to be given, scored, and Standardized tests to be given, scored, and interpreted in a standard way.interpreted in a standard way.

• Researched with respect to psychometric properties Researched with respect to psychometric properties to ensure accurate measures of learning.to ensure accurate measures of learning.

• Are sensitive to improvement in brief intervals of Are sensitive to improvement in brief intervals of time.time.

• Tell us how students earned their scores (qualitative Tell us how students earned their scores (qualitative information).information).

• Designed to be as short as possible to ensure “do-Designed to be as short as possible to ensure “do-ability.”ability.”

• Are linked to decision making for promoting positive Are linked to decision making for promoting positive achievement and problem-solving.achievement and problem-solving.


2121

Curriculum Based Curriculum Based MeasurementMeasurement• CBM has been shown to posses high CBM has been shown to posses high

levels of reliability levels of reliability – Reliability - the extent to which the Reliability - the extent to which the

measurements of a test remain measurements of a test remain consistent over repeated tests of the consistent over repeated tests of the same subject under identical conditions same subject under identical conditions

• 42 one-minute CBM type assessments 42 one-minute CBM type assessments in reading, math, and written in reading, math, and written expression for grade K-5 were found expression for grade K-5 were found to have reliability coefficients between to have reliability coefficients between .90-.99 with just three one-minute .90-.99 with just three one-minute administrations (Jenkins, 2002)administrations (Jenkins, 2002)

2222

Curriculum Based Curriculum Based MeasurementMeasurement• Discriminant Validity - Discriminant Validity - Does it appear to Does it appear to

measure what it’s supposed to measure? measure what it’s supposed to measure? AndAnd

Doesn’t associate with constructs that shouldn’t be Doesn’t associate with constructs that shouldn’t be related.related.

• Several studies have demonstrated the ability Several studies have demonstrated the ability

of CBM to differentiate between students of CBM to differentiate between students receiving special education services, students receiving special education services, students receiving Chapter 1 services, and students not receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).Tindal, Spira, and Marston, 1987).

2323

NC CBM Norming Project NC CBM Norming Project • Completed during the 2004-05 school Completed during the 2004-05 school

yearyear• 3 days in a row in October, January, and 3 days in a row in October, January, and

AprilApril• Each pilot county (New Hanover, Bertie, Each pilot county (New Hanover, Bertie,

Harnett, Guilford, Burke) completed a Harnett, Guilford, Burke) completed a project for K through 5project for K through 5thth grades grades

• Random sample, 120 students per Random sample, 120 students per grade levelgrade level

• Probes were given in reading, math, and Probes were given in reading, math, and writing (Basic Skill Builders)writing (Basic Skill Builders)

2424

North Carolina NormsNorth Carolina NormsPracticePractice

• Norm (mean) for a kindergartener in Norm (mean) for a kindergartener in number identification in the fallnumber identification in the fall

• Norm for a fifth grader in written Norm for a fifth grader in written expression in the springexpression in the spring

• Norm for a first grader in sight word Norm for a first grader in sight word reading in the winterreading in the winter

• Norm for a first grader in sight word Norm for a first grader in sight word reading in the springreading in the spring

2525

Other types of CBMs - Other types of CBMs - DIBELSDIBELS• Dynamic Indicators of Basic Early Dynamic Indicators of Basic Early

LiteracyLiteracy Skills Skills

• Assesses similar skills as Skill Assesses similar skills as Skill Builders, a little more in depthBuilders, a little more in depth

• Benchmark assessment data is Benchmark assessment data is available (national data)available (national data)

• Can use for benchmark assessments Can use for benchmark assessments (3 or 4 times a year) as well as for (3 or 4 times a year) as well as for progress monitoringprogress monitoring

2626

Other types of CBMs - Other types of CBMs - DIBELSDIBELS• Kindergarten – FallKindergarten – Fall

– Initial Sound FluencyInitial Sound Fluency– Letter Naming FluencyLetter Naming Fluency

• Kindergarten – WinterKindergarten – Winter– Initial Sound Fluency Initial Sound Fluency – Letter Naming FluencyLetter Naming Fluency– Phoneme Segmentation FluencyPhoneme Segmentation Fluency– Nonsense Word Fluency Nonsense Word Fluency

• Kindergarten – SpringKindergarten – Spring– Letter Naming Fluency Letter Naming Fluency – Phoneme Segmentation FluencyPhoneme Segmentation Fluency– Nonsense Word FluencyNonsense Word Fluency

2727

Other types of CBMs - Other types of CBMs - DIBELSDIBELS• First – FallFirst – Fall

– Letter Naming FluencyLetter Naming Fluency– Phoneme Segmentation FluencyPhoneme Segmentation Fluency– Nonsense Word FluencyNonsense Word Fluency

• First – WinterFirst – Winter– Phoneme Segmentation FluencyPhoneme Segmentation Fluency– Nonsense Word FluencyNonsense Word Fluency– Oral Reading Fluency Oral Reading Fluency

• First – SpringFirst – Spring– Phoneme Segmentation FluencyPhoneme Segmentation Fluency– Nonsense Word FluencyNonsense Word Fluency– Oral Reading Fluency Oral Reading Fluency

2828

Other types of CBMs - Other types of CBMs - DIBELSDIBELS• Second – FallSecond – Fall

– Nonsense Word FluencyNonsense Word Fluency– Oral Reading Fluency Oral Reading Fluency

• Second – WinterSecond – Winter– Oral Reading Fluency Oral Reading Fluency

• Second – SpringSecond – Spring– Oral Reading Fluency Oral Reading Fluency

2929

Other types of CBMs - Other types of CBMs - AIMSwebAIMSweb• Similar reading probes to DIBELSSimilar reading probes to DIBELS

• Adds a reading comprehension piece Adds a reading comprehension piece (MAZE)(MAZE)

• Also has math computation, early Also has math computation, early numeracy and writing componentsnumeracy and writing components

• Product is for purchase, probes not Product is for purchase, probes not available for freeavailable for free

Other types of CBMs: Other types of CBMs: www.interventioncentral.orgwww.interventioncentral.org

• Website has many Website has many CBM probes CBM probes available for available for freefree

• You can create You can create multiple forms of multiple forms of early literacy and early literacy and numeracy probesnumeracy probes

• Many national Many national norms available for norms available for comparisoncomparison

3030


• NumberflyNumberfly

• Create probes for Create probes for Quantity Quantity Discrimination, Discrimination, Missing Number, Missing Number, Number Number IdentificationIdentification

3131

• ChartDogChartDog

• Create charts for Create charts for student progressstudent progress

3232


3333

Wrap Up of CBMWrap Up of CBM• Pros and Cons of each measure we discussedPros and Cons of each measure we discussed

• How to use these measures as a school-wide How to use these measures as a school-wide screening tool (universal screening)screening tool (universal screening)

• Benefits of a using universal screeningBenefits of a using universal screening

• Need for a good data management systemNeed for a good data management system

• Think about going paperless!Think about going paperless!

1 FUNCTIONAL ASSESSMENT: Curriculum-Based Measurement Progress Monitoring NC Department of Public...

Documents

Transcript of 1 FUNCTIONAL ASSESSMENT: Curriculum-Based Measurement Progress Monitoring NC Department of Public...