USING FORMATIVE AND BENCHMARK ASSESSMENTS IN AN RTI SYSTEM Abby Potter Education Consultant, Title I...
-
Upload
drusilla-copeland -
Category
Documents
-
view
214 -
download
0
Transcript of USING FORMATIVE AND BENCHMARK ASSESSMENTS IN AN RTI SYSTEM Abby Potter Education Consultant, Title I...
USING FORMATIVE AND BENCHMARK ASSESSMENTS IN AN RTI SYSTEM
Abby PotterEducation Consultant, Title I School Support Team
John HumphriesSchool Psychologist, Student Services/Prevention & Wellness
Agenda: 60 minutes
Abby & John: Introduction and Background
John: Benchmark Assessment—Screening Abby: Formative (Ongoing) Assessment John: Benchmark Assessment—Progress
Monitoring Abby & John: Q & A, Discussion as time
allows
Response to Intervention (RtI)
RtI is a process forachieving higher levels of academic and behavioral
success for all students through:
High Quality Instructional Practice
Continuous Review of Student Progress (multiple measures)
Collaboration
Response to Intervention (RtI)
A Systemic Approach forConstant Inquiry
Continuous Review of Student Progress
To Assess:• How all students are performing (screening)• How they are responding to differentiated core instruction
(ongoing assessment)• How they are responding to intervention/additional supports
(monitoring progress)
Topics for Assessment Types
Definitions Purposes/Rationale Strengths & Limitations Common features Research Resources for getting started
Balanced Assessment System
Key Components: Continuum of assessments Multiple users Multiple information sources, used to
create a complete picture of student progress
Each assessment type has a primary purpose, as well as strengths and limitations
Balanced Assessment System
Formative Benchmark Summative
Daily Ongoing Evaluation Strategies Periodic Diagnostic/Progress Assessments Large-Scale Standardized Assessments
Immediate Feedback Multiple Data Points Across Time Annual Snapshot
Student-Centered Classroom/School-Centered School /District/State-Centered
8
CLASSROOM INTERIM LARGE SCALE What do I learn from this assessment?
What learning comes next for this student? How are students progressing? How well is this program working?
How are schools, districts, and/or states progressing?
Who will use the information?
Primary Users Students Teachers
Primary Users Students Teachers/ Teaching teams Parents Schools/Districts
Primary Users Secondary Users Schools/Districts Teachers State Parents Federal Students Independent Evaluators
What is the frequency of the assessment?
Short Cycle - occurring within and between lessons
Medium Cycle - occurring within and between instructional units
Long Cycle – occurring annually or bi-annually
What action will be taken with the results?
Within the instructional activity, information is used to change or adjust teaching
Students receive frequent and meaningful feedback on their performance
Teachers engage students in the monitoring of their own learning
Re-teach decisions
Professional learning communities use data to identify strengths and gaps in instruction and curriculum
Curriculum may be changed/refined Teachers may modify instruction for
individuals and groups of students based on their progression towards outcome goals (selected, targeted options)
Strategic long term evaluation of curriculum/programming
Determination of Adequate Yearly Progress
What professional development is needed for the primary users?
Need for a solid understanding of learning progressions and the feedback loop
Concrete examples of formative assessment strategies
Collaborative work time to share effective teaching strategies
Opportunities to practice and perfect usage Collaboration time plan for assessments
If district or school developed – identification of learning outcomes and time to develop items
If using a commercial product – time to learn features of the tool and how the results link to district curriculum and state standards
Time to disaggregate and interpret data Collaborative work time to share effective
teaching strategies
Understanding and interpretation of large scale assessment
Time to disaggregate data and evaluate progress at building level and grade level
Opportunities for revising instructional practices and/or curriculum
What is the connection to content standards?
Lessons are connected to the broader curriculum Curriculum is linked to grade level standards Grade level standards are linked to the framework
Examples Examples: feedback questioning, interviews, exit questions observations discussions un-graded class work
Examples: end of unit assessments district assessments (6-traits) benchmark assessments commercial products (See
www.studentprogress.org))
Examples: WKCE WAA NAEP MAPS?
Summative/ Large-Scale
Purpose : To determine how students in schools,
districts, and states are progressing
To inform curriculum and instruction
To determine Adequate Yearly Progress (AYP)
Benchmark Assessment
Purpose: To determine to what extent all students
are progressing (screening)
To determine how well additional supports or services are working before too much time passes (monitoring progress)
Formative Assessment
Purpose:
To consider what learning comes next for students
To improve learning while there is still time to act – before the graded event
Current Practices
What are you doing now to assess your students?
How is it working? WKCE and AYP WKCE Definition of Proficiency Differences WKCE and NAEP Achievement Gaps
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 20140%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
f(x) = 0.000285714285714286 x + 0.860666666666668R² = 0.0171428571428512
f(x) = − 0.00342857142857143 x + 0.458666666666667R² = 0.280519480519481
f(x) = 0.00514285714285715 x + 0.508666666666667R² = 0.15779220779221
% 4th Grade Proficient/Advanced WKCE Reading Statewide
White
Linear (White)
Black
Hispanic
SwD
Linear (SwD)
Econ. Dis.
ELL
Linear (ELL)
NCLB requires 100% by 2014.Does not include WAA.Data from WINNS, 9/10/08
RtI: From SPED to Gen Ed to Every Ed
Perspective: “The Art and The Science of Teaching”
Why RtI? Why now? One difference is the major advances in assessment and intervention technologies that allow us to make better decisions and intervene more appropriately. Moving from “true” CBM to more standardized measures.
Benchmark Assessment:Screening
Definitions Purposes/Rationale Strengths and Limitations Common features Research Resources for getting started: Academics
& Behavior
Screening: Definition
Screening is characterized by fast, inexpensive, repeatable data collection about critical skills, beliefs, or behaviors.
Screening usually identifies students who need further assessment or provides information for future planning activities.
Screening: Purposes/Rationale
The purpose of screening is to identify students who are “at-risk” of a poor outcome
Rationale: Use a screener with strong statistical properties along with other data to identify students you want to learn more about
Don’t wait until it’s too late. WKCE is a poor screener for this reason.
Screening: Strengths & Limitations
By definition, easy, quick, repeatable
Immediate results Guide
programming Predictive validity
Diagnostically Guiding
instruction Administrators Teachers
Absent good PM and Formative Asmt.
Statistical limitations
Strengths Limitations: How Misused
Selected Research on Screening
Jenkins, J. R., Hudson, R. F., & Johnson, E. S. (2007). Screening for service delivery in an RTI framework: Candidate measures. School Psychology Review, 36, 560-82.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103) New York: Macmillan.
Riedel, B. W. (2007). The relationship between DIBELS, reading comprehension, and vocabulary in urban first-grade students. Reading Research Quarterly, 42, 546–567.
Ritchie, K. D., & Speece, D. L. (2004). Early identification of reading disabilities: Current status and new directions. Assessment for Effective Intervention, 29(4), 13–24.
Snellen Eye Chart (1862).
World Health Organization: Principles of Screening (1968)
The condition should be an important health problem. There should be a treatment for the condition. Facilities for diagnosis and treatment should be available. There should be a latent stage of the disease. There should be a test or examination for the condition. The test should be acceptable to the population. The natural history of the disease should be adequately understood. There should be an agreed policy on who to treat. The total cost of finding a case should be economically balanced in
relation to medical expenditure as a whole. Case-finding should be a continuous process, not just a one time
project. Wilson JMG, Jungner G. Principles and Practice of Screening for
Disease. WHO Chronicle 1968;22(11):473
Resources for Screening
BASC CBCL Office Referrals Teacher Nomination TeenScreen GAIN-SS Online SOS
Go to National Center on Student Progress Monitoring
Also see The ABCs of CBM by Hosp etc.
MAP?
Behavioral Screening Academic Screening
Formative (Ongoing) Assessment
Definitions Purposes/Rationale Strengths and Limitations Common features Research Resources for getting started: Academics
& Behavior
Formative (Ongoing) Assessment
Definition:“Formative assessment is an intentional and
systematic process used by teachers and students during instruction that provides feedback to adjust on-going teaching and learning to improve students’ achievement of the intended instructional outcomes.”
CCSSO, 2007
Formative (Ongoing) Assessment
intentional systematic process feedback adjust on-going intended instructional outcomes
Purpose:• To consider what learning comes next
for the student• To improve learning while there is still
time to act – before the graded event
Formative (Ongoing) Assessment
Formative (Ongoing) Assessment
Examples: • Teacher observations• Teacher questioning & class discussions• Analysis of student work (graded & non-
graded)• Exit questions• Teacher feedback• Student self-assessment• KWLs• Student Journals
Strengths:• Informs day-to-day instruction• Informs intervention• Instant information• Student self-assessment• Provides information about on-going student progress• Designed & evaluated by those who know the students
best• Provides a huge volume of qualitative, descriptive data
Formative (Ongoing) Assessment
Formative (Ongoing) Assessment
Limitations: Time Informal/not standardized Overabundance of information May be challenging to ‘grade’ When used to the exclusion of other types
of assessment
Formative (Ongoing) Assessment
Essential components of effective formative assessment:
Learning Progressions: clearly articulate the sub-goals of the ultimate learning goal
Learning Goals and Criteria for Success: clearly identified and communicated to students
Descriptive Feedback: provided to studentswith evidence-based feedback that is linked to the intended instructional outcomes and criteria for success.
CCSSO, 2008
Formative (Ongoing) Assessment
Essential components of effective formative assessment (continued):
Self- and Peer-Assessment: important for providing students an opportunity to think metacognitively about their learning.
Collaboration: A classroom culture in which teachers and students are partners in learning should be established.
CCSSO, 2008
Formative (Ongoing) Assessment
Research•Inside the Black Box: Raising Standards Through Classroom AssessmentBy Paul Black and Dylan Wiliam (1998)
•New assessment beliefs for a new school missionBy Rick Stiggins (2004)
•Implementing Formative Assessment at the District Level: An Annotated Bibliography (New England Comprehensive Center)
Formative (Ongoing) Assessment
Resources for getting started: Academics & Behavior
Set learning goals and criteria for success
Select assessment techniques (teacher and students)
Determine how feedback is provided Organize information from formative
assessment (teacher and students)
Formative (Ongoing) Assessment
“Assessment FOR learning turns the classroom assessment process and its results into an instructional intervention designed to increase, not merely monitor, student learning.”
Richard Stiggins
Benchmarks: Progress Monitoring
Definitions Purposes/Rationale Strengths and Limitations Common features Research Resources for getting started: Academics
& Behavior
Progress Monitoring: Definition Progress monitoring (PM) is a
scientifically-based practice used to assess student performance and evaluate the effectiveness of instruction.
PM: Purposes/Rationale
PM has two purposes: Determine whether students are
progressing appropriately from additional supports and intervention
Build more effective supports and interventions
Rationale: Use PM to closely monitor whether what we’re doing is effective!
PM: Strengths & Limitations
High frequency Sensitive to
change Guide
programming more than screening
May have to make your own PM tools
Improper tools give invalid, unreliable results
Statistical limitations
Used in isolation
Strengths Limitations: How Misused
Research on Progress Monitoring
A substantial research literature Support a wide range of educational
decisions Beginning in 1977 as Data-Based Program
Modification (Deno & Mirkin, CEC) "Developments in Curriculum-Based
Measurement" by S.L. Deno, 2003, The Journal of Special Education, 37. 3., 184-192.
Progress Monitoring and SLD Federal regulations currently require: Information demonstrating that the student
received repeated assessments of achievement reflecting student progress §300.309(b)(2)
Data-based documentation is “…an objective and systematic process of documenting a child’s progress.” “…data documenting a child’s progress are systematically collected and analyzed…” Comments, Page 46657
National Perspective
President Obama on Monday: “…far too few districts are emulating the example of Houston and Long Beach, and using data to track how much progress a student is making and where that student is struggling - a resource that can help us improve student achievement.”
Resources for PM
Frequency of difficulties in school
Self-rating Parent/teacher
rating Determined by
treatment providers
Go to National Center on Student Progress Monitoring
Also see The ABCs of CBM by Hosp
Behavioral PM Academic Screening
Summary
Everyone has an important role in selecting assessments for RtI schools
Reading specialists have expertise in teaching and assessing reading skills
Teachers have expertise in aligning assessment with curriculum
School psychologists have expertise in tests and measurement for academics and behavior
How Does it Fit Together?
Addl.Diagnostic
Assessment
InstructionResults
Monitoring
IndividualDiagnostic
IndividualizedIntensive
weekly
All Students at a grade level
Fall Winter Spring
UniversalScreening
None ContinueWithCore
InstructionGuided byFormative
Assessment
SUMMATIVE•Grades•Discipline•AYP Measures
GroupDiagnostic
SmallGroupDifferen-tiatedBy Skill
2 times/month
Step 1Step 2 Step 3 Step 4
SupplementalServices
1-5%
5-10%
80-90%CoreInstruction
IntensiveSupport
Courtesy of Dave Tilley, Heartland AEA
Questions in Test Selection
How does this map to our data system? Does this test have adequate technical properties
for our intended use? Reliability & Validity Frequency Scale Alignment with our curriculum
How will we use the collected data? Don’t use a test outside of its how intended.
If using for SLD, federal regulations § 300.304 require use “for the purposes for which the assessments or measures are valid and reliable”
Achieving Balance
Thinking about your assessment system… Is your system balanced?
If not, are you okay with imbalance? How do the assessments support and
inform one another? Do all users know the purpose, strengths
and limitations of the assessments? What do you do with the results of the
different assessments? Does everyone play an important role? How deal with disagreement? Collaborate!
Myths and Misperceptions
Myth: “Running Records have no role in RtI” Myth: “DIBELS does not work—no research” Myth: “PM is SPED, no need in Gen Ed” Myth: “WKCE is worthless” Myth: “We already assess too much and now
RtI is going to make us assess more” Myth: “School Psychologists will rule the world
and take over reading programming” Myth: “Reading Specialists will rule the world
and take over school psychology programming”
Additional Research Sources
Response to Intervention: Research for Practice. 2007, NASDSE
www.rti4success.org www.rtinetwork.org (NASP and IRA)
Response to Intervention: A Research Review. Hughes & Dexter
Contact Information
Abby Potter [email protected] (608) 267-7338
John Humphries [email protected] (608) 266-7189
USING FORMATIVE AND BENCHMARK ASSESSMENTS IN AN RTI SYSTEM
Abby [email protected]
John [email protected]
(800) 441-4563 dial 6