Data Wise TeamsThe Step By Step Guide to Using

Assessment Results to Improve Teaching and Learning

Pamela Askew

*(2012) Askew,P.

Data Wise Improvement Process

Act6. Develop Action Plan

7. Plan to Assess Progress8. Act and Assess

Prepare1. Organize for Collaborative Work

2. Build Assessment Literacy

Inquire3. Create Data Overview

4. Dig Into Data5. Examine Instruction

Creating a Data TeamWho’s on theTeam?

Principal School Counselor Instructional Specialist Academic Engagement Administrator (AEA) Support Staff Teachers

Principal

Provide ongoing support to classroom teachers, school counselor, instructional specialist, teachers, administrators and support staff with targeted technical assistance, coaching, and mentoring through: Formalized workshops Literacy leadership team meetings Data study groups Teacher-teacher assistance Modeling, demonstration lessons, coaching and more…

School Counselor

Collaboratively analyze data and participate in problem solving

Coordinate the completion of diagnostic assessment

Coordinate the RCT of designated students Assist in determining the need for Student

Service referral and/or a request for assistance from the School Support Team

Instructional Specialist

Coordinate the screening/progress monitoring of the specified groups of students in their school

Collaboratively analyze data Support teachers in implementing

interventions prescribed for their students Assist in establishing small group instruction

and interventions for identified students

Teachers/*AEA/School Counselor

Conduct screening/progress monitoring and diagnostic assessment of specified students

Assist teachers with the provision of small group intervention

Conduct the RCT of designated students *Monitor data use to drive decisions *Engage in data dialogues

Pair-Share Protocol-Data Teams Share with the person next to you

What personal challenges lie before you in participating in your Team role?

What resources are available to you to help you in your Team role?

School Data Inventory

Data Source Content orArea

Dates of Collection

Students Assessed

Accessibility Current Data Use

More Effective Use

DIBELS Reading: Phonological Awareness, Phonics,Fluency,Comprehension

PM:September,January, May

RCT: Ongoing

K-5 and Burst K-3 in grades 4,5 (TRC)

RCT: At-risk Students

Administration; Teacher

Data is used to form guided Readinggroups

Analyze data to guide instruction, intervention, and professional development of teachers.

Burst Reading:VocabularyLanguage

Oct-May K-3 Administration Data is used to differentiate instruction

Inform instruction regarding vocabulary development

MEAP ELAMathScienceSocial Studies

Sept-March 3-5 Administration; Teacher

Retention Decisions/Interventions

InformInstruction

MAP Reading: Fluency, Comprehension;Math

September, January, May

2-5 Administration Guide instruction/Intervention

Analyze data to guide instruction and PD

Instructional Learning Cycle,Process, Perception, Demographic

Reading:Comprehension;Attendance,

Ongoing PK-5 Teachers, Adm Benchmarking Inform instruction,Review patterns,Provide Interventions

Comprehensive Research-Based Wayne LiteracyPlan (CRWLP) Single Core reading program for all elementary

students(Open Court) Provides the basis for reading instruction and facilitates

the meaningful connection to supplemental materials and intervention programs (Renaissance Learning)

Required use of research-based intervention materials as approved by the district

CRWLP delineates a plan to assess and monitor student progress that is used to drive educational decisions: DIBELS, Burst, TRC

Open Court A reading program designed for K - 5th graders who read below or at grade

level

Focus: Phonemic Awareness and Phonics, Comprehension, and Writing

Goal: Build phonemic awareness (all sounds) Link sounds to printed text

(the 1st step toward grasping the alphabetic principle) Help build rapid decoding skills

Time Frame: 90 minutes of instruction 30 minutes of workshop

Daily lessons patterned after a 3-step model: Rereading for Fluency Reading the (new) Book (reading strategies and phonics applications) Working with Words and Writing Sentences (phonics application)

Star Reading A supplemental intervention reading program designed for students in

grades K-8 who read below or at grade level

Focus: to accelerate learning for all children of all ability levels and ethnic and social backgrounds, worldwide

Goal: Apply decoding skills and build comprehension skills

Time Frame: 30-40 minutes of intervention followed by testing

Daily lessons patterned after 5-step model: Revisiting (fluency and comprehension) Reviewing (comprehension and application of its 4 reading strategies) Rehearsing (background knowledge and relevant vocabulary) Reading and Reciprocal Teaching (comprehension) Responding/Reflecting (comprehension)

DIBELS DIBELS are procedures and measures for assessing the acquisition of early

literacy skills from K-6 grade. They are designed to be short (one minute) fluency measures used to regularly monitor the development of early literacy and early reading skills.

Focus: phonemic awareness, alphabetic principle, accuracy and fluency with connected text, reading comprehension, and vocabulary

Key Features: Progress Monitoring (on-going, targeted skill development) Variety of reports generated by student, by group, for parents Identifies for the teacher who needs additional literacy intervention via

immediate feedback

Levels:o RED - At risk/Deficit o YELLOW - Some risk/Emerging o GREEN - Low risk/Established

Time Frame: follow recommended schedule

Burst

Designed for students K through third grade Provides intervention via instruction and

practice in small groups: phonemic awareness language enrichment letters and sounds decoding and early spelling reading pace and style improves writing skills

TRC (recommended for 3rd-6th grades)

Combines DIBELS skills with Text, Reading, & Comprehension identifies a student’s instructional reading level so that lessons are

specific to those needs. includes: print concepts, reading behaviors, and oral

comprehension. Report out for data collection

Building Assessment Literacy (summative and formative) Assessment concepts critical to data teams

Discrimination Reliability Validity Measurement Error Score Inflation

Sampling, Discrimination andSensitivity to Change Sampling

Items are chosen that best measure an overall concept or are of content

Covers the “Broad” with the “Narrow” Discrimination

The ability of an item to differentiate between high and low performance

Sensitivity to change The ability to timely measure change

Test Qualities

Reliability The consistency of test scores

Validity Does the test score mean what it says it means? Can you generalize from the test score to

something real?

Score InflationA Major Threat to Validity Increases in scores that do not reflect a change in abilities What practices contribute to score inflation on formative?

Not using summative assessments to frame meaningful performance goals.

Not showing criteria and models in advance. Not showing criteria and models in advance. Not offering appropriate choices. Not providing feedback early and often. Not encouraging self-assessment and goal setting. Not allowing new evidence of achievement to replace old

evidence.

Oral Reading Fluency

240220200180160140120100806040200

Re

ad

ing

FC

AT

-SS

S S

core

550

500

450

400

350

300

250

200

150

100

50

Buck, J., & Torgesen, J. (2003). The relationship between performance on a measure of oral reading fluency and performance on the Florida Comprehensive Assessment Test (Technical Report 1). Tallahassee, FL: Florida Center for Reading Research,.

Above 110, the odds are 91% the student will rank “adequate” on State Assessment.

Below 80, the odds are 19% the student will rank “adequate” on State Assessment.

Linkage of Oral Reading Fluency to State Reading Outcome Assessments

PMRN

The Progress Monitoring and Reporting Network (PMRN) is a Web-based data management system for the recording, storing, and reporting of student gains in reading. The PMRN is:

a convenient place for recording and organizing the results of student reading assessments

a secure, centralized, easily accessible location for the storage of student information

a tool that provides timely and helpful reports to assist educators effectively analyze, plan, and communicate

*There are minimum system requirements. Details are available through the Privacy Statement link on the Home Page of the PMRN Website.

PMRN

Recording and reporting student progress in the PMRN is easy and can be completed from any computer with internet access*.

Assess – Assessment teams monitor student progress four (3) times per year

Enter Data – designated Data Entry Level (DEL) Users Submit the scores into the PMRN through the Website’s data entry page

Compute – The PMRN aggregates the data Access – Teachers access their students’ reports

immediately

Recommended Instructional Level

Green – Initial – the current reading program is meeting the needs of the student.

Yellow – Strategic – the student has weaknesses that are specific and need to be identified and addressed to reach grade level reading

Red – Intensive - the student needs Immediate Intensive Intervention (iii) in order to make the gains required to reach grade level reading

Initial

Strategic

Intensive

Interpreting DIBELS scores Above Average: At or above the 60th percentile

Low Risk: At grade level-above the 40th percentile

Moderate Risk: Moderately below grade level-between the 20th and 40th percentile

High Risk: Serious difficulties in reading-below the 20th percentile

Class Status Report

The Class Status Report is the primary report for the teacher. The student’s risk level for the individual progress monitoring measures are provided, as well as the Recommended Instructional Levels.

The report can be sorted in a variety of ways and provides links to numerous other reports.

Data Analysis Activity #1

Which students are in most need of additional intervention?

Which students are on the fence or need watchful eye? Which students are in most need

of an intervention program with a strong phonemic awareness component?

What does the data say about student Rcct? Pogh?

Which students appear to require more intense level of intervention?

School Status Report

Progress Report

The student’s scores are plotted against the distribution of scores for the class. The flag indicates the students score and the level of risk.

Target

Risk Level

1. Describe the progress of the class in relation to the standard from Assessments 1 to 2 and from 2 to 4. 2. Describe the progress of the student in relation to the class and the standard from Assessments 1 to 2 and from 1 to 4.

Data Analysis Activity # 2

Comparison Reports

A Comparison Report is a Box & Whiskers graph representing the range of scores, on a specific measure, for a class and 10 other classes OR a grade at a school and five (5) other schools that have similar demographics. The median score of the target class/school is plotted against the comparison group. The populations of the comparison groups are SIMILAR, but not identical.

What are the general trends for the comparison groups in relation to the standard?

What is the general trend for the target class in relation to the standard? In relation to the comparison groups?

Data Analysis Activity # 3

Levels of Service DT to SIT: Levels of Service DT to SIT: Linking Curriculum/Instruction with RTI

Core Reading Program (PM)

Intervention (PM and OPM)

Supplemental Materials/Guided Reading

Individually Designed Intervention Using Problem Solving and OPM

Evaluation ReferralSchool Improvement

Team (SIT)

DT

What Does Your Data Say?

Evaluating the Core – What Happens with the Initials Across Time? School Status Report (Progress Report)

Interventions – Who Should Receive Them? Class Status Report

Referral to SST – The Filtering Process Analyzing the Three Rules Individual Progress Report

Below Standard Comparison to Peer Group Rate of Growth

Evaluating the Core

Look at School Status Report (Progress Report) Across the Benchmarks

Analyze what happens with the Initials across time

What patterns exist across the grade levels? What conclusions can you draw regarding

the effectiveness of the core curriculum?

Interventions

Using a Class Status Report from kindergarten or first grade, determine who should have received interventions and which interventions they should have received.

Which students could have benefited from differentiated instruction?

Did the identified students actually receive differentiated instruction and/or interventions?

What questions should you ask if you have a student with this profile?

Begin Data Dialogue

Engage all staff/team members Each staff person or team member must come up with three

questions about the data: constructive and collaborative/problem solving

Determine the outcome for the discussion Keep focus on improvement, not blaming Provide adequate time ( recommended: 30-45 minutes) Be careful not to end discussions too soon. Take time to analyize

the data Make sure that everyone know the target