Guide to Test Interpretation Using DC CAS Score Reports to Guide Decisions and Planning District of...

Guide to Test InterpretationUsing DC CAS Score Reports to Guide Decisions and

Planning

District of ColumbiaOffice of the State Superintendent of Education

CTB/McGraw-Hill

September 26–27, 2013

Introductions

OSSE Jessica Enos

Data Analysis Manager Jeff Noel

Director of Data Management Tommy Shen

Senior Data Analyst

CTB Amy Dement-Dorey

Senior Program Manager Glenn Gage

Program Associate Gabriel Martinez

Content Development Lead Rick Mercado

Standard Setting Specialist

2

Copyright © 2013 CTB/McGraw-Hill LLC.

Objectives

Participants will be prepared to – describe the information found in DC CAS score

reports accurately;– share this information with their colleagues and

stakeholders; and– use the information on those reports to make well-

informed, practical decisions.

3


About This Presentation

Overview the Guide to Test Interpretation (GTI) Describe the reports available in the GTI Discuss best practices for interpreting and using

the reports Provide hands-on experience and discussion

with colleagues

4


Agenda

About the DC CAS and the GTI Discuss the tools and information needed to

make good decisions Discuss the types of score reports available Provide hands-on experience using score

reports Discuss how to use the information to effect

positive changes in the classroom Closing discussion

5


About the DC CAS

Statewide standards-based tests for:– Reading (Grades 2–10)– Mathematics (Grades 2–8 & 10)– Science (Grades 5 & 8), Biology (High School)– Composition (Grades 4, 7, & 10)– Health (Grades 5, 8, & High School)

6


What does the DC CAS measure?

DC CAS:– is aligned to the DC content standards– assesses students’ proficiency in the

knowledge and skills specified in the DC content standards

– communicates student proficiency using performance levels and performance level descriptors (PLDs)

7


Performance Level Descriptors

Abbreviated PLDs are printed on the backs of score reports.

Full PLDs can be found:– in Appendix A of the GTI – on the OSSE website at

osse.dc.gov/service/dc-cas-performance-level-descriptors

8


http://osse.dc.gov/service/dc-cas-performance-level-descriptors



Guide to Test Interpretation

The GTI provides information about how results are reported and how those results can be interpreted and acted upon.

It arrives in schools in the fall, following the spring test administration.

9


Inside the GTI

Sample DC CAS score reports

PLDs for each tested grade and content area

Accommodations that are allowable for Students with Disabilities (SWDs) and English Language Learners (ELLs)

10


Important Companion Document

The DC CAS Resource Guide includes:– Descriptions of the

reporting categories– Content standards that

are targeted in DC CAS– Scoring rubrics and

scoring guides

11


DC CAS Resource Guide

The 2013 DC CAS Resource Guide is now available on the CTB website:www.ctb.com/dc-cas– Then click on “Training

and Ancillary Materials”

The 2014 DC CAS Resource Guide is scheduled to be posted later in the year.

12


http://www.ctb.com/dc-cas


13

Three Types of Score Reports

Stude

nt

•Reading•Mathematics•Science•Biology•Composition

Cla

ssroom

•Roster Report•Content Strands Roster Report•Scoring Categories Roster Report

School

•Content Strands Summary Report•Item Analysis Summary

Score Report Terminology

Raw scores– Number (or percentage) of score points achieved

Scale scores (SS)– Derived from raw scores– Enable comparison across

school years, in the same grade– Do not enable comparison

across grades or content areas

14



15

Score Report Terminology (cont.)

Percent index (PI) scores– PI scores can be interpreted very much like percent

correct scores and range 0–100.– They have been shown to

be more reliable than percent correct scores, representing an expected percent correct if students had been given all possible items measuring the content area.

– PI scores are computed using the scale scores derived from item response theory (IRT).


Content strands– Subcategory of related content standards in a content area– Usually synonymous with “reporting category”

Performance levels– Levels of performance on

DC CAS, with a focus on “proficiency”

Performance level descriptors (PLDs)– Summarize the knowledge, skills, and abilities expected of

students in each performance level

16



Roster– List of student scores organized by class or grade

Disaggregated results– Test results reported for separate student subgroups

Item analysis summary– Information on student performance for each test item

17


Performance Level Descriptors

Example of the back of a Grade 2 student report.Also shown on page 11 of the 2013 GTI.

18

Student Report, Reading/Math

Example of a Grade 2 Student Report for Reading and Mathematics.As shown on page 10 of the 2013 GTI.

19

Using the Student Report

Parents and teachers can use this report to:– Assess a student’s proficiency (i.e., knowledge and

skills) in the DC CAS content standards using the PLD for the student’s performance level

– Compare the student’s content strand scores to state average content strand scores

20


Student Report Information

Scale scores and performance levels– Scale scores and performance levels are comparable

across years for that content area and grade level.– Scale scores and performance levels are not

comparable across grades.

Content strand percent correct scores– Students’ content strand performance can be

compared with the average State performance.

21


Student Report, Composition

Example of a Grade 4 Student Report for Composition.As shown on page 13 of the 2013 GTI.

22

Content Strands Summary Report

Example of a Grade 2 Mathematics Content Strands Summary Report.As shown on page 17 of the 2013 GTI.

23


24

Using the Content Strands Summary Report

Teachers and principals can use this report to:– Evaluate grade-level performance on the content

strands by comparing the school’s performance to state performance on each content strand.

– Identify content strands where instructional approaches, instructional materials, or emphasis (e.g., number of instructional hours, homework) may need to be reconsidered.


25

Content Strands Summary Report Information

PI score for the school and state, for each content strand– You cannot compare these PI scores across content

strands or across years.– PI scores are not adjusted for the varying difficulty

levels of the items in each strand.

26

Student Roster Report

Example of a Grade 6 Roster Report.Also shown on page 15 of the 2013 GTI.

Using Student Roster Reports

Roster reports list and summarize the information found in other reports.

The student roster report provides:– Alphabetical list of students, by class and grade. – For multiple grades in one class, a separate report is

produced for each grade.

27


Content Strands Roster Report

Example of a Grade 8 Mathematics Content Strands Roster Report.Also shown on page 19 of the 2013 GTI.

28


29

Content Strands Roster Reports Information Part 1: Roster

– Alphabetical list of students, by class and grade.– Scores for the total test: scale score, raw score, percent

correct, scale score error range, performance level, PI.– Scores for the content strands: number of items (and #

points possible), minimum PI for proficiency, and performance indicator/PI score.

• Performance indicators allow for comparison of student proficiency across content strands

Part 2: Summary– Summary by class and grade (i.e., mean PI, mean scale

score, mean raw score).

Content Strands Roster Report Summary

Example of a Grade 8 Mathematics Content Strands Roster Report Summary.Also shown on page 21of the 2013 GTI.

30

Item Analysis Summary

Example of a Grade 8 Mathematics Item Analysis Summary.Also shown on page 25 of the 2013 GTI.

31

Using the Item Analysis Summary

Teachers and principals can use this report to:– Evaluate grade-level performance on the objectives

within each content strand– Identify objectives where instructional approaches,

instructional materials, or emphasis (e.g., number of instructional hours, homework) may need to be reconsidered

– Compare the school’s performance to state performance on each objective

– Compare performance on multiple choice and constructed response items

32


Item Analysis Summary Information

Percent correct score for the school and state, for items in each tested standard– Do not compare percent correct scores across items

or across years; they are not adjusted for difficulty.

33



34

Transition to the Common Core

2011 •Transition to the Common Core State Standards began

2012 •DC CAS Reading and Writing were aligned to Common Core•DC CAS Mathematics focused on essential knowledge and skills students needed to be successful with Common Core

2013 •Mathematics was aligned to Common Core

2014 •All DC CAS assessments are aligned to Common Core

35

Practice with Reports

In your own or in pairs, use the reports in the GTI to answer the three questions on the Practice with Reports worksheet.– We’ll review the answers in several minutes.



1. How many students in my class scored Proficient or above in Math overall?a. Which report(s) can I use to answer the question?

• Roster Report (p. 15); or Content Strands Roster Report, Part 2 (p. 21).

b. What is the answer using the report in the GTI? • Roster Report, 4 students; or CSRR2, 26 students.

36



2. For which content strand(s) did my class have the most difficulty reaching proficiency?a. Which report(s) can I use to answer the question?

• Content Strands Roster Report, Part 1 (p. 19)b. What is the answer using the report in the GTI?

• Geometry (5 students did not score proficient)

37


Practice with Reports (cont.)

3. For which content strand(s) is my school’s performance higher than the state (DC public schools, including charters), for the grade level shown?a. Which report(s) can I use to answer the question?

• Content Strands Summary Report (p. 17)

b. What is the answer using the report in the GTI? • Measurement & Data shows the school ahead of State.

38


Small Group Discussion

1. On the Data Exploration Worksheet, identify one or two instructional challenges you face this school year with a class or grade level of students.

2. Identify specific information in the reports that might help you examine these challenges.

3. Use the information to sketch out a plan for addressing the challenges this year, and what information would be expected to change if there is improvement.

39

Note: Some types of questions (e.g., growth) are computed separately from the reports, and are not able to be answered using these reports.

Large Group Discussion

Share your insights, comments, and unanswered questions:– What challenges did you identify?– What information in the score reports was useful?– What thoughts and questions do you have about the

score reports?– What actions, interventions, or other next steps are

you considering?– What additional information or support do you need?

40


Closing Discussion

Planning:– What are your next steps?– Will DC CAS reports help with your planning?– Will additional information be needed to plan for the

2013–2014 school year?

Please give feedback on today’s presentation on the workshop evaluation.

41


For Further Information

For further assistance regarding score reports or the Guide to Test Interpretation, contact:

For more DC CAS information and resources, visit the CTB website at:www.ctb.com/dc-cas

42


http://www.ctb.com/dc-cas

Thank you!

Rick Mercado [email protected]

Gabe Martinez [email protected]

Glenn Gage [email protected]

Amy Dement-Dorey [email protected]

43


Guide to Test Interpretation Using DC CAS Score Reports to Guide Decisions and Planning District of...

Documents

Transcript of Guide to Test Interpretation Using DC CAS Score Reports to Guide Decisions and Planning District of...