Post on 04-Jan-2016
Guide to Test InterpretationUsing DC CAS Score Reports to Guide Decisions and
Planning
District of ColumbiaOffice of the State Superintendent of Education
CTB/McGraw-Hill
September 26–27, 2013
Introductions
OSSE Jessica Enos
Data Analysis Manager Jeff Noel
Director of Data Management Tommy Shen
Senior Data Analyst
CTB Amy Dement-Dorey
Senior Program Manager Glenn Gage
Program Associate Gabriel Martinez
Content Development Lead Rick Mercado
Standard Setting Specialist
2
Copyright © 2013 CTB/McGraw-Hill LLC.
Objectives
Participants will be prepared to – describe the information found in DC CAS score
reports accurately;– share this information with their colleagues and
stakeholders; and– use the information on those reports to make well-
informed, practical decisions.
3
Copyright © 2013 CTB/McGraw-Hill LLC.
About This Presentation
Overview the Guide to Test Interpretation (GTI) Describe the reports available in the GTI Discuss best practices for interpreting and using
the reports Provide hands-on experience and discussion
with colleagues
4
Copyright © 2013 CTB/McGraw-Hill LLC.
Agenda
About the DC CAS and the GTI Discuss the tools and information needed to
make good decisions Discuss the types of score reports available Provide hands-on experience using score
reports Discuss how to use the information to effect
positive changes in the classroom Closing discussion
5
Copyright © 2013 CTB/McGraw-Hill LLC.
About the DC CAS
Statewide standards-based tests for:– Reading (Grades 2–10)– Mathematics (Grades 2–8 & 10)– Science (Grades 5 & 8), Biology (High School)– Composition (Grades 4, 7, & 10)– Health (Grades 5, 8, & High School)
6
Copyright © 2013 CTB/McGraw-Hill LLC.
What does the DC CAS measure?
DC CAS:– is aligned to the DC content standards– assesses students’ proficiency in the
knowledge and skills specified in the DC content standards
– communicates student proficiency using performance levels and performance level descriptors (PLDs)
7
Copyright © 2013 CTB/McGraw-Hill LLC.
Performance Level Descriptors
Abbreviated PLDs are printed on the backs of score reports.
Full PLDs can be found:– in Appendix A of the GTI – on the OSSE website at
osse.dc.gov/service/dc-cas-performance-level-descriptors
8
Copyright © 2013 CTB/McGraw-Hill LLC.
Guide to Test Interpretation
The GTI provides information about how results are reported and how those results can be interpreted and acted upon.
It arrives in schools in the fall, following the spring test administration.
9
Copyright © 2013 CTB/McGraw-Hill LLC.
Inside the GTI
Sample DC CAS score reports
PLDs for each tested grade and content area
Accommodations that are allowable for Students with Disabilities (SWDs) and English Language Learners (ELLs)
10
Copyright © 2013 CTB/McGraw-Hill LLC.
Important Companion Document
The DC CAS Resource Guide includes:– Descriptions of the
reporting categories– Content standards that
are targeted in DC CAS– Scoring rubrics and
scoring guides
11
Copyright © 2013 CTB/McGraw-Hill LLC.
DC CAS Resource Guide
The 2013 DC CAS Resource Guide is now available on the CTB website:www.ctb.com/dc-cas– Then click on “Training
and Ancillary Materials”
The 2014 DC CAS Resource Guide is scheduled to be posted later in the year.
12
Copyright © 2013 CTB/McGraw-Hill LLC.
Copyright © 2013 CTB/McGraw-Hill LLC.
13
Three Types of Score Reports
Stude
nt
•Reading•Mathematics•Science•Biology•Composition
Cla
ssroom
•Roster Report•Content Strands Roster Report•Scoring Categories Roster Report
School
•Content Strands Summary Report•Item Analysis Summary
Score Report Terminology
Raw scores– Number (or percentage) of score points achieved
Scale scores (SS)– Derived from raw scores– Enable comparison across
school years, in the same grade– Do not enable comparison
across grades or content areas
14
Copyright © 2013 CTB/McGraw-Hill LLC.
Copyright © 2013 CTB/McGraw-Hill LLC.
15
Score Report Terminology (cont.)
Percent index (PI) scores– PI scores can be interpreted very much like percent
correct scores and range 0–100.– They have been shown to
be more reliable than percent correct scores, representing an expected percent correct if students had been given all possible items measuring the content area.
– PI scores are computed using the scale scores derived from item response theory (IRT).
Score Report Terminology (cont.)
Content strands– Subcategory of related content standards in a content area– Usually synonymous with “reporting category”
Performance levels– Levels of performance on
DC CAS, with a focus on “proficiency”
Performance level descriptors (PLDs)– Summarize the knowledge, skills, and abilities expected of
students in each performance level
16
Copyright © 2013 CTB/McGraw-Hill LLC.
Score Report Terminology (cont.)
Roster– List of student scores organized by class or grade
Disaggregated results– Test results reported for separate student subgroups
Item analysis summary– Information on student performance for each test item
17
Copyright © 2013 CTB/McGraw-Hill LLC.
Performance Level Descriptors
Example of the back of a Grade 2 student report.Also shown on page 11 of the 2013 GTI.
18
Student Report, Reading/Math
Example of a Grade 2 Student Report for Reading and Mathematics.As shown on page 10 of the 2013 GTI.
19
Using the Student Report
Parents and teachers can use this report to:– Assess a student’s proficiency (i.e., knowledge and
skills) in the DC CAS content standards using the PLD for the student’s performance level
– Compare the student’s content strand scores to state average content strand scores
20
Copyright © 2013 CTB/McGraw-Hill LLC.
Student Report Information
Scale scores and performance levels– Scale scores and performance levels are comparable
across years for that content area and grade level.– Scale scores and performance levels are not
comparable across grades.
Content strand percent correct scores– Students’ content strand performance can be
compared with the average State performance.
21
Copyright © 2013 CTB/McGraw-Hill LLC.
Student Report, Composition
Example of a Grade 4 Student Report for Composition.As shown on page 13 of the 2013 GTI.
22
Content Strands Summary Report
Example of a Grade 2 Mathematics Content Strands Summary Report.As shown on page 17 of the 2013 GTI.
23
Copyright © 2013 CTB/McGraw-Hill LLC.
24
Using the Content Strands Summary Report
Teachers and principals can use this report to:– Evaluate grade-level performance on the content
strands by comparing the school’s performance to state performance on each content strand.
– Identify content strands where instructional approaches, instructional materials, or emphasis (e.g., number of instructional hours, homework) may need to be reconsidered.
Copyright © 2013 CTB/McGraw-Hill LLC.
25
Content Strands Summary Report Information
PI score for the school and state, for each content strand– You cannot compare these PI scores across content
strands or across years.– PI scores are not adjusted for the varying difficulty
levels of the items in each strand.
26
Student Roster Report
Example of a Grade 6 Roster Report.Also shown on page 15 of the 2013 GTI.
Using Student Roster Reports
Roster reports list and summarize the information found in other reports.
The student roster report provides:– Alphabetical list of students, by class and grade. – For multiple grades in one class, a separate report is
produced for each grade.
27
Copyright © 2013 CTB/McGraw-Hill LLC.
Content Strands Roster Report
Example of a Grade 8 Mathematics Content Strands Roster Report.Also shown on page 19 of the 2013 GTI.
28
Copyright © 2013 CTB/McGraw-Hill LLC.
29
Content Strands Roster Reports Information Part 1: Roster
– Alphabetical list of students, by class and grade.– Scores for the total test: scale score, raw score, percent
correct, scale score error range, performance level, PI.– Scores for the content strands: number of items (and #
points possible), minimum PI for proficiency, and performance indicator/PI score.
• Performance indicators allow for comparison of student proficiency across content strands
Part 2: Summary– Summary by class and grade (i.e., mean PI, mean scale
score, mean raw score).
Content Strands Roster Report Summary
Example of a Grade 8 Mathematics Content Strands Roster Report Summary.Also shown on page 21of the 2013 GTI.
30
Item Analysis Summary
Example of a Grade 8 Mathematics Item Analysis Summary.Also shown on page 25 of the 2013 GTI.
31
Using the Item Analysis Summary
Teachers and principals can use this report to:– Evaluate grade-level performance on the objectives
within each content strand– Identify objectives where instructional approaches,
instructional materials, or emphasis (e.g., number of instructional hours, homework) may need to be reconsidered
– Compare the school’s performance to state performance on each objective
– Compare performance on multiple choice and constructed response items
32
Copyright © 2013 CTB/McGraw-Hill LLC.
Item Analysis Summary Information
Percent correct score for the school and state, for items in each tested standard– Do not compare percent correct scores across items
or across years; they are not adjusted for difficulty.
33
Copyright © 2013 CTB/McGraw-Hill LLC.
Copyright © 2013 CTB/McGraw-Hill LLC.
34
Transition to the Common Core
2011 •Transition to the Common Core State Standards began
2012 •DC CAS Reading and Writing were aligned to Common Core•DC CAS Mathematics focused on essential knowledge and skills students needed to be successful with Common Core
2013 •Mathematics was aligned to Common Core
2014 •All DC CAS assessments are aligned to Common Core
35
Practice with Reports
In your own or in pairs, use the reports in the GTI to answer the three questions on the Practice with Reports worksheet.– We’ll review the answers in several minutes.
Copyright © 2013 CTB/McGraw-Hill LLC.
Practice with Reports
1. How many students in my class scored Proficient or above in Math overall?a. Which report(s) can I use to answer the question?
• Roster Report (p. 15); or Content Strands Roster Report, Part 2 (p. 21).
b. What is the answer using the report in the GTI? • Roster Report, 4 students; or CSRR2, 26 students.
36
Copyright © 2013 CTB/McGraw-Hill LLC.
Practice with Reports
2. For which content strand(s) did my class have the most difficulty reaching proficiency?a. Which report(s) can I use to answer the question?
• Content Strands Roster Report, Part 1 (p. 19)b. What is the answer using the report in the GTI?
• Geometry (5 students did not score proficient)
37
Copyright © 2013 CTB/McGraw-Hill LLC.
Practice with Reports (cont.)
3. For which content strand(s) is my school’s performance higher than the state (DC public schools, including charters), for the grade level shown?a. Which report(s) can I use to answer the question?
• Content Strands Summary Report (p. 17)
b. What is the answer using the report in the GTI? • Measurement & Data shows the school ahead of State.
38
Copyright © 2013 CTB/McGraw-Hill LLC.
Small Group Discussion
1. On the Data Exploration Worksheet, identify one or two instructional challenges you face this school year with a class or grade level of students.
2. Identify specific information in the reports that might help you examine these challenges.
3. Use the information to sketch out a plan for addressing the challenges this year, and what information would be expected to change if there is improvement.
39
Note: Some types of questions (e.g., growth) are computed separately from the reports, and are not able to be answered using these reports.
Large Group Discussion
Share your insights, comments, and unanswered questions:– What challenges did you identify?– What information in the score reports was useful?– What thoughts and questions do you have about the
score reports?– What actions, interventions, or other next steps are
you considering?– What additional information or support do you need?
40
Copyright © 2013 CTB/McGraw-Hill LLC.
Closing Discussion
Planning:– What are your next steps?– Will DC CAS reports help with your planning?– Will additional information be needed to plan for the
2013–2014 school year?
Please give feedback on today’s presentation on the workshop evaluation.
41
Copyright © 2013 CTB/McGraw-Hill LLC.
For Further Information
For further assistance regarding score reports or the Guide to Test Interpretation, contact:
For more DC CAS information and resources, visit the CTB website at:www.ctb.com/dc-cas
42
Copyright © 2013 CTB/McGraw-Hill LLC.
Thank you!
Rick Mercado ricardo_mercado@ctb.com
Gabe Martinez gabriel_martinez@ctb.com
Glenn Gage glenn_gage@ctb.com
Amy Dement-Dorey amy_dement-dorey@ctb.com
43
Copyright © 2013 CTB/McGraw-Hill LLC.