Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose •...
Transcript of Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose •...
![Page 1: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/1.jpg)
Assessment Standards Setting
Process
TNReady and Social Studies Transition
Dr. Nakia Towns, Assistant Commissioner, Data & Research
![Page 2: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/2.jpg)
Standards Setting: Purpose
• To make recommendations to the Tennessee State Board
of Education regarding cut scores for all grade 3-11 state
assessments.
‒ Standards Setting Committee recommendations will be presented to
State Board for first read in July/August 2016.
‒ The second read and approval will be August/September 2016.
‒ This may require a called meeting depending upon SBE proposed
summer meeting schedule.
‒ Cut scores must be approved prior to final score reporting for
TNReady.
• To ensure recommendations are consistent with
expectations stated in the Tennessee Academic
Standards.
2
![Page 3: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/3.jpg)
• The minimum scale score a student must earn to be
considered at a certain performance level
• Three cut scores result in four levels of performance.
Standards Setting: What is a Cut Score?
3
Cut
Score
A
Cut
Score
B
Cut
Score
C
![Page 4: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/4.jpg)
Cut Scores Matter: TCAP Proficiency Trends
4
![Page 5: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/5.jpg)
Cut Scores Matter: NAEP Proficiency Trends
5
27% 28%26%
34% 34%
0%
10%
20%
30%
40%
50%
60%
70%
NAEP Proficiency Trends: 4th Grade READING
2007 2009 2011 2013 2015
29% 28%30%
40% 41%
0%
10%
20%
30%
40%
50%
60%
70%
NAEP Proficiency Trends:4th Grade MATH
2007 2009 2011 2013 2015
![Page 6: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/6.jpg)
• Standards setting determines student performance and
reporting.
• Tennessee educators and community members
engage in standards setting to make recommendations
about how we define expectations for student mastery
of our state standards.
• Before the standards setting process can begin, we must
have clearly defined those student expectations through
Performance Level Descriptors.
Standards Setting Process
6
![Page 7: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/7.jpg)
Performance Level
Descriptors (PLDs)
![Page 8: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/8.jpg)
Types of Performance Level Descriptors
8
• Policy PLDs
• Range PLDs
• Threshold PLDs
• Reporting PLDs
![Page 9: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/9.jpg)
9
Types of Performance Level Descriptors
Policy PLDs
• High-level descriptors that are the same for all subjects and grade levels
• Committee developed a draft December 14, 2015
• Finalized by January 31, 2016
Range PLDs
• Grade-level, content specific descriptors which link back to the standards
• Developed by a committee in March to be finalized by April 30, 2016
Threshold PLDs
• Occurs during standards setting in summer 2016
• Specifically defines what it takes for a student to attain each achievement level
• Applications can be submitted until 5 pm April 25, 2016
Reporting PLDs
• Puts Range PLDs in public friendly language for reports provided to TN parents
• Created by content experts
• Finalized by June 30, 2016
![Page 10: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/10.jpg)
Policy Performance
Level Descriptors (PLDs)
![Page 11: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/11.jpg)
15 total participants
• Central Office Staff: 2
• Principals: 3
• Teachers: 2
• Parent: 1
• Higher Education: 3
• State Board Staff: 1
• CORE Office Consultant: 1
• TDOE Staff: 2
Policy PLD-Committee
11
![Page 12: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/12.jpg)
The Tennessee Academic Standards require that students
demonstrate particular skills and knowledge in each
grade level and course area, with the purpose of preparing
students for postsecondary success. Based on an aligned
assessment that reflects current results, student
performance is reported along a continuum of four levels.
Policy PLDs: Final Version
12
![Page 13: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/13.jpg)
Levels:
4 - Performance at this level demonstrates that the student
has an extensive understanding and expert ability to
apply the grade/course level knowledge and skills
defined by the Tennessee academic standards.
3 - Performance at this level demonstrates that the student
has a comprehensive understanding and thorough
ability to apply the grade/course level knowledge and
skills defined by the Tennessee academic standards.
Policy PLDs: Final Version
13
![Page 14: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/14.jpg)
Levels:
2 - Performance at this level demonstrates that the student
is approaching understanding and has a partial ability
to apply the grade/course level knowledge and skills
defined by the Tennessee academic standards.
1 - Performance at this level demonstrates that the student
has a minimal understanding and has a nominal
ability to apply the grade/course level knowledge and
skills defined by the Tennessee academic standards.
Policy PLDs: Final Version
14
![Page 15: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/15.jpg)
Range Performance
Level Descriptors (PLDs)
![Page 16: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/16.jpg)
• Demonstrate how the skill described in a standard
changes and becomes more sophisticated across
performance levels
• Define the increasing content, reasoning, and
contextual evidence needed to categorize students into
a particular performance level.
Range PLDs
16
![Page 17: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/17.jpg)
54 committee members total with six educators per
committee representing the following subject areas and
grade bands:
• ELA 3-5
• ELA 6-8
• ELA High School
• Math 3-5
• Math 6-8
• Math High School
• SS 3-5
• SS 6-7
• SS 8 and US History
Range PLDs: Committee
17
Educator Committee
Members:
Classroom Teachers
Instructional Coaches
Supervisors of Instruction
![Page 18: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/18.jpg)
• March 16-18: Draft Range PLDs created
• April: Drafts revised for consistency of language and
stakeholder feedback gathered
• May 1: Range PLDs finalized
Range PLDs: Spring Timeline
18
![Page 19: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/19.jpg)
Threshold Performance
Level Descriptors (PLDs)
![Page 20: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/20.jpg)
• Set the stage for the standards setting process
• Developed by the standards setting committee to guide
the process of cut score determination
• Specifically define what it takes for a student to attain
each achievement level for each subject and
grade/course
Threshold PLDs
20
![Page 21: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/21.jpg)
Standards Setting
![Page 22: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/22.jpg)
Committee members will be selected from multiple
stakeholder groups (educators, higher education partners,
parents, community members)
Standards Setting Committee
22
ELA Mathematics Social Studies
Grades 3-5 Grades 3-5 Grades 3-5
Grades 6-8 Grades 6-8 Grades 6-7
High School High School: Alg. 1 and 2
High School: Geometry
High School: Integrated
Math 1, 2, and 3
Grade 8 and US History
![Page 23: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/23.jpg)
• Knowledgeable in tested content
• Familiar with tested students
• Diverse backgrounds, demographics, and experience
Gender
Years of experience
Region
Ethnicity
Student population
Standards Setting Committee
23
![Page 24: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/24.jpg)
• The minimum scale score a student must earn to be
considered at a certain performance level
• Three cut scores result in four levels of performance.
Standards Setting: What is a Cut Score?
24
Cut
Score
A
Cut
Score
B
Cut
Score
C
![Page 25: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/25.jpg)
Standards Setting: Ordered Item Booklets (OIB)
• The difficulties of these
items are based on actual
student performance.
• The actual responses of
students who have taken
the tests are used to
compute the item
difficulty
• Then these items are put
in order from least
difficult to most difficult.
25
![Page 26: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/26.jpg)
Standards Setting: Bookmarking Process
• Based on the threshold PLD,
think of target student for
each performance level, from
lowest to highest.
• The committee will go through
the OIB and consider, item
by item, whether the target
student would be able to
answer the item correctly.
• If not, the cut score marker is
placed before that item.
26
Level 1
Level 2
Level 3
Level 4
![Page 27: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/27.jpg)
Questions?
Feedback?
![Page 28: Assessment Standards Setting Process - Tennessee · 2017. 12. 5. · Standards Setting: Purpose • To make recommendations to the Tennessee State Board of Education regarding cut](https://reader034.fdocuments.us/reader034/viewer/2022051906/5ff87637698fff69111c4757/html5/thumbnails/28.jpg)
Districts and schools in Tennessee will
exemplify excellence and equity such that all
students are equipped with the knowledge and
skills to successfully embark on their chosen
path in life.
Excellence | Optimism | Judgment | Courage | Teamwork