THE GOOD, BAD, AND UGLY OF MULTIPLE MEASURES FOR …ccsso.confex.com/ccsso/2017/webprogram...THE...
Transcript of THE GOOD, BAD, AND UGLY OF MULTIPLE MEASURES FOR …ccsso.confex.com/ccsso/2017/webprogram...THE...
THE GOOD, BAD, AND UGLY OF MULTIPLE MEASURES FOR SYSTEM IMPROVEMENT AND ACCOUNTABILITY: WHAT THREE PIONEERING STATES ARE LEARNING
David T Conley, University of Oregon/EdImagine
Keric Ashley, California Department of Education
David Cook, Kentucky Department of Education
Paul Leather, New Hampshire Department of Education
Overview of the Session
■ Introduction (5 mins.)– David Conley
■ Each State’s Context (35 mins.)– California, Keric Ashley– Kentucky, David Cook– New Hampshire, Paul Leather
■ Rapid Fire Round (35 mins.)– Respondents
■ Conclusion (5 mins)– David Conley
■ Questions (10 mins.)
2
California- Keric Ashley
The California WayState Driven, not Federally DrivenThe California Way rests on the belief that educators want to excel, trusts them to improve when given the proper supports, and provides local schools and districts the flexibility to deploy resources so they can improve.
3
Accountability Framework
■ Performance■ Equity■ Improvement■ Transparency■ One system to meet local, state,
and federal needs
4
Local Control Funding Formula (LCFF) & the Local Control Accountability Plan (LCAP)
■ In 2013, Governor Brown signed the LCFF into law, along with a new local accountability system to:
– provide resources more equitably to students with learning and socio-economic barriers, and
– provide greater flexibility for educators to serve and respond to the student’s needs as addressed in their LCAP.
5
Three Statutory Purposes for the California School Dashboard§ To support LEAs in identifying strengths,
weaknesses, and areas for improvement;
§ To assist in determining whether LEAs are eligible for technical assistance; and
§ To assist the Superintendent of Public Instruction in determining whether LEAs are eligible for more intensive state support/intervention.
6
State Indicators
§ Assessment - ELA and math § English Learner Progress § Chronic Absenteeism § Graduation Rates § Suspension Rate § College/Career Readiness (multiple
measures)
7
Blue HighestGreenYellowOrangeRed Lowest
State Performance Levels
Schools and districts receive one of five color-coded performance levels on the state indicators. Each color represents a combination of status and growth combined.
8
Reference ChartsPerformance levels are calculated using percentiles to create a 5 by 5 reference chart that combine Status and Change.
Change
Status
9
Local Indicators
Data is not collected at the state level - LEAs will measure and report (met/not met) on their progress through the Dashboard based on locally collected data:– Basic Services – Implementation of State Academic Standards – Parent Engagement – School Climate
v Local indicators apply to districts, not individual schools.
10
11
Linking the LCAP and the Dashboard
LEAs are required to use data from the Dashboard to inform the development of the: – goals, actions and services – Plan Summary that includes greatest
progress, greatest needs, and performance gaps
– Annual Update
12
Kentucky- David CookThe Good: 6,000 people provided input on what multiple measures should be
The Bad: 6,000 people provided input on what multiple measures should be
The Ugly: 6,000 people provided input on what multiple measures should be
Kentucky’s new accountability system places emphasis on several important concepts that considered together promote a strong educational experience for all of Kentucky’s students, including:
• readiness for the next step in education or life with the indicators of proficiency and transition readiness;
• growth, with incentives for attending to the growth of students at the lower levels of performance;
• reduction of achievement gaps specifically;
• support to schools with very low-performing student groups; and
• access and opportunity of students to experience rich curriculum, equitable access, and support for the whole child more broadly than just through tests and tested content areas.
• These access and opportunity measures are tied strongly to equity because they help ensure that all students have robust opportunities and access to the precursors (“leading indicators”) of growth and high achievement.
13
Overall Rating- Kentucky
Star Rating Supplemental Labels
(Five Star) Gap Closure: Supplemental designation for closing the differences in achievement between students in historically lower-performing groups with the school
(Four Star)
(Three Star) Gap Issue: Supplemental designation for schools with very large achievement gaps and low-performing schools(Two Star)
(One Star)
14
Innovations in New Accountability System- KentuckyLocal Measure (District and Charter Only)
■ Locally-defined measure that highlights an area for improvement or objective of a charter school
■ Each district shall propose to the Kentucky Department of Education the targeted multi-year goal or objective that is SMART (specific, measurable, achievable, relevant, and time bound)
■ Each public charter school will include a measurable objective related to the charter contract that will be included in the charter school’s rating
15
Innovations in New Accountability System: Competency Pilot- Kentucky
■ Based on state standards, locally organized■ Connects curriculum, instruction and assessment ■ Evidence of mastery gained from a variety of sources ■ Upon demonstration of technical quality, evidence may be
used in lieu of state assessment results in accountability
16
New Hampshire- Paul Leather
■ PACE is a learning system designed to capitalize on the latest advances in understanding how people learn.
■ The goal is to structure learning opportunities that allow students to grapple with gaining meaningful knowledge and skills at a depth of understanding that can they can transfer to new real-world situations.
Current Status:
■ 3 years of waiver implementation, and ESSA planning to result in Sec. 1204 Application & statewide scaling.
■ Our goal is to scale PACE across our state of New Hampshire and to inspire other states to accept a similar challenge.
NH PACE -- Performance Assessment of Competency Education
17
PACE Theory of Action
Local Educators help design and implement accountability system
Reciprocal support for local districts including technical, policy, and practical guidance
Use of Competency based instruction, learning, and assessment
Development/use of curriculum embedded high quality performance assessments
Fosters transformational change through internally driven motivation
Builds local capacity of teachers & leaders
Restructures rigor and content representation of curriculum, instruction, and assessment
Provides specific feedback to students, parents, and teachers on student progress towards proficiency
Creates positive changes to the
instructional core of
classroom practice
Students are Career
and College Ready
18
Combining Multiple Measures
PACEComparable
Annual Determinations
PACE Common Performance Task
District-Level Competency
Scores
Competency 1Local performance
assessments
Competency 2Local performance
assessments
Competency 3Local performance
assessments
Competency 4Local performance
assessments
State Assessments in select grades
19
DISTRICT
ASSESSMENT
PLAN
Note Weight of Local Assessments!
A Particular Challenge – “Body of Work”
CLASSROOM COMPETENCY GRADING[All courses and disciplines]
DISTRICT STATE
Grade Level
COURSE COMPETENCY COMMON ASSESSMENTS COMPETENCY ASSESSMENT
COMPETENCY COMMON ASSESSMENT
Assessment Type 1. NUMBERS & QUANTITIES, 2. ALGEBRA, 3. FUNCTIONS, 4. GEOMETRY, 5. STATISTICS & PROBABILITY
4Unit Summative
• Place value, rounding, addition, subtraction
• Measurement conversions, addition, subtraction
• Fractions with like denominators
• Multiplication/ division facts,
• Multi-digit multiplication, division (multi-digit)
• Geometry
• Fractions with unlike denominators
• Decimal fractions
• Geometry & symmetry
NWEA (MAP) 212.5
SMARTER BALANCE
Performance TaskTri 1: Mapping Migrating Monarchs
5Unit Summative
• Place Value
• Multiplication
• Division
• Fraction Review
• Addition/Subtraction
• Multiplication of Fractions
• Division of Fractions
• Area
• Volume and Capacity
• Algebraic Expressions
• Data and Analysis
• Geometry
NWEA (MAP) 221.0
PACE: Algebra Quantities, Creating Equations
Performance Task Tri 1: Summer Olympics
6 Unit Summative• Ratios , Rates and Measurement Conversions, Order of Operations,
Exponents and Algebraic ExpressionsNWEA (MAP) 225.6 PACE: Algebra, Equalities
and Equations
HumRRO Key Evaluation Findings ■ Integrated competency-based education system with performance events are a
dramatic shift from traditional schooling
■ Substantial progress toward this goal
■ Buy-in– Educators determine what is assessed, how it is assessed, and how the tasks are scored
– Assessments integrated into instruction– Replaces Smarter Balanced assessments and requires deep knowledge
■ Collaboration– Cross-district activities (task development sessions, professional development, scoring
sessions, standard-setting, etc.)
■ Teaching & Learning– Increased depth of knowledge– “Testing to what is taught” vs. “teaching to the test”
– Learning while testing
■ Context– Previous experience with competency-based education– District size
– Substantial work for teachers 21
PACE Resource
Provides detailed information on the technical aspects of the PACE assessments including the validity evidence for within-district and cross-district comparisons of performance assessment scores.
Rapid Fire RoundIssue Respondent(s)
1. Technical problems including differential reliability across measures, combining unlike measures, scaling, cut scores, and performance levels
Paul Leather
2. Political realities including maintaining consistency across changing administrations, and different philosophies of privacy and the right of government to collect information on citizens
All
3. Operational challenges of data collection including collecting data across educational units, using common definitions and specifications for data categories, and local capacity to collect data
Paul Leather
4. Meeting federal requirements including the inconsistency of ESSA requirements for multiple measures and federal expectations of a single measure of accountability
Keric Ashley
5. Potential and actual benefits including the expectations and perspectives of educators David Cook
6. Parental considerations including reporting complex data to parents and helping them interpret school performance results
Keric Ashley
7. Equity dimensions including addressing the special and sometimes unique needs of severely underperforming schools in relation to all schools
David Cook
Conclusion: Multiple measures
■ No one measure can capture the complexity of student learning or of school performance
■ While multiple measures have their drawbacks, they represent the future of state accountability systems
■ States will need to experiment with various approaches before a set of best practices emerges
■ In the meantime, robust, valid and reliable measures exist to meet ESSA requirements and to go beyond those requirements
24
Questions
■ To contact the presenters:– California: Keric Ashley: [email protected]– Kentucky: David Cook [email protected]– New Hampshire: Paul Leather [email protected]– David Conley: [email protected]
25
For a copy of this presentation, email:[email protected]
For more information, visit edimagine.com
@drdavidtconley26