Educator Effectiveness: Instruction and Assessment

Linda Barker,Director of Teaching and Learning

1Goal for today..Provide an overview of how quality standards, instruction, assessment fit into a system for improving practices of educators and student learning results for students.

An invitation

Requirements: SB 191State law requirements: Conduct performance evaluations for all teachers and principals at least once each school year.

Base at least half of each teachers and principals evaluation on multiple measures of students academic growth (CSAP, as appropriate, plus other growth data).

Requires all teachers and principals in one of the performance standards, highly effective, effective, partially effective, or ineffective.

Requirements contState law requirements: Award non-probationary status to teachers with three consecutive years of effective performance and remove it for those who are not effective for two consecutive years.

Consider factors such as student mobility and the numbers of students with disabilities or at risk of failing school.

Require mutual consent of teachers and principals to teacher assignments.

Factor in teacher effectiveness before seniority when considering district-level layoffs. The WHY.Purpose of EvaluationAccording to the rules for administration of a state system to evaluate the effectiveness of licensed personnel, the basic purposes of this system are:

To ensure that all licensed personnel are evaluated using multiple, fair, transparent, timely, rigorous and valid methods, 50 percent of which is determined by the academic growth of their students and 50% on practice

To ensure that all licensed personnel receive adequate feedback and professional development support to provide them a meaningful opportunity to improve their effectiveness

To ensure that all licensed personnel are provided the means to share effective practices with other educators throughout the state

Key Priorities of the System:Data should inform decisions, but human judgment will always be an essential component.

The implementation and assessment of the evaluation system must embody continuous improvement.

The purpose of the system is to provide meaningful and credible feedback that improves performance.

The development and implementation of educator evaluation systems must continue to involve all stakeholders in a collaborative process.

Educator evaluations must take place within a larger system that is aligned and supportive.

1. Human judgment Data should inform decisions, but human judgment will always be a part of the process Processes and techniques are recommended to improve individual judgment and minimize errors and bias

2. Embodiment of continuous improvement by monitoringPilot and rollout intended to capture what works and what doesntChanges in assessment practices and toolsEmerging research and best practices3. Providing credible and meaningful feedback with:Actionable informationOpportunities for improvementIdea that this is a process and not an event

4. Involves all stakeholders in a collaborative processFamilies, teachers, related service providers, administration, school board

5. Takes place within a larger, aligned and supportive systemAll components of the system must serve to increase the number of educators and students who are successful

81. Evaluation System Goals2. Stakeholder Investment and Communication Plan3. Selecting Measures4. System Structure5. Evaluators6. Data Integrity7. Using Results & Professional Development8. System EvaluationComponents of an Effective Evaluation System

92. Annual Orientation3.Self-Assessment4.Review of Annual Goals and Performance Plan5.Mid-Year Review6.Evaluator Assessment7.End-of-Year Review8.Final Ratings9.Goal-Setting and Performance Planning1.TrainingA Process, not an Event!

Evaluation CyclePrincipal/AssistantPrincipals and TeachersMay 15End of September.End of September.Prior to the beginning of Spring Semester

Train: Prior to the beginning of School.Orient: Within the first week of School.End of MayMid-JuneEnd of JuneReview and finalize compiled results of measures of student learningInclude any new district or school decisions for measuring Student LearningConfirm measures used to determine Student Learning Finalize baseline informationReview available information from measures to determine if students are on trackMay 15When you conduct this annual orientation with your staff, we want to remind you of the evaluation cycle outlined in the state model system. It is critically important that you maintain the fidelity of this cycle during this first year of implementation so well take a few minutes to review the evaluation cycle and include both the professional practices and measures of student learning.

As a reminder, the dates in the center blue circle represent an example of when these steps should occur. You should adjust the dates to meet your particular school calendar.

Step 1: Every evaluator involved in using the Colorado State Model Evaluation System should be trained. This process helps to ensure reliability and validity and makes certain that everyone has the same foundational knowledge to apply to this high stakes decision making process.

Step 2: Each district should provide an orientation on the evaluation system, including all measures to which the educator will be held accountable, no later than the end of the first two weeks of school each year. This will ensure that educators who are new to the system will have the knowledge they need to actively participate in their own evaluations. It will also provide aforum for district staff to review the system and learn of any changes made since the previous year.

Step 3: Each educator should complete a self assessment by the end of the first month of the school year. This step in the process provides the person being evaluated with an opportunity to reflect on personal performance over the course of theprevious year and in the context of the students, teachers and school for the current year.

Step 4: As soon as the educators self assessment has been completed, the evaluator and person being evaluated should review the schools annual goals to ensure alignment with the goals included in the educators performance plan. This step allows the teacherto consider the unique context for that year with respect to the schools culture, student body, community issues andchanges in district initiatives and to adjust professional and school goals.

Step 5: The educator being evaluated and the evaluator should schedule time to review progress toward achieving school and personal goals. As a result of this review, every person being evaluated should have a clear understanding of their potential effectiveness rating based on evidence available to date.

Step 6: Evaluators should review the performance of educators being evaluated throughout the year and record their ratings on the rubric as such information is collected. This is not an end of the year activity, but rather one that is conducted in a consistent and ongoing manner. The evaluator should complete the rubric prior to the end-of-year review.

Step 7: The evaluator and educator being evaluated discuss the educators performance ratings on the rubric and measures of student learning, self-assessment ratings, artifacts and any evidence needed to support either the self-assessment orevaluator ratings.

Step 8: Should the evaluator and the educator being evaluated not agree on the final ratings during the end-of-year review,they should determine what additional evidence is needed in order to arrive at the correct rating. The suggested two weekperiod provides adequate time to collect and summarize the evidence and have a discussion to determine final ratings.

Step 9: Using the element and standard ratings, comments and artifacts discussed during the end-of-year review and theestablishment of final ratings, the educator being evaluated will develop a professional growth plan and new studentlearning targets designed to address any areas in which growth and development are needed, professional development ortraining required and other resources needed to fully implement the professional growth plan.

We recognize that some districts may have additional steps or processes in your evaluation cycle and we encourage you to include those in your orientation so that the evaluation cycle is clear and transparent to all educators. 10Teacher Evaluation Framework

11Definition of Teacher Effectiveness Effective Teachers in the state of Colorado have the knowledge, skills, and commitments needed to provide excellent and equitable learning opportunities and growth for all students. They strive to support growth and development, close achievement gaps and to prepare diverse student populations for postsecondary and workforce success. Effective Teachers facilitate mastery of content and skill development, and employ and adjust evidence-based strategies and approaches for students who are not achieving mastery and students who need acceleration. They also develop in students the skills, interests and abilities necessary to be lifelong learners, as well as for democratic and civic participation. Effective Teachers communicate high expectations to students and their families and utilize diverse strategies to engage them in a mutually supportive teaching and learning environment. Because effective Teachers understand that the work of ensuring meaningful learning opportunities for all students cannot happen in isolation, they engage in collaboration, continuous reflection, on-going learning and leadership within the profession.

Circle 3 phrases that jump out at you.

Step Out: S.B. 191 required that Teacher Effectiveness be defined. The State Council members created this definition and the definition drove what was included in the rubric.

Turn and talk with a partner about what words have been circled. Discuss why they resonant you?

12STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Teachers Definition of Teacher EffectivenessI. Know Content50% Professional Practice Standards50% Student Growth MeasuresWeighting: How Much Does Each Standard Count Towards Overall Performance?Observations of Other Measures Teaching Aligned with CDE GuidelinesState Other Assessments Other Measures Summative for Non-tested Aligned Assessments Areas CDE Guidelines Match of test to teaching assignmentsWeighting:Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance?Performance StandardsIneffectivePartially Effective Effective Highly EffectiveQuality StandardsII. Establish EnvironmentIII. Facilitate LearningIV. Reflect on PracticeV. Demonstrate LeadershipVI. Student GrowthAppeals ProcessIn the rules, the minimum weighting requirement was replaced by each standard shall have a measurable influence.

13Look for the first unchecked professional practice.Move one column back to identify the rating for the element.Understanding the Scoring Business Rule

So lets review the business rule for scoring the rubric:Look for the first unchecked professional practice.Move one column back to identify the rating for the element.The rating for each element is the lowest rating for which all professional practices are marked. As illustrated, the teacher would be rated as Proficient for Standard 1, Element a.

14

BEFOREAFTERIn this example before there were more practices in the first column and the language is made clearer and more concise in each of the professional practices highlighted in yellow9/24/201315Redundancies have been eliminated.

Revised Teacher Rubric16

Example: Data is now represented in Std. IV, Element a.The next revision to the rubrics well highlight is the elimination of redundancies. In this example, youll note that data is now represented in Std. 4, Element a where before you might find data under several different elements.9/24/201316Rubric Structure and Rating Level Focus

The focus of the Basic rating level is the educator whose performance does not meet state performance standards and who is not achieving at expected levels.The focus of Partially Proficient and Proficient levels is what educators do on a day-to-day basis to achieve state performance standards and assure that students are achieving at expected levels. The focus of Accomplished and Exemplary ratings shifts to the outcomes of the educators practices, including expectations for staff, students, parents and community members, as a result of practices exhibited under rating levels 2 and 3. NOWNot Evident and the stem was removed and the term BASIC was used to capture foundational professional practices that every teacher should have to be in front of students. That now means for every rating level, you check if the professional practice is a part of the teachers body of evidence and the scoring rule remains the same for every level.In other words, Basic items are scored exactly as all of the other rating levels. This eliminates the double negatives and dual scoring rules. 17Teacher EvaluationsI. Mastery of contentII. Establish learning environmentIII. Facilitate learningIV. Reflect on practiceV. Demonstrate leadershipMeasured using multiple measures on multiple occasions, including: (1) observations; and (2) at least one of the following: student perception measures, where appropriate and feasible, peer feedback, feedback from parents or guardians, or review of teacher lesson plans or student work samples. May include additional measures.This slide can be used to mention the other 50% of the evaluation process. You can add your own thinking about what that 50% will look like once your district has made some decisions around it.Here is another way to show the framework for the State Model Evaluation System. The Quality Standard that accounts for student academic growth accounts for 50 percent of the total rating. The professional practice Quality Standards account for the other 50 percent.

Principals are evaluated on:Teacher inputTeacher evaluation ratingsTeacher improvement for Professional Practice StandardsAND ON SPF dataAt least one other measure of student academic growth

18Its All About the Practice

CDE tool for Observable PracticesThe TeacherStudents:Element a:Uses lesson plans that reflect:Daily review and revision.Instructional objectives appropriate for students. Explicit connections to specific learning objectives and approved curriculumImplements lesson plans based onStudent needsColorado Academic Standards.Districts plan of instruction.Stated learning objectives. Collaborates with other school staff to vertically and horizontally align, articulate, and deliver the approved curriculum.

Interact with the rigorous and challenging content.Perform at a level consistent with or above expectations.Discuss strengths and next steps regarding their learning with their teachers.

Teacher Self-Assessment Ratings Distribution These slides are the aggregration of the pilots self-assessments. Have them identify which standards had the biggest bar span of proficient and partially proficient. Is that a surprise that is was instruction. Why or why not? Lets dig deeper into the practices to see what the teachers stated as their need under instruction.21

Teacher Self-Assessment Ratings-Standard 3-plan and deliver effective instruction.Looking at the practices under Instruction, which practices show up as proficient and partially proficient. What does this data tell us?22Theory into Practiceare rubrics real?Portscheller, Schechty, Marzano

Power of Non-Fiction WritingStudent Learning Outcomes

Heres a simple truth that is easy to forget. School is not about grading. School is about

Learning24STATE COUNCIL FOR EDUCATOR EFFECTIVENESSFramework for System to Evaluate Teachers Definition of Teacher EffectivenessI. Know Content50% Professional Practice Standards50% Student Growth MeasuresWeighting: How Much Does Each Standard Count Towards Overall Performance?Observations of Other Measures Teaching Aligned with CDE GuidelinesState Other Assessments Other Measures Summative for Non-tested Aligned Assessments Areas CDE Guidelines Match of test to teaching assignmentsWeighting:Scoring Framework: How Do Measures of Quality Standards Result in a Determination of Individual Performance?Performance StandardsIneffectivePartially EffectiveEffectiveHighly EffectiveQuality StandardsII. Establish EnvironmentIII. Facilitate LearningIV. Reflect on PracticeV. Demonstrate LeadershipVI. Student GrowthAppeals ProcessUnder the State Model Evaluation System, teachers and principals will be evaluated based on Quality Standards. Note that in the teacher evaluation framework, there is an appeals process.

Teachers will be rated on six Quality Standards:Half of the rating will be based on the five Quality Standards that measure professional practice: content knowledge, establish environment, facilitate learning, reflect on practice and demonstrate leadership.

- The sixth Quality Standard, the student growth measures standard, will account for the other half of their rating. The student growth measures standard will be based on multiple measures of growth including several different assessments. To support this part of the evaluation, the Colorado Content Collaboratives, a group of P-12 educators from around the state, are identifying and creating a resource bank of high-quality assessments in 10 content areas. The assessments are aligned to the new Colorado Academic Standards and, while not required, provide an optional resource for measuring student growth.

Each district will have some flexibility in deciding how much each of the five professional practice standards is weighted. Each district will have some flexibility in deciding how much each of the six professional practice standards is weighted with a measurable influence for each standard.

25Teacher EvaluationsVI. Responsibility for student academic growthEvaluated using the following:(1) a measure of individually-attributed growth, (2) a measure of collectively-attributed growth; (3) when available, statewide summative assessment results; and (4) for subjects with statewide summative assessment results available in two consecutive grades, results from the Colorado Growth Model. Refers to outcomes on a measure that are attributed to an individual licensed person,e.g. DRA2 growth measures for a 1st Grade Teachers studentsRefers to outcomes on a measure are attributed to two or more licensed personnel, e.g. 10th gr. Math TCAP All Secondary math teachers in schoolMention a secondary and elementary example for each.

Make sure: These rules come directly from SB191 Individual The teacher is directly responsibleCollective/Attributed team / school / co-teacherCGM available for 4-10 r/w/m Collectively characterized as shared among grade level, dept., school or provider (Sped, Intervention)

26Its All About the Questions?

How do measure student learning outcomes in your district/school?Student Learning Outcomes

Have on chart and record thinking. This can be used later in creating templates/pie charts.9/24/201328Do you understand what % your district (1338) has assigned for collective or individual attribution in your evaluation system. Collective vs. Individual Attribution

(Know ahead of time where this district is in terms of their balance of collective and individual attribution)

From the guidance doc:an overly high percentage of collective attribution will decrease the ability at the school or district levelto recognize high-performing teachers (who may be held back by the average) and to identifystruggling teachers (who may be propped up by the average). Therefore, it is imperative that districts understand the importance of finding the right balance between collective and individual attribution.9/24/201329Example: Categorizing TeachersNo Summative/CGMAll Other Teachers

70%

Colorado Growth Model (CGM)Reading, Writing, Math (Gr. 4-10)

2.

State SummativeReading, Writing, Math (Gr. 3 & 1st Yr. Gr. 4-10) Science (Gr. 5, 8, 10) Social Studies (Gr. 4, 7, HS)ACT (HS)WIDA ACCESS (ELL)

What are the pros and cons of creating groups of like teachers based on the types of measures that must be included in their body of evidence?

Based on what is expected in law, what categories of teachers are possible?9/24/201330Select and weight multiple measures of student learning to be included in educator evaluations.

What can we do as a district to build comparable bodies of evidence in like groups of teachers?

Questions to consider :What assessments must we include?How much influence will they have on the evaluation?

Student Learning Outcomes Tool(SLOT) Student Learning Outcomes

Use one of the categories of teachers they talked about and run through how you might work with it in the SLOT.

9/24/201331Districts, BOCES and schools may decide after completing the Assessment Inventory that a locally created assessment would likely be included as a measure in evaluation. Which ones?

Districts and BOCES can use the Assessment Review Tool to show they are taking steps to ensure that the locally created measurements they select are fair, valid, and reliable.Using Local Assessments in Evaluation

Helps by validating assessments9/24/201332Decision Points How will our district identify teacher roles and the types of assessments that exist?Who will participate in the process? What do we need to know in order to make good decisions?What resources (internal and external) do we have/need?How will we know if we have made good decisions?Categorizing, Weighting, and Selecting Measures Reflection

9/24/201333What are Student Learning Targets?

Method of measuring student growth, progress or mastery of the standards Long-term academic goal set by educators for studentsBased on available prior student learning data and information Set at the beginning of the year/course by teachers in collaboration with their supervisor or evaluator Can be shared by teachers of the same course/grade level or individually

Setting Student Learning Targets9/24/201334Student Learning Target Infrastructure NWEA, targets, %, grades, SLOs, District & School Performance FrameworksThrough the Colorado Educational Accountability Act of 2009 (SB09-163)CDE annually evaluates districts and schools based on student performance outcomes.All districts receive a District Performance Framework (DPF). This determines their accreditation rating.All schools receive a School Performance Framework (SPF). This determines their school plan types.Provide a common framework through which to understand performance and focus improvement efforts.

Accreditation & Plan TypesAccreditation designations:Accredited with Distinction (10%)Accredited (50%)Accredited with Improvement Plan (25%)Accredited with Priority Improvement Plan (10%)Accredited with Turnaround Plan (5%)School plan types:Performance Plan (60%)Improvement Plan (25%)Priority Improvement Plan (10%)Turnaround Plan (5%)

Whats my school rating?CDE Summer Symposium - June 2012 - Accountability & Improvement39

Unified Improvement PlanningThrough the Colorado Educational Accountability Act of 2009 (SB09-163)All schools and districts must annually develop and submit an improvement plan. The plan streamlines federal and state accountability systems and reporting requirements.Plans must include: Data Analysis: trends, performance challenges, root causesAction Planning: targets, interim measures, major improvement strategies, action steps, implementation benchmarksAligned with ESEA (Titles I, IIA, III), State Graduation Completion Plans and various grant requirements.

CDE Summer Symposium - June 2012 - Accountability & Improvement43Improvement Planning ProcessGather and Organize DataReview Current PerformanceDescribe Significant TrendsPrioritize Performance ChallengesIdentify Root CausesSetPerformanceTargetsIdentify Interim MeasuresIdentify Major Improvement StrategiesIdentify Implementation BenchmarksSection III: Data Analysis and Data NarrativeSection IV: Target SettingSection IV: Action PlanningOngoing: Progress MonitoringPreparing to PlanThis chart reflects the UIP process. Its comprised of a few key components:Gather and organize data: Collecting and presenting dataData analysis & data narrative: Describing trends, prioritizing performance challenges, determining root causesTarget setting: Setting targets and interim measures for where you want to be in your challenge areasAction planning: Identifying major improvement strategies to respond to the root causes and set implementation benchmarks as milestonesProgress monitoring: Check your progress towards your targets multiple times a year to make adjustments where necessaryCDE Summer Symposium - June 2012 - Accountability & Improvement44Dr. Harvey Silvers work on connections9 yearsformative assessment leads to dramatic gains in achievement, one of the most powerful tools a teacher can use.Assessment and Instruction/Practice

What assessments of student learning do you currently use in your classroom, caseload practice or school? Standardized TestsTextbook-based TestsTeacher-developed Unit Tests: e.g. quadratic equations, Catcher in the Rye, photosynthesis.Mid-Year & FinalsCommon across subject (all students in the grade/subject take the same test)Common across schools (all students in the grade/subject across all schools take the same test)Performance Assessment: lab experiment, speaking a second language, research project, serving a volleyball, solving a math problem with written explanation.Writing Tasks: compositions, essays, reportsPortfolios: collection of student work over timeExhibitions: oral report, art show, musical performance, DECA competition, science fair, capstone projects.Use of assessmentdiagnosticinterim/benchmarksummative Non-cognitiveValue JudgmentDo we value the assessment(s)? Why?

Assessment Inventory

for each type of teacher at your table46Think about your assessments/purpose/audience:Feedback/Instruction FOR learningData/Trends (after T and L).OF learningExit ticketsSponge padsWork foldersWriting logsDiscussion recordsJournalsWriting samplesTCAPSATACTPrevious years gradesDistrict testsFinal examsChapter quizzes and tests

All grades, all levels..Your job is to uncover understanding, not cover content.

Assessments

Science and Social Studies Summative Assessments

Colorado is piloting new science and social studies assessments. These assessments will be computer-based and will be designed to measure the CAS. These assessments will be administered in 2014.

The science assessments will be administered in grades 5, 8, and 12and

social studies assessments will be administered in grades 4, 7, and 12th grade. Practice testing environments, also referred to as ePATs (electronic Practice Assessment Tools), that help to familiarize students with the testing environment can be accessed at:http://www.pearsonaccess.com. Partnership for Assessment of Readiness of College and Careers PARCCs next-generation assessment system will provide students, educators, and the public with the tools needed to identify whether students from grade 3 through 11th grade in math and language arts/literacy are on track for postsecondary success.PARCC in 2014-15Summative assessments aligned to College and Career Readiness StandardsComputer adaptiveMight not be a set test windowResults will show achievement and growthQuestion types:1. Selected Response2. Constructed Response 3. Extended Response 4. Performance Based Tasks 5. Technology-Enhanced 6. Technology-Enabled

How will the new assessments be different from CSAP/TCAP?

52Drag the words from the word box into the correct locations on the graphic to show the life cycle of a butterfly as described in How Animals Live.Words:

Grade 3 Technology-Enhanced Constructed-Response Item

Grade 7 Analytical Constructed ResponseYou have read three texts describing Amelia Earhart. All three include the claim that Earhart was a brave, courageous person. The three texts are:

Biography of Amelia Earhart Earhart's Final Resting Place Believed Found Amelia Earharts Life and Disappearance

Consider the argument each author uses to demonstrate Earharts bravery.

Write an essay that analyzes the strength of the arguments about Earharts bravery in at least two of the texts. Remember to use textual evidence to support your ideas.

CCSS-use of evidence, summary of text, complex texts, writing to explain/inform, writing coherently, drawing evidence from texts, grammar and conventions, comparisons of authors, relevant info from multiple sources.

AssessmentsHow do these questions differ from the questions we currently ask on TCAP?

Interdisciplinary, online, not one choice, multiple possible answers, defend and justify their answers, mental manipulation, inference, transfer.not your mom/dad or even older brother test!

What are the implications for classroom instruction?Still

Rubrics are currently being piloted for the following Specialized Service Professional groups:

SSP Pilot Year 2013-2014 Pilot sites will pilot the evaluation system for those SSP groups they have selected SSP Implementation and Validation Study Year 2014-2015 All districts will evaluate their SSPs under SB 191 requirementsCDE will conduct a validation study on the evaluation rubrics Specialized Service Professionals57AudiologistsCounselors NursesOccupational TherapistsOrientation and Mobility SpecialistsPhysical TherapistsSchool PsychologistsSchool Social WorkersSpeech Language Pathologists

The first tools that will be available for you focus on our Specialized Service Professionals. 9/24/201357Districts decide58Timeline of Implementation Review the timeline for implementation. Currennly we are in Year Three or the 2013-14 pilot and rollout year. During this year, CDE will provide statewide technical assistance on the rollout of the teacher and principal evaluation systems, support all districts with implementation through resources, trainings and tools and finally, during Year three, we will validate both the teacher and principal rubrics.

During year 4, or the 2014-2015 full statewide implementation year will continue implementation efforts while analyzing data, making needed adjustments and providing recommendations to the State Board of Education in order to continuously improve the evaluation system. 59Questions?Resources?http://www.cde.state.co.us/educatoreffectiveness/statemodelevaluationsystemTools, webinars, fact sheets, FAQs, videosUser guide-all the answers!

http://www.cde.state.co.us/educatoreffectiveness/coloradolegacyfoundation