· Web viewReview Handout #1 to link the conceptual framework of a three phase process for ......

STUDENT LEARNING OBJECTIVES (SLOs)Annotated Notes

M2-Building

Slide No. NotesSlide 1 The Student Learning Objective (SLO) Process is comprised of three (3) phases: Design,

Build, and Review. Student Learning Objectives provide indicator of teacher effectiveness through student performance outcomes based on standards.

Welcome to Training Module 2: “Building”

Concept

Within the Building Module, the trainer will guide participants to create Student Learning Objectives and performance measures that will guide instruction and provide evidence of student mastery or growth. During this phase, the participants will complete the development of SLOs and the Performance Measures associated with each goal.

Key Points for Trainers

1. Orient the user to where in the process the “building” activities take place.a. After teachers have completed the Design Phase (which includes the

development of a goal statement, identifying underlying content standards, and creating a blueprint) they will move from this thinking/designing stage to actually “building” the Performance Measures needed to support the SLO process.

2. Articulate that using the SLO Process Template requires an initial step of i) reviewing the data definitions, ii) referencing SLO models, and iii) examining the structure of the SLO Template.

a. During this module, participants should review the following:1) SLO Process Template-referenced in Slide 6 (template found in

SLO/Build/Templates).2) The Art and Physical Education Models-referenced in Slide 7 (Note:

Additional sample models are available).3) The following definitions:

i. PDE’s SLO Definition: A process to document a measure of educator effectiveness based on student achievement of content standards. (Orientation Module)

ii. Goal Statement: Narrative articulating the “Big Idea” upon which the SLO is based. (Orientation Module)

iii. Standards: Targeted content standards used in developing SLOs, which are the foundation of performance measures. (Orientation Module)

iv. Assessment Literacy: The skills, knowledge, and concepts associated with sound assessment practices, including the critical

© Pennsylvania Department of EducationPDE SLO Annotated Notes-M2 Building – May 2014-R 1

Slide No. Notesreview of quality evidence. (Orientation Module)

v. Rationale statement: Narrative providing reasons why the goal statement and the aligned standards address important learning. (Orientation Module)

vi. Blueprint: Visual depiction of the relationship among key SLO components. (Orientation Module)

vii. Performance Measure: As assessment tool used to measure the knowledge and skills acquired by students. (Orientation Module)

viii. Performance Indicator: Statement of the expected level of achievement on each performance measure. (Orientation Module)

3. Clarify completion of SLO Process Template, including the development and/or selection of performance measures aligned to the selected content standards.

Learning Activity________________________________________________________________________

Technical Notes“Structure”

1. Concept – “What is this slide telling the audience?”

2. Key Points – “What/Where are the details ‘needed for teaching’?”

3. Learning Activity – “How can the participant’s learning be enhanced?” (This item will not be populated for every slide.)

Slide 2 Goal & Objectives

Concept

The Goal of the Build Module is for teachers to create SLOs and identify and/or develop Performance Measures for each Performance Indicator that can be used in guiding instruction and determining student mastery and/or growth as part of Pennsylvania’s Educator Effectiveness System. The SLO Process is the mechanism which the Elective Data portion in the PA evaluation system is developed.


1. Explain the relationship between the SLO Process Template and the “Help Desk” definitions.

a. The “Help Desk” defines each element within the SLO Process Template. When using the online SLO Process Template, the “Help Desk” document also displays the correct format for responses with the template’s elements.

b. The “Help Desk” also provides a response example. Additional examples are found among the numerous SLO models developed by PA teachers.


Slide No. Notes

2. Ensure the audience recognizes that performance measures are developed or selected during the “Build” phase.

a. This is the time where teachers will take their fully developed blueprints and identify and/or develop the required performance measures for each SLO.

Learning Activity

Slide 3 Helpful Tools

Concept

As you work through the Build Module, you will need to access both SLO Models and Templates. These materials will provide focus for the work and help ensure the development of complete SLOs and Performance Measures.

All SLO training materials can be found at both the PDE’s SAS portal and Research in Action’s Homeroom learning platform. Go to http://www.pdesas.org. Once there you can login to the Homeroom learning portal.


1. Clarify which tools can assist in completing the SLO Process template:a. Resources have materials such as handouts, rules of thumb, etc. that

reinforce content presented in the videos. In this module, models are provided to guide new users in what a completed SLO, including developed and/or selected performance measures, looks like.

b. Templates are used to complete each phase of the process. The key template is the SLO Process Template. This template is provided as an online tool. Template #4 is a replication of SLO Process Template, provided as a downloadable Word document. Template #5, the Performance Task Framework, is also provided for performance tasks that may be culminating events over multiple days (e.g., student projects) or other types of teacher and district developed assessments. Additional development templates are found within the Assessment Literacy Series: Quick Start PA

c. During the Build Phase, teachers will review the following templates:i. Template #1 – Goal Statement

ii. Template #2 – Targeted Content Standardsiii. Template #3 – SLO Blueprint

d. During the Build Phase, teachers will complete:i. Template #4 – SLO Process Template (must be completed in this


Slide No. Notesphase); and MAY need,

ii. Template #5 – Performance Task Framework used as a resource when creating performance measures that are culminating events or accumulation of students’ work products. (Recommended for teacher or district developed assessments, but the use of Template #5 is optional.)

Learning Activity

1. Have participants go to www.pdesas.org to review the resources that are available on the SAS portal and the Homeroom learning portal.

Slide 4 Outline of the Build Module

Concept

This figure is designed to help visualize the various components associated with the “building” phase of the SLO Process.


1. Apply this “learning map” into the beginning of the presentation to provide an outline of both the “building” and SLO process, including some techniques (steps) to begin the work, including the presentation, refinement, and review of created SLOs.

2. SLO Process Componentsa. BUILD: This component is the “action” step in the process that focuses on

completing the SLO Process Template and creating and/or selecting performance measures.

b. Preview the SLO Process Template (Template #4) and “Help Desk” Definitions. These will provide guidance around the classroom context, SLO goal, Performance Measures, Performance Indicators, and Elective Rating.

i. Performance Measures – An assessment tool used to measure the knowledge and skills acquired by students.

ii. Performance Indicators – Statement of the expected level of achievement on each performance measure.

iii. Elective Rating – As part of the SLO process and the Educator Effectiveness Rating Tool, teachers will be required to set four (4) levels that describe the number of students expected to meet the targets listed for each Performance Indicator. These levels include: Distinguished, Proficient, Needs Improvement, and Failing. Educators determine the percentages within each rating prior to the beginning of instruction. These performance ratings will be examined at the end of the evaluation period and be used to determine the Elective Rating that is applied to the overall final rating for each teacher.


Slide No. Notes

Learning Activity

*Template #4 – SLO Process Template

Slide 5 SLO Template Preview

Concept

This section of the Module provides a preview of the materials used in developing SLOs.


1. Make sure the participants have the Templates and applicable SLO Models needed for working through the Building phase of the SLO process. They will need the following:

• Template #4 – SLO Process Template (SLO/Build/Templates)• Template #5 – Performance Task Framework (SLO/Build/Templates)• Help Desk Definitions (SLO/Build/Templates)

Learning Activity

1. Allow time for participants to gather the above mentioned templates, SLO models, and resources from both their previous work during the Design phase and the Homeroom learning portal link found on the SAS Portal.

Slide 6 SLO Template Preview

Concept

To begin the Build phase of the SLO, it is important to review some previously completed documents. Make sure that teachers review the Goal Statement (Template #1), the Targeted Content Standards (Template #2), and their completed SLO Blueprint (Template #3). This information is needed because the SLOs are based upon the targeted content standards chosen during the Design phase of the process.


1. Orient the user to the SLO Process Template (Template #4): The Template’s front page is focused on the demographics, goals, and performance measures; the back page is the performance indicators defining achievement levels on the assessments. The back page of the template also articulates (see Section 5: Elective Rating) the aggregated performance expectation for his/her identified students.

a. Identify what data (demographics) (performance scores) (performance measures) is needed to complete the template.

b. Identify the group of students the SLO will be based upon, including any


Slide No. Notes“focused” target group.

2. Ensure the definitions for each data element (i.e., each data field within the template) are understood with focus on the technical definitions and the examples provided.

The SLO Process Template reflects the entirety of the SLO Process: Design (think about the goals and targeted standards and indicators), Build (identify/build performance measures for each performance indicator selected), Review (refine/edit/improve completed SLOs).

Learning Activity

Slide 7 SLO Process Template

Concept

Model #1-Grade 8 Art and Model #2-Grade 3 Physical Education provide participants with examples (i.e., training aids) of a completed SLO Process Template. These SLO models, included sample Performance Measures were developed by PA educators as demonstrations, NOT exemplars. They provide participants with concrete examples of each phase of the SLO process: Design, Build, and Review.


1. Review Handout #1 to link the conceptual framework of a three phase process for SLOs with the operational structure of the material.

2. Ensure the audience understands that this is a modular design. This means that, given work and experience with SLO, individuals can engage in the work at any phase of the process, not necessarily at the “implied” beginning [Design].

3. Reinforce, via model review, how these models reflect the three C’s of quality:a. Completeness: Template is “completed” according to pre-established

business rules.b. Comprehensiveness: The performance measures are “comprehensive”

assessments of the targeted content standards.c. Coherency: The SLO’s focus on a “Big Idea” within the PA Standard is aligned

with an integrated set of standards, performance measures, and performance indicators.

Learning Activity

1. Using the Art and Physical Education models, have participants review and identify the various sections of the completed SLO. (These sections will be unpacked in upcoming slides. Information to support Section 3, quality performance measures, is fully explained in the Assessment Literacy Series: Quick Start PA.)


Slide No. Notes

Slide 8 Section 1: Classroom Context

Concept

Section 1 of the SLO Process Template contains basic information and provides focus for the work.


1. Make sure that teachers are using the “Help Desk” Definitions as a guide. This document (found in SLO/Build/Templates) provides the format and expectations when filling out each section. It is important that the SLO Process Template is filled out correctly.

2. Emphasize, especially for high school teachers, that the SLO is subject/content specific. This means that the educator teaching multiple subjects, (e.g., Algebra I, Geometry, Integrated Math III) will identify one subject, and then include all students enrolled in that particular subject/course (unless otherwise directed by the LEA).

Learning Activity

1. Using Template #4 – SLO Process Template and the “Help Desk” Definitions, guide participants through filling out Section 1 of the template.

Slide 9 Section 1: Classroom Context

Concept

This slide provides the definitions for each part in Section 1 of the SLO Process Template. Completely filling out this section of the template first will provide context applicable to working on the specific performance measures that will be used for each SLO.


1. Even though this chart is here and clarifies each part of Section 1, please make sure that teachers refer to the “Help Desk” definition document (found in SLO/Build/Templates) as it provides the correct formatting along with examples. It is important that each section is filled out properly.

2. Example (as found in SLO Model 1, Art):a. 1d. Class/Course Title – Full Name(s) - Art


Slide No. Notesb. 1e. Grade Level – Numeric Values/Text – 8c. 1f. Total # of students – Numeric values only – 230d. 1g. Typical Class Size – Numeric values only – 25e. 1h. Class Frequency – (# of sessions) per (week, 6 day cycle) for (year,

semester, 35 day rotation) equaling a total of (#) sessions – daily for one quarter (42 sessions)

f. 1i. Typical Class Duration – Numeric values only – 45 minutesLearning Activity

Slide 10 Section 2: SLO Goal

Concept

Referring to Template #1 and Template #2 from the Design module, teachers will be able to refresh themselves about the Goal Statement, Targeted Content Standards, and the Rationale developed during the Design phase. Using these templates will help teachers complete Section 2 of the SLO Process Template.


1. Remind participants that the Goal Statement is integral to the development of an SLO. It is the narrative that articulates the “Big Idea” upon which the SLO is based. This statement must be aligned with PA standards. Including national or professional standards is acceptable, but not as a replacement for PA standards.

2. Targeted Content Standards are those that have been selected for use with the performance measures that will be developed. These selected standards should represent the “Big Ideas” within the content area. Refer participants to the Curriculum Framework for the Course/Content area for which the SLO is being written.

3. Example from the SAS Portal – Curriculum Framework: Mathematics – 3rd gradea. Big Idea – The likelihood of an event occurring can be described numerically

and used to make predictions.b. Essential Question – How can using graphs help us to solve problems and

describe data we collect?c. Concepts – Graphical displays of data: Frequency tables, bar graphs, picture

graphs, line plotsd. Competencies – Construct and analyze frequency tables, bar graphs, picture

graphs, and line plots and use them to describe data and solve problems.e. Standards/Eligible Content – 2.6.3.A, 2.6.3.B, 2.6.3.C, 2.6.3.D, 2.6.3.E,

2.7.3.D, 2.8.3.F, CC2.4.3.A.4, M3.E.1.1.1, M3.E.1.1.2, M3.E.1.2.1, M3.E.1.2.24. Teachers can cut and paste the language that has already been created by the

Pennsylvania Department of Education when developing goal statements. All the information is easily accessible on the SAS Portal.


Slide No. NotesLearning Activity

Slide 11 Section 2: SLO Goal

Concept

This slide provides the definitions for each part of Section 2 of the SLO Process Template. This section can be filled out using the information from Templates #1 (Goal Statement) and #2 (Targeted Content Standards).


1. Remind teachers that they should use the “Help Desk” Definition document (found in SLO/Build/Templates) when filling each Section out as it provides the correct formatting as well as example statements. Teachers can also refer to the Art and Physical Education Models that were completed by PA educators.

2. Step-by-Step process to select “Big Idea” from SAS Portal:a. Go to www.pdesas.orgb. Select Curriculum Frameworkc. Select Subject Area/Grade Level from the drop-down screen.d. Click Searche. “Big Ideas” are listedf. Choose a “Big Idea”g. Click on a specific “Big Idea”h. Complete framework for the “Big Idea” provides teachers with Essential

Questions, Concepts, Competencies, and Standards/Eligible Content available for that particular “Big Idea/ (Enduring Understanding).”

i. Click on specific standard to drill down to materials, resources, and assessments available to specific content standard.

Learning Activity

1. Have participants work with the curriculum framework element on the SAS Portal. Choose a particular subject area and grade level. Locate the various statements related to the specific subject and grade level. (Big Idea, Essential Question, Concepts, Competencies, and Standards/Eligible Content).


Slide No. NotesSlide 12 Section 3: Performance Measures

Concept

Each section of the SLO Process has a specific task and each has its own importance. However, Section 3: Performance Measures is a critical and probably the most challenging work to complete. Selecting and/or developing Performance Measures that are of high-quality is essential for demonstrating student achievement of the selected content standards.


1. Make sure the participants understand that performance measures must allow equitable opportunities for students to demonstrate learning.

2. Discuss the strengths and weaknesses of status (mastery) and growth metrics.a. Status metrics have absolute standards and are easily understood; however,

they do not reflect changes (improvement) in student learning.b. Growth metrics are sensitive to changes in learning; however, they are more

unstable and can be limited for high performing students.3. Principles of Well-Developed Measures:

a. Be built to achieve the designed purpose;b. Produce results that are used for the intended purpose;c. Align to targeted content standards;d. Contain a balance between depth and breadth of targeted content;e. Be standardized, rigorous, and fair;f. Be sensitive to testing time and objectivity; and,g. Be valid and reliable.

Learning Activity


Slide No. NotesSlide 13 Section 3: Performance Measures

Concept

This slide provides the definitions for each part of Section 3 of the SLO Process Template. This section will take some time to complete and teachers need to consider many aspects of the performance measure to ensure that they select/develop a performance measure that fulfills the purpose of the assessment. Section 3 breaks out each aspect that teachers must consider when selecting/developing performance measures.


1. Referring teachers to the “Help Desk” Definition Guide and models will be useful toward filling out this section of the SLO Process Template.

2. Allow teachers time to fully consider all aspects of the Performance Measures and complete Section 3.

3. 3c. Purpose Statement: states “what” the performance task is measuring, “how” the results (scores) can be used, and “why” the performance measure was developed. Here is an example statement:

Elementary Pre-Post – Checkpoints in Mathematics are assessments intended to measure student proficiency of grade-level expectations in the sequence of the district’s curriculum, including different depths of knowledge. This grade-level assessment is provided to all students in the fall and spring of each year. Item and strand-level scores are reported to educators. Scores will be used by the district, schools, and teachers to monitor growth in student achievement.

4. 3d. Metric – teachers must ensure that the metric used by the Performance Measure is aligned with the Performance Indicator 4a. Is the Performance Indicator determining Growth (change in student performance across two or more points in time), Mastery (attainment of a defined level of achievement), or Growth and Mastery (Mastery for All Student Groups/Growth for a Focused Student Group).

a. Art/Grade 8 Modeli. PI Target #1 – achieve Advanced or Proficient on all three dimensions

of the Mood Portrait Rubricii. PI Target #2 – achieve Advanced or Proficient on all three dimensions

of the Demuth Oil Pastel Drawing Rubriciii. PI Target #3 – achieve Advanced or Proficient on all four dimensions

of the Clay Architectural Structure Rubrici. All three of these PI Targets describe a “Mastery” metric – an

attainment of a defined level of achievement.3e. Administration Frequency – if it is a pre-post performance measure – the frequency could be noted as “at the beginning of the semester/at the end of the semester; if it is a portfolio – the frequency could be noted as “during a six-week period”; if it’s a culminating activity/event – the frequency could be noted as “once a


Slide No. Notessemester.”

a. Art-Grade 8 Modeli. PM #1: Mood Portrait – once a quarter

ii. PM #2: Demuth Oil Pastel Drawing – once a quarteriii. PM #3: Clay Architectural Structure – once a quarter

3f. Adaptations/Accommodations – unique accommodation needed because of the performance measure’s design.

a. Art-Grade 8 Modeli. IEP, Gifted IEP, ELL – Additional time out of class is offered for those

who need more time to complete the projects. All other adaptations will be developed based on an IEP and/or specified district policy.

Learning ActivitySlide 14 Section 3: Performance Measures (cont.)

Concept

This slide continues with the various aspects associated with Performance Measures.


1. Resources and equipment are those materials needed for the assessment, not the instruction of the content. This data element is frequently misunderstood.

2. Scoring personnel (including second scorers) and the assessment developer and administrator are often the same teacher, especially for very unique performance measures. Here, the audience must be aware and articulate ways to mitigate rater/observer bias. This can be helped by having well-defined scoring criteria.

Learning Activity

The next slides provide an overview of the assessment literacy process, a process that supports the creation and/or review of performance measures that teachers will build and/or select to be used for the purposes of the SLO process. The activities below are provided to create a scenario in which the participant will want more information and subsequently recognize the purpose of the Assessment Literacy Series. (Adult Learning Theory, Robert Mager: Adults want to know why they should learn.)

1. Ask participants to write administration instructions for a task that they frequently administer, pretending that they might ask a colleague from their content area and grade- level to administer the performance measure. Follow this activity with a tour of the materials found in the Quick Start modules.

2. Ask participants to write scoring rubrics for a task that they frequently administer, pretending that they might ask a colleague from their content area and grade-level to score student work. Follow this activity with a tour of the materials found in the Quick Start modules.


Slide No. NotesSlide 15 Section 4: Performance Indicators

Concept

Performance indicators are a description of the expected level of achievement on each measure used in the SLO. An understanding of the scoring tool used to describe student achievement for any given performance measure is necessary to write a performance indicator statement. Sample performance indicator statements are listed below:

Physics(1) Roller Coaster Energy Project(2) Achieve 6 out of 9 points using the roller coaster project rubric. [Mastery]

US History(1) US History Final Exam(2) Achieve an 85% or higher on the final exam. [Mastery]

5th Grade ELA(1) DRA(2) Using the DRA text gradient chart, one year of reading improvement. [Growth]

Referring to SLO Design/Template #3-SLO Blueprint, teachers can begin to fill out Section 4: Performance Indicators of the SLO Process Template. When teachers filled out the SLO Blueprint, they identified the targets for each Performance Measure selected and/or to be developed.


1. Remind teachers that Performance Indicators offer a great deal of flexibility in the system. Performance Indicators can be linked – meaning a student must meet a specific achievement level across two or more Performance Measures in order to meet the standard. Also, Performance Indicators can be weighted if there is more than one Performance Indicator.

2. Performance Indicators should be specific, measurable, and ambitious, but attainable.

Learning Activity


Slide No. NotesSlide 16 Section 4: Performance Indicators

Concept

Again, this slide provides the definitions for each part of Section 4 of the SLO Process Template. However, referring to the “Help Desk” Definition Guide will be very useful toward completing this section.

[Remind teachers that each Section of the SLO Template builds upon each other. You will not be able to fill out Section 4 if you have not completed the work in order to fill out Section 2.]

Key Points for TrainersParticipants learning the SLO process often confuse and intertwine the Performance Indicator statement (4a) with the Level statement (5a), especially when the Performance Indicator statement is described in percentages (i.e., achieves an 80% on a test).

• Right: scoring 4 out of 5 on the “my awesome project rubric”• Right: achieving 80% on a final exam• Wrong: 80% of the students in the sample will score a 4 on the PM Rubric

4a. ALL STUDENT GROUP1. Ensure teachers are clear that PIs are not “performance expectations” for a group of

students but rather as a single indicator of a performance on the assessment.

4b. FOCUSED STUDENT GROUP [optional]1. This function allows teachers to differentiate their instruction and assessment of

various students within a SLO population.a. PI Targets: Focused Student Group

i. PI Target #1 – Students who score below 2 on the pre-test will improve a minimum of one level on the post-test.

Learning Activity

Slide 17 What is “Assessment Literacy”?

Concept

Assessment Literacy means understanding the basic principles of quality assessment practices in order to assess students effectively.

“Research suggests that teachers spend from one-quarter to one-third of their professional time on assessment-related activities. Yet almost all do so without the benefit of having learning the principles of sound assessment” (Stiggins, 2007).


Slide No. Notes“Governmental agencies and others involved in test development activities must be held to the same high expectations as test publishers and professional assessment companies with regard to following established requirements, adhering to industry standards, and implementing best practices related to established requirements and standards for products and services as test publishers and professional companies. A crucial part of this process is to agree upon standards for Quality Assurance (QA) and Quality Management (QM). Because testing is such an important enterprise with results that impact students, teachers, and schools, every step related to assessment development, administration, and scoring must be clearly documented and correctly implemented.” (Research in Action, Inc., Quality Assurance Techniques for Developing Measures of Student Achievement-Standard Operating Procedures ‘Smart Book’, April 2011).


1. Focus on common understanding and misunderstanding about assessment literacy.2. Assessment is part of the instructional process; Testing is an event.3. Assessment literacy is a unitary concept; meaning that educators are never

“assessment literate”, they simply have a greater understanding of assessment concepts and procedures.

Learning Activity

Ask one of the following:1. How is assessment different than testing?2. Explain the relationship between curricula, instruction, and assessment.3. What is the role of formative vs. summative assessment?4. Which assessment techniques should be avoided in the SLO process and why?

Slide 18 Assessment Life Cycle

Concept

Each step of the Assessment Life Cycle outlines the sequence required to ensure high-quality assessments that measure and validate student achievement. Details associated with each step of the Life Cycle will be outlined further in the Quick Start training.


1. Stress the importance for participants to understand and follow this step-by-step process, as it will ensure the development of high-quality assessments.

2. Ensure that teachers recognize the assessment life cycle as a continuous sequence that allows for additional evidence and corrective action to be implemented from administration to administration. This “process” creates a body of evidence about the score inference by which teachers are, in part, regarded as “effective”.

3. Clarify the advantages and limitations associated with educators creating their own


Slide No. Notescustomized assessments for use in the SLO process.

Learning Activity

Slide 19 Overview of the Quick Start PA Process

Concept

Just like the Student Learning Objective Process, the Quick Start training is comprised of three phases: Design, Build, and Review.


1. Delineate that the purpose of Quick Start is to engage teachers who are unfamiliar with the assessment development process.

2. Clarify that the ALS (full model) guides the development of assessments typically associated with district, common assessments. For existing measures, the ALS Review section will outline the quality criteria the assessment must adhere to, in order to be considered of sufficient quality (i.e., sufficient quality for use in the SLO process).

3. Each phase of the process deals with specific steps within the Assessment Life Cycle. Working through the three phases (Design, Build, and Review) will provide participants the opportunity to develop measures that meet crucial criteria for each step and ensure the development of high-quality assessments/measures of student achievement.

4. In many cases, educators will be using vendor-produced assessments for their SLOs. In these instances, the Review section is most applicable.

Learning Activity

Slide 20 Principles of Well-Developed Measures

Concept

Working through the steps associated with the Assessment Life Cycle focuses test developers on the principles of well-developed measures. The Performance Measure Rubric for Teachers (see Review Phase materials) will guide participants through the process of selecting/developing high-quality performance measures. The rubric has 18 technical aspects related to the basic principles of quality assessment. They are organized into three strands to align with the Design, Build, Review phases of the SLO Process. Using this rubric allows teachers to validate the performance measures they have determined will effectively measure student progress toward the goals identified in their SLO.



Slide No. Notes

Measures must:1. Be built to achieve the designed purpose – “For all types of assessments, the first

step is to clearly define the purpose. Teachers must specify exactly what the assessment is intended to measure, characteristics of intended test takers, and types of scores to be reported, and how the information derived from the assessment will be used.” (RIA “Smart Book”, 2011)

2. Produce the results that are used for the intended purpose – does the assessment measure what we really want to measure? Evaluate the final assessment against the intended purpose.

3. Aligned to targeted content standards – “The desired attributes and characteristics of the assessment need to be identified, specified, and documented. The test design framework and blueprint provide information necessary to guide the item/task development process. This is a critical activity in establishing validity.” (RIA “Smart Book”, 2011)

4. Contain a balance between depth and breadth of targeted content – “Several considerations are taken into account during the assessment design phase. First, to satisfy accepted standards of reliability and validity, a minimum number of items and score points are required within each subtest and for the overall assessment. Second, the assessment must include items with a range of difficulty levels if the assessment’s purpose is to provide information about student achievement at different levels on the performance continuum.” (RIA, “Smart Book”, 2011)

5. Be standardized, rigorous, and fair – “A number of technical and editorial issues are related to item and test specifications. Obviously, crucial quality processes are needed in assessment development, such as adequate content coverage and the development of items and test forms to meet best practice requirements. The desired psychometric properties for the items, such as difficulty and discrimination, as well as desired test properties such as overall test difficulty and reliability need to be targeted and met. Detailed item specifications must include: item types, number of items, response options, difficulty levels, language load, and artwork. Test specifications guide the process of developing forms and likewise need to be detailed to address issues of content coverage, overall difficulty level, balance of items, targeted distribution of item difficulties, rules for sequencing items, total number of items, and timing.” (RIA “Smart Book”, 2011)

6. Be sensitive to testing time and objectivity a. The total time to administer the performance measure is developmentally

appropriate for the test-taker. Generally, this is 30 minutes or less for young students and up to 60 minutes per session for older students (high school).

7. Have score validity and reliability evidence – “General scoring criteria and methods for scoring both short-constructed response (SCR) and extended-constructed


Slide No. Notesresponse (ECR) should be outlined when the guidelines for writing items are developed.”

a. Cut scores are established for each performance level. Performance level descriptors describe the achievement continuum using content-based competencies for each assessed content area.

b. As part of the assessment cycle, post administration analyses are conducted to examine such aspects as items/tasks performance, scale functioning, overall score distribution, rater drift, content alignment, etc.

c. The performance measure has score validity evidence that demonstrated item responses were consistent with content specifications. Data suggest the scores represent the intended construct by using an adequate sample of items/tasks within the targeted content standards. Other sources of validity evidence such as the interrelationship of items/tasks and alignment characteristics of the performance measure are collected.

d. Reliability coefficients are reported for the performance measure, which includes estimating internal consistency. Standard errors are reported for summary scores. When applicable, other reliability statistics such as classification accuracy, rater reliabilities, and others are calculated and reviewed.

Learning Activity

Slide 21 Section 5: Elective Rating

Concept

As part of the SLO Process, educators will be required to set expected levels of student learning based on the Performance Indicators and their Performance Measures. The levels are identified as “Distinguished, Proficient, Needs Improvement, and Failing.” This Teacher Expectation Rating will determine the overall SLO rating which will be used as the Elective Rating on the Pennsylvania Educator Effectiveness Rating Tool.


1. Participants learning the SLO process often confuse and intertwine the Performance Indicator Statement with the Teacher Expectation Statement, especially when the Performance Indicator statement is described in percentages.

Examples:Performance Indicator Statement: achieves at minimum an 80% on the post-test.Elective Rating Statement: 80% of students meet the performance indicator (implies that 80% of students achieve at minimum an 80% on the test).

1. The SLO Process allows for a great deal of control and flexibility over the performance measures and the expected levels of growth and/or mastery associated


Slide No. Noteswith each measure that will be used to fulfill the requirements of the Elective portion of the overall teacher evaluation.

2. These levels are established by educators prior to the evaluation period and each performance level is populated with a percentage range such that 0% to 100% meeting expectations is distributed among the levels.

3. The SLO Rating for teachers is not completed until after performance data are collected, reviewed, and evaluated against each Performance Indicator.

Learning Activity

Slide 22 Section 5: Elective Rating (continued)

Concept

This slide provides an example of the different ranges of performance across the four rating levels.


1. Once the performance data has been collected, reviewed, and evaluated against each Performance Indicator, the number of students meeting expectations can be charted and totaled. Then an overall percentage can be determined by a simple division problem: the total number of students meeting expectations/the total number of students in the SLO population across all indicators.

2. The same student may be included across all indicators (which are demonstrated in the slide); however, some variation in student counts across time will exist.

3. Remember, Teacher Expectations and Performance Indicators are two different things:

a. Performance Indicators are the descriptions of the expected level of achievement for each student in the SLO population on a particular Performance Measure – Physical Education-3rd grade Model

b. 5a. Levels – are the four levels of projected performance regarding the PI, reflecting a continuum established by the educator prior to the evaluation period. Each performance level (i.e., Failing, Needs Improvement, Proficient, and Distinguished) is populated with a percentage range such that 0% to 100% meeting expectations is distributed among the levels.

i. Failing – 0% - 60% of students will meet the PI targets.ii. Needs Improvement – 61% to 84% of students will meet the PI

targets.iii. Proficient – 85% - 94% of students will meet the PI targets.iv. Distinguished – 95% - 100% of students will meet the PI targets

4. Another problematic area is where to set the values among the different performance levels. A “trade-off” exists between the rigor of the performance targets and expectation of teachers that students will meet those standards.

a. In looking at the Physical Education – Grade 3 Model, the different PI


Slide No. NotesTargets for the focused student group is reflective of the “trade-off” between the rigor of the performance targets and expectation of teachers that students will meet those standards.

b. Because the PI Targets are based on both Mastery and Growth – the teacher expectations are different for different students.

Learning Activity

1. Using the Physical Education-Grade 3 Model, have participants create student results and from that data determine Section 5: Teacher Expectations

Slide 23 Section 5: Elective Rating (continued)

Concept

This slide provides the definitions for each part in Section 5 of the SLO Process Template.


1. Even though this chart is here and clarifies each part of Section 5, please refer to the “Help Desk” definition guide (found in SLO/Build/Template) as it provides the correct formatting along with examples. It is important that each section is filled out correctly.

2. One thing that needs to be considered is what data will be used.a. In the (Physical Education-Grade 3 Model) the teachers choose to use data from

teacher-developed measures.a. Teachers will need to be clear on what and how many teacher-developed

measures were used and how the results were combined to determine the percentages of students who met the PI Targets.

b. In the (Art – Grade 8 Model) teachers choose to use data from district-designed measures and examinations and student projects.

i. Teachers will be gathering information from different performance measures. Will the same weight be given to each measure? Will one measure be weighted more than the other?

3. Another factor to consider is how the data are aggregated (how they are gathered and summarized).

4. Element 5b SLO Rating is not determined until after performance data are collected, reviewed, evaluated, and reported.

5. Element 5b also has a section for Notes/Explanations – this provides an opportunity for teachers to offer additional information related to student outcomes.a. In-depth analysis of the data that will provide goals for future implementation

and improvement of student achievement through the SLO.b. Recommendations as to how analysis of the achievement data will inform future

teaching practice as defined by Danielson’s Framework for Teaching.c. Recommendations for further SLO development to support student achievement

of the targeted content standards.


Slide No. Notes

Learning Activity

Slide 24 Summary & Next Steps

Concept

During Module 2: Building, teachers have worked through the development of the SLO and selected and/or developed performance measures. Completing the following templates provided guidance through this phase of the SLO process.• Template #4-SLO Process Template• Template #5-Performance Task FrameworkUsing the “Help Desk” Definition Guide (found in SLO/Build/Template) assisted in the completion of these templates.

The next and final step of the SLO Process is outlined in training Module 3: REVIEWING SLOs. During this module, educators will conduct an extensive quality assurance review of their developed SLOs and all applicable performance measures.


1. Completed Templates #4 and #5 (if applicable) will be needed in order to move ahead to Module 3: Reviewing, where educators will review the completed SLO and the Performance Measures associated with each Performance Indicator to ensure that they meet all the criteria for high-quality SLOs and Performance Measures.

Learning Activity

1. Make sure participants have completed the above mentioned Templates before leaving Module 2: Building.

2. Answer any questions related to the Build Phase of the SLO Process. Template #5-Performance Task Framework


· Web viewReview Handout #1 to link the conceptual framework of a three phase process for ......

Documents

Transcript of · Web viewReview Handout #1 to link the conceptual framework of a three phase process for ......