PBMAS 101 What’s the Score??? Go for the Touchdown with ESC-2.
-
Upload
alisha-wilcox -
Category
Documents
-
view
214 -
download
0
Transcript of PBMAS 101 What’s the Score??? Go for the Touchdown with ESC-2.
PBMAS 101What’s the Score???
Go for the Touchdown with ESC-2
PBMAS=
• Performance
• Based
• Monitoring
• Analysis
• System
PBMAS Report• Automated data system that reports on the performance of
school districts and charter schools in the 4 Program Areas below:
– Bilingual– Career and Tech– No Child Left Behind– Special Education
• http://www.tea.state.tx.us/pbm/regrpts.html
TEA’s Monitoring Systems:Historical Context
•78th TX Legislative 2003 Session
•Resulted in a realignment of TEA’s functions
•Emphasis on a coordinated approach to TEA’s monitoring to districts
PBM Overall Coordination
Performance-Based
Monitoring
Program Monitoring and Interventions
Program Areas
Historical Context Different Monitoring Components
•PBMAS
•Data Validation System
•Federal Compliance
•State Accountability System (AA, AU, etc.)
•Federal Accountability System (AYP)
Historical Context Different Monitoring Components
•Financial Integrity Rating System (FIRST)
•Financial Audits
•Complaints
•Due Process Hearings
•Governance
•Other Monitoring such as Office for Civil Rights
A Look into One of the TEA Monitoring Components:
PBMAS
PBMAS Indicators• Types of indicators generally fall into these
categories:
– Student performance indicators •(ie TAKS Passing Rate; annual dropout
rate)
– Program effectiveness indicators •(ie TAKS Participation Rate)
10 Guiding Principles of PBM
1. School District Effectiveness
2. Statutory Requirements
3. Valid Indicators of Performance
4. Maximum Inclusion
5. Individual Program Accountability6. High Standards
7. Annual Statewide Focus
8. Public Input and Accessibility
9. Systems Evolution
10. Coordination
PBMAS Components
1. Data Sources
2. Filters3. No Data Available for an Indicator
4. Rounding
5. Masking
6. Standards & Performance Levels
PBMAS Components
7. Report Only Indicators
8. Minimum Size Requirements and Special Analysis
9. Required Improvement
10.System Safeguards
Component #1Data Sources
•Student Assessment•Dropouts•Annual Graduation Data•Longitudinal Graduation Data•AEIS•PEIMS
Component #2Filters
May be necessary before
Performance Indicators are calculated
i.e. Residential Facilities increase
% of special education identified students in a program
Component #3No Data Available for an Indicator
• District can’t be evaluated for absence of data.
• Data validation analysis and reviews are conducted by TEA to ensure proper coding…
i.e. coding for migrant students: Is the district identifying and coding students appropriately?
Component #4Rounding
Calculations rounded to one decimal point
Component #5Masking
To protect student identity
Component #6Standards and Performance Levels (PL)
• PL = Level System in which a standard is applied to a performance on an indicator:• 0 = Met Standard• 1• 2• 3= farthest from meeting standard
• SA= Special Analysis• PJSA=Professional Judgment Sp Analysis
Component #6
Performance Levels (PL) Example:
TAKS Reading Standard = 70%
If a district meets or exceeds the 70% in example, it would receive a
“0” PL
Component #6
Standards and Performance Levels (PL)
2 Types of Standards:
Absolute and Relative
Component #6Standards and Performance Levels (PL)
Absolute Standards:
• 0 PL = Met Standard
• “1” PL = 0.1 - 10% below standard
• “2” PL = 10.1 -20.% below standard
• “3” PL = 20.1% or below
Component #6Standards and Performance Levels (PL)
Relative Standards:
based on the distribution of scores to the population being evaluated
ESC-2 Example of Relative Standard
Description
Indicator 12
2007
State
ID Rate
2007
ESC-2
ID Rate
2007
PL
Assigned
Calculation
Sped ID
8.5% 12.1% 2 12,844
105,828
Example of Relative StandardPL =
Not Assigned
PL = 0 PL = 1 PL = 2 PL = 3
8.5% or lower!
8.6% - 12% 12.1%-16% 16.1% or up
Changes to Standards• Changes are made after careful consideration.
Following list includes a few: – Actual improvement over time on an indicator– Overall impact on PBMAS of increased standards– External input– Whether a state or federal goal has been identified
See chart on page 8 of manual
Component #7: Report Only Indicators
Gives a district a “preview” for planning purposes.
No PL assigned for 2 years
Component #8: Minimum Size Requirements
and Special Analysis
•At least 30 students are needed for calculations.
•Can be met in current year or over 2 recent years.
•Always in the favor of the district
Component #8: Minimum Size Requirements
and Special Analysis
•If a district meets standard regardless of amount of students, then district receives a “0” PL.
Component #8: Minimum Size Requirements
and Special Analysis
•Special Analysis: for districts and charters that do NOT meet the minimum size requirements.
Component #8 (continued)
• There are two types of special analysis in PBMAS:
–Automated special analysis–Professional judgment special analysis (PJSA)
Component #8 (continued)
• GROUP SIZE DETERMINES TYPE OF ANALYSIS:– 30 or more in current year or over two
years: STANDARD ANALYSIS is applied.– SPECIAL ANALYSIS is applied in different
ways depending on the following group sizes:• 15-29• 5-14• Less than 5
Component #8 (continued)
• Special analysis does not apply to indicators for which the district has NO students in a particular student group; these districts receive a designation of No Data (ND).
• Example: Neutral ISD has no CTE students. The district receives a designation of ND – No Data for the CTE TAKS Passing Rate indicator.
Component #8 (continued)
• Special analysis (continued)
– Does not apply to indicators for which the district performance level is 0 – Met Standard, regardless of student group size.
• Example: Terrific ISD has four CTE students, and all four of these students met the TAKS passing standards. The district performance level on the CTE TAKS Passing Rate indicator is 0 – Met Standard.
Key Monitoring Concepts (continued)
• Example #1 for Group Size 15-29: District’s Performance on a TAKS Reading Indicator– Current Year = 4/10 (PL2) – Prior Year = 4/10 (PL2) – PL = 2SA because of group size and PL
is the same for both years.
Key Monitoring Concepts (continued)
• Example #2 for Group Size 15-29: District’s Performance on a TAKS Reading Indicator– Current Year = 4/10 (PL2)– Prior Year = 7/15 (PL1)– PL = 1SA because neither is a PL3 so
the better of two PLs is assigned.
Key Monitoring Concepts (continued)
• Example #3 for Group Size 15-29: District’s Performance on a TAKS Reading Indicator (50% = 0)– Current Year = 1/10 = 10% (PL3)– Prior Year = 4/10 = 40% (PL2)– PL = determined by professional judgment
special analysis.
Key Monitoring Concepts (continued)
• Example #4 for Group Size 15-29: If the same performance results were for Sped Year After Exit rather than Reading, the PL=NA (Not Assigned) because professional judgment special analysis is not currently available.
Key Monitoring Concepts (continued)
• Example #5 for Group Size 15-29: District’s Performance on a TAKS Reading Indicator – Current Year = 1/10 (PL3)– Prior Year = 8/10 (PL0) – PL = NA because PLs are more than one
level apart over two years.
Key Monitoring Concepts (continued)
• Example #1 for Group Size 5-14: District’s Performance on a TAKS
Reading Indicator – Current Year = 5/11 (PL1)– Prior Year = 2/2 (PL0) – PL = NA because size of student group.
Key Monitoring Concepts (continued)
• Example # 2 for Group Size 5-14: District’s Performance on a TAKS Reading Indicator – Current Year = 5/11 (PL1) – Prior Year = 0/2 (PL3)– PL = NE because PLs are more than one
level apart.
Key Monitoring Concepts (continued)
• Example #3 for Group Size 5-14: Current District’s Performance on a TAKS Reading Indicator– Current Year = 0/7 (PL3) – Prior Year = 2/7 (PL3) – PL = determined by professional judgment
special analysis.
Key Monitoring Concepts (continued)
• Example #4 for Group Size 5-14: If the same performance results were for Social Studies rather than Reading, the PL=NE because professional judgment special analysis is not currently available for TAKS Social Studies.
Key Monitoring Concepts (continued)
• Example #1 for Group Size Less Than 5: District’s Performance on a TAKS Reading Indicator – Current Year = 0/2 (PL3) – Prior Year = 0/2 (PL3) – PL = NE because of size of student group.
Component #9: Required Improvement
•Built-in system to help districts that demonstrate improvement from one year to the next.
•List of indicators on pages 15-16
Component #9: Required Improvement
Example: Indicator 12 Sped Identification
•Calculate actual difference from 2008- 2007:
15% - 25% = -10%
•Calculate Required Improvement for the sped identification rate: 8.5% (state) – 25%
2 (2009-2007)
•Compare the two numbers to see if there is an actual change < or = to RI. -10.0 < -8.25 = 0 PL
= -8.25
Component #10: System Safeguards
• To ensure “Integrity”
• Uses Data Validation Analyses:
1.Leaver Data
2.Student Assessment Data
3.Discipline Data
4.Other Means
System Safeguard “Other Means”:
• Accuracy of data reporting!• Implementation of intervention activities!!• Implementation of plans for improvement or
correction• Accuracy of findings made through PBM process.
Let’s Jump to INTERVENTION
LEVELSThese are different than the
PL #’s for each indicator within each program.
Intervention Levels
• Levels are assigned per program area as an overall “stage” level:– Bilingual– CTE– NCLB– Special Education
• Districts must work through intervention activities and upload documents to TEA on different deadlines.
Basic Intervention Activities
•Focused Data Analysis•Program Effectiveness Review•LEA Public Meeting•Compliance Review•Continuous Improvement
Planning
Accessing PBMIntervention Documents
• After reviewing the district PBMAS reports / Performance Levels, click on the following link for Interventions:
• http://www.tea.state.tx.us/pmi/
Who Ya Gonna Call???
• Bilingual: Velma Escamilla [email protected]
• CTE: Bob Chapman [email protected]
• NCLB: Andi Kuyatt [email protected]
• Special Ed: Barbie Tumlinson [email protected]