www.FLDOE.org

2017 Annual Assessment and Accountability Meeting

Bureau of Accountability Reporting Presentations

August 31, 2017

1

www.FLDOE.org

Accountability Update

Presented by:Jane Fletcher

Assistant Deputy CommissionerOffice of Accountability and Policy Research

Division of Accountability, Research, and Measurement2

www.FLDOE.org

Overview

• 2017 School Grade Results• School Grades• School Improvement Rating• District Grades• ESSA Timeline

3

www.FLDOE.org

2017 School Grade Results

983

877

1166

223

430

200

400

600

800

1000

1200

1400

A B C D F

4

www.FLDOE.org

2016 and 2017 School Grades

763 768

1,292

386

111

983

877

1166

223

430

200

400

600

800

1000

1200

1400

A B C D F2016 2017 2016 2017 2016 2017 2016 2017 2016 2017

5

www.FLDOE.org

Statewide Highlights

• The percentage of schools earning an “A” or “B” increased to 57 percent (1,860 schools), up from 46 percent (1,531 schools) in 2015-16.

• Elementary schools saw the largest percentage point increase in “A” schools

• 30 percent (546 schools) of elementary schools earned an “A” in 2016-17, up from 21 percent (386 schools) in 2015-16.

• 1,608 schools maintained an “A” grade (669 schools) or increased their grade (939 schools) in 2016-17

• “F” schools decreased by more than half (61 percent), dropping from 111 schools in 2015-16 to 43 schools in 2016-17.

6

www.FLDOE.org

70% of Schools Graded “D” or “F” in 2016 Improved Their Grade in 2017

Increased Their Grade 70%

Did Not Increase

Their Grade 30%

7

www.FLDOE.org

78% of F Schools Improved Their Grade

2 Improved to an “A” 10 Improved to a “B”

36 improved to a “C”

16 Improved to a “D”

18 Remained an “F”

8

www.FLDOE.org

Florida’s Focus on Low Performing Schools is Paying Off

573

497

266

0

100

200

300

400

500

600

700

2015 2016 2017D F

9

www.FLDOE.org

District Grades for 2017

10

www.FLDOE.org

School Grades

11

www.FLDOE.org

School Grades Model(A maximum of 11 components)

English Language Arts Mathematics Science Social Studies Graduation

RateAcceleration

Success

Achievement(0% to 100%)

Achievement(0% to 100%)

Achievement (0% to 100%)

Achievement(0% to 100%)

Overall,4-year

Graduation Rate

(0% to100%)

High School (AP, IB, AICE, dual

enrollment or industry

certification) (0% to 100%)

Learning Gains (0% to 100%)

Learning Gains (0% to 100%)

Blank Blank

Learning Gains of the

Low 25%(0% to 100%)

Learning Gains of the

Low 25%(0% to 100%)

Blank Blank Middle School (EOCs or industry

certifications)(0% to 100%)

12

www.FLDOE.org

Percent Tested

• Must test 95% of students• Calculated for each assessment and then aggregated.• Schools that do not test 95% of students will receive grades of “I”• Superintendents can appeal the “I” by demonstrating that the

data accurately represents the school’s progress or requesting that late reporting assessment results be included.

• Commissioner will review data to determine if the performance data is representative of the school’s progress.

• If the Commissioner determines the data is representative, she will release grades for these schools at the end of the appeals period.

13

www.FLDOE.org

Learning Gains in School Grades• 2014 legislature established a new framework for learning gains requiring that

learning growth toward achievement levels 3, 4, and 5 is demonstrated by students who scored below each of those levels in the prior year (s. 1008.34(3)(b), F.S.)

Method for 2002 to 2014 Method Used Since 2016

Improve one or more achievement levels from one year to the next (e.g., move from Level 1 to Level 2; Level 2 to Level 4, etc.)

Same

Maintain a Level 3, Level 4, or Level 5 from one year to the next

Same, except for Level 3 and Level 4, in addition to maintaining the level, the student’s scale score must have improved from one year to the next

For students who remain in Level 1 or Level 2, demonstrate a specified scale score gain

For students who remain in Level 1 or Level 2, demonstrate a learning gain by increasing their score to a higher subcategory within the Level (e.g., move from the bottom third of Level 1 to the middle third of Level 1)

14

www.FLDOE.org

Learning Gains of the Lowest 25%

• Calculated for both English Language Arts and Mathematics

• Applies the same learning gains methodology to the lowest performing 25% of students

• Determining the lowest performing 25% of students

• Uses the performance of students in the prior year calculated at each grade level to identify the lowest performing 25% of students (EOCs not by grade level)

• Low 25% is no longer limited to students in Achievement Levels 1 and 2

15

www.FLDOE.org

Middle School Acceleration• The percentage of eligible students who passed one or more high

school level statewide, standardized end-of-course (EOC) assessments or attained industry certifications identified in the industry certification funding list

• Calculated for all schools that include grades 6, 7, and 8 or grades 7 and 8

• Eligible students include full-year-enrolled students, who are current year grade 8 students who scored at or above Achievement Level 3 on the Mathematics statewide assessments (FSA & EOC) in the prior year, or are full-year-enrolled students in grades 6, 7, or 8 that took high school level EOC assessments or industry certifications (industry certification data is the most recent available and lags by one year)

• Students must be enrolled in the course to be included• A student is included in the calculation no more than once

16

www.FLDOE.org

Graduation Rate

• The most recent 4 year cohort graduation rate measured according to 34 § CFR 200.19

• Calculated for all schools that include grades 9 to 12, grades 10 to 12, and grades 11 and 12

• Also calculated for combination schools that include these grade levels

• Beginning in 2016-17 grad rate, students who withdraw to a private school that the district has a contract with will remain in the graduation cohort for their last public school

17

www.FLDOE.org

College and Career Acceleration

• Cohort-based calculation using the graduates from the graduation rate calculation as the denominator

• The percentage of graduates who, while in high school

• Were eligible to earn college credit through AP, IB, or AICE examinations

• Earned a C or better in dual enrollment or• Earned a CAPE industry certification

18

www.FLDOE.org

School Grades Scale

19

Grade Scale

A 62% of total points or higher

B 54% to 61% of total points

C 41% to 53% of total points

D 32% to 40% of total points

F 31% of total points or less

• The State Board of Education sets the scale and must, per state law, periodically review the scale to determine whether the expectations should be raised to encourage increased student achievement • If the Board adjusts the grading scale upward, it must inform the public and

the school districts of the reasons for the adjustment and the anticipated impact on school grades

www.FLDOE.org

Calculating the School Grade

• The school’s grade is determined by • Summing the points earned for each component (each

component is worth 100 points) and dividing by the sum of total points available for all components with sufficient data

• The percentage resulting is the percentage of points the school earned from all applicable components

• This percentage is compared to the scale set by the State Board of Education to determine a school’s grade

20

www.FLDOE.org

School Grades ModelOther Topics

• Per state law, if two or more schools operate at the same facility (collocated schools), and at least one of the collocated schools does not earn a school grade or a rating because of insufficient data, the performance data across all the schools at the same location are combined to calculate a school grade (s. 1008.34(3)(a)3, F.S.)

• This provision results in more schools being included in school accountability

21

www.FLDOE.org

Florida Standards Alternate Assessment

• Results will be included in the 2017-18 school grades

• Will go through the rule making process • Set the passing score to use in the achievement

measures for US History and Civics• Determine the learning gains methodology• Workshops in the Fall• Likely to the State Board in early 2018

22

www.FLDOE.org

District Grades

23

www.FLDOE.org

District Grades

• Districts receive grades based on all of the components in the school grades model

• Students who were not full-year enrolled in a school but were full-year enrolled in the district will be included in the district grade in addition to students included in schools’ grades.

24

www.FLDOE.org

District Grades Model(A maximum of 11 components)

English Language Arts Mathematics Science Social Studies Graduation

RateAcceleration

Success

Achievement(0% to 100%)

Achievement(0% to 100%)

Achievement (0% to 100%)

Achievement(0% to 100%)

Overall,4-year

Graduation Rate

(0% to100%)

High School (AP, IB, AICE, dual

enrollment or industry

certification) (0% to 100%)

Learning Gains (0% to 100%)

Learning Gains (0% to 100%)

Blank Blank

Learning Gains of the

Low 25%(0% to 100%)

Learning Gains of the

Low 25%(0% to 100%)

Blank Blank Middle School (EOCs or industry

certifications)(0% to 100%)

25

www.FLDOE.org

District Report Cards

• State law also requires the department to develop a district report card that includes the district grade and additional data points

• District and school grades are currently available on the department’s EdStats portal (https://edstats.fldoe.org)

• The remaining report card elements will be available in the near future at that same portal

26

www.FLDOE.org 27

www.FLDOE.org 28

www.FLDOE.org

School Improvement Ratings

29

www.FLDOE.org

School Improvement Rating

• Alternative schools and Exceptional Student Education (ESE) Center schools choose whether to receive a school grade or a school improvement rating

• If the school chooses to receive a rating, its students’ performance information is used in both the school’s rating and the students’ home-zoned school’s grade

• The department provides the district a list of schools from which to verify the schools that are eligible to choose a rating

30

www.FLDOE.org

School Improvement Ratings

• The school improvement rating shall identify an alternative school as having one of the following ratings:

• Commendable: a significant percentage of the students attending the school are making learning gains

• Maintaining: a sufficient percentage of the students attending the school are making learning gains

• Unsatisfactory: an insufficient percentage of the students attending the school are making learning gains

31

www.FLDOE.org

Percent Tested

• Schools must assess 80% of students to receive a rating

• Schools that assess less than 90% of students are not eligible to receive a rating of Commendable

32

www.FLDOE.org

Rating Components

• Learning Gains in English Language Arts (100 points)• Learning Gains in Mathematics (100 points)

• Learning gains are calculated using the method described in the school grades rule

• Eligible students include students enrolled in membership survey 2 or 3 and tested

• Retake assessments are included when first-time assessments are not available for a student

• FCAT 2.0 Reading• Concordant and Comparative Scores

33

www.FLDOE.org

Calculation of the Rating

• Schools will be rated on only those components for which they have sufficient data

• The rating is based on the percentage of possible points earned by each school

• Commendable – 50% of points or higher• Maintaining – 26% to 49% of points• Unsatisfactory – 25% of points or less

34

www.FLDOE.org

Three Year School Improvement Ratings

• “If an alternative school does not meet the requirements for the issuance of a school improvement rating in the current year, and has failed to receive a school improvement rating for the prior 2 consecutive years, the school shall receive a rating for the current year based upon a compilation of all student learning gains for all grade levels, for those 3 years.” (s. 1008.341 (2) F.S.)

• 2017-18 would be the first year this would take effect

35

www.FLDOE.org

Every Student Succeeds Act

36

www.FLDOE.org

Superintendent Workgroup

• Review decision areas and options• Workgroup members seek input from fellow

superintendents on these decision areas and share recommendations for development of draft State Plan

• Provide input on the draft State Plan

37

www.FLDOE.org

Timeline

• Posted draft state plan for public comment June 30th

• Revised state plan based on input received• Submitted revised state plan to Governor for review

August 17 (30 days)• State Plan due by Monday, September 18• USED has 120 days to review and approve the plan

38

www.FLDOE.org

State Plan• At this point, waiting on feedback from the

Governor before we determine that the plan is final • Reevaluated the need for separate waiver requests

at this time• We described our current accountability system within

the state plan template, and provided our rationale for why it is best for our students to continue on this path

• Will use state, district, and school report cards as part of our accountability system to provide information to the public about subgroup performance

39

www.FLDOE.org

State Plan (continued)

• Increased the reporting period for former ELLs from two years to four years

• Established long-term goals for ELLs on the ACCESS for ELLs 2.0 English Language Proficiency assessment

• Clarified and added more specificity to the Title IV, Part A section on allowable use of funds for new Student Support and Academic Enrichment grants

40

www.FLDOE.org

Nine State Plan Sections

A. Title I, Part A: Improving Basic Programs Operated by Local Educational Agencies

B. Title I, Part C: Education of Migratory ChildrenC. Title I, Part D: Prevention and Intervention Programs for

Children and Youth Who Are Neglected, Delinquent, or At-RiskD. Title II, Part A: Supporting Effective InstructionE. Title III, Part A: English Language Acquisition, Language

Enhancement, and Academic Achievement

41

www.FLDOE.org

Nine State Plan Sections

F. Title IV, Part A: Student Support and Academic Enrichment Grants

G. Title IV, Part B: 21st Century Community Learning CentersH. Title V, Part B, Subpart 2: Rural and Low-Income School

ProgramI. Title VII, Subpart B of the McKinney-Vento Homeless Assistance

Act: Education for Homeless Children and Youth Program

42

www.FLDOE.org

Appeals Review and Guidelines, Graduation Rate Process, and Accountability Match Process

Presented by:Tracy Halley

Director of School GradesBureau of Accountability Research

Office of Accountability and Policy ResearchDivision of Accountability, Research, and Measurement

43

www.FLDOE.org

Today’s Topics

• Part 1: • Appeals

• Part 2:• Graduation Rate Process for 2016-2017

• Part 3:• Accountability Match Process for 2017-2018

44

www.FLDOE.org

Part 1 – Appeals

•2016-2017 Highlights•Review Guidelines for Submitting Appeals

45

www.FLDOE.org

2015-2016 Appeals vs. 2016-2017

• Post-Appeals School Grades has been posted to schoolgrades.fldoe.org

• The number of School Grade Appeals submitted decreased from 162 in 2015-2016 to 18 in 2016-2017

46

www.FLDOE.org

Guidelines for AppealsRule 6A-1.09981

(7)(e) Districts shall be afforded an opportunity to contest or appeal a preliminary school grade within thirty (30) days of the release of the preliminary school grade.

47

www.FLDOE.org

Guidelines for Appeals Rule 6A-1.09981

(7)(f) A successful grade appeal requires that the district clearly demonstrate the following:1. Due to the omission of student data, a data miscalculation, or a special circumstance beyond the control of the district, a different grade would be assigned to a school; or

• Examples of what to submit:• Late Reporting• Districtwide internal system malfunction• A middle school student who has a passing high school industry

certification

48

www.FLDOE.org

Guidelines for Appeals Rule 6A-1.09981

(7)(f) A successful grade appeal requires that the district clearly demonstrate the following:1. Due to the omission of student data, a data miscalculation, or a special circumstance beyond the control of the district, a different grade would be assigned to a school; or

• Examples of what NOT to submit:• A Request for DOE to do something that is contrary to the statute or rule.• Student did not perform as expected, exclude their scores.• Student took the writing at School A but reading at school B, exclude their scores.• Include EOC test scores for students who were not reported as enrolled in the

course.• Student withdrew in late April, exclude the test they took in March.• Include College and Career Acceleration tests that are not on the approved list. • Any change that would not result in a different (increased) grade.

49

www.FLDOE.org

Guidelines for Appeals Rule 6A-1.09981

(7)(f)2. Where the percent of students tested is less than ninety-five (95) percent at a school and the school did not receive a grade, that the student data accurately represents the progress of the school.

• If the students that were not tested were counted as not satisfactory, the grade would remain the same.

• If the students that were not tested were counted as satisfactory, the grade would remain the same.

50

www.FLDOE.org

Guidelines for Appeals Rule 6A-1.09981

(7)(g) An appeal shall not be granted under the following circumstances:1. It was not timely received;2. It was not submitted by the district superintendent;3. It would not result in a different grade, if granted;4. It relies upon data that the district had the opportunity to correct but failed to do so, under the process described in paragraph (7)(c) of this rule or the data reporting processes as defined in Rule 6A-1.0014, F.A.C., Comprehensive Management Information Systems.

51

www.FLDOE.org

Tools for Submitting Appeals

• Appeals Check List• Details the rule and specific requirements.

• Calculation Guide Sheet• Provides a clear and easy way to demonstrate that

the appeal, if granted would change the school’s grade.

52

www.FLDOE.org

1

Appeals Submission Checklist SCHOOL INFORMATION

District Name: School Name: District School Number: Filled out by:

Rule 6A-1.09981, F.A.C. provides for two circumstances under which a district may successfully appeal

(7)(f) A successful grade appeal requires that the district clearly demonstrate the following:

1. Due to the omission of student data, a data miscalculation, or a special circumstance beyond the control of the district, a different grade would be assigned to a school; or

2. Where the percent of students tested is less than ninety-five (95) percent at a school and the school did not receive a grade, that the student data accurately represents the progress of the school. [For school improvement ratings, the school must test at least eighty (80) percent of its students.]

Question 1. If the appeal is granted, will it change the school’s grade/school improvement rating?

Yes. (Please provide the school grades/rating calculation form.) No. (If no, then your appeal cannot be granted.)

Question 2. If the school tested less than the required percentage of students: Does the data used to calculate the school grade/school improvement rating accurately represent the progress of the school?

Yes. If the students that were not tested were counted as not satisfactory, the grade/rating would remain the same. Yes, but changes should be made to the calculation. (For example, include late reported results.) No. Changes should be made to the calculation, which will result in a different grade than the grade/rating that was calculated. (For example, include late reported results or remove students from the tested denominator due to a district/school mistake.)

No. The data does not represent the progress of the school, please leave the school as an “I”. An explanation is required.

Rule 6A-1.09981, F.A.C. provides circumstances under which an appeal will not be granted

(7)(g) An appeal shall not be granted under the following circumstances:

1. It was not timely received;

2. It was not submitted by the district superintendent;

3. It would not result in a different grade, if granted;

4. It relies upon data that the district had the opportunity to correct but failed to do so, under the process described in paragraph (7)(c) of this rule or the data reporting processes as defined in Rule 6A-1.0014, F.A.C., Comprehensive Management Information Systems.

Question 3. Are all of the following statements true? Each must be true for an appeal to be granted: The appeal and supporting documentation were uploaded to the district’s school grades ShareFile folder no later than: Monday, July 31, 2017.

The appeal was submitted by the district superintendent. The changes requested will result in a different grade/improvement rating, or release of a school’s grade/improvement rating that is currently listed as an “I” due to fewer than the required percentage of students tested, if granted.

The appeal does NOT contain a request to correct data that should have been corrected during survey reporting or updated during the web application processes.

The appeal does NOT request changes that would contradict law or rule.

Data Submission for the Appeal (without this information we cannot evaluate the appeal appropriately)

The appeal includes the district of enrollment (DISTENRL), school of enrollment (SCHLENRL), and student ID number (SID) from the INDV file for each student included in the appeal.

The appeal includes specific and clear details about what changes are being requested for each student included in the appeal. The student level data submitted for the appeal has been discussed with the Bureau of Accountability Reporting and a format has been agreed upon, which will provide the data needed to evaluate the appeal. (We recommend this be done as early in the appeals window as possible, giving the district as much time as necessary to provide useful data.)

53

www.FLDOE.org

District School Number: District Name: School Name: School Grades Calculation School Grade Appeals Calculation

School Grades Components Sum of the

Numerator from the INDV file

Sum of the Denominator from

the INDV file Component

Value Recalculated

Appeal Numerator Recalculated Appeal

Denominator Recalculated

Appeal Component Value

English Language Arts Achievement

1-A 1-B 1-C 1-D 1-E 1-F

English Language Arts Learning Gains

2-A 2-B 2-C 2-D 2-E 2-F

English Language Arts Learning Gains of the Lowest 25%

3-A 3-B 3-C 3-D 3-E 3-F

Mathematics Achievement 4-A 4-B 4-C 4-D 4-E 4-F

Mathematics Learning Gains 5-A 5-B 5-C 5-D 5-E 5-F

Mathematics Learning Gains of the Lowest 25%

6-A 6-B 6-C 6-D 6-E 6-F

Science Achievement 7-A 7-B 7-C 7-D 7-E 7-F

Social Studies Achievement 8-A 8-B 8-C 8-D 8-E 8-F

Middle School Acceleration 9-A 9-B 9-C 9-D 9-E 9-F

Graduation Rate 2015-16 10-A 10-B 10-C 10-D; cannot be appealed.

10-E; cannot be appealed.

10-F

College and Career Acceleration 2015-16

11-A 11-B 11-C 11-D 11-E 11-F

Total Points Earned Recalculated Appeal Total Points Earned

Total Components Recalculated Appeal Total Components

Percent of Total Possible Points Recalculated Appeal Percent of Total Possible Points

2017 Grade Recalculated Appeal 2017 Grade

Percent Tested 12-A 12-B 12-C 12-D 12-E 12-F

*If adding or removing students to the gain calculation, all students must be re-ranked for the gains in the lowest 25%. Adding and removing students from the lowest 25% without re-ranking can cause inaccurate results.

54

www.FLDOE.org

District School Number: District Name: School Name: School Improvement Rating Calculation School Improvement Rating Appeals Calculation

School Improvement Rating Components

Sum of the Numerator from

the INDV file

Sum of the Denominator from

the INDV file Component

Value Numerator Denominator Component Value

English Language Arts Learning Gains

1-A 1-B 1-C 1-D 1-E 1-F

Mathematics Learning Gains

2-A 2-B 2-C 2-D 2-E 2-F

Total Points Earned Recalculated Appeal Total Points Earned

Total Components Recalculated Appeal Total Components

Percent of Total Possible Points Recalculated Appeal Percent of Total Possible Points

2017 School Improvement Rating Recalculated 2017 School Improvement Rating

Percent Tested 3-A 3-B 3-C 3-D 3-E 3-F

55

www.FLDOE.org

Areas that could use improvement• Marking students as additional school year

students on the Student Demographic format, Additional School Year Student data element.

• Not doing this could cause a school to be collocated when it shouldn’t be.

• EOC Course enrollment as reported on Student Data Updates.

• Reporting Student ID consistently across all formats and surveys.

56

www.FLDOE.org

Part 2 – Graduation Rate Process for 2016-17

•Review the methodology•Explain the review process

57

www.FLDOE.org

New for 2016-2017 Graduation Rate• HB 7069 states

3. A high school must include a student in its graduation rate if the student transfers from then high school to a private school with which the school district has a contractual relationship.

• A new withdrawal code will be created.• In 2016-2017 any student that meets this criteria needs to be

identified by the district using the new cohort corrections application.

• The new withdrawal code will be collected on surveys starting the 2017-2018 reporting year.

58

www.FLDOE.org

Federal Uniform Graduation Rate• All students must be accounted for.• Only standard diplomas count.• Does not remove transfers to adult education

programs (remain in denominator).• Assigns DJJ students back to their most recent regular

high school.

59

www.FLDOE.org

Federal Uniform Graduation Rate

• Key for the 1617 Graduation Rate• Year0 – 1213• Year1 – 1314• Year2 – 1415• Year3 – 1516• Year4 – 1617

60

www.FLDOE.org

Formats Used for the Cohort Build• Final Survey Data

• Format: Student Demographic Information• Survey 2/Year1• Survey 5/Year1-Year3

• Format: Student Course Transcript Information• Survey 2/Year1

• Format: End of Year Status• Survey 5/Year0-Year3

• Format: Prior School Status/Student Attendance• Survey 5/Year1-Year3

• Format: Exceptional Student• Survey 2/Year1• Survey 5/Year1-Year3

• Format: Federal/State Indicator Status• Survey 2/Year1• Survey 5/Year1-Year3

61

www.FLDOE.org

Formats Used for the Cohort Build

• Snapshot Survey Data• Format: Student Demographic Information

• Survey 5/Year4• Format: End of Year Status

• Survey 5/Year4• Format: Prior School Status/Student Attendance

• Survey 5/Year4• Format: Exceptional Student

• Survey 5/Year4• Format: Federal/State Indicator Status

• Survey 5/Year4

62

www.FLDOE.org

Building the Unadjusted Cohort

Key Data Elements• Student Number Identifier, Florida • School Number, Current Enrollment• Withdrawal Code• Diploma Code• Withdrawal Date• Grade Level

63

www.FLDOE.org

Building the Unadjusted Cohort

• Include all first-time 9th graders in fall 2013 membership in your district (Year1).

• From Survey 2 Demographic and Course.

• Add incoming transfers on the same schedule to graduate found in Survey 5 Demographic and End of Year StatusoNew 9th graders in Year1oNew 10th graders in Year2oNew 11th graders in Year3oNew 12th graders in Year4

64

www.FLDOE.org

2016-2017 Process

• A 3-year file was made available in April 2017.• Prelim 3 Year GradRate File 1617 DIST## (ShareFile)

• Survey 5 will be pulled for the initial cohort purposes in October.

• The deadline for Federal Reporting and other important metrics is October 6, 2017

• Initial 4-year cohort file will be created immediately following the initial pull.

• New Graduation Rate Cohort Corrections web application will begin once the cohort file has been created.

65

www.FLDOE.org

2016-2017 Process• Results will be available on ARM’s High School

Graduation Rates EDStats Tool:• https://edstats.fldoe.org/• Click on the green button for “PK-12 Public Schools.”• Click on High School Graduation Rates under “Interactive

Reports - PK-12.”

66

www.FLDOE.org

2016-2017 Review ProcessGraduation Rate Review Process

• Starting with the 2015-2016 calculation, districts had the opportunity to review district and school graduation rate numerator and denominator prior to the close of the cohort corrections process.

• This process will be different in 2016-2017, as the numerator and denominator will be available on the web application and will update throughout the process.

• The new web application for the Graduation Rate Cohort Corrections process will serve as the review process.

67

www.FLDOE.org

Part 3 –Accountability Match Process

•2017-2018 Changes•Basic Information: Student Database Reporting

68

www.FLDOE.org

2017-2018 Changes• In May 2017 the State Board of Education approved the

shortening of Survey reporting periods. • This was announced at FAMIS in June.• Survey 2:

• Survey Week: October 9-13, 2017 • Due Date: October 20, 2017 • State Processing: October 16 – November 3, 2017 • Final Update/Amendment Date: December 15, 2017

• Survey 3: • Survey Week: February 5-9, 2018 • Due Date: February 16, 2018 • State Processing: February 12 – March 2, 2018 • Final Update/Amendment Date: April 15, 2018

69

www.FLDOE.org

2017-2018 Changes• What does this mean for the Accountability process?

• Survey 2 for accountability purposes will be final in December 2017.• Districts will no longer be able to make corrections to Survey 2

during the accountability match process, which happens during the Survey 3 state processing window.

• Bureau of Accountability Reporting will be providing files during Survey 2 state processing.

• Deleted Records for Survey 2• Survey 2 for Accountability Purposes

• Separate manuals will be provided for the Survey 2 accountability process and the Survey 3 accountability process.

• Final Survey 3 data will be used.• Applications using Survey 3 data will not open before April 15,

2018.

70

www.FLDOE.org

Florida DOE Student Database Records Reporting for Accountability Match Process

• Essential for calculating and reporting school and district accountability outcomes.

Critical Functions:• Allows for matching of membership records (Survey 2 and 3) to establish

full-year enrollment.• Allows for matching of membership records to assessment records.• Determines race/ethnicity classification.• Determines lunch status.• Determines English language learner status and length of time in school in

the U.S. • Determines the status of students with disabilities.• Identifies students enrolled with Virtual Instruction Providers.

71

www.FLDOE.org

Student Database ReportingSurveys 2 & 3:• Key record formats:

- Student Demographic Information - Exceptional Student - English Language Learner Information - Federal/State Indicator Status - Prior School Status/Student Attendance - Student Course Schedule

72

www.FLDOE.org

Student Database Reporting Student Demographic Information:• Critical for the matching of membership records

(Survey 2 and 3) to establish full-year enrollment.

Student Demographic - Key Data Elements:• School of Enrollment (School Number, Current

Enrollment): The school to which assessment scores of students are credited.

• Lunch Status• Race/Ethnicity

73

www.FLDOE.org

Student Database Reporting

Student Demographic - Key Data Elements (continued):• English Language Learners, PK-12• Student Number Identifier, Florida• Student Number Identifier – Alias, Florida• Florida Education Identifier (FLEID)• Student Name• Birth Date• Grade Level

74

www.FLDOE.org

Student Database Reporting

Student Demographic - Key Data Elements (continued):• Additional School Year Student• District Number, Zoned School

• Reported on Survey 2 and Survey 3• School Number, Zoned School (home-zoned school)

• Reported on Survey 2 and Survey 3• English Language Learners: Date Entered United States School

• Establishes the date on which a student entered school in the U.S. Length of time from the initial date of testing (FSA Writing) affects eligibility for inclusion in school grades proficiency components.

• If this date is not reported then the student will be eligible to be included in the calculation of the achievement and learning gains components.

75

www.FLDOE.org

Student Database Reporting

Exceptional Student - Key data elements:• Primary Exceptionality (Exceptionality, Primary)• Other Exceptionality (Exceptionality, Other)• Exceptional Student, IDEA Educational Environments

• Used to help determine ESE Center Schools.

English Language Learner Information:• English Language Learners: ESOL Entry Date

76

www.FLDOE.org

Student Database Reporting

• Prior School Status/Student Attendance:• Withdrawal Date • Withdrawal Code• Entry Code • Entry Date

• Federal/State Indicator Status:• Dropout Prevention/Juvenile Justice Programs

• Provides a code indicating whether the student is in a dropout retrieval program (code R) or is in an alternative to expulsion program (code E). For students reported with either of these codes who are enrolled in an alternative school or ESE center, their test scores are not included in school improvement ratings, school grades, or district grades.

77

www.FLDOE.org

Student Database Reporting Student Course Schedule:• A student must have at least one course reported at the school of

enrollment in order to be included in the accountability process.• Two exceptions are

• Dual Enrollment (Dual Enrollment Indicator Code of A, B, C, or E)• Hospital Homebound (Exceptionality of M)

• Key for compilation of EOC course records (Surveys 4, 1, 2, and 3):- Percent Tested- EOC inclusion in all components- Middle School Acceleration

• Accountability Reporting does not receive the student’s records unless a course record and a demographic record are submitted.

• Also identifies Virtual Instruction Providers (VIPs).

78

www.FLDOE.org

Student Database ReportingStudent Course Schedule:• Virtual Instruction Provider (data element)

• The providers themselves do not have the ability to report data to the Department of Education’s student database; the providers must depend on districts to accurately report enrollment for them.

• Provider codes must be reported correctly in Surveys 2 and 3 to ensure provider accountability.

79

www.FLDOE.org

Reporting Enrollments for Virtual Instruction Providers (VIPs)

• Full-time VIP students are assigned the 4-digit school number 7001.

• Provider codes are assigned to approved providers.• Provider Code Table is available in Appendix CC of

student database manual.• For students with a school number reported as 7001,

the provider code(s) should be reported accurately on the applicable Student Course records.

80

www.FLDOE.org

Reports Available on ShareFile for 2017-2018

• Deleted Records (deletion reasons)• M = McKay student (3518)• H = Home education student (N998)• P = Private school student (N999)• D = Other District, Instruction• E = Migrant Non-Attendees (9997)• C = No course record was found for this student• A = Additional School Year student• V = 7001 but no provider • W = Withdrew prior to survey week

• Matched Records• Matched by School• Matched by District

81

www.FLDOE.org

Reports Available on ShareFile for 2017-2018

• Available during Survey 2 State Processing:• Survey 2 for Accountability• Survey 2 Deleted Records

• Available during Survey 3 State Processing:• Survey 3 for Accountability• Unmatched Survey 3 Records• Unmatched Survey 2 Records• Survey 3 Deleted Records• Matched Records

82

www.FLDOE.org

Virtual Instruction Providers Reports

• State Virtual Providers will be provided their own reports.

• Districts will no longer need to provide reports to the VIPs.• State VIPs will still be responsible for communicating to the

districts which changes need to be made.• Districts will still be responsible for making changes

requested by the VIP.

83

www.FLDOE.org

Highlights• Reports will be available on ShareFile during Survey 2 state

processing and during Survey 3 state processing.• A detailed instruction manual will be available for each

process.• Students with withdrawal dates prior to the survey week will

be added to the Deleted Records file.• In order for a student to be included in the accountability

process they must be taking at least one course at the school of enrollment.

• Two exceptions only:• Dual Enrollment• Hospital Homebound

84

www.FLDOE.org

ShareFile Policies• ShareFile is for data transfer only, not data storage. • Bureau of Accountability Reporting will be

implementing the new procedures:• During an open process files will remain on ShareFile until

the process has closed.• Survey 2 and Survey 3 Accountability Match processes• Appeals (pre-appeals INDV file)

• One time posting of files will remain on ShareFile for no more than two weeks

• 3 year prelim graduation rate• Post appeals INDV file• Please pull files down within this window

• We will be happy to repost files upon request but the files will only be available until the close of business the day of the request. 85

www.FLDOE.org

Questions and Answers

Questions concerning Appeals and Graduation Rate can be directed to the Bureau of Accountability Reporting

at evalnrpt@fldoe.org or (850) 245-0411

86

www.FLDOE.org

2016-17 Graduation Rate Cohort Corrections

Accountability Web ApplicationPresented by:

Danielle BoudreauxDistrict Communications Liaison

Bureau of Accountability ResearchOffice of Accountability and Policy Research

Division of Accountability, Research, and Measurement87

www.FLDOE.org

Login using Single Sign On

88

www.FLDOE.org

District Home Page/School List Page

Headers:• Select• School Number• School Name• Unadjusted Cohort • Withdrawn to Another District• Withdrawn Out-of-State• Withdrawn to Private School• Withdrawn to Home Education• Deceased • Still Enrolled (K-12)

• Transferred to Adult Education• Earned Special Diploma• Earned Certificate of Completion• Earned a GED• Other Nongraduates• Dropouts• Graduates• Adjusted Cohort• Total Graduates/Adjusted Cohort• DJJ

89

www.FLDOE.org

District Statewide Duplicates Tab

• Only students enrolled in user’s district during the cohort.• Most recent flag indicates if the record is the most recent record (i.e., the top record in

the Statewide Dups file).• Not available to school users.Headers:

• Most Recent Record• School Number• School Name• Student Last Name• Student First Name• Student ID• Alias ID• Date of Birth• Sex

• Race• Grade Level• Withdrawal Code• Withdrawal Date• Withdrawal Definition• Statewide Duplicate• DJJ• Adjusted Cohort

90

www.FLDOE.org

School Page/Student List Page

• Each School Page has the same five tabs for both school and district users.• All Unadjusted Cohort• Transfers Out of Cohort• Nongraduates• Graduates• View Records with Upload Errors

Headers:• Select • School Number• School Name• Student Last Name• Student First Name• Student ID

• Alias ID• Date of Birth• Sex• Race • Grade Level• Withdrawal Code

• Withdrawal Date• Withdrawal Definition • Statewide Duplicate• DJJ• Adjusted Cohort

91

www.FLDOE.org

Add New Student Form

• Only available to district users.• Should only be used in those cases where a student who was

removed in a prior year needs to be added.

92

www.FLDOE.org

Student Edit Form

• Shows original withdrawal codes and dates from the preliminary 4-year file alongside updated codes and dates.

• Withdrawal date box will appear when withdrawal code entered.

• DEL, DUP, and NOT can still be used.• Validation rules will apply.

93

www.FLDOE.org

Other Functionality• Automatically updates graduates and adjusted cohort

counts.• Search

• Works the same as other applications• Searches within district and current application only

• Uploads• Can only upload changes to existing records• Adding records can only be done manually

• Reports:• One report available for school users

• All Unadjusted Cohort• Two reports available for district users

• Statewide Duplicates (no student IDs or alias IDs will be provided)• All Unadjusted Cohort

94

www.FLDOE.org

DJJ Accountability Rating System

Presented by:Jason GaitanisBureau Chief

Bureau of Accountability ResearchOffice of Accountability and Policy Research

Division of Accountability, Research, and Measurement95

www.FLDOE.org

Agenda• DJJ Accountability Rating System Background• DJJ Measure Walkthrough• DJJ Simulation and Rule Development Timeline

96

www.FLDOE.org

DJJ Accountability System Rating Background

97

www.FLDOE.org

Rule Authority

Section 1003.52(16), F.S., Educational services in Department of Juvenile Justice programs• When establishing this rule, the Department of

Education must consult with• Department of Juvenile Justice • District school boards • Providers

98

www.FLDOE.org

System Requirements

• Rating must be based on objective and measurable student performance measures that evaluate a student’s educational progress

• Limited to prevention, day treatment and residential programs

• Measures must be based on appropriate outcomes for all students in juvenile justice education programs

• Must take length of stay into consideration

99

www.FLDOE.org

Measure Requirements

Performance measures shall include outcomes that relate to: • Student achievement of career education goals• Acquisition of employability skills• Receipt of a high school diploma or its equivalent• Grade advancement• The number of CAPE industry certifications earned

100

www.FLDOE.org

DJJ Measures

101

www.FLDOE.org

Rating Framework

• Three ratingsa) Commendableb) Acceptablec) Unsatisfactory

• Similar to school improvement rating system

102

www.FLDOE.org

Rating System

• Up to 11 components included• Only components for which a program has

sufficient data are included• All are based on a percentage calculation• For most components students must have

been in the program for at least 40 days to be included in a measure (s. 1003.52(5), F.S.)

• Additional requirements for some components

103

www.FLDOE.org

1 - Attendance Rate

The percentage of students who showed improvement in their attendance rate following re-enrollment• Based on an exit cohort• Only includes students who were in the

program for at least 40 days• Must have been enrolled in school prior to and

following release from the DJJ school• Based on days attended of those enrolled • Credit awarded if at 95% upon return,

regardless of prior rate

104

www.FLDOE.org

Currently Proposed Cuts for Attendance

Program Type

Commendable Acceptable Unsatisfactory

Prevention 100% to 60% 59% to 35% 34% to 0%

Intervention 100% to 60% 59% to 35% 34% to 0%

Non-Secure Residential

100% to 60% 59% to 35% 34% to 0%

Secure Residential

100% to 60% 59% to 35% 34% to 0%

105

www.FLDOE.org

2 - Graduation Rate

The percentage of students whose last school attended was a DJJ school who graduated or earned a GED within 5 years of entering 9th grade• Based on a 9th grade entry Graduation Cohort• DJJ school must have been the last school they

attended or the prior to last school where the last school had less than 30 days of attendance

• Includes an additional year follow up for Graduation/GED status

106

www.FLDOE.org

Currently Proposed Cuts for Graduation Rate

Program Type

Commendable Acceptable Unsatisfactory

Prevention 100% to 50% 49% to 15% 14% to 0%

Intervention 100% to 50% 49% to 15% 14% to 0%

Non-Secure Residential

100% to 50% 49% to 15% 14% to 0%

Secure Residential

100% to 50% 49% to 15% 14% to 0%

107

www.FLDOE.org

3 - Certified Teachers

The percentage of core courses taught by teachers certified for the subject area• Only includes core courses as identified by

the Course Code Directory

108

www.FLDOE.org

Currently Proposed Cuts for Certified Teachers

Program Type

Commendable Acceptable Unsatisfactory

Prevention 100% 99% to 50% 49% to 0%

Intervention 100% 99% to 50% 49% to 0%

Non-Secure Residential

100% 99% to 50% 49% to 0%

Secure Residential

100% 99% to 50% 49% to 0%

109

www.FLDOE.org

4 - Postsecondary Enrollment

The percentage of students attending postsecondary institutions• Based on an exit cohort• Only includes students who were in the

program for at least 40 days• Only includes students with a diploma or

GED at the time of exit• Enrollment can be anytime within one year

of exit

110

www.FLDOE.org

5 - Employment

The percentage of students employed• Based on an exit cohort• Only includes students who were in the

program for at least 40 days• Only includes students who were at least 16

at the time of release and not attending a post-secondary institution

• Employment can be anytime within one year of exit

111

www.FLDOE.org

Currently Proposed Cuts for Employment

Program Type

Commendable Acceptable Unsatisfactory

Prevention 100% to 80% 79% to 50% 49% to 0%

Intervention 100% to 70% 69% to 30% 29% to 0%

Non-Secure Residential

100% to 70% 69% to 30% 29% to 0%

Secure Residential

100% to 70% 69% to 30% 29% to 0%

112

www.FLDOE.org

6 - Learning Gains – ELA

The percentage of students who make learning gains on ELA assessments• Only includes students who were in the

program for at least 40 days prior to the assessment

• Learning gains will be similar to the school grades calculations for school improvement ratings.

113

www.FLDOE.org

7- Learning Gains - Mathematics

The percentage of students who make learning gains on Mathematics assessments• Only includes students who were in the

program for at least 40 days prior to the assessment

• Learning gains will be similar to the school grades calculations for school improvement ratings.

114

www.FLDOE.org

8 - CAPE

The percentage of students who earn a CAPE Industry Certification (High School) or Digital Learning Tools (Middle School) Certificate• Only applies to programs with a designed LOS of 9

months or longer• Based on an exit cohort• Only includes students who were in the program for

at least 40 days• Certificate can be earned anytime within one year of

exit• Currently there is limited Digital Learning Tools data

115

www.FLDOE.org

9 - Common Assessment Learning Gains - Reading

The percentage of students who make learning gains on the DJJ Common Assessment• Based on an exit cohort• Only includes students who were in the program for at

least 40 days• Student must have both a pre- and a post- assessment• Learning gain means student increases a level or

maintains if already at the highest level• The new Common Assessment went live November 2016• A full school year’s worth of data, therefore, for all 11 measures

to use to develop recommended cut scores will not be available until July 2018. This will delay rule adoption until Fall 2018 and rollout of first scores under the approved system until Spring 2019.

116

www.FLDOE.org

10 - Common Assessment Learning Gains - Math

The percentage of students who make learning gains on the DJJ Common Assessment• Based on an exit cohort• Only includes students who were in the

program for at least 40 days• Student must have both a pre- and a post-

assessment• Learning gain means student increases a level or

maintains if already at the highest level

117

www.FLDOE.org

11 - Data Integrity

The percentage of students who have pre- and post-test data on the common assessment• Based on an exit cohort• Only includes students who were in the

program for at least 40 days• Based on new assessment, so data for this

measure will not be available until 2016-17

118

www.FLDOE.org

Procedures for Calculating Scores and Ratings

• Three ratings include Commendable, Acceptable and Unsatisfactory

• Components will be rated on a 3-point scale• Rating will be the simple (unweighted) average of

the components with sufficient data• If a DJJ education program doesn’t have sufficient

data to generate a rating for three years in a row, the prior three years of data will be aggregated to produce a rating

119

www.FLDOE.org

State Board of Education Will Need to Establish the DJJ Ed. Program Grading Scale

• Measures are classified before computing final rating to standardize scale among measures to promote fairness

• Classification scale for each measures is based on actual relative performance among program type

• Classification scale is set for each measure by program type independently

120

www.FLDOE.org

Overall Ratings

Program Type

Commendable Acceptable Unsatisfactory

Prevention 3.0 to 2.5 2.4 to 1.6 1.5 to 1.0

Intervention 3.0 to 2.5 2.4 to 1.6 1.5 to 1.0

Non-Secure Residential

3.0 to 2.5 2.4 to 1.6 1.5 to 1.0

Secure Residential

3.0 to 2.5 2.4 to 1.6 1.5 to 1.0

121

www.FLDOE.org

Measures and Simulation Plan• The rating is based on the percentage of total points earned, and programs are graded

based only on the components for which they have sufficient data• Measures are classified on a 3 point scale before being combined• Cut-scores used to determine classification are specific to each program type/measure

combination

122

www.FLDOE.org

What’s Next• Fall 2017/Winter 2018 –

• DOE completes simulation 2 and 3 of the DJJ ratings, based on a cohorts of students served during 2014-15 and 2015-16 and the follow-up years of 2015-16 and 2016-17, adding FSA ELA, FSA Mathematics, Industry Certifications, and Employment measures to the calculation.

• Spring/Summer 2018 –• DOE Gathers requirements and begins development of processes

for programs to make limited data updates corrections similar to those allowed for school grades and school improvement ratings.

123

www.FLDOE.org

What’s Next• Fall 2018/Winter 2019 –

• DOE completes informational baseline Year One DJJ ratings, based on a cohort of students served during 2016-17 and the follow-up year of 2017-18, including the 8 existing measures for which complete data are available, and a partial year of data (November 2016-June-2017) of data for the 3 measures based on the Common Assessment.

• Rule development process resumes with public workshops held to seek input on Rules 6A-1.099812, and 6A-1.099813.

• Recommendations submitted to the State Board of Education for approval.

• Spring 2019 –• DOE publishes informational baseline 1st year DJJ ratings using

approved cut scores.

124

www.FLDOE.org

Questions

Please email the Bureau of Accountability Reporting at evalnrpt@fldoe.org

125

www.FLDOE.org

www.FLDOE.org