FLDOE 2011 Instructional Validity Study

Kati PearsonNovember 29, 2011

PDCA Instructional Cycle

PLAN

ACT

DO

CHECK

• Data Disaggregation

• Calendar Development• Direct Instructional

Focus

• Tutorials

• Enrichment

• Assessment

• Maintenance

• Monitoring

Date: November 29th, 2011

Bell work: Scale Reflection

Benchmark: 1982: Debra P. vs. Turlington

Objective: Today we will define the FLDOE Instructional Validity Study by reviewing background information and analyzing artifacts for consistency.

Essential Question: How does instructional validity connect to FCIM?

Vocabulary: Instructional Validity

Agenda:

I do: Review Background and Define Instructional Validity StudyWe do: Analyze artifactsYou do: Collect artifacts, monitor fidelity, and complete Instructional Validity Checklist

Summarizing Activity: Revisit Scale with Reflective Prompt

Homework: Review Instructional Validity Study with staff and monitor and collect artifacts to ensure fidelity.

Common Board Configuration

Scale: School-Based Instructional Validity Implementation

Learning Goals 4.0 3.0 2.0 1.0 0.0

I am able to identify, understand and articulate the alignment of curriculum, instruction, and assessment as well as, produce evidence and student data to support this practice.

In addition to Score 3.0 performance, in-depth inferences and applications of the alignment of curriculum, instruction, and assessment along with evidence and student data to support the practice.

Demonstrates ability to communicate to others knowledge and understanding of the alignment of curriculum, instruction, and assessment along with evidence and student data to support the practice.

Demonstrates knowledge and understanding of the alignment of curriculum, instruction, and assessment along with, evidence to support the practice.

With help, a partial understanding or knowledge of the alignment of curriculum, instruction, and assessment is evident.

Even with help, no understanding or knowledge of the alignment of curriculum, instruction, and assessment is evident.

I Do

• Review Background• Define Instructional Validity Study

PDCA Instructional Cycle

PLAN

ACT

DO

CHECK

• Data Disaggregation

• Calendar Development• Direct Instructional

Focus

• Tutorials

• Enrichment

• Assessment

• Maintenance

• Monitoring

Background

• In 1982, the Florida Department of Education implemented the nation's first instructional validity study.

• This study was required by the court in the Debra P. vs. Turlington case challenging the Florida high school graduation testing requirement. – African American students who failed a statewide test required for

a diploma in Florida challenged the testing requirements as racially based.

– Approximately 2% of the white seniors had not passed, compared to approximately 20% of the African-American seniors.

– http://www.fldoe.org/asp/hsap/hsap1983.asp

Outcome

• Basically, the courts held that the State of Florida could require students to earn a passing score on a graduation test only if they had been given instruction on the content covered by the test, and have multiple opportunities to meet the testing requirement.

Vocabulary Development

• The term "instructional validity" means that any student who is seeking a regular high school diploma has been given the opportunity to learn the content measured by each required assessment.

• It is NOT sufficient to substantiate that the required benchmarks have been adopted and are included as topics in textbooks (i.e., curricular validity); we must document that the content is actually taught. – The concept also applies to ESE, 504, and LEP students who

take these tests with the appropriate accommodations as well as to adult high school students.

Change in Requirements

• Based on the Debra P. vs. Turlington case, two major requirements for diploma sanction testing were established.– 1. Adequate Notice: requires that students are told

what a graduation test will cover several years before the test is implemented.

– 2. Curricular Validity: means that the schools are teaching what is being tested.

• Under Debra P., “the state must collect data to demonstrate curricular validity." (pp. xviii-xix)

Essential Question: How does instructional validity connect to FCIM?Alignment Between Curriculum, Instruction, and Assessment

Curriculum: Things in the course

Instruction: Things taught and studied

Assessment: Things on the Exam

Curriculum: Things in the course

Instruction: Things taught and studied

Assessment: Things on the Exam

Why Now?

• We’ve adopting new standards and must ensure that we are teaching those standards.

• Students need to be given sufficient notice that they will be required to learn new standards.– Next Generation Sunshine State Standards (NGSS)– Standards that apply to End of Course

Assessments (EOC).– Common Core Standards

NGSSS: Next Generation Sunshine State Standards

• The NGSSS were adopted by the State Board of Education and, thus, represent mandated state policy that all districts must implement.

• In July 2010, State Board of Education Rule 6A-1.09422 was amended to require new assessment graduation requirements based on the NGSSS and course requirements.

• The Grade 10 FCAT 2.0 Reading and EOC assessments are the required assessment instruments that measure selected aspects of the NGSSS for the purposes of high school graduation.

District Responsibility

• The Department's FCAT 2.0/EOC instructional validity study will place primary responsibility on each school and every district to document that instruction on the benchmarks measured by the FCAT 2.0/EOC assessments is provided to students across grade levels.

• While the instructional validity study may be generally successful, any individual school or district could be challenged if there is unclear evidence that students are being afforded the opportunity to learn the required course and tested content.

Major Elements

• In its most general sense, there are four questions being asked by the study:

1. Are people adequately informed? 2. Is the curriculum aligned with the NGSSS course

requirements? 3. Are students receiving instruction in the aligned

curriculum? 4. Are students provided remedial instruction if they

have not earned passing scores on the Grade 10 FCAT 2.0 Reading or EOC assessment?

Course Descriptions & Item Specifications

• Above all, teachers, parents, and students should know that all assessments items are derived primarily from the following sources.

1. Course Descriptions: FCAT 2.0 Course Descriptions: http://www.floridastandards.org/courses/coursedescriptionsearch.aspx EOC Course Descriptions: http://fcat.fldoe.org/eoc/

2. Test Item Specification DocumentsFCAT 2.0 Test Item Specifications available at: http://fcat.fldoe.org/fcat2/itemspecs.asp EOC Test Item Specifications Available at: http://fcat.fldoe.org/eoc/itemspecs.asp

We Do

• Analysis of Artifacts– Lesson plans– Common Board Configuration– Teacher Created Tests

PDCA Instructional Cycle

PLAN

ACT

DO

CHECK

• Data Disaggregation

• Calendar Development• Direct Instructional

Focus

• Tutorials

• Enrichment

• Assessment

• Maintenance

• Monitoring

Lesson Plans

• Make sure plans include NGSSS instead of SSS (Common Core for Kindergarten)

Common Board Configuration

Testing and Assessment

Test Items Should Align to Standards

• Please monitor to ensure there is not an overly excessive utilization of:- True False Items- Fill in the Blank Items- Use of Word Banks- Low complexity multiple choice items,

etc.

What Grade Level and What Standard?1. In the sentence below choose the correct homonyms.

It (would, wood) be (great, grate) if you (would, wood) put more (would, wood) on the fire, but be sure to close the (great, grate) when (you’re, your) done!

Example found on a grade 10 teacher created test.Standard: LA.910.1.6.8 • The student will identify advanced word/phrase relationships and their meanings. • Homonyms not tested and not part of the grade level standard• Content Limits: the terms synonym and antonym should not be used in stem

construction. Wording should be most similar in meaning or most opposite in meaning.

Homophones and homographs are part of the grade 5 standard.Standard: LA.5.1.6.8• The student will use knowledge of antonyms, synonyms, homophones, and

homographs to determine meanings of words.

Grade 10: LA.910.1.6.8 FCAT 2.0 Example

Read the quote from the passage.• Because most narwhal live in the

arctic regions of northern Canada, the general public is less familiar with them. . . Due to their obscurity, they have not been subject to hunters as other whales have been.

Obscurity means

A. remoteness.B. darkness.C. anonymity.D. density.

FCAT 2.0 Example

Read the sentence from the passage:• Narwhal are social animals and tend

to live in groups of three or four, though some truly gregarious narwhal have been found in pods of twenty.

What does gregarious mean?A. enjoying the company of othersB. traveling at set times of the yearC. happy to be separated from others

D. dependent on others for protection

You Do• School Responsibility

Instructional Validity Self Study– Collect artifacts and monitor fidelity– Complete Instructional Validity Checklist

PDCA Instructional Cycle

PLAN

ACT

DO

CHECK

• Data Disaggregation

• Calendar Development• Direct Instructional

Focus

• Tutorials

• Enrichment

• Assessment

• Maintenance

• Monitoring

What Should Be Collected From SchoolsExamples include, but are not limited to:• SAC agendas• Letters to parents/students• Information posted to school websites• Email notification and presentations to teachers regarding course

descriptions, test item specifications, instructional focus calendars, pacing guides, curriculum maps, benchmark assessments, etc.

• Faculty agendas that deal with instructional validity• School newspapers/newsletters• Documentation of remedial services• School process or policy on reviewing and maintaining lesson plans• Teacher signature sheet acknowledging that they have received

course descriptions and test item specifications• PLC agendas that involve the implementation of new standards• Policy for creating syllabus

Instructional Validity Checklist

Scale: School-Based Instructional Validity Implementation

Learning Goals 4.0 3.0 2.0 1.0 0.0

I am able to identify, understand and articulate the alignment of curriculum, instruction, and assessment as well as, produce evidence and student data to support this practice.

In addition to Score 3.0 performance, in-depth inferences and applications of the alignment of curriculum, instruction, and assessment along with evidence and student data to support the practice.

Demonstrates ability to communicate to others knowledge and understanding of the alignment of curriculum, instruction, and assessment along with evidence and student data to support the practice.

Demonstrates knowledge and understanding of the alignment of curriculum, instruction, and assessment along with, evidence to support the practice.

With help, a partial understanding or knowledge of the alignment of curriculum, instruction, and assessment is evident.

Even with help, no understanding or knowledge of the alignment of curriculum, instruction, and assessment is evident.

Reflective Prompt: What next steps will I now take to ensure instructional validity?