Oregon’s Alternate Oregon’s Alternate Assessment: Past, Present, Assessment: Past, Present,

and Future Tenseand Future Tense

Oregon Department of Oregon Department of EducationEducation

Dianna Carrizales, PhD

Office of Student Learning and PartnershipsOffice of Assessment and Instruction

In conjunction withUniversity of Oregon: Behavioral Research and Teaching

Group

In the Beginning In the Beginning

• Accepting the Accepting the NCLBNCLB challengechallenge– Meaningful assessment of students with Meaningful assessment of students with

significant cognitive disabilitiessignificant cognitive disabilities

• Defining the population Defining the population – Functional life skills Functional life skills – Some academic skills Some academic skills

The Evolution of a System The Evolution of a System

• 2001-20052001-2005 • 2006-20072006-2007

10 CIM

8

7

6

5

4

3

10 CIM

8

7

6

5

4

3

Extended CLRAS

Exte

nd

ed

A

ssessm

en

t

Scaff

old

ed

A

dm

inis

trati

on

Accom

mod

ate

d A

ssessm

en

t

Gen

era

l A

ssessm

en

t

Targ

ete

d A

ssessm

en

t

Juri

ed

Assessm

en

t

Accom

mod

ate

d A

ssessm

en

t

Mod

ified

Assessm

en

t

Extended K - 2

Change and RefinementsChange and Refinements

• Accepting the Accepting the peer review challengepeer review challenge– Grade level contentGrade level content– Comparability of formsComparability of forms– Technical documentationTechnical documentation– Sound alternate achievement standards Sound alternate achievement standards

based on research-based and based on research-based and documented proceduresdocumented procedures

• General AssessmentGeneral Assessment

• General Assessment with General Assessment with accommodationsaccommodations

• Extended AssessmentExtended Assessment– ElementaryElementary– MiddleMiddle– HighHigh

• Scaffold Administration option of Scaffold Administration option of Extended AssessmentExtended Assessment

Current Conclusion Current Conclusion (2007-(2007-2008) 2008)

• 2007-20082007-2008

10 CIM

8

7

6

5

4

3

Exte

nd

ed

A

ssessm

en

t

Scaff

old

ed

A

dm

inis

trati

on

Accom

mod

ate

d A

ssessm

en

t

Consider General Assessment (with Consider General Assessment (with or without accommodations) ifor without accommodations) if

• Student:Student:– Performs at or around grade levelPerforms at or around grade level– Has academic difficulties that primarily surround reading but Has academic difficulties that primarily surround reading but

may be average or close to average in other subject areasmay be average or close to average in other subject areas– Has academic difficulties in areas other than reading that are Has academic difficulties in areas other than reading that are

“mild to moderate” and can typically be addressed by using “mild to moderate” and can typically be addressed by using simplified languagesimplified language

– Is reading within two to three grades of his or her enrolled Is reading within two to three grades of his or her enrolled levellevel

• Instruction:Instruction:– Is primarily general curriculum instruction (but may also use a Is primarily general curriculum instruction (but may also use a

specialized curriculum in some areas)specialized curriculum in some areas)

Consider Extended Assessment Consider Extended Assessment ifif• Student:Student:

– Performs well below grade levelPerforms well below grade level– Is significantly below grade level in readingIs significantly below grade level in reading– Has academic difficulties that are generalized (to all subject Has academic difficulties that are generalized (to all subject

areas) and are significantareas) and are significant– Benefits from specialized individual supportsBenefits from specialized individual supports

• Instruction:Instruction:– Is primarily a specialized curriculum or Is primarily a specialized curriculum or – From general curriculum must be significantly reduced in From general curriculum must be significantly reduced in

breadth, depth, and complexity breadth, depth, and complexity

Consider Scaffolded Administration Consider Scaffolded Administration of Extended Assessment if:of Extended Assessment if:

• Student:Student:– Performance is significantly impacted by a disability– Does not read– Has academic, mobility, and receptive and expressive

language difficulties that are generalized and significant– Relies on individual and significant supports to access reduced

content materials

• Instruction:Instruction:– Is from a specialized curriculum and has functional Is from a specialized curriculum and has functional

components components and/orand/or– Includes academic goals that are significantly reduced in Includes academic goals that are significantly reduced in

depth, breadth, and complexity from grade level contentdepth, breadth, and complexity from grade level content

Participation by Subject Participation by Subject (2006-(2006-2007)2007)

Subject Elementary Middle/High

Total number of students participating per subject area

Total number of elementary

participants per subject

Percentage of elementary

participants by subject

Total number of middle/high

participants per subject

Percentage of middle/high

participants by subject

Reading 5444 3178 58%* 2266 42%

Mathematics 4720 2483 53% 2237 47%

Writing 2174 1006 46% 1168 54%

Science 1562 552 35% 1010 65%

*Read: 58% of students taking an Extended Reading Assessment were elementary students.

Participation by Gender Participation by Gender (Reading(Reading

exemplarexemplar))

Gender Elementary Middle/High

N % N %

Female 1077 34%* 827 36%

Male 2101 66% 1439 64%

*Read: 34% of the students taking Elementary Extended Reading are female.

2006-2007 Extended 2006-2007 Extended Assessment Participation by Assessment Participation by DisabilityDisability• ElementaryElementary

– Mental Retardation (25.5%)Mental Retardation (25.5%)– Specific Learning Disabilities Specific Learning Disabilities

(20.75%)(20.75%)– Autism Spectrum Disorders Autism Spectrum Disorders

(16.5%)(16.5%)– Communication Disorder Communication Disorder

(12.25%)(12.25%)

• Middle/HighMiddle/High– Mental Retardation (42%)Mental Retardation (42%)– Autism Spectrum Disorder Autism Spectrum Disorder

(18%)(18%)– Specific Learning Disabilities Specific Learning Disabilities

(12.25%)(12.25%)– Other Health Other Health

Impairments(9.5%)Impairments(9.5%)

Extended Assessment Extended Assessment DescriptionDescription

• Performance assessmentPerformance assessment• Individually administeredIndividually administered• Response options or selected response Response options or selected response

(not multiple choice)(not multiple choice)• Scaffold vs. standard administrationsScaffold vs. standard administrations

– Direct instruction principlesDirect instruction principles– ““Multiple choice” optionsMultiple choice” options– Accommodations research Accommodations research

• Based on grade level contentBased on grade level content

Development of Grade-Level Development of Grade-Level Items for Oregon’s Extended Items for Oregon’s Extended AssessmentAssessment• Grade-level content standardsGrade-level content standards

– Reduction in Complexity (one slide)Reduction in Complexity (one slide)– Reduction in breadth (one slide)Reduction in breadth (one slide)– Reduction in depth (one slide)Reduction in depth (one slide)

• http://www.ode.state.or.us/teachlearn/real/newspaper/http://www.ode.state.or.us/teachlearn/real/newspaper/

• Field and pilot testingField and pilot testing– Item level piloting (with individual students)Item level piloting (with individual students)– Population piloting (filming and piloting)Population piloting (filming and piloting)

Reduction in ComplexityReduction in Complexity• Development:Development:

– Select words with the lowest number of Select words with the lowest number of syllables syllables

– Reduce number wordsReduce number words– Monitor clause structure Monitor clause structure – Allow opportunities for modeling Allow opportunities for modeling – Provide more examples when possibleProvide more examples when possible– Test rules rather than exceptionsTest rules rather than exceptions– Monitor sequencing to avoid confusionMonitor sequencing to avoid confusion

• Administration:Administration:– Appropriate pacingAppropriate pacing– Performance-neutral praise statementsPerformance-neutral praise statements

Reduction in DepthReduction in Depth

• Evaluate:Evaluate:– Alignment between depth of knowledge in Alignment between depth of knowledge in

content standards and depth of knowledge in content standards and depth of knowledge in assessment itemsassessment items

– Representation of skill-sets required by each Representation of skill-sets required by each item item • Recognition and reproductionRecognition and reproduction

• Skill or conceptSkill or concept

• Strategic thinkingStrategic thinking

• Extended thinkingExtended thinking

Reduction in BreadthReduction in Breadth

• Creation of grade bands by typical Creation of grade bands by typical developmental divisions developmental divisions – Elementary, grades 3 – 5Elementary, grades 3 – 5– Middle, grades 6 – 8Middle, grades 6 – 8– High, grades 9 – 12 (10)High, grades 9 – 12 (10)

• Vertical alignment of grade level Vertical alignment of grade level content standardscontent standards– Standards that span the Standards that span the

breadth of the grades breadth of the grades assessed assessed

• IdenticalIdentical

• ProgressiveProgressive

Grade 2 Grade 3 Grade 4

Read, write, order, model, and compare whole numbers less than 100

Read, write, order, model, and compare whole numbers less than one thousand.

Read, write, order, model, and compare whole numbers to one million, common fractions, and decimals to hundredths.

ProgressiveProgressive

IdenticalIdenticalGrade 4 Grade 5 Grade 6

Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.

Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.

Identify and/or summarize sequence of events, main ideas, and supporting details in literary selections.

Elementary Reading Elementary Reading ExampleExample

Item Content Prompts

Say: Here are some words with other words that are related.

1 Show me happy. Show me the opposite of happy.

[1 = either answer correct / 2 = both answers correct]

2 Show me over. Show me the opposite of over.

[1 = either answer correct / 2 = both answers correct]

3 Show me sea/ see . (Note: either word is correct as both sound the same). Show me another word that sounds the same. [1 = either answer correct / 2 = both answers correct]

4 Show me go. Show me another word that means go.

[1 = either answer correct / 2 = both answers correct]

5 She raised her hand so her teacher would call on her. Show me another word that means raised.

[1 = reached out her hand / 2 = lifted her hand]

Test DescriptionTest Description

• Content prompts: items reflecting content Content prompts: items reflecting content of the test based on grade-level standardsof the test based on grade-level standards

• Prerequisite skills: items taken as a lead-in Prerequisite skills: items taken as a lead-in to the test to determine level of supportto the test to determine level of support

• Access level: Permission of supports (other Access level: Permission of supports (other than accommodations) that can be than accommodations) that can be provided during the performance provided during the performance assessmentassessment

• Scaffold administration: to administer the Scaffold administration: to administer the version with additional prompts, version with additional prompts, supportive language, and graphics supportive language, and graphics

Standard SettingStandard Setting

• Book-marking selectedBook-marking selected

• 32 teachers, 3 days, board-approved32 teachers, 3 days, board-approved

• Standards set by gradeStandards set by grade

• Rigorous outcomesRigorous outcomes

• Unanswered Mathematics questions remainUnanswered Mathematics questions remain– DifficultyDifficulty– PopulationPopulation– Administration/trainingAdministration/training

Impact dataImpact data

• Outcomes by gradeOutcomes by grade

Extended Assessment Reading Extended Assessment Reading OutcomesOutcomesTable 1: Extended Reading Impact by Grade, 2006-2007

Grade

Does Not Yet

MeetNearly

Meets Meets ExceedsMeets or

Exceeds

3 19% 18% 36% 26% 62%

4 29% 15% 32% 24% 56%

5 30% 18% 35% 17% 52%

6 30% 15% 31% 24% 54%

7 32% 18% 38% 12% 50%

8 38% 19% 31% 13% 43%

10 39% 24% 25% 13% 37%

Extended Assessment Extended Assessment Mathematics OutcomesMathematics OutcomesTable 2: Extended Mathematics Impact by Grade, 2006-2007

Grade

Does Not Yet

MeetNearly

Meets Meets ExceedsMeets or

Exceeds

3 28% 36% 24% 12% 36%

4 52% 12% 15% 21% 36%

5 54% 17% 19% 10% 28%

6 63% 23% 12% 3% 15%

7 69% 15% 14% 2% 16%

8 77% 13% 7% 2% 10%

10 78% 13% 9% 0% 9%

Extended Assessment Writing Extended Assessment Writing OutcomesOutcomes

Table 3: Extended Writing Impact by Grade, 2006-2007

Grade

Does Not Yet

MeetNearly

Meets Meets ExceedsMeets or

Exceeds

4 33% 27% 34% 6% 40%

7 67% 11% 16% 6% 22%

10 66% 3% 23% 9% 31%

Extended Assessment Science Extended Assessment Science OutcomesOutcomes

Table 4: Extended Science Impact by Grade, 2006-2007

Grade

Does Not Yet

MeetNearly

Meets Meets ExceedsMeets or

Exceeds

5 32% 23% 37% 8% 45%

8 33% 48% 11% 9% 20%

10 40% 40% 13% 7% 21%

Training structure Training structure

• 2007-20082007-2008– 8 regional trainings to train trainers (Oct-Nov)8 regional trainings to train trainers (Oct-Nov)– Held at host sites across the stateHeld at host sites across the state– 300 (2006-2007) assessors will be re-trained 300 (2006-2007) assessors will be re-trained

on the assessment, decision-making principles, on the assessment, decision-making principles, and web-training system to train others in their and web-training system to train others in their home-districtshome-districts

– Self-contained web-training siteSelf-contained web-training site– Data-base of trainersData-base of trainers– Training, self-testing, proficiency component Training, self-testing, proficiency component

monitored at a state level monitored at a state level

ChallengesChallenges

• Instructional content vs. accountabilityInstructional content vs. accountability

• Difficulty levelDifficulty level

• Working with the field toward a new Working with the field toward a new mode of thinking about assessing mode of thinking about assessing students with significant cognitive students with significant cognitive disabilitiesdisabilities

• Who is/is not being tested with this Who is/is not being tested with this assessment? assessment?

Lessons learnedLessons learned

• Expectations can be increasedExpectations can be increased• Teachers are open to research-based change that Teachers are open to research-based change that

is results orientedis results oriented• Biggest changes to the assessment since last year: Biggest changes to the assessment since last year:

– Use of Prerequisite SkillsUse of Prerequisite Skills• Biggest changes to training since last year: Biggest changes to training since last year:

– ShorterShorter• The four “P’s” The four “P’s”

– ProcessProcess– PopulationPopulation – ParallelismParallelism – Peer-reviewPeer-review

New directionsNew directions

• Possibly scaling to general Possibly scaling to general assessmentassessment

• Reducing the number of items while Reducing the number of items while increasing the depth and breadth of increasing the depth and breadth of standards coveragestandards coverage

• ReportingReporting

Contact InformationContact Information

• Dianna.Carrizales@state.or.usDianna.Carrizales@state.or.us