For Each

Student

Instruction

GoalsAssessment

For All Students

OR Reading First: Review of OR Reading First: Review of Comprehensive ProgramsComprehensive Programs

2

Objective of Reading FirstObjective of Reading First

(1) “To provide assistance to State educational agencies

and local educational agencies in establishing reading

programs for students in kindergarten through grade 3

that are based on scientifically based reading research to

ensure that every student can read at grade level or

above not later than the end of grade 3.”

NCLB, 2001, Part B, Sec. 1201.

3

Why Focus on Why Focus on Reading Programs NowReading Programs Now

Unprecedented convergence about what children need to be successful readers

National syntheses provide scientific evidence on which to base practice

Much classroom practice is shaped by textbooks

State standards are embracing the science

Publishers respond to the marketplace and need

Window of opportunity to align what we know, what we use, and how we teach to attain critical results

4

ComprehensiveComprehensive Reading Programs Reading Programs

Purpose: to provide sufficient instruction in the core

components of reading (enough of the “right stuff” in a systematic design)

to provide instruction that enables the majority of students to meet or exceed grade-level standards on all the key Reading First elements

to serve as the primary reading program for a school within and between grades (K-3)

5

Why Adopt A ComprehensiveWhy Adopt A Comprehensive Reading Program? Reading Program?

Increases continuity, coherence, and community of

effort within and between grades (all teachers are aware and working toward the same goals)

Creates more “buying power” regarding professional development

Affords greater differentiation of instruction for children (can share children within and between grades)

Reading instruction is rocket science (Moats, 1999) and expecting teachers to construct and instruct is unreasonable and too important to leave to chance.

6

Oregon Reading First Oregon Reading First Curriculum ReviewCurriculum Review

Focus: Comprehensive reading programs Purpose: To determine alignment with

SBRR & provide consumers guidance Curriculum Review Panel Tool/Criteria: Consumer’s Guide (Simmons

& Kame’enui) Review Process

7

Curriculum Review Panel: Curriculum Review Panel: Selection of MembersSelection of Members

Former State Superintendent of Public Instruction, Stan Bunn, invited 25 educators across the state to serve on the Curriculum Review Panel.

5 additional members from the U of O, Eugene Research Institute, and Oregon Research Institute and 3 members from ODE were identified in Oregon’s Reading First grant.

8

Curriculum Review Panel: Curriculum Review Panel: 17 final members17 final members

2 district administrators 2 district curriculum specialists 1 classroom teacher 4 university faculty from three state universities

(U of O, Pacific, Univ. of Portland) 2 doctoral students from the U of O 4 researchers from ERI 1 researcher from ORI 1 educational specialist from ODE

9

Oregon Reading First Oregon Reading First Curriculum Review Panel MembersCurriculum Review Panel Members

Julie Anderson Educational Specialist/English Language Arts, ODE Scott Baker, Ph.D Researcher, Eugene Research Institute Lynette Doht Reading Specialist, Portland Public Schools Hank Fien Research Assistant, University of Oregon Barbara Gunn, Ph.D Researcher, Oregon Research Institute Arlene Hett, Ph.D Director of Teacher Education, University of Portland Sara Johnson Principal, Henry L. Slater Elementary School Edward J. Kame’enui , Ph.D Professor, University of Oregon Kristen MacConnell Research Assistant, University of Oregon Anita McClain, Ph. D Professor, Pacific University Janet Otterstedt Research Assistant, Eugene Research Institute Sandra Pellens, Ph.D Director of Instruction, Molalla River SD 35 Michael Rebar, Ph.D Researcher, Eugene Research Instiitute Deborah C. Simmons, Ph.D Professor, University of Oregon Sylvia Smith, Ph.D Researcher, Eugene Research Institute Carrie Thomas-Beck, Ph.D Curriculum Specialist, Springfield SD 19 Deborah White Teacher of the Year, Lyle Elementary School

10

TOOL: Consumer’s Guide to TOOL: Consumer’s Guide to Evaluating a Core Reading Program Evaluating a Core Reading Program

Developers: Drs. Deborah C. Simmons and Edward J. Kame’enui, University of Oregon

Why Developed: To assist states, districts and schools in the selection of research-based tools

When Developed: As part of National Center to Improve the Tools of Educators’ scope of work (1990-2000)

Purpose: To document and quantify the design and delivery features of core reading programs.

11

Consumer’s Guide: Consumer’s Guide: OrganizationOrganization

Programs Evaluated by Grade

Within Grade by Essential Component: Phonemic Awareness Phonics Fluency Vocabulary Comprehension

12

Essential Components by GradeEssential Components by Grade

K 1 2 3

PA NA NA

Phonics

Fluency NA

Vocab.

Compre-hension

13

Consumer’s Guide:Consumer’s Guide:Organization (cont.)Organization (cont.)

For each Essential Component:High Priority ItemsDiscretionary Items

Overarching Design Items for Each Grade

14

Consumer’s Guide:Consumer’s Guide:Sample ItemsSample Items

KINDERGARTEN Phonemic Awareness

High Priority Items: #1 Progresses from the easier phonemic

awareness activities to more difficult (e.g., isolation, blending, segmentation, and manipulation) (ss)

#2 Teaches skills explicitly and systematically (w) #4 Integrates letter-sound correspondence

instruction to phonological awareness (w) #5 Focuses on segmentation or the combination of

blending and segmenting for greatest transfer (ss)

15

Consumer’s Guide:Consumer’s Guide:Sample ItemsSample Items

KINDERGARTEN Phonemic Awareness

Discretionary Items: #1 Focuses beginning instruction on the phonemic

level of phonological units with short words (two to three phonemes; e.g., at, mud, run)

#3 Focuses first on the initial sound (sat), then on the final sound (sat), and lastly on the medial sound (sat) in words.

16

Consumer’s Guide:Consumer’s Guide:Sample ItemsSample Items

KINDERGARTEN Design Features 1. Coordinates and integrates phonemic awareness

and phonics instruction and student materials. 2. Provides ample practice on high-priority skills. 3. Provides explicit and systematic instruction. 4. Includes systematic and cumulative review of high

priority skills. 5. Demonstrates and builds relationships between

fundamental skills leading to higher order skills.

17

Scoring CriteriaScoring Criteria

Use the following criteria to score each item:

= Program consistently meets/exceeds criterion

= Program partially meets/exceeds criterion

= Program does not satisfy the criterion

When evaluating individual elements, slash ( / ) the respective circle that represents your rating (e.g., ).

18

Analysis of High Priority ItemsAnalysis of High Priority Items

Type of Review

1. (w) = Within a sequence of lessons. A specified element isbest analyzed by reviewing a particular lesson or a series of 2-3successive lessons.

2. (ss) = Scope and sequence. A specified element is bestanalyzed by reviewing the programÕs scope and sequence.

3. (st) = Skills trace. A specified element is best analyzed bycompleting a skills trace.

19

Within a Sequence of Within a Sequence of Lessons Analysis (W)Lessons Analysis (W)

High Priority Items - Phonemic Awareness Instruction - K

Rating

Criterion Evidence

Initial Week __

(Unit 4)

Week __

(Unit 11)

2.Teaches skills explicitly and systematically. (w)

Unit 2, Day 2: Introduce /m/ + 1st sound isolation

Day 3 /m/ pictures and objects - 1st sound

Unit 4, Day 3: Auditory Blending

Also see Days 4 and 5.

Unit 11, Day 2: isolate final phoneme.

Day 3: final phoneme

Day 4: Final phoneme

20

Scope and Sequence Scope and Sequence Analysis (SS)Analysis (SS)

High Priority Items - Phonemic Awareness Instruction - K

Rating Criterion Evidence

Initial Instruction

Week __

(Unit 4)

Week __

(Unit 8/9)

1.Progresses from the easier phonemic awareness activities to the more difficult. (ss)

Unit 2, Day 2 & Day 3 Isolate initial /m/.

Day 4

Isolate initial /a/.

Day 3:

Intro auditory blending

Day 5: Practice auditory blending

Day 7: auditory blending

Day 4: Segmentation

Day 9: 1st sound manipulation + segmentation

Unit 9:

Day 4: segmentation

21

Skills Trace of Skills Trace of Letter-Sound Review (ST) - KLetter-Sound Review (ST) - K

Sequence, Instruction, Review

1 - Lesson/Day

2 -Lesson/Day

3 - Lesson/Day

4 -Lesson/Day

5 -Lesson/Day

Day or Lesson

Unit 5, Day 2 Unit 5, Day 3 Unit 5, Day 4 Unit 5, Day 5 Unit 5, Day 6

Instruction j y q z none

Review Cycle

Ss review all sounds.

j (X2) +

Ss review all sounds.

j, y

d, j, n, s, y

all sounds

q / k, p, r, v/

q / all sounds

k, p, r, v, g

z

q and z

Sequence, Instruction, Review

6 - Lesson/Day

7 - Lesson/Day

8 - Lesson/Day

9 - Lesson/Day

10 - Lesson/Day

Day or Lesson

Unit 5, Day 7 Unit 5, Day 8 Unit 5, Day 9 Unit 5, Day 10

Unit 6, Day 1

Instruction v x none none none

Review Cycle

all sounds j / o, b, v / v / all sounds

x

all sounds

all sounds g, h, w, c, p, l

22

Review ProcessReview Process

Announcement posted on Oregon Reading First and WOATRA’s websites inviting publishers to submit core reading programs for review. (February 14, 2003 deadline for submissions)

23

Call for Call for Comprehensive ProgramsComprehensive Programs

Comprehensive Programs: (a) include materials for all grades K-3, (b) comprehensively address the “five

essential components” of the Reading first legislation in scientifically based beginning reading instruction: phonemic awareness, phonics, reading fluency, vocabulary, and reading comprehension.

24

Comprehensive Programs Comprehensive Programs Reviewed:Reviewed:

15 programs submitted / 9 reviewed as comprehensive programs: Harcourt Houghton-Mifflin Macmillan/McGraw-Hill Open Court Reading Mastery Rigby Scott Foresman Success For All Wright Group

25

Review Process: Review Process: Program AssignmentProgram Assignment

OR CRP members provided with 6 hours of training by OR RF staff (February 21, 2003)

CRP members randomly assigned to programs to review. Each reviewer assigned to either a K/1 or 2/3 grade. No

reviewer evaluated an entire program K-3. Each member reviewed 1 to 4 programs based on

availability. CRP members were not permitted to review any program

for which they were an author, consultant, or advisor. (Reviewers signed statements of disclosure.)

Initial review completed March to May, 2003.

26

Review Process:Review Process:

Each section of a program (K/1 or 2/3) was reviewed by 2 independent reviewers.

The same two reviewers were never paired more than once.

Thus, each program was reviewed by 4 different members of the review panel.

Members spent from 8 to 30 hours to complete a program assignment (K/1 or 2/3)

All review work was completed at the OR Reading First Center.

27

Review Process:Review Process:Reliability Between ReviewersReliability Between Reviewers

Upon completion of the review, scores between the two reviewers were compared.

Each rating was assigned a point value: = 2 points = 1 point = 0 points Items that were off by one were averaged (e.g.

full circle and partial circle = 1.5) For items that were off by more than one (e.g. full

circle and empty circle), a third reviewer was asked to reconcile the items.

28

Review Process:Review Process:Third ReviewerThird Reviewer

Third reviewers assigned to “reconcile” did not conduct the initial review of the program.

Third reviewers examined the documented evidence from the 2 initial reviewers and the program materials to determine which of the two scores best represented the selected item as the final score.

29

Review Process:Review Process:Summarizing ResultsSummarizing Results

Result of the review process is one averaged/ rectified score for each item for a program.

Final Report includes a completed Consumer’s Guide for each program (item by item).

Final Report summarizes results by program, by essential components, and by grade.

For each program, results are summarized by high priority, discretionary, and design items.

30

Final Report:Final Report:Sample Consumer’s Guide Sample Consumer’s Guide

High Priority Items - Phonemic Awareness Instruction

Rating Criterion

1. Progresses from the easier phonemic awareness activities to the more difficult (e.g., isolation, blending, segmentation,and manipulation).(ss)

+ 2. Teaches skills explicitly and systematically. (w)

3. Models phonemic awareness tasks and responses orally and follows with students’ production of the task. (w)

4. Integrates letter-sound correspondence instruction to phonological awareness. (w) [NRP, pg. 2-41]

+ 5. Focuses on segmentation or the combination of blending and segmenting for greatest transfer. (ss) [NRP, pg 2-41]

31

Final Report:Final Report:Sample Consumer’s Guide (cont.)Sample Consumer’s Guide (cont.)

Kindergarten Phonemic Awareness Instruction - High Priority

Tally the number of elements with each rating.

2 2 + 1 __ + __ (2 pts) (1.5 pts) (1 pt) (.5 pts) (0 pts)

Total Points/Total Possible Points 8/10 = 80%

32

Sample: Summary of Sample: Summary of Kindergarten RatingsKindergarten Ratings

High Priority ItemsPhonemic Awareness Instruction (5) 80%

Phonics Instruction (9)

Letter-Sound Association Instruction(3)

Decoding Instruction (5)

Irregular Words Instructions (1)

75%

Vocabulary Instruction (3) 0%

Comprehension Instruction (4) 38%

Kindergarten High Priority Totals 58%

33

Next StepsNext Steps

Final report distributed to districts, schools, and publishers for guidance in program selection at IBR I in June 2003.

Final report discussed in detail on Day 3 of IBR I. Programs will be on display throughout the four

days of IBR I for school staff to examine. Schools will need to allot their professional

development funds to pay a team of educators to examine the programs more closely during the last week of June.

34

Next Steps Next Steps (cont.)(cont.)

Schools will need to notify ODE of their program selection by early July to secure approval then order materials.

Schools will select those programs that were reviewed favorably by Oregon’s RF Curriculum Review Panel.

Schools will then need to arrange for professional development on their new program early in the fall.