Southern Regional Education Board 1 Preparing for the 2014 HSTW Assessment Mike Ross Consultant,...

SouthernRegionalEducationBoard

1

Preparing for the

2014HSTW Assessment

Mike Ross Consultant, HSTW / MMGW SW Ohio

Susanne DavisHSTW Assessment Coordinator, Ohio


2

Topics Covered

What is the HSTW Assessment?

Administration Nuts & Bolts

Best Practices for Successful Assessments


3

What is the HSTW Assessment?

Required in even-numbered years Subject tests in Reading,

Mathematics and Science based on NAEP frameworks

Student SurveyCourse-taking patternsSchool and classroom experiences

Teacher Survey Provides schools with

comprehensive school-level data breaking out student achievement by experiences


Examples of How Schools Use the Results

Identify improvement needs Serves as basis for Site Review

Process (SRP) Gain support for school

improvement Engage the entire faculty in school

improvement Set priorities and goals for school

improvement Improve classroom practices

4


5

New for 2014

There will be no individual Student Reports!


6

Five Components

Reading testMathematics testScience testStudent survey

Course-taking patternsSchool and classroom

experiencesTeacher survey (online)


Testing Times

Reading 75 min. Mathematics 80 min. Science 55 min.

Total 3.5 hours

Student Survey Section 1 - staff person completes

Sections 2 & 3 - untimed (about 35-45 min.)

7


8

Test Content

Used 2009 NAEP frameworks as a starting pointModified to reflect college and career

readiness goals

Reading – focus on informational texts

Mathematics – topics through Algebra II and some pre-calculus

Science – life, physical and earth science


9

Reading Test

75 minutes80% informational texts; 20%

literary nonfictionIncludes meaning vocabularyIncludes paired texts (to assess

critical analysis)Does not include literary fiction

or poetry


10

Reading Test

Passage length: 500-1500 words

Cognitive targets15% locate/recall55% integrate/interpret30% critique/evaluate


11

Mathematics Test

80 minutes

Scientific or graphing calculators permitted for one sectionType of calculator will neither advantage

nor disadvantage student

TopicsPre-algebra through Pre-calculus,

including Algebra II


12

Mathematics Test

Content11% - Number Properties and

Operations29% - Measurement/Geometry24% - Data Analysis, Statistics

and Probability36% - Algebra


13

Mathematics Test

Cognitive Targets50% - Low Complexity

recall routine procedures

45% - Moderate Complexitymake connections within

mathematics or between mathematics and real world

5% - High Complexity

analyze a problem mathematically


14

Science Test

55 minutesContent

43% - Life science 42% - Physical science15% - Earth and space science


15

Science Test

Science Practices70%

• Identifying science principles• Using science principles

30%• Using scientific inquiry• Using technological design


16

Student Survey Section 1 – Course Experience

Staff member should complete this section for each student

Based on Course Matching and Transcript Analysis

Section 2 – School ExperienceStudents complete this sectionSchool and classroom experiencesBased on HSTW key practices

Section 3 – General InformationStudents complete this section


17

Teacher Survey

Online survey! School and classroom experiences

School missionHigh expectations and extra helpGuiding and supporting studentsCurriculum content and engaging

students in learningTransitionsLeadership and professional growth


18

Results Reporting

1. School Report (June, 2014) PDF file on CD

2. HSTW Award of Educational Achievement (by May 2, 2014)

3. Student Reports - Discontinued


19

HSTW Assessment Report

Mailed to schools June, 2014 Executive Summary -“3-minute

report”One page summary of

performance and implementation Overview -“30-minute report”

Slightly longer summary Full report -“3-hour report”

Tables breaking out achievement by experiences

Teacher survey results


20

School Report

Executive SummaryOne-page snapshotGraphs and chartsIncludes:

• Implementation summary• Percentage of students

meeting readiness goals: 250, 257, 258

• Percentage of students at each proficiency level

• Blank quadrant for state data


21


22

School Report

The OverviewApproximately 15 pagesGraphs and chartsIncludes:

• Mean scores• Percent meeting readiness goals• Percent at each proficiency level• Performance on HSTW indices• More detailed implementation

summary


23


24

School Reports

Full report

All those tables!

Student achievement broken out by survey questions

Teacher survey results


25


26

HSTW Award of Educational Achievement Award certificates mailed to

schoolsby May 2, 2014

Awarded to students who:Meet all three readiness goals

(reading 250, math 257, science 258)

Complete at least 2 of 3 HSTW recommended curriculum goals• English/language arts,

mathematics, scienceComplete a concentration

• Career/technical, mathematics/science, humanities


27

HSTW Award of Educational Achievement


28

Award of Educational Achievement

Opportunities for High Schools

Local community colleges

Local employersAdditional incentives for

students


29

HSTW Assessment Administration

Nuts & Bolts


30

2014 Student Assessment and Teacher Survey

Window

January 24– February 14(3 weeks)

Plan ahead to get the best results!


31

Conducting the Assessment Process

OrderingPreparingAdministeringReturning materials


32

Submit orders today if possible, but no later than Friday, Sept. 27.

Do NOT order online! Do NOT send order to SREB!

The good news :

There is no cost to your school! HSTW SWO pays for the entire

Assessment!

Ordering the Assessment


33

Unless there are special reasons, we encourage you to test no more than 60 students.

You will receive extra test sets. However, when submitting scan sheets to be scored, do not exceed the number ordered!

Schools testing more than 60 must pay for each additional student: $44 per student.

A cancellation fee may be charged if you cancel testing or miss the return deadline.

Other Ordering Information


34

Test Administration Guide Sampling Instructions Student Roster Form Checklist Test Description and Sample

Question Booklet Tasks and Timelines Course Description Guide Course Matching Chart

Administration Materials Online www.sreb.org


Preparing for the Assessment

Administration materials are now available online at: www.sreb.org

Booklets & scan sheets will arrive by January 17, 2014

35


36

Tasks & Timeline

Per Regional Office instructions


37

Identify Students for the Assessment Select one of two options:

1. RANDOM SAMPLE OF 60 SENIORS

- OR -

2. ALL SENIORS (ENTIRE CLASS)

Select a true random sample by using the instructions provided. You must also select five alternates at random.



38

Other Tips for Selecting Students

Only include currently enrolled seniors.

Only exclude students if their IEP exempts them from state assessments.

Same accommodations provided for state assessments should be provided for this assessment.


39

Other Tips for Selecting Students

Administrator Guide will contain suggested accommodations for English-language learners.

Contact ETS by October 25 to request large-print or audio CD versions .

Be reasonably assured that all students can complete ALL 4 components – reading, math, science, AND student survey.


40

Random Sampling Options

What is a “random sample”?

What is the easiest way to draw a random sample?

How to use random number tables


Random Sampling

Simple random sample - every time a selection is made, every student on the list has an equal chance of being the one selected.

Selection Bias – anything that contaminates or distorts the process.


What’s the easiest way to select a random

sample?

Draw names from a hat, box, bowl, or drum!

Draw until you have selected 60 + 5 names!

42


43

Group ActivityRandom Sampling

Procedures: Random Number Table

1. Develop a list of all eligible students (delete names of students who moved or have IEP’s stating they may not participate).

2. Number the remaining names of students on the list

3. Use the random number sheet to select students, starting with column one and working down.

4. The table provided works for classes of up to 350. For larger classes, obtain a larger random number table from SREB.

5. Add student names to the roster until you have the appropriate number.


CAREER CENTERS and HOME SCHOOLS:

How to handle “double-selected” students

44

Career Center selects random sample.Home School selects random sample.Center and home school should compare lists

to see if any students were “double-selected” (i.e., randomly selected by BOTH schools).

If a student is double-selected, test the student at the site where he/she takes senior academic courses.

Contact ETS (Lisa Rion – [email protected] 609-734-5646) to obtain a special student code.

Enter that special code on the double-selected student’s answer document. That student’s results will go into both reports.


45


Complete the Student Roster - VERY IMPORTANT!!

Fully complete form including assessment answer document ID numbers.

File with school testing records for five years! Also give one copy to principal.

• You will need this fully completed roster form in Spring 2015 to conduct the follow-up survey of graduates

• Also needed to identify students who fail to include their names (for student reports and awards)


Student Roster

46


47


NotificationNotify students, parents and teachersProvide information on assessment

and/or schedule informational meetingRefer to your Board of Education policy

where applicable

SchedulingSchedule time and place (SREB

recommends administering over 2-3 days)

Include make-up sessionsIdentify assistants (follow recommended

ratios)


48


Course-Taking Preparation

Work with department chairs to complete course-matching chart (may update previous work)

Prepare student transcripts (including Fall grades and Spring schedule)


49


50

Other Tips for Course-Matching1. Get each department head for English,

math and science to help with this process.

2. Include any courses the seniors have taken from grades 9-12 within English, math, science or career-technical areas.

3. Using the course description guide, match each course from your school’s master schedule to the SREB course title on the course-matching chart. Example from the course-matching guide

Regular, Advanced, or College Prep Algebra I:“Algebra I courses include the study of properties and

operations of the real number system, evaluating rational algebraic expressions …”


51


Receive Assessment Materials

Booklets & scan sheets arrive at site- by January 17, 2014

Verify contents according to instructions

Keep materials secure (locked)


52

Administering the Assessment

Prepare Assessment MaterialsAssign a numbered answer

document to each student• Record the i.d. numbers on the student

roster

A staff person should complete Student Survey Section One – “Course Experience” for each senior

– Reduces testing time– More accurate


Administering the Teacher Survey

Detailed instructions will be provided, including:

• Sample letter to teachers• Survey and log-on instructions

Teachers must enter the correct HSTW site code. Ohio site codes are 5-digit numbers beginning with “3 6”

53


54


Review Test Arrangements

Confirm dates, times, locations and assistants

Remind students, teachers and parents

Prepare Students

Practice items and use of calculators


Idea Exchange #1

What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to prepare for the Assessment?

55


56


January 24 – February 14, 2014

Schedule make-up sessions for absent students.

It is crucial that students complete ALL FOUR components!!


57

Administration Procedures

Assemble materials and set up the room.

Supervise student check-in. Supervise the session using the

administration scripts. You may answer questions about the

content of the student survey, but not the three subject tests.

Keep materials secure at all times. Schedule makeup sessions. Check i.d. information on scan sheets.


58


59

Administering the Teacher Survey

January 24– February 14, 2014Schedule early in the window!Include all full- and part-time classroom

teachers 9th-12th

RECOMMENDED – Administer during a staff meeting or other function to guarantee completion. Use computer lab.

Plan on one hour (although most teachers will finish in about 45 minutes).

Surveys are anonymous.Teachers shared by MMGW & HSTW

should take survey version for site at which they are most involved.


Idea Exchange #2

What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to administer the Assessment?

60


Educational Testing Service Test Processing Center

Ewing, New Jersey

• ETS processes thousands of tests every day!• Misplaced materials can be very hard to locate.• Use the pre-paid, bar-coded shipping labels to ensure

that your materials are delivered to the right location!

Returning Materials


62

Returning MaterialsPack according to instructions!

Use the original ETS boxes. Use the ETS pre-paid mailing labels.Return items in their designated box

or envelope:

White Box Brown Box “To Be Scored” “NOT To Be

Scored” 60 answer sheets All test booklets plus header sheet


Returning Materials

VERY IMPORTANTDo not return more student tests

than you ordered!

For example, if you ordered 60, do not return more than 60 in the

white box.

Your school will be responsible for paying $44 for each additional

answer document sent to ETS for scoring.

63


Returning Materials

Include control forms

Use the pre-paid shipping labels

Keep receipts and UPS tracking numbers

64


Returning Student Materials

1. Postmark deadline to mail materials to ETS: February 14, 2014

2. Use the prepaid labels, original boxes and the envelope provided. Include the completed survey control form.

3. If you misplace mailing materials, contact your Regional Office as soon as possible.

4. Return everything! Pack all test books AND extra scan sheets (used or unused) in original boxes and return as instructed.

5. Make sure your materials are picked up.

**KEEP YOUR UPS RECEIPT- this is the only method of tracking your materials.**

65


66

Teacher Surveys

All Teacher Surveys must be completed and submitted online by February 14,

2014.


67

Dates to Remember Orders to Regional Office

TODAY! (no later than Friday, Sept. 27) Administration materials available online

Now! Requests for large print or CD versions to

Educational Testing Serviceby October 25, 2014

Testing materials arrive at sites by January 17, 2014

Student & Teacher administration window January 24 – February 14, 2014


68

Dates to Remember

Student materials postmark deadlineFebruary 14, 2014

Award certificates and student reports mailed to sitesby May 2, 2014

School reports mailed to sitesJune, 2014


69

Top Ten Mistakes (in no particular order)

1. Returning extra student scan sheets (i.e., more than ordered)

2. Missed deadlines• holding make-up tests late in window• shipping too close to deadline• forgetting to have materials picked up

3. Site code errors • Site code zip code !• All Ohio site codes are 5-digit numbers

beginning with “3 6”

4. Incorrect shipping • shipping to wrong address• shipping in wrong box or envelope• damaging scan sheets by using tape,

rubber bands, or paper clips


70

Top Ten Mistakes (continued)

5. Failure to administer the Student Survey• No subject tests can be scored without the

Student Survey!

6. Incorrectly marking the “Assessment Status” box on student answer sheets

• Student tests marked “absent” will not be read by the scanner

7. Random sampling errors -“Frankensamples”

• Adjusting the sample to match site demographics, programs, feeders, etc.

• Picking every 3rd or 4th student• Asking for volunteers• Drawing sample from selected classes only


71

Top Ten Mistakes (continued)

8. Using incorrect test materials • using Xeroxed copies of scan sheets• using old scan sheets

9. Lost items • student roster• pre-paid mailing labels• original shipping cartons• UPS tracking numbers

10. Incomplete student information• names left blank on scan sheets• incorrect demographic information


72

Make the 2014Assessment a

Success! Communication

Letters to students and their parentsAnnouncements to faculty and staff

CooperationImportance of the assessment to the

school’s improvement goalsImportance of students, administration

and faculty taking it seriouslyHandling make-up workCooperation from teachers whose

classes might be interrupted by the assessment


73

Make the 2014 Assessment a

Success! Encourage all students to do their best.

Build enthusiasmBe honest – explain how this helps

the school and future studentsMake students feel special for

participating in the assessment and thank them for their participation


Recommended Next Steps

to Get Started1. Place order.

2. Download and review materials from www.sreb.org .

3. Assemble a small planning team.

4. Schedule dates, times, locations.

5. Announce preliminary plans to faculty and staff.

74


75

Previous participation in an assessment Name, phone and email for the assessment

contact person (will be receiving materials for a year!)

Complete shipping address (no P.O. boxes) Principal’s name, phone and email Number of students to be tested and

sampling method SWO Region pays for 60 seniors at your

site. You must pay for any additional students beyond 60 or special composites.

Week you plan to administer the assessment

Number of teachers to be surveyed

Ordering Information Needed

from You


76

Contact Information Susanne Davis, HSTW SW Ohio

[email protected](513) 677-5641

Missy Kramer, HSTW SW [email protected](513) 619-2330

Fran Cowart, [email protected](404) 879-5611

Lisa Rion, Educational Testing [email protected](609) 734-5646

Southern Regional Education Board 1 Preparing for the 2014 HSTW Assessment Mike Ross Consultant,...

Documents

Transcript of Southern Regional Education Board 1 Preparing for the 2014 HSTW Assessment Mike Ross Consultant,...