Proof of Concept Study: Grade 5 Mathematics Grade 6 ELA/Reading Nancy Carolan, Hope Lung NC...
-
Upload
madeline-rosa-wilkinson -
Category
Documents
-
view
216 -
download
0
description
Transcript of Proof of Concept Study: Grade 5 Mathematics Grade 6 ELA/Reading Nancy Carolan, Hope Lung NC...
Proof of Concept Study:
Grade 5 MathematicsGrade 6 ELA/Reading
Nancy Carolan, Hope LungNC Department of Public Instruction
August 18, 2015
Proof of Concept Study
Purposes and Design
2
Purposes–Grades 3-8Valid and reliable measures of content
standardsGrowth data for educator effectiveness reportsAssessments throughout the year that inform
and guide instruction and that predict performance on future assessments
A stand-alone summative assessment at the end of the year that may be connected to the interim assessments
3
Grades 3-8 Concept DesignStudents take 3 interim assessments during the
school year The interim assessments are not included in
accountability calculationsStudents take a shortened end-of-year test
End-of-grade test without embedded field test items
The modified end-of-year test is included in accountability calculations
4
5
Interim 1 Interim 2 Interim 3 End-of-Year
(Assess entire year)
Grades 3-8 Concept Design
Note: Design could be altered based on outcome of study
Timeline Based on Current Design
Year Administration Grade Levels Purpose
2015–16 Proof of Concept(sample population)
Grade 5: MathGrade 6: ELA/Reading
Determine feasibility of concept
Summer 2016: Decision-point of how to proceed
2016–17 Field Test(sample population)OrRelease RFP for assessment
Grades 3–8: Math Grades 4–8: ELA/Reading
Test items for inclusion in operational test forms
2017–18 Pilot Operational Year(statewide)
Grades 3–8: Math Grades 4–8: ELA/Reading
Full Implementation
6
Proof of Concept Study
Research Questions
7
Research QuestionsDetermine the feasibility of implementing the
concept across grades 3–8 data analysis to affirm reliability input/feedback on the integration of the
assessments in instruction logistical/operational (delivery mode, scoring) professional development requirements possibility of different item types
8
Research QuestionsIs it valid and reliable to combine results on
the interim assessments for proficiency and growth reporting; thereby, eliminating an end-of-year summative assessment?
Do interim results provide teachers and students with useful information to inform and improve the delivery of instruction?
Will interim assessments predict performance on the end-of-year test?
9
Research QuestionsAre the interim assessments required of all
students or are some of the interim assessments optional?
Does the structure of the content standards require adjusting design of the through-grade model?
Is it feasible to incorporate constructed-response items or writing prompts on the interim assessments?
10
Research QuestionsIs the performance affected by the delivery of
through-grade assessment versus a traditional summative assessment model?
What information will be available for student-level and teacher-level reports and how is such information best delivered and presented?
Is the professional development adequate?
11
Research QuestionsIs it feasible to deliver the assessments both
online and paper/pencil?Does the through-grade model provide
parents with useful information and do they view the model as an effective way to assess students?
12
Proof of Concept Study
Reporting
13
Information for Teachers
14
Student-Level Reports Number of items correct by content standard Number of overall items correct Item-level reporting
• Aligned content standard• Student response• Correct answer• Item type• Selection type• Calculator active/inactive
Proof of Concept Study
Sample and Specifications
15
16
Sample―Grades 5 and 6• Grade 5 Math — 45 schools, 4,021 students
• Grade 6 ELA/Reading ― 35 schools, 4,859 students
• Generalizable to the entire state
• Stratified random sample pulled based on variables
• Demographic variables: region, ethnicity, gender, economically disadvantaged
• School-level achievement variable: mean scale score
17
Interim―Test Specifications Test specification development is step 2 in the test development
process
Separate meetings held late June and early July Invitees selected by NCDPI Curriculum and Instruction Mathematics
and ELA Active grade 5 Math and grade 6 ELA classroom teachers, instructional
coaches, and LEA curriculum and instruction leaders First half of meeting―Training by NCDPI Curriculum and Instruction Second half of meeting—NCDPI Test Development collected feedback
and recommendations
Test Development team discusses teacher recommendations with NCDPI Curriculum and Instruction to finalize test specifications https://center.ncsu.edu/ncaccount/
Feedback will be collected from sampled schools throughout the year
Interim Design—Grade 5 Math• Calculator active and inactive sections
• 25 Items• 21 multiple-choice items (8 calc. inactive, 13 calc. active)• 4 gridded-response items (calc. inactive)
• Interim 1: 5.NBT.2, 5.NBT.5, 5.MD.5.b, 5.MD.5.c
• DRAFT Interim 2: 5.NF.1, 5.NF.2, 5.NF.3, 5.NBT.6, 5.NBT.7
• DRAFT Interim 3: 5.NF.2, 5.NF.4 a & b, 5.NF.6, 5.NF.7 a, b & c, 5.NBT.7
• Assessment Specification documents https://center.ncsu.edu/ncaccount/
• Answer sheets scanned and reports distributed by the LEA Test Coordinator
18
Interim 1 Design—Grade 6 ELA• Interim 1 Design
20 multiple-choice itemsSelections: Poetry (1), Informational (1), Literature (1)Answer sheets scanned and reports distributed
immediately by the LEA Test Coordinator
• Standards assessed on each ELA interim assessment RL.1, RL.2, RL.3, RL.4, RL.5, L.4a, L.5.a RI.1, RI.2, RI.3, RI.4, RI.5, RI.6, RI.8
19
Interims 2 & 3 Design—Grade 6 ELA
• DRAFT Interims 2 and 3 19 multiple-choice items, 1 constructed response (CR) item
CR item will be short-answer and can typically be answered in a paragraph or less. Students must write on lines provided on answer sheet.
Interim 2 selections: Informational (1), Literature (2)
Interim 3 selections: Informational (2), Literature (1)
Answer sheets returned to the LEA Test Coordinator to be shipped for central scoring, reports returned within 8 days
• Standards assessed on each ELA interim assessment RL.1, RL.2, RL.3, RL.4, RL.5, L.4a, L.5.a
RI.1, RI.2, RI.3, RI.4, RI.5, RI.6, RI.820
Interim Design—Math and ELAPaper/pencil only to ensure participation
Up to 90 minutes maximum test administration time
Items pulled from the EOG item pool (if available)
One form for each interim assessment
Assessment Specification documents https://center.ncsu.edu/ncaccount/ May need to create a new NC Education account to access
21
Proof of Concept Study
Policy and Administration
22
Interim Assessment Policy Testing Windows (3)
Interim 1 October 1–30, 2015 Interim 2 December 8, 2015–January 22, 2016 Interim 3 March 3–31, 2016
Districts/charters can determine the testing days within the designated windows
Interims are not required to be administered to all students on same day, but should be administered within the same week
Make-up administrations are optional but strongly recommended Should be administered by the classroom teacher Proctors are not required Administration does not require the removal of displays
23
24
Interim Assessment PolicyEligibility Mathematics Grade 5
All students enrolled in grade 5 at sampled schools who participate in the EOG mathematics assessment
English Language Arts/Reading Grade 6 All students enrolled in grade 6 at sampled schools who participate
in the EOG English Language Arts/ reading assessment
Both Transfer students−−Take the interim(s) and the modified end-of-
year (EOY) test
No opt out
25
Interim Assessment PolicyStudents Not Eligible to Participate Students with disabilities whose IEP documents
participation in the NCEXTEND1 alternate assessment
LEP students who scored below Level 4.0 Expanding on the W-APT and are in their first year in U.S. schools are not eligible to participate in the grade 6 ELA study, but they are eligible to participate in the grade 5 math study
Students who are granted a medical exception from the Division of Accountability Services for the EOG assessments
26
Interim Assessment PolicyAccommodations IEP, 504, and/or LEP teams do not have to reconvene
and document the accommodations for the Proof of Concept Studies. Students may use the accommodations that are specified on
their current IEPs, Section 504 Plans, or LEP documentation for the EOG for the modified end-of-year test.
Instructional accommodations may be used for the interims except for the Read Aloud and Signing/Cueing accommodations for grade 6 ELA/Reading. Reading aloud or signing/cueing the selections, questions, or answer choices invalidates results because the interims measure reading skills.
27
Interim Assessment PolicySpecial Print Versions Braille, Large Print (LP), One Item Per Page (OIPP), and
Large Print One Item Per Page Editions (LP/OIPP) can be ordered from TNN Accommodation Notification Request Forms for special
print versions do not need to be sent in for interim assessments
Submit orders at least 30 working days before the actual administration date
28
Interim Assessment PolicyMaterials Assessment Guide
– guide for interims (separate guides: 1 for ELA/Reading; 1 for Math)
– guide for end-of year (separate guides: 1 for ELA/Reading; 1 for Math)
Answer sheets
Test books (separate test books for the subjects)
29
Interim Assessment PolicyRequired Supplemental Materials English Language Arts/Reading
Blank paper
Mathematics Blank paper
Graph paper (auto shipped for interims and end-of-year)
Calculators
– Any four-function calculator with memory key
30
Interim Assessment PolicySecurity Assessment guides are not secure test materials
Store at the school until all interims have been administered, then securely destroy
Following the administration, interim assessment booklets are to be kept at the schools for 4 weeks, then securely destroyed Booklets must remain in the school Booklets should be accounted for at all times Local decision where booklets are stored at the school (not accessible to
students) Teachers may (should) use the booklets with students in reviews Parents may view the booklets only within the school setting. The teacher may
share with parents the student’s scores on the items through customary communication (i.e., individual parent/teacher conferences at the school)
Interim assessments (booklets, items, content) cannot be shared with other schools
Modified End-of-Year Assessments for Concept
Study Participants
31
32
End-of-Year (EOY) Test Specification—Math Grade 5
Number of Multiple-
Choice Items
Number of Gridded
Items
Total Number of Items
Modified EOY Math G5 (Concept Study participants only)
38 6 44
End-of-Grade Grade 5 Math 46 8 54
33
End-of-Year (EOY) Test Specification—ELA/Reading Grade 6
Number of Multiple-Choice Items
Number of Constructed
Response Items
Total Number of Items
Total Number of Selections
Modified EOY ELA/Reading G6 (Concept Study participants only)
48 0 48 6
End-of-Grade Grade 6 ELA/Reading
56 0 56 7
EOY Test Policy—Concept Study
End-of-Year assessments follow the same guidelines as the End-of-Grade Assessments
– Multiple forms– Test window– Test security– Supplemental materials– Paper/pencil administrations only– Submit accommodation notification forms– Scanned locally– Reporting– Scores included in accountability calculations
34
Proof of Concept StudyResources
35
36
NC Education Page
Use the following link to access presentations, memos and resources
https://center.ncsu.edu/ncaccount
Proof of Concept Study
Curriculum & Instruction Professional Development
37
38
ELA Professional Development Webinars
Part One: Standards Study − includes resources for understanding, expectations for students, progression, integrating the strands.
Date: August 19 3:00‒4:00 p.m.https://attendee.gotowebinar.com/register/7149308645817868290
Part Two: Text Complexity − includes how it should be considered within instruction, scaffolding, and determining the complexity of a text.
Date: August 20 3:00–4:00 p.m. https://attendee.gotowebinar.com/register/542795173790925826
39
If you were unable to attend math professional development…
• All materials—PowerPoint and handouts will be available on the
DPI Mathematics Wiki:
http://maccss.ncdpi.wikispaces.net/Home
See “Proof of Concept” Page on wiki for registration links as well.