Post on 17-Jan-2016
description
Using the 2011 NECAP Science Results
New England Common Assessment Program
2
Dr. Kevon Tucker-SeeleyAssessment Specialist – NECAP Manager
Office of Instruction, Assessment & Accountability1-401-222-8494
Peter McLarenScience and Technology Specialist
Office of Instruction, Assessment & Accountability1-401-222-8454
Jennifer GoleniaScience and Technology Specialist
Office of Instruction, Assessment & Accountability1-401-222-8406
Lindsay WepmanAssessment Specialist
Office of Instruction, Assessment & Accountability1-401-222-8254
Welcome and RIDE Introductions
NECAP
Service Center
1-877-632-7774
Elliot Dunn
NECAP Science Program Manager
603-749-9102 ext. 2126
dunn.elliot@measuredprogress.org
Harold Stephens
NECAP Program Director
603-749-9102 ext. 2235
stephens.harold@measuredprogress.org
Carole Soule
NECAP Program Manager
603-749-9102 ext. 2450
soule.carole@measuredprogress.org
3
Measured Progress Introductions
NECAP Service Center: 1-877-632-7774
Kellie Beaulieu: NECAP Program Assistant603-749-9102 ext. 2156
beaulieu.kellie@measuredprogress.org
Mellicent Friddell: NECAP Program Assistant603-749-9102 ext. 2355
friddell.mellicent@measuredprogress.org
Alison Cady: NECAP Program Assistant603-749-9102 ext. 2149
cady.alison@measuredprogress.org
4
Measured Progress Introductions
• Review the different types of NECAP Science reports and share State results
• Demonstrate a simple data analysis protocol using NECAP Science Data
• Share some observations from the NECAP benchmarking process
5
Goals of the Workshop
6
Workshop Reference Materials
7
Types of NECAP Science Reports
Student Report (Confidential)Information for Parents
Item Analysis Report (Confidential)School level by student
Results Report (Public) School and District level
Summary Report (Public) District/State level
Student Level Data Files (Confidential)Excel/csv files by grade on district and school confidential site
8
http://iservices.measuredprogress.org
Accessing Your Confidential Reports
Select “Interactive”
to view Interactive
Reports
Select “Reports” to view Static Reports
9
Accessing Your Confidential Reports
NECAP Analysis and Reporting System (NARS) Account Creation Hierarchy
Password Assistance: Use the following list to determine whom to contact for assistance with your User Name and Password:Superintendents – Contact NECAP Service Center at 877-632-7774Principals – Contact NECAP Service Center at 877-632-7774Teachers – Contact School Principal
http://www.ride.ri.gov/assessment/Results.aspx
Using NECAP Science Data
• Focus is improving student learning– The goal is to increase student achievement
• Engage in collaborative discourse about data– Using relevant, timely data to influence educational
decisions is considered a best practice
• Discussion of data requires a structured approach– Protocols, ground rules, and shared talk time
• Remember, NECAP Science data is for assessing school-level achievement, NOT individual student achievement – Look for trends and patterns
NECAP Science Data
Today we’ll take a look at:
• Demo District Results
• Demo School Results
• Test Items and Item Analysis Reports
• State Results
Phases of Collaborative
Inquiry
Adapted from N. Love
Using NECAP Science Data
Using Data―A Simple Approach
1. Observe the data
– Look at the data with your table.
– Make objective statements about the data. Avoid statements like, “It seems…” or “I think…”
– What do you notice?
Using Data―A Simple Approach
2. Discuss
– Talk with your colleagues at your table.
– Why do think the data might look this way?
Using Data―A Simple Approach
3. Action steps
– What are some other sources that could confirm or refute this data?
– What does the data mean for my school regarding curriculum and instruction?
– How can I dig deeper?
Using Data―A Simple Approach
4. Reflection (time permitting)
– What did you learn from this activity?
– Are there new things that you’d like to bring back to your school?
Tying it Together
• Limitations – This is one source of data. You can’t make major
programmatic changes from this alone.
– Look to other sources of data to confirm or refute.
– Think about curricular and instructional action steps.
• Look for professional development around data use in the 2012–13 and 2013–14 school years.
Percentage of Rhode Island Students by Achievement Level
Grade Year Subject SBP PP P PwD P+PwD CHGAverage
SS
4 10/11 SCI 18 38 43 1 44 0 438
4 09/10 SCI 20 36 43 1 44 4 437
4 08/09 SCI 20 40 40 <1 40 --- 437
8 10/11 SCI 29 46 24 1 25 3 833
8 09/10 SCI 31 47 22 <1 22 5 832
8 08/09 SCI 38 45 17 <1 17 --- 831
11 10/11 SCI 25 49 25 1 26 5 1134
11 09/10 SCI 32 48 20 1 21 1 1133
11 08/09 SCI 36 45 19 1 20 --- 1132
10/11 Comparison of NECAP States by Achievement Level
Grade State Subject SBP PP P PwD P+PwD CHGAverage
SS
4 RI Sci. 18 38 43 1 44 0 438
4 NH Sci. 8 36 54 1 55 1 441
4 VT Sci. --- --- --- --- --- --- ---
8 RI Sci. 29 46 24 1 25 3 833
8 NH Sci. 21 52 27 1 28 1 835
8 VT Sci. --- --- --- --- --- --- ---
11 RI Sci. 25 49 25 1 26 5 1134
11 NH Sci. 26 47 26 1 27 3 1134
11 VT Sci. --- --- --- --- --- --- ---
Note: Vermont’s data is embargoed until 9/28. RI will do a public release on 9/27
Benchmarking Short Answer and Constructed Response Items
•Short answer items receive a score from zero to two.
•Constructed response items receive a score from zero to three or zero to four.
•Zeros are employed when a student produces some work, but the work is totally wrong or irrelevant or if he or she leaves the item blank.
•For purposes of aggregating item results, blanks and zeros both count as zero points towards a student’s score.
Preparation for Benchmarking
The work in preparation for scoring student responses included:
• Development of scoring guides (rubrics) •Content specialists from the NH, RI, and VT Departments of Education •Measured Progress’s test developers
•Selection of “benchmark” responses•Examples of student work at different score pointsfor each item•Used in training and continuous monitoring of scorer accuracy
Scorer Training
Scorer training consisted of:
•Review of each item and its related content and performance standard
•Review and discussion of the scoring guide and multiple sets of benchmark responses for each score point
•Qualifying rounds of scoring in which scorers needed to demonstrate a prescribed level of accuracy.
NECAP Benchmarking Process
Lessons Learned from NECAP Benchmarking
• What does it mean – to identify?– to describe?– to explain?
• How does this look over grade levels?
• What does it mean to compare and contrast?
Lessons Learned
Grade 4
• What is a characteristic?
• What observations of information can be gained from charts and diagrams?
• What patterns can be distinguished from graphs?
• How can students better use evidence from data in their explanations?
Lessons Learned
Grade 8
• When should line graphs be used? When should bar graphs be used?
• How can students better support or refute their predictions/hypotheses using evidence from data?
• Why is it important to use multiple trials in an investigation?
Lessons Learned
Grade 11
• What are typical sources of error? – How can they affect the outcomes of an investigation?
• How can skills in the analysis of data be demonstrated more deeply?
• By 11th grade, students should know about the use of proper units of measurement.
• Development of Assessment Targets
• Participation in annual test item review committee & bias and sensitivity review committee
• Use of classroom teacher judgment data
• Participation in standard setting panels
• Technical Advisory Committee work
29
Involvement of Local Educators in NECAP Science
3030
Use the data• Share progress with parents• Identify gaps in curriculum, instruction, and assessment
Identify students in need of accommodations• Anticipate types of accommodations• NimbleTools will be an option – Spring 2012
Plan for administration now• Review what went well and what went not so well during the
2011 administrationBegin preparing your students
• Share the reference sheets• Released Items documents• Preparing Students for NECAP:
Tips for Teachers to Share with Students
Preparing for 2012 NECAP Science
3131
Questions?
Peter McLarenScience and Technology Specialist
Office of Instruction, Assessment & AccountabilityPeter.mclaren@ride.ri.gov
1-401-222-8454
Jennifer GoleniaScience and Technology Specialist
Office of Instruction, Assessment & AccountabilityJennifer.golenia@ride.ri.gov
1-401-222-8406