Central Mass. Readiness Center Part II Maintaining Quality Creating Cut Scores Preparing for June...
-
Upload
jeffry-richards -
Category
Documents
-
view
220 -
download
0
Transcript of Central Mass. Readiness Center Part II Maintaining Quality Creating Cut Scores Preparing for June...
Central Mass. Readiness Center Part
IIMaintaining Quality
Creating Cut Scores
Preparing for June Report
Preparing for 2015 Full Implementation
Dr. Deborah Brady
Ribas Associates
Good Morning!
DO NOWReview today’s Agenda, Handout, and
PowerPointFill out a Name TagNote cards for questions and/or exit slipsHave a cup of coffeeWiki Page has today’s materials as well as
Day 1 Materials
Topic Comments on Exit SlipsDDMs Examples of good DDMs for all areas: science, math, elementary, middle school Assessing quality of specific DDMs Developing rubrics for DDMs; (H-M-L); rubric examples Developing Exemplars PARCC examples Processes June report Roll out to teachers: best practices Sample DDM Plan for one educator Calibration of scores and scorers; assuring growth scores are comparable Developing cut scores Planning for full implementation Team Time Assessing the “holes” in DDM in the district PARCC/Writing to Text/Text Sets
Text sets: finding, developing, assessing the rigor and appropriateness
Parking Lot For unresolved issues
Size of classes that are too small to determine student growth (DESE)
Agenda Day 2February 18. 2014
Where are you on your journey now? (Carousel, Living Likert)
Preparing for June report and for next year Local Curriculum Quality: Checklist for DDM Indirect and Direct Measures
Good, Bad, Questionable Examples from local districts
Calibration Protocols and options
Juried Resources—Assessments and More elaborate Lessons with performance assessemtns
Documenting 2 DDMs per Educator (Documentation Form for 2 DDMs)
Scoring Assessments and Developing Local Cut Scores Pre-Test, Post-Test Cut Score Example
Rubric Scoring in 100% system
Introduction to protocols for assessment: samples
Time to research, work with team, plan for June, plan for next year
The Steps Necessary to Get Ready for June Report and After
• Adapting present assessments
• Creating new assessments
• Writing to text for HSDevelopin
g and Piloting
Assessments
• Alignment of Content
• Rigorous and appropriate expectations
• Approval of assessments
Assessing Quality
and Rigor
• Security• Calibration
of standards and of assessors
• Rubric quality
• Analysis of results: High-M-Low Growth
Piloting
• 2 DDMs per educator
• JUNE REPORT• Directions for
teachers • Directions for
students• Organizing for
the actual assessments
• Storing, tracking the information
2015 Full Implementatio
n
• Data storage
• Data Analysis
• L-M-H Growth
• Roster Verification
• Data team timeInterpreting the results
Student Impact
Carousel Walk and Living Likert Scale
Carousel Walk Take a marker for your team (or yourself if you are on your own)
Visit each of the 5 “stages” of DDM development
Put a check to the right of each item that you have completed
After viewing each stage, return to the stage that most represents you and/or your district
Living Likert Go to your “stage”
What are the barriers? What are the strengths that your district has?
Discuss the barriers and strengths with the others at your stage.
Be prepared to report out as a whole group.
Preparing for the June ReportCurriculum Quality Checklist
Documenting 2 DDMs per educator
Checklist for DDM Direct and Indirect Measures Check all Items that are
completedDefinition Please fill in this column as completely as you
can Course This information is necessary for June Report
Grade(s) of DDM Grade level(s) that this assessment will cover
Check one
Direct Measure Indirect Measure
Direct Measure= Assessment of student growthIndirect Measure=Assessment of something that connects indirectly to student growth (attendance, for example)
Content Area List content area.Indirect measure include the job category/categories of those involved, for example, SLPs or guidance counselors
Alignment to State and/or District Standards
For direct measures: List at least 2 standards that will be assessed so that this is a “substantial” assessment.For indirect measures, 1) What are the substantial, important, essential areas that you are assessing? 2) How does this indirect measure connect with student growth?
Please list the two (or more) standards using standards language.1. 2.
Rigor: Check the levels of Blooms that are assessed
The original Bloom is the first word on the list. The new Bloom (all verbs) is the second. Note, in the new Blooms, Creating is on the highest level, above evaluating.Indirect Measures: NA; skip Types of Questions and Duration (below), but fill in the remaining categories.
Check all that apply: Knowledge, Remembering Comprehension, Understanding Application, Applying Analysis, Analyzing Synthesis, Creating Evaluation, Evaluating
Check all that apply
Type(s) of questions
Multiple Choice, fill in, short answer (recall items from content area) Multiple Choice, fill in, short answer (text dependent questions) Open Response (short answer) Essay (long response). Type:
Narrative Informational Text Argument with claims and proof
One text is read Two texts are read Performance Assessment (CEPA) Other_______ (Fill in at right.)
Indicate the percentage of the assessment for each question type, for example, multiple choice=50%; 2 open responses=50% (25% each) Multiple Choice _____%Open Response _____%Essay _____________%
Performance Assessment: Describe briefly:
Duration of assessment
Assessments can take place in a class period or over a period of days. Indirect Measures please fill this section out.
Necessary for planning for full implementation.
When assessment(s) will take place
Provide approximate month or window for assessment(s), for example, end of first trimester, September.Provide multiple dates if the assessment is a pre-post or is administered more than once.Indirect Measures please fill this out.
Components of assessment that are completed so far.
Directions to teacher for administering Directions to students Graphic organizers (optional) The assessment Scoring guide Rubric Security Calibration protocol if this assessment has a rubricIndirect measures: Describe your proposed process; you will need to create many of these components
Rubric Not Yet Does not apply
How was the rubric created? For example,adapted from DESE’s CEPA rubric, or developed by the department.
Please attach rubric along with the assessment as far as you have gone.
Quality Tools Core Curriculum Objectives: Model Curriculum Units and Curriculum Embedded Performance
Assessment Rubrics: DESE Cognitive Complexity and Common Core: Daggert and Hess Juried assessments: www.EngageNY.org Juried units/assessments: www.achieve.org, MA MCU Juried rubrics: CEPA, Hess (ELA, math, History, Science),
Delaware’s K-12 rubrics for 3 Common Core text types: argument, narrative, informational text
Calibration and Looking at Student Work (LASW) Protocols (on handout) plus http://Nsfharmony.org/protocol/a_z.html
Achieve.org has Common Core aligned and annotated student sample essays at http://www.achievethecore.org/page/507/in-common-effective-writing-for-all-students
Demonstrating Growth(when accuracy of computation may be a concern)
Essay Prompt from Text
Read a primary source about Mohammed based on Muhammad’s Wife’s memories of her husband.
Essay: Identify and describe Mohammed’s most admirable quality based on this excerpt. Select someone from your life who has this quality. Identify who they are and describe how they demonstrate this trait.
What’s wrong with this prompt?
Science Open Response from Text
Again, from a textbook, Is this
acceptable?
Scoring Guides from Text
Lou Vee Air Car built to specs (50 points)
Propeller Spins Freely (60 points)
Distance car travels
1m 70
2m 80
3m 90
4m 100
Best distance (10,8,5)
Best car(10,8,5)
Best all time distance all classes (+5)
235 points total
A scoring guide from a textbook for
building a Lou Vee Air Car. Is it good enough to ensure
inter-rater reliability?
Technology/Media Rubric
A multi-criteria rubric for technology. What is
good, bad, problematical?
PE Rubric in Progress.Grade 2 Overhand
throw.Looks good?
Music: Teacher and Student Instructions
World Language Scoring Guide and Rubric
Table Talk
Curriculum Quality
Local Standards
2 DDMs per EducatorMEPID Last Name First Name Grade/Dept. DDM1 DDM2 DDM3
(optional) Jones Brigit ELA 6 MCAS 6 Growth
Score ELA (SGP)ELA 6 DDM (writing to
text)
Smith Marion 9-12 library Library Search Tools DDM
Indirect: Increase teachers who do
research in library.
Watson Elsbeth 5 ELA team Fountas and Pinnell DDM
MCAS 5 Growth Score History Unit Exam
DDM Holmes Sharon Grade 2 Fountas and
Pinnell DDMGalileo DDM
NOTE: Consider developing a system for naming DDMs for storing, locating, assuring consistency as you report in June
F&P.6; M.Gal.6ELA.SGP.6
Math.6.1; Math.6.2; Math.6.GalileoPE.K-5 I, PE.K-5 2
Bio.9-10.1; Bio.9-10.2; Bio.9-10.H1; Bio.9-10.H2
Table Talk
June Reporting
Scheduling for Full Implementation
Calculating Cut ScoresPre- Post- Calculation
Rubric Inaccuracy when translated to a percentage
Cut Scores for L-M-H Growth
Pre-testPost test
Difference
%age growth
Diff/pre Student Scores Sorted low to high
%diff/pre
Teacher score is based on the MEDIAN Score of her class for
each DDM Teacher L-M-H Rank
20 35 15 75% 20% 5Cut score? LOW Growth
25 30 5 20% 42% 15 bottom 20%30 50 20 67% 42% 20
35 60 25 42% 50% 25 Moderate Growth
35 60 25 42% 60% 25 median teacher score
40 70 35 87% 62% 25 median Teacher score 40 65 25 62% 67% 25
50 75 25 50% 70% 30
50 80 30 60% 75% 35 Top 20%?
50 85 35 70% 87% 35 Cut score? HIGH GROWTH
Rubrics Converted to %Roobrix.com
1. Number of levels in Rubric
2. Number of criteria in Rubric
3. Minimum passing grade if student did the work (very poorly)
4. Student’s score
4 Levels
2. Criteria: Topic/Conventions (weighed 2:1=3 criteria)
3. 50
4. Proficient times 2/Advanced
Result=89%
Topic
4 3 2 1
Topic 4 3 2 1
Conv. 4 3 2 1
Total 4 6
Table TalkLocal Cut Scores
High-Moderate-Low Student Growth
Teacher’s Growth Score is the Median Score for each class included in the DDM (not the
average)
Begin with top and bottom 10% then go to the “body of the work,” i.e., the student work
Rubrics and percentages
Calibration ProtocolsInter-Rater Reliability
Singleton Teacher and Fairness
Rhode Island http://www.ride.ri.gov/Portals/0/Uploads/Documents/Teachers-and-Administrators-Excelent-Educators/Educator-Evaluation/Online-Modules/Calibration_Protocol_for_Scoring_Student_Work.pdlf
Select Facilitator Individually
sort your students’ work into
High, Medium, and
Low piles
Individuallydescribe the characteristi
cs of each pile with
rubric or list of criteria.
As a group,share
characteristics.
As a group,develop
rubric or list of criteria
and exemplars(possibly)Score all
Assessments
As a group, develop an action plan (next step)
for each level and for all students
Reflect on process.
Share reflections.
START HERE
PROTOCOLSProtocols for Calibrating Inter-Rater ReliabilitySee handout
Develop local Cut Scores
Assign L-M-H Growth to
each Student
Sort student scores to
each TeacherDetermine
Median scoreAs teacher’s Growth Score
Modification for
DDM Process
Brenda is making tree costumes for a play. The list below shows the amounts of the different colors of cloth Brenda will use to make one tree costume.
yards brown cloth
yards orange cloth
yard yellow cloth
1. What is the difference, in yards, between the amount of orange cloth and the amount of brown cloth that Brenda will use to make one tree costume? Show or explain how you got your answer.
2. Brenda plans to use brown cloth for the trunk and branches of the tree, and orange and yellow cloth for the leaves.
3. What is the total amount of cloth, in yards, Brenda will use to make the leaves of one tree costume? Show or explain how you got your answer.
4. Brenda wants to make two tree costumes.5. What is the total amount of cloth, in yards, Brenda will use to
make two tree costumes? Show or explain how you got your answer.
MCAS Rubrics and Examples at all levelsGrade 6 Math
MA DESE Student Work at http://www.doe.mass.edu/mcas/student/2013/
2
The student response demonstrates a fair understanding of the Number and Operations-Fractions concepts involved in adding and subtracting fractions with unlike denominators, including mixed numbers, by replacing given fractions with equivalent fractions in such a way as to produce an equivalent sum or difference of fractions with like denominators. While some aspects of the task are completed correctly, others are not. The mixed evidence provided by the student merits 2 points
K-5 Same Prompt; 6-12 Same Prompt
http://www.achievethecore.org/page/507/in-common-effective-writing-for-all-students
Grade 4
On-Demand Writing- Uniform Prompt
Which is Better as a Pet, a Dog or a Cat?
Many people have a dog for a pet. Some people have cats. Wich is better? I say dog. Maybe you say cat. I just might be able to persaude you in the following. Dogs are great companions for lonely people. They can go for a rousing walk in the park, or a good long nap. Playing games of catch or fetch every day makes good fun. Even a jog on the hottest day could even be enjoyable too. Dogs don’t just provide fun though. They can also provide protection.
Dogs are very intelligent. They can be trained to find people or save them. Some don’t even need to be trained. For instance, if someone is trying to break in, your dog might bark and scare them off. Dogs are great for many different reasons. Overall, dogs are awesome pets to have. Have I convinced you though? If you are, then great! If your not then thats okay. It's really up to you. So which one is it going to be?
AnnotationsIntroduces a topic clearly States an opinion Provides a concluding section related to the opinion presented Creates an organizational structure in which related ideas are grouped to support the writer’s purpose Provides reasons that are supported by facts and details
Links opinion and reasons using words and phrases
Annotated Exemplar ORQ: How does the author create the mood in the poem?
Answer and explanation in the student’s words
Specific substantiation from the text
4 Texture: Crunchy outside; moist inside cookie and slightly melting chocolateAppearance: Glossy from the use of real butter; fits in the palm of your hand, variation in shapes and modulation of surface perhaps of gourmet or home made origin; see substantial chocolate chunks.Taste: Cookie melts in your mouth and the chocolate is high quality, in chunks, and possibly imported dark chocolateTemperature: Warm, just out of the oven and cooling on a cooling rack
3 Texture: Cake-like, chocolate is at room temperatureAppearance: Flat surface, slightly smaller than the palm of your hand, some variation in sizes and surface modulation indicates it may be home made, see some possible bites without chocolate. Looks like commercial chocolate bits.Taste: Cookie is mostly crunchy. The chocolate tastes like commercial chocolate chips.Temperature: Room temperature.
2 Texture: Somewhat dry.Appearance: Flat surface, small, almost skimpy size; maybe from a cafeteria. Many bites without chips.Taste: Somewhat dry. Chocolate is not of high quality; may be old.Temperature: Cool, maybe from cafeteria refrigerator or freezer.
1 Texture: Dry, cold, crumbling Appearance: Porous, skimpy portion, uniform shapes, perhaps commercially madeTaste: Dry and chocolate has a white coating as if it is old or of poor qualityTemperature: Cold
Chocolate Chip Cookie/Holistic Rubric
copyright 2013 Ribas Associates
“Don’t let perfection get in the way of good.”
Team Time to Plan
The Steps Necessary to Get Ready for June Report and After
• Adapting present assessments
• Creating new assessments
• Writing to text for HSDevelopin
g and Piloting
Assessments
• Alignment of Content
• Rigorous and appropriate expectations
• Approval of assessments
Assessing Quality
and Rigor
• Security• Calibration
of standards and of assessors
• Rubric quality
• Analysis of results: High-M-Low Growth
Piloting
• 2 DDMs per educator
• JUNE REPORT• Directions for
teachers • Directions for
students• Organizing for
the actual assessments
• Storing, tracking the information
2015 Full Implementatio
n
• Data storage
• Data Analysis
• L-M-H Growth
• Roster Verification
• Data team timeInterpreting the results
Student Impact