1 · Web viewItems and Rubrics 1 p. 2 Rubric A (Information) . p. 4 Rubric B (URL) p.
Module 21 Using Rubrics to Assess Learning. Module 22 Our Journey A Road Map Student Involvement =...
-
Upload
helen-gordon -
Category
Documents
-
view
214 -
download
0
Transcript of Module 21 Using Rubrics to Assess Learning. Module 22 Our Journey A Road Map Student Involvement =...
Module 2 1
Module 2
Using Rubrics to Assess Learning
Module 2 2
Our JourneyA Road Map
Student Involvement
= Rest Stop, time for reflection
Creating a Rubric
Reliability and Validity
Target-Method-Match
Module 2 3
Assessment
Large-Scale
Assessment
Classroom Assessmen
t
Module 2 4
Assessment
Large-Scale Assessmen
t
Classroom Assessment
CLASSROOM ASSESSMENT
Formative Assessment
(for Learning)
Summative Assessment
(of Learning)
Purpose
Performance Assessment
Selected Response
Personal Communicatio
nEssay
Type
Module 2 5
Assessment
Large-Scale Assessmen
t
Classroom Assessment
CLASSROOM ASSESSMENT
Formative Assessment
(for Learning)
Summative Assessment
(of Learning)
Purpose
Performance Assessment
Selected Response
Personal Communication
Essay
Type
Module 2 6
Do you hear this at your school?
Why did I get a B for this assignment?
You didn’t tell us that spelling counts.
Will this be on the test?
Why do we have to learn this?
My friend said my “A” in America History isn’t the same as his “A”.
Module 2 7
Outcomes• Teachers will review their
understanding of quality assessment and its critical role in instruction.
• Teachers will know how to identify and develop appropriate criteria for use in rubrics and checklists.
• Teachers will know a process to develop performance assessment rubrics that are valid and reliable and inform instruction.
Module 2 8
In tr o d u c tio n /L e a d e r s h ip P r o f e s s io n a l
D e v e lo p m e n t M o d u le s H a w a ii S ta n d a r d s Im p le m e n ta tio n P r o c e s s
S ix S t e p M o d e l Id e n t if y R e le v a n t
S t a n d a r d s D e t e r m in e A c c e p t a b le e v id e n c e
D e t e r m in e L e a r n in g E x p e r ie n c e s
T e a c h a n d C o lle c t E v id e n c e
A s s e s s s t u d e n t w o r k /in f o r m in s t r u c t io n
E v a lu a t e /ju d g e / c o m m u n ic a t e le a r n in g r e s u lt s
# 1 . S t a n d a r d s Im p le m e n t a t io n P r o c e s s
H C P S T o o lkit: C u r r ic u lu m
F r a m e w o r ks In s tr u c tio n a l G u id e
: T h e p r o c e s s 6 s te p s C y c lic a l N a tu r e B u ilt o n s y s te m o f
s ta n d a r d s
# 2 U s in g R u b r ic s t o A s s e s s L e a r n in g
H C P S T o o lkit: C u r r ic u lu m
F r a m e w o r k In s tr u c tio n a l G u id e
L e a r n in g d r iv e n b y
s ta n d a r d s
K n o w w h e n
le a r n in g h a s b e e n a c h ie v e d
K n o w w h a t le a r n in g lo o ks like
E v id e n c e a lig n e d to c u r r ic u lu m a n d in s tr u c tio n
U s e in fo r m a tio n fo r g o a l s e ttin g
# 3 C u r r ic u lu m M a p p in g
H C P S T o o lkit: S c o p e a n d
S e q u e n c e G L P I
C u r r ic u lu m m a p s a n d a s s e s s m e n t d a ta
S c h o o l a n d
d is tr ic t r e fo r m a p p r o a c h
C o lle c t d a ta
r e g a r d in g a c tu a l te a c h in g
# 4 . S t a n d a r d s - B a s e d G r a d in g a n d R e p o r t in g
H C P S T o o lkit R C F A Q s /W e b s ite P a r e n t B r o c h u r e
A s s e s s m e n t G r a d e s
B a s e d o n S ta n d a r d s
A lig n m e n t w ith
s ta n d a r d s b a s e d in s tr u c tio n a n d a s s e s s m e n t
G r a d in g
P r a c tic e s P u r p o s e fo r
G r a d in g Is s u e s /S p e
c ia l N e e d s
G u id e lin e s H I Im p le m e n ta tio n
Module 2 9
Assess TheAmerican Idol
After viewing the following performances, give the performer a letter grade (A-F).
Module 2 10
American Idol: Got Assessment?
3
1
0
10
0
2
3
0
0
0
13
0
5
6
17
0 5 10 15 20
William
Jasmine
Fantasia
ABCDF
Module 2 11
Standards Implementation Planning Model Adapted from WestEd’s Learning from Assessment
Module 2 12
Student Involvement
Module 2 13
Student-Involved Assessment
• Are partners in development• Describe expectations for success• See how close they are now• Create a clear path to success
Module 2 14
Student-Involved Communication
• Student-led parent/teacher conferences
• Greater sense of responsibility• Pride in accomplishment• Greater achievement
Module 2 15
Creating a Rubric
Module 2 16
4-Step Process for Setting Criteria with
Students• Step 1: Brainstorm
• Step 2: Sort & Categorize
• Step 3: Create a T-chart
• Step 4: Add, Revise, & Refine
Module 2 17
Step 1: Brainstorm
• Pose a question
• Record all ideas on the chart paper
• Add in your own ideas to make sure
that all the important features are
included and your goals are met
Module 2 18
Step 2: Sort & Categorize
• Have students look at the list and see if any ideas fit together. Ask “Do you see any similar items?”
• Circle or mark the similar items using colored pens.
• Decide on headings using big ideas.
Module 2 19
Step 3: Create a T-Chart
• Draw a T-Chart• Transfer the items
from Step 2.• Put the big ideas or
criteria on the left.• Put the details or
specifics on the right.
Criteria
American Idol
Details
Module 2 20
Now that you have the criteria, what do you do with it??
Rubrics
Module 2 21
What is a Rubric?
• An assessment that evaluates student performance
• Uses specific criteria for assessment
• Defines levels of performance for each criterion
Module 2 22
Why Use Rubrics
• Assessment is more objective and consistent.
• The target is clearer – shows what is expected and how work will be evaluated.
• It gives feedback on the effectiveness of the instruction.
• It provides benchmarks against which to measure and document progress.
Module 2 23
Common Features of Rubrics
• Measures a stated objective (performance, behavior, or quality)
• Uses a range to assess performance
• Contains specific performance characteristics arranged in levels indicating the degree to which a criterion has been met.
Module 2 24
Terms to use in measuring range/scoring
levels• Needs Improvement … Satisfactory,
Good … Exemplary• Needs work … Good … Excellent• Beginning … Developing …
Accomplished… Exemplary
• Performance grid 5…3…1
Guidelines for Rubric Developmenthttp//edweb.sdsu.edu/triton/july/Rubrics/Rubric_Guidelines.html
Module 2 25
Step 4: Add, Revise, Refine
• Have students review the T-Chart.• Ask students, “Are there any new
items or criteria that we need to add?”
• Make changes as necessary.
Module 2 26
• What are the implications of involving students in assessment?
REST STOP
Module 2 27
Target
Method
Match
Module 2 28
Five Standards of Assessment Quality
Standard 1Clear Targets
Standard 2Clear and
Appropriate Users and UsesStandard 3
Appropriate Assessment Methods
Standard 4Sampling
Standard 5Potential Sources of Bias and Distortion
Module 2 29
TUMS Bias
T = Clear TargetsU = Appropriate Users &
UsesM = Method of
AssessmentS = SamplesBias = Minimize Bias
Module 2 30
The Various Possible Links Between AchievementTargets and Assessment Methods
TARGET TO
BE ASSESSED
ASSESSMENT METHOD
SELECTED RESPONSE
ESSAY PERFORMANCE
ASSESSMENT PERSONAL
COMMUNICATION
KNOWLEDGE MASTERY
Multiple choice, true/false, matching, and fill-in can sample mastery of elements of knowledge
Essay exercises can tap understanding of relationships among elements of knowledge
Not a good choice for this target--Three other options preferred
Can ask questions, evaluate answers and infer mastery--but a time-consuming option
REASONING
PROFICIENCY
Can assess understanding of basic patterns of reasoning
Written descriptions of complex problem solutions can provide a window into reasoning proficiency
Can watch students solve some problems and infer about reasoning proficiency
Can ask student to Òthink aloudÓ or can ask follow up questions to probe reasoning
SKILLS
Can assess mastery of the knowledge prerequisites to skillful performance--but cannot rely on these to tap the skill itself
Can observe and evaluate skills as they are being performed
Strong match when skill is oral communication proficiency; also can assess mastery of knowledge prerequisite to skillful performance
ABILITY TO CREATE
PRODUCTS
Can assess mastery of knowledge prerequisite to the ability to create quality products--but cannot use these to assess the quality of products themselves
A strong match can assess; (a) proficiency in carrying out steps in product development, and (b) attributes of the product itself
Can probe procedural knowledge and knowledge of attributes of quality products--but not product quality
DISPOSITIONS
Selected response questionnaire items can tap student feelings
Open-ended questionnaire items can probe dispositions
Can infer dispositions from behavior and products
Can talk with students about their feelings
Links Between Achievement Targets and Assessment Methods
Reproduced with permission from Rick Stiggins
Module 2 31
WHAT DO WE WANT TO ASSESS? — ACHIEVEMENT
TARGETS• Knowledge and
understanding• Reasoning• Performance
Skills• Products• Dispositions
Module 2 32
GENERAL LEARNER OUTCOMES
1. Self-Directed Learner2. Community Contributor3. Complex Thinker4. Quality Producer5. Effective Communicator6. Effective and Ethical User of
Technology
Module 2 33
WHAT ARE WAYS OF ASSESSING? —
ASSESSMENT METHODS• Selected response
• Essay
• Performance assessment
• Personal communication
Module 2 34
Selected Response Example
Benchmark: Explain the phases of the moon and eclipses.
A lunar eclipse occurs when:A. The moon passes between the earth and the sun.B. The earth passes between the moon and the sun.C. The sun passes between the earth and the moon.D. Every month when there is a “new moon”.E. Both answers A and D.
Module 2 35
Short ResponsesBenchmark: Describe and explain the effects of multiple forces acting on an object.Prompt: There are at least two forces that affect the speed of your car, forward motion and resistance to forward motion.
– Explain how these forces affect the performance of your solar car and give an example to support your answer.
Module 2 36
Personal Communications
Type of Target AchievementDispositions
Assessment MethodPersonal CommunicationCan talk with students about their feelings.
Fine Arts:Integrated Stds, Grade 1 Performance Indicator: Discusses personal feelings about a variety of works.
Make an animal sculpture with clay starting with a pinch pot. Tell how you feel about the creature that you made.
Module 2 37
Task+ Criteria
= Performance Assessment
Module 2 38
Task• Any activity, project,
prompt, problem or assignment that is given to the student to do.
• The quality of the student response to the task is what is assessed using the criteria.
Module 2 39
Criteria for SelectingPerformance Tasks
• Measure several standards or outcomes
• Involve critical thinking skills• Use instructional time wisely• Focus on credible real world tasks• Minimize bias and is fair• Is feasible• Is clearly defined-- Adapted from Herman, Aschbacher and Winters (1992)
Module 2 40
Activity1. Partner with someone and identify a grade
level.
2. Using the Curriculum Framework at your table,
identify standard(s) and the grade level
performance indicators (GLPI).
3. Based on the GLPI, identify the most
appropriate assessment target and method.
4. Develop performance task.
Module 2 41
What are your ah-has and questions on Target-Method-Match?
REST STOP
Module 2 42
Five Standards of Assessment Quality
Standard 1Clear Targets
Standard 2Clear and
Appropriate Users and UsesStandard 3
Appropriate Assessment Methods
Standard 4Sampling
Standard 5Potential Sources of Bias and Distortion
Module 2 43
Know your students
Observations
Communications Products
Module 2 44
Triangulation of Evidence: Assessment Data from
Many Sources
Watching Students
Interviews, Writings,Discussions
Student Work,Evidence
Module 2 45
Create Content Area Rubric Instructions
• In your groups, choose a sample performance assessment task in an Instructional Guide or use the task created in the prior activity.
• Create a content area rubric using the 4 Steps process for a performance assessment task.
Module 2 46
Decisions About Performance Criteria
Performance Criteria
Task Specific Generalized
Holistic (on overall
Score)
Analytical Trait -
several dimensions scored separately
Quantity or Quality
...Amount of Detail
Number of Score Points
Module 2 47
General or Task Specific?
• GENERALIZED or Generic - can be used across similar performances.
• TASK-SPECIFIC- each one can only be used for a single task.
Module 2 48
Holistic or Analytical?
HOLISTIC•gives a single score or rating for an entire product or performance based on an overall impression of a student’s work.
ANALYTICAL TRAIT
• divides a product or performance into essential traits or dimensions so that they can be judged separately.
Module 2 49
Holistic or Analytical Trait
HolisticUse :• Quick snapshot of overall
status• When speed of scoring is
important• Simple products or
performances
Disadvantages:• 2 students can get same
score for different reasons
• Can’t identify strengths & weaknesses
• Not useful for students
AnalyticalUse• Planning instruction - show
relative strengths & weaknesses
• Details to teach student quality
• Detailed feedback• Precision more important
that speed:Disadvantages:• Scoring is slower• Takes longer to learn
Module 2 50
Options for Selecting Rubrics
• Create your own - build from scratch• Adopt - use an existing rubric• Adapt - Modify or combine existing
rubrics– Reword parts– Drop or change one more scales– Omit criteria not relevant– “Mix” and Match”– Divide a holistic rubric into several scales– Add a “no response” category– Modify for different grade levels
Module 2 51
RELIABILITY AND VALIDITY
Reliability and
Validity
Module 2 52
Five Standards of Assessment Quality
Standard 1Clear Targets
Standard 2Clear and
Appropriate Users and UsesStandard 3
Appropriate Assessment Methods
Standard 4Sampling
Standard 5Potential Sources of Bias and Distortion
Module 2 53
Module 2 54
Board Policy 4501
Grades recorded by teachers must meet the dual criteria of validity and reliability.
•Validity: grades have been based on HCPS.
•Reliability: – Sufficient number of grade entries to allow
students ample opportunities over a grading period.
– Records are maintained accurately, legibly and support the grades given.
Module 2 55
ACTIVITY—Assessing student work with a rubric or checklist
1. Individually read and score student work samples one, two, and three using the rubric.
2. Share the scores in your group.3. Discuss the criteria on the rubric to develop
common understanding.4. Individually read and score student work
samples four to seven using the rubric.5. Share the scores in your group.
Module 2 56
Reflect on the following:
What do your teachers need to know to use rubrics effectively?
REST STOP
Module 2 57
Validity
A good rubric must possess “validity.”Definition: • the extent to which what is being
measured by an instrument is actually what is intended. Are the test and rubric actually measuring the desired performance-outcomes?
Module 2 58
Reliability
A good rubric must possess “reliability.”
Definition: • the extent to which the measuring
instrument yields responses that are consistent across time and between different scorers.
Module 2 59
Reliability• Factors that have nothing to do with the
purpose of the assessment can impact scores a student can get. Examples:– Rater fatigue– Rater’s mood– Failing students may get more critical look
than “good” students
• Well-designed rubrics respond to this concern by establishing a description of the scoring criteria in advance. Scorer should revisit the established criteria to ensure consistency.
Module 2 60
Reliability (cont.)
• Clarity of the rubric– Are the scoring categories well-defined?– Are the differences between scoring
categories clear?– Would two independent raters arrive at the
same score for a given response?
• Use anchor papers.• Whenever possible, share scoring
rubrics with students in advance.
Module 2 61
Process for Training and Managing Scorers
1. Duplicate a sampling of all tests and have all scorers evaluate and score each test.
2. Ask all scorers to review each other’s scoring.
3. Ask them to discuss discrepancies.4. Arrive at consensus on interpretation
and application of rubric criteria.5. Ask scorers to jointly-rescore tests.6. Periodically review each other’s tests.
Module 2 62
A Final Caution
• Establishing reliability is a prerequisite for validity.
• Although a valid assessment is by necessity reliable, the contrary is not true (e.g. you may have a reliable test that focuses too much on an element that is not relevant to the purpose of the assessment).
• On a reliable test, a student should get the same score regardless of when the test was taken, when it was scored and who scored it.
Module 2 63
What concerns or issues regarding reliability and validity need to be considered when developing and using rubrics?
REST STOP
Module 2 64
Rubric Development: Step 4
REVISE• Look at your American Idol rubric.• Consider these questions: - “Are there any new criteria we need to add?” - “Is there anything we’ve listed that someone doesn’t understand?” - “Have we included any criteria that are not
significant?”
• Add, Revise, Refine your rubric.
Module 2 65
Activity In your groups:• Select a song that can be performed
in 1 minute. • Practice the song so that you meet all
the performance criteria in the American Idol rubric.
• Perform the song to the large group.• Audience will assess the performance
using the American Idol rubric.
Module 2 66
Strategies for Using Rubrics
as Instructional Tools1. Understandable vision of the learning
target2. Use Models3. Descriptive Feedback4. Student Self-Assessment5. Focused on One Aspect6. Focused Revision7. Student Self-Reflection and
Communication About Learning
Module 2 67
Debrief• When is it appropriate to use rubrics?• How can I use Target-Method-Match to
guide my classroom assessment?• What are some strategies for involving
students in assessment?
Module 2 68
Part V: Planning Next Steps
Complex/District Planning Time…