1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.
-
Upload
maximilian-elliott -
Category
Documents
-
view
213 -
download
0
Transcript of 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.
![Page 1: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/1.jpg)
1
You’re the Judge Now…Getting the Scores Right
Calibration Training, continued
![Page 2: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/2.jpg)
Determining an Indicator’s Rating
Each Indicator often has several definitional narratives for each level of proficiency. However, Principals must enter only one (1) score as to each Indicator (1, 2, 3, 4, 5, NO, NA).
2
3
![Page 3: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/3.jpg)
Determining an Indicator’s Rating
“Best fit” approach
3
![Page 4: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/4.jpg)
The Goal of Rater Accuracy
To move from subjectivity to higher levels
of objectivity and insight.
4
![Page 5: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/5.jpg)
How Do You Become More Accurate?
Practice, discipline, and
intentional study of the rubric.
5
![Page 6: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/6.jpg)
Let’s Review!!
• Indicator 1• Indicator 6• Indicator 7
• Indicator 11• Indicator 12
• “I think that’s a Revulsion!”• “I like Chuck E. Cheese too!”• “If you can express your answer in
the numeric way, you can also express it in the geometric way and in words.”
• “I luv puzzles!!”• “In science, you have to be
flexible. Everyone be flexible”6
![Page 7: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/7.jpg)
Calibration Activity One
7
![Page 8: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/8.jpg)
Your First Job…
• Using your rubric, working in PAIRS, sort observation evidence into their appropriate indicator category or categories:– Indicator 1– Indicator 6– Indicator 7– Indicator 11– Indicator 12
8
![Page 9: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/9.jpg)
Answer Key to Sorting the Evidence, Calibration Activity #1
9
EvidenceIndicator 1 B, O, R, TIndicator 6 C, F, H, L, Q (also I)Indicator 7 D, E, I, M, S (also H)Indicator 11 N, P (also A, G, J and S)Indicator 12 A, G, J, K
![Page 10: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/10.jpg)
Your Second Job…
• Using your rubric, working in PAIRS, determine the “best fit score” for each piece of evidence. – Note that the evidence
typically relates to just one component of the indicator. Right now, you should ignore the other components of the indicator/rubric that are not addressed on the evidence sheet.
10
![Page 11: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/11.jpg)
Suggested* Scores
11
Evidence Indicator ScoreA 11, 12 3B 1 1-2C 6 1D 7 2E 7 5F 6 2G 11, 12 4H 6, 7 5 (Ind. 6), 3-4 (Ind. 7)I 6, 7 1-2J 11, 12 1 (Ind. 12); 2 (Ind. 11)
![Page 12: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/12.jpg)
Suggested* Scores
12
Evidence Indicator ScoreK 12 5L 6 4M 7 4N 11 2O 1 1-2P 11 2-3Q 6 3-4R 1 3-4S 7, 11 5T 1 3
![Page 13: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/13.jpg)
Wait…there’s something we haven’t talked about!
13
![Page 14: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/14.jpg)
Make Your Own “Professional Preference” Lists
• Pet Peeves
and
• Preferences
14
![Page 15: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/15.jpg)
Common Pet Peeves/Preferences
Pet Peeves/Negative Bias--messy rooms--sloppy attire--food/drink in the classroom--accents, loud speech--typohs --particular strategies
Preferences/Positive Bias--neat room--attractive appearance--humor--casual demeanor--students in uniforms--particular strategies
15
![Page 16: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/16.jpg)
Give me a 5 if you’re ready for a break!
16
![Page 17: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/17.jpg)
We’re Back…ready or not!!!
17
![Page 18: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/18.jpg)
Three Additional Indicators for Calibration
• Indicator 2: Clearly defined behavior
• Indicator 9: Active learning, questioning, incorporation of interests, and involving all students
• Indicator 13: Monitoring for progress
18
![Page 19: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/19.jpg)
19
Indicator 2
![Page 20: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/20.jpg)
Indicator 2—Teacher clearly defines expected behavior.
• Ensures that students are engaged and clear as to the expectations of the classroom with few reminders given.
• Monitors the behavior of students during whole-
class, small group and seat work activities and during transitions between instructional activities.
• Stops inappropriate behavior promptly and consistently with an appropriate voice level / word choice.
20
![Page 21: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/21.jpg)
Clearly Defining Expected Behavior: What Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level?
• students to be doing/saying if teachers are performing at an “effective” level?
21
![Page 22: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/22.jpg)
22
Indicator 9
![Page 23: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/23.jpg)
Indicator 9—Engagement of learners: active learning/ involvement/ questioning/ incorporation of interests.
• Engages most students in active learning experiences 80 percent of the class time.
• Uses questioning techniques throughout the lesson, scaffolding to at least the mid-level of Bloom's taxonomy. Provides wait time for some student response and does random checking to ensure the involvement of all learners.
• Engages students by incorporating their general skills and interests into the lesson.
23
![Page 24: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/24.jpg)
Engagement of LearnersWhat Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level?
• students to be doing/saying if teachers are performing at an “effective” level?
24
![Page 25: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/25.jpg)
25
Indicator 13
![Page 26: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/26.jpg)
Indicator 13—Teacher checks to determine if students are progressing toward stated objectives.
• When appropriate, moves to all areas of the room while students are working on guided practice to promote and reinforce positive student behaviors.
• Uses different types of student response techniques, both individual / group.
• Uses student response techniques to increase active engagement.
• Student’s understanding is evaluated by feedback. • Uses wait time of 3-5 seconds (more for more complex
questions) after voicing the question. Provides opportunity for students to formulate more thoughtful responses and allows time for the student to consider supporting evidence.
26
![Page 27: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/27.jpg)
Checks/Monitors for ProgressWhat Do you Expect…
• teachers to be doing/saying if they are performing at an “effective” level…
• students to be doing/saying if teachers are performing at an “effective” level…
27
![Page 28: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/28.jpg)
The Three-Step Process of Scoring
• Step One: Gather evidence
• Step Two: Sort evidence into indicators, as appropriate
• Step Three: Determine rating
28
![Page 29: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/29.jpg)
29
![Page 30: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/30.jpg)
Calibration Activity Two
30
![Page 31: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/31.jpg)
• During the video– Step One, Part A: Take
notes, documenting the evidence you observed relevant to indicators 2, 9 & 13.
– We will pause briefly at the 5 min. marker to give you add’l time to write your evidence down on notepaper
• After the video– Step One, Part B:
Complete your notes and review them for the most significant evidence relevant to indicators 2, 9 & 13.
– Transfer the key evidence onto sticky notes—one per piece of evidence.
31
Practice Video 1…just indicators 2, 9 & 13Step One
Independent Work
![Page 32: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/32.jpg)
Additional Notes About the Video
• Your “universe of evidence” is the video alone.– “consistently implements” = consistently within
the video– Disregard rubric requirements that require
knowledge outside of video (stricken in the rubric)
• Video format is similar to what will be used on the calibration test.– One screen on the board; one panoramic screen
on the class.
32
![Page 33: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/33.jpg)
33
Everyone Ready?
![Page 34: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/34.jpg)
Calibration Activity #2…Brief Practice Video …just 2, 9 & 13
CCOSA--Tulsa Model Video for Calibration Activity No. 2 (7th grade ELA) -- view first 12 minutes only
34
![Page 35: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/35.jpg)
• During the video– Step One, Part A: Take
notes, documenting the evidence you observed relevant to indicators 2, 9 & 13.
– We will pause briefly at the 5 min. marker to give you add’l time to write your evidence down on notepaper
• After the video– Step One, Part B:
Complete your notes and review them for the most significant evidence relevant to indicators 2, 9 & 13.
– Transfer the key evidence onto sticky notes—one per piece of evidence.
35
Practice Video 1…just indicators 2, 9 & 13Step One
Independent Work
![Page 36: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/36.jpg)
Step Two, Part A:
Using the worksheets as placemats, sort the evidence (on sticky notes) into the appropriate indicator category•If evidence applies to more than one indicator, you may copy the evidence onto multiple sticky notes
Step Two, Part B:
Share worksheets with Table Team
Consolidate the table’s findings evidence onto one placemat per indicator.
36
Practice Video 1…just indicators 2, 9 & 13Step Two
Independent Work Table Work
![Page 37: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/37.jpg)
Pause and Share Evidence with the Room
• What evidence did you have for Indicator 2?
• What evidence did you have for Indicator 9?
37
![Page 38: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/38.jpg)
Pause and Share Evidence with the Room
• What evidence did you have for Indicator 13?
38
![Page 39: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/39.jpg)
Practice Video 1…just indicators 2, 9 & 13Step Three
Step Three
Using the placemats and the rubric, determine the appropriate rating for each indicator and write the “best fit” rating on the placemat in the designated spot.
39
Table Work
![Page 40: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/40.jpg)
Pause and Share Ratings with the Room
• What rating did you have for Indicator 2?
Target: 2
• What rating did you have for Indicator 13?
Target: 2
• What rating did you have for Indicator 9?
Target: 2
40
![Page 41: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/41.jpg)
41
![Page 42: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/42.jpg)
Calibration Activity #3
Practice Test video!
42
![Page 43: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/43.jpg)
Practice Video…eight indicators
• Rating only Indicators 1, 2, 6, 7, 9, 11, 12 & 13• Viewing just the first 20 minutes of an
Observation Engine video (secondary Math).– We will NOT stop at the video while we view it.
• We will:– Independently categorize evidence, and – Independently rate the indicators; then– Share our findings with table teams and report.
43
![Page 44: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/44.jpg)
44
Everyone Ready to Cue Music?
![Page 45: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/45.jpg)
Link to Practice Test Video (Secondary Math)
CCOSA--Tulsa Model Video Practice Test (Secondary Math) -- view first 20 minutes only
45
![Page 46: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/46.jpg)
Practice Test -- Secondary
46
• Ind. 1: 3• Ind. 2: 3• Ind. 6: 3• Ind. 7: 2
• Ind. 9: 2• Ind. 11: 3• Ind. 12: 4• Ind. 13: 2
![Page 47: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/47.jpg)
Target Scores…
47
![Page 48: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/48.jpg)
Are We Ready?
48
![Page 49: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/49.jpg)
Calibration Test Basics
• Required by the state, but it’s TPS-developed• Online platform called Observation Engine.• The testing time will last 90 minutes in all.• Scoring only the 20 minutes of 3 videos– Indicators 1, 2, 6, 7, 9, 11, 12, & 13– Customized by elementary/secondary (your choice)• Either test provides K-12 calibration certification
– After each video, you will have 10 minutes to score before we proceed with the next video.
49
![Page 50: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/50.jpg)
Calibration Test Basics, continued
• Target scores determined by expert panel• Your “grade” is determined by comparing your
scores against the target scores.• Goal is 80% on target• The tests will be graded electronically using
the scantrons on which you enter your scores.• Pass/Fail data will be available in about 2
weeks, if not earlier.
50
![Page 51: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/51.jpg)
Context of Calibration Test
• Your “universe of evidence” is the video alone.–“consistently implements” = consistently within the video–Disregard rubric requirements that require knowledge
outside of video (stricken in the rubric)
• Tips–Use the truncated rubric form for each video, focusing on
each required indicator separately. –Don’t stress! You will do fine!
– Re-takes are available (with new videos)
–Don’t let your professional preferences bias you.
![Page 52: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/52.jpg)
Are we Ready? …. Questions?
52
![Page 53: 1 You’re the Judge Now…Getting the Scores Right Calibration Training, continued.](https://reader031.fdocuments.us/reader031/viewer/2022032801/56649dda5503460f94ad0d50/html5/thumbnails/53.jpg)
For Tulsa Model or Calibration Questions…
Jana Burk – [email protected] S. Kleindienst - [email protected]
Katy Ackley – [email protected] Gripado – [email protected]