Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve,...
-
Upload
mabel-lucinda-watson -
Category
Documents
-
view
221 -
download
0
Transcript of Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve,...
![Page 1: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/1.jpg)
Jazzin’ Up Assessment:
Writing Items that are Relevant and Sensitive
Scott Strother, Mike Sieve, John Kellermeier
11/19/15
![Page 2: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/2.jpg)
Quantway and Statway:Assessment Team (A-Team)
Summative AssessmentsItem BanksCurriculum Item Improvement
![Page 3: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/3.jpg)
Assessment Expert Convenin
g
Assessment Team & PD
1. Diversity of Item Types
2. High Quality Assessments
3. High Quality Items
1. Improved Network Capacity2. Leadership within Colleges3. Support and
Collaborationwithin and between colleges
![Page 4: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/4.jpg)
Assessment Expert Convenin
g
Assessment Team & PD
1. Diversity of Item Types
2. High Quality Assessments
3. High Quality Items
1. Improved Network Capacity2. Leadership within Colleges3. Support and
Collaborationwithin and between colleges
![Page 5: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/5.jpg)
High Quality Assessments Items
▪Context
▪Bias and Sensitivity
▪Relevance
▪Goal of Item (e.g. productive struggle)
▪Clear and concise
▪Depth of Knowledge
▪Cognitive Load
▪Difficulty (ensuring the appropriate spread)
▪Discrimination (of various ability levels)
▪Design Specifications
![Page 6: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/6.jpg)
High Quality Assessments Items
▪Context
▪Bias and Sensitivity
▪Relevance▪Goal of Item (e.g. productive struggle)
▪Clear and concise
▪Depth of Knowledge
▪Cognitive Load
▪Difficulty (ensuring the appropriate spread)
▪Discrimination (of various ability levels)
▪Design Specifications
![Page 7: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/7.jpg)
Writing High Quality Items
6 week course focused on:
Diverse Item Types Characteristics of High Quality Assessment Items
Understanding Depth and Rigor of Items Purpose of Assessment and Alignment
Reliability and Validity Assessment Construction
7
Carnegie’s Network Improvement Community Assessment Literacy Training
![Page 8: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/8.jpg)
The Challenge
8
How can we write items that are relevant for todays
students?
![Page 9: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/9.jpg)
The Solution
9
Carnegie’s Quantway and Statway lessons revolve around 3
central topics:
Citizenship – information about your society, government and world that can impact many
decisions you make
Medical Literacy – information about health issues and medical treatments
Personal Finance – financial information and how to use it to make decisions in your life
![Page 10: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/10.jpg)
The Solution
10
Replace problems like this….
![Page 11: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/11.jpg)
The Solution
11
Replace problems like this….
With problems like this….
![Page 12: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/12.jpg)
12
Learning Outcome: Compare two quantities using a percent
Traditional Assessment Item:
The number 36 is what percent of 56?
Making the item more relevant (Medical Literacy):
According to the CDC, there are an estimated 56 million deaths each year worldwide. Of these, 36 million deaths are caused by non-
communicable diseases, such as cardiovascular disease, cancer, diabetes and chronic lung diseases. What percent of deaths worldwide
are caused by non-communicable diseases?
http://www.globalissues.org/issue/587/health-issues
Making Items Relevant
![Page 13: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/13.jpg)
Learning Outcome: Estimate ratios of large numbers
Traditional Assessment Item:
The number 124,358,000 is larger than the number 10,342.Approximately how many times larger is it?
Making the item more relevant (Citizenship):
Throughout the past several years, the growth of the internet has exploded. According to Netcraft, an internet services company that
provides web hosting market-share analysis, there were 23,603,183 active web sites in June of 2004. This number had increased to
181,414,052 active web sites by June of 2014. Estimate how many times larger the number of active websites was in 2014 compared to 2004.
Round your answer to the nearest whole number.
13
Making Items Relevant
![Page 14: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/14.jpg)
14
Making Items Relevant
Learning Outcome: Identify and interpret the vertical/horizontal intercepts and the slope in context
Traditional Assessment Item: Use the graph pictured below, to answer the following questions:
What is the y-intercept of the graph?
What is the x-intercept of the graph?
What is the slope of the graph?
![Page 15: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/15.jpg)
15
Making Items Relevant
Making the item more relevant (Personal Finance):
What is the vertical intercept? Interpret its meaning in the context of this problem. Be sure to use quantitative information along with the writing principle that was presented in class.
![Page 16: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/16.jpg)
16
Making Items Relevant
What is the vertical intercept? Interpret its meaning in the context of this problem. Be sure to use quantitative information along with the writing principle that was presented in class.
What is the slope of the line? Be sure to include units in your answer. Write a sentence that describes the slope in the context of this problem.
Making the item more relevant (Personal Finance):
![Page 17: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/17.jpg)
The Hidden Curriculum
▪All assessment items have an overt curriculum and a hidden curriculum
▪The overt curriculum is the goal of the item
▪The hidden curriculum is the context of the item
17
![Page 18: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/18.jpg)
The Hidden Curriculum
▪Every context has a cultural location
▪Not every student (or instructor) understands that culture
▪Every context has implications for power, privilege and inequity.
18
![Page 19: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/19.jpg)
The Basketball Problem
▪The National Basketball Association (NBA) announced that a new synthetic basketball would be used for the 2006–2007 season.
▪Players in the NBA complained about the new ball. The NBA announced that the traditional leather ball would be used again beginning January 1, 2007.
▪The players’ complaints suggested that they thought that the new ball impacted their game performance. If this is true, the new ball would have a measurable effect on the games. In this task, you will develop a plan to answer the question, “Did the synthetic ball affect game performance?”
▪What variables could you look at to answer the question? Be specific about what you would measure. Think about aspects of the basketball game that you could use as variables.
19
![Page 20: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/20.jpg)
The Zodiac Problem
▪Can someone’s birth date influence his or her personality traits? If you wanted to gather information about birth dates and personality, what kind of information would you look for?
▪Your have a list of personality traits grouped by birthdays. Each group has three sets of personality traits. Only one set actually goes with that birthday group (according to the Zodiac calendar).
▪Look for your birthday. Read the three sets of personality traits for your birthday group. Choose the set of personality traits that you think matches your own personality. We want to investigate if the Zodiac’s personality predictions are likely to be correct.
20
![Page 21: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/21.jpg)
Intentional Use of the Hidden Curriculum
▪Increase awareness and cultural competence
▪Address issues of equity and diversity
21
![Page 22: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/22.jpg)
Sex and Gender
Members of an advocacy organization for transgender rights were asked what sex they were assigned at birth (female or male) and how the identify their gender presentation now (woman, man or other). The results are summarized in the table below
What percent of those who were assigned female at birth now present as other than man or woman?
22
Gender PresentationSex Assigned at Birth
Woman
Man Other Total
Female 10 62 9 81Male 75 13 11 99Total 85 75 20 180
![Page 23: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/23.jpg)
Poverty Rates of Elderly Americans
Data from 2001 gives the poverty rates of elderly Americans (65 years and older) broken down by race (U.S. Census Bureau, 2002). The poverty rate is the percent of people in a group who have income below the poverty line.
What percent of poor elderly Americans are white?
23
Poverty rates of elderly Americans by race
Race Poverty rate Percent of total elderly population
Black 21.9% 8.4%
White 8.1% 82.8%
Other 17.7% 8.7%
![Page 24: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/24.jpg)
Conclusions
24
Initially, students struggled with the format of these items compared to traditional assessment items – They needed to overcome their fear of word problems!
Soon, students began to see real value in developing their mathematical skills– The proverbial “lightbulb” came on when concepts were introduced in a relevant context!
Finally, students began using these skills across disciplines – They showed a detailed understanding of ideas by applying these skills in other classes!
![Page 25: Jazzin’ Up Assessment: Writing Items that are Relevant and Sensitive Scott Strother, Mike Sieve, John Kellermeier 11/19/15.](https://reader035.fdocuments.us/reader035/viewer/2022062322/5697bf9e1a28abf838c941b0/html5/thumbnails/25.jpg)
25
Questions