Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students
description
Transcript of Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students
![Page 1: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/1.jpg)
Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students
Adam ZwickleTomas KoontzAndrew BodineKristina Slagle
Environmental and Social Sustainability Lab
![Page 2: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/2.jpg)
2
Overview Need for a systematic way of measuring knowledge of sustainability concepts
Developing an assessment
Results
Where we go from here
![Page 3: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/3.jpg)
3
Need for measuring knowledge Lacking in the scientific community
“Sustainability” has been defined and measured a myriad of ways Often focused merely on the environmental aspect
Commonly tangled with values, attitudes, and beliefs
Needed to operationalize abstract goals…
![Page 4: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/4.jpg)
4
Need for measuring knowledge University goals-
More along the lines of: “Become carbon neutral by 2050”
Less common: “Create sustainably minded citizens of tomorrow”
![Page 5: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/5.jpg)
5
Developing an assessment Built upon the “triple bottom line”, the
“three legged stool”, the “3 p’s” Environmental (planet) Economic (prosperity) Social (people)
Basic knowledge from each knowledge domain?
~OR~ Knowledge bridging the different spheres?
![Page 6: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/6.jpg)
6
Developing an assessment Replicated questions used in the past
Coyle, 2005. “Environmental Literacy in America.”
Solicited topics and questions from experts
Held expert focus groups Pilot tested among professors, graduate,
and undergraduate students Narrowed down to 30 questions
![Page 7: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/7.jpg)
7
Conceptualizing sustainability knowledge
Social
Economic
Environmental
Sustainability
![Page 8: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/8.jpg)
8
Phase 1 Administered 30 question assessment
Through an online survey
Sent to over 10,000 currently enrolled undergraduates 14.3% response rate
![Page 9: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/9.jpg)
9
Phase 1 - Analysis Utilized Item Response Theory (IRT)
To remove 14 questions
IRT- Developed in the educational testing field Used for standardized tests (GRE)
![Page 10: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/10.jpg)
10
An example of IRT…
Information
Correct
Incorrect
50/50threshold
“Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?”
![Page 11: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/11.jpg)
11
Questions - Environmental
What is the most common cause of pollution of streams and rivers?
Ozone forms a protective layer in the earth's upper atmosphere. What does ozone protect us from?
What is the name of the primary federal agency that oversees environmental regulation?
What is the primary benefit of wetlands? Which of the following is an example of sustainable
forest management? In the U.S., what do we currently do with the
nuclear waste generated by nuclear power plants?
![Page 12: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/12.jpg)
12
Questions - Social
Which of the following is the most commonly used definition of sustainable development?
The wealthiest 20% of people in the U.S. own approximately what percent of the nation’s privately held wealth?
Over the past 3 decades, what has happened to the difference between the wealth of the richest and poorest Americans?
Higher levels of education generally lead to... Which of the following populations has the
highest rate of growth?
![Page 13: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/13.jpg)
13
Questions - Economic
Many economists argue that electricity prices in the U.S. are too low because…
Which of the following countries has now passed the U.S. as the biggest emitter of the greenhouse gas carbon dioxide?
Which of the following is a leading cause of the depletion of fish stocks in the Atlantic Ocean?
Which of the following is the most commonly used definition of economic sustainability?
Which of the following is the primary reason that gasoline prices have risen over the last several decades in the U.S.?
![Page 14: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/14.jpg)
14
Results Environmental
6 questions
Mean: 4.39/6SD: 1.48
73%
Social5 questions
Mean: 3.55/5SD: 1.2371%
Economic5 questions
Mean: 3.03/5SD: 1.2761%
Total16 questions
Mean: 11.08/16SD: 3.21
69%
![Page 15: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/15.jpg)
15
Example Analysis
Mean1
4
7
10
13
16
9.5710.57 10.95 11.34
Freshman
Sophomore
Junior
Senior
ANOVAF(3,1330)=8.09, p<.001
![Page 16: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/16.jpg)
16
Example Analysis Compare sub score by major
Economic score Economics: 3.82
Aerospace engineering: 4.00 Animal science: 3.15 Finance: 3.02 English: 2.80 Accounting: 2.63
![Page 17: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/17.jpg)
17
Phase 2 Combined our shortened set with a
separate assessment developed at the University of Maryland
Administered survey at both campuses
![Page 18: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/18.jpg)
18
Phase 2 - Analysis IRT to compare information richness of
questions across campuses using local descriptives (“Fishermen on Lake Erie”
vs. “Fishermen in the Chesapeake Bay”) different ways of asking about the same topic
Structural equation modeling Compare question types (multiple answer?) Knowledge structures (3 or 1 domain?)
![Page 19: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/19.jpg)
19
Structural equation modeling
Social
Economic
Environmental
Sustainability
![Page 20: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/20.jpg)
20
Survey Implementation Best practices to maximize response
rates: Invitation script (from authority vs.
student) Page length vs. number of pages Use of incentives Time of academic calendar Time of day, day of week Reminders
![Page 21: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/21.jpg)
21
Moving Forward Adjust the instrument:
Some answers change over time Develop test bank of validated questions –
try this on your campus and send us questions
Be clear about limitations: Multiple choice format Measures knowledge only
Publish Phase 1 – International Journal of
Sustainability in Higher Education, 2014
![Page 22: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/22.jpg)
22
Moving Forward Goals:
Evaluate sustainability themed majors, programs, and courses
Serve as an indicator for comparison within institutions
Encourage friendly competition
Hopes: Help solidify the concept of
“sustainability” Provide some credence for more abstract
educational goals
![Page 23: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/23.jpg)
23
AcknowledgementsFunded by:
The Ohio State Office of Sustainability http://sustainability.osu.edu/
OSU’s School of Environment
& Natural Resources http://senr.osu.edu/
![Page 24: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/24.jpg)
24
Thank You!
Environment and Social Sustainability Lab www.ess.osu.edu Contains:
This presentation The 16 phase 1 questions with multiple choice
answers
Email [email protected] [email protected]
Questions?
![Page 25: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/25.jpg)
25
Appendix Phase 1 -16 Question Sustainability
Literacy Assessment
![Page 26: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/26.jpg)
26
Questions - Environmental What is the most common cause of pollution of streams and
rivers? Dumping of garbage by cities Surface water running off yards, city streets, paved lots,
and farm fields Litter near streams and rivers Waste dumped by factories Don't know
Ozone forms a protective layer in the earth's upper atmosphere. What does ozone protect us from? Acid rain Climate change Sudden changes in temperature Harmful UV rays Don't know
![Page 27: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/27.jpg)
27
Questions - Environmental What is the name of the primary federal agency that
oversees environmental regulation? Environmental Protection Agency (the EPA) Department of Health, Environment, and Safety (the DHES) National Environmental Agency (the NEA) Federal Pollution Control Agency (the FPCA) Don't know
What is the primary benefit of wetlands? Promote flooding Clean the water before it enters lakes, streams, rivers, o
r oceans Keep the number of undesirable plants and animals low Provide good sites for landfills Don't know
![Page 28: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/28.jpg)
28
Which of the following is an example of sustainable forest management? Setting aside forests to be off limits to the public Never harvesting more than what the forest produces in new
growth Producing lumber for nearby communities to build affordable housing Putting the local communities in charge of forest resources Don’t know
In the U.S., what do we currently do with the nuclear waste generated by nuclear power plants? Use it as nuclear fuel Sell it to other countries Dump it in landfills Store and monitor the waste Don't know
Questions - Environmental
![Page 29: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/29.jpg)
29
Questions - Social Which of the following is the most commonly used definition of
sustainable development? Creating a government welfare system that ensures universal
access to education, health care, and social services Meeting the needs of the present without compromising the a
bility of future generations to meet their own needs Setting aside resources for preservation, never to be used Building a neighborhood that is both socio-demographically and economically diverse Don’t know
The wealthiest 20% of people in the U.S. own approximately what percent of the nation’s privately held wealth? 20% 35% 50% 85% Don't know
![Page 30: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/30.jpg)
30
Questions - Social Over the past 3 decades, what has happened to the
difference between the wealth of the richest and poorest Americans? The difference has increased The difference has stayed about the same The difference has decreased Don’t know
Higher levels of education generally lead to... Lower levels of voter turnout Greater annual earnings Larger family size Higher self esteem Don’t know
![Page 31: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/31.jpg)
31
Questions - Social Which of the following populations has the highest rate of
growth? North America Europe China Africa Don’t know
![Page 32: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/32.jpg)
32
Questions - Economic Many economists argue that electricity prices in the U.S. are
too low because… They do not reflect the costs of pollution from generating t
he electricity Too many suppliers go out of business Electric companies have a monopoly in their service area Consumers spend only a small part of their income on energy Don’t know
Which of the following countries has now passed the U.S. as the biggest emitter of the greenhouse gas carbon dioxide? China Sweden Brazil Japan Don’t know
![Page 33: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/33.jpg)
33
Questions - Economic Which of the following is a leading cause of the depletion
of fish stocks in the Atlantic Ocean? Fishermen seeking to maximize their catch Reduced fish fertility due to genetic hybridization Ocean pollution Global climate change Don’t know
Which of the following is the most commonly used definition of economic sustainability? Maximizing the share price of a company's stock Long term profitability When costs equal revenue Continually expanding market share Don’t know
![Page 34: Developing an Assessment of Sustainability Knowledge (ASK) for undergraduate students](https://reader034.fdocuments.us/reader034/viewer/2022051401/56814af2550346895db8015c/html5/thumbnails/34.jpg)
34
Questions - Economic Which of the following is the primary reason that gasoline
prices have risen over the last several decades in the U.S.? Growing percentage of gas stations owned by large corporations Increasing oil discoveries overseas Higher rates of state and federal gasoline tax Increasing global demand for oil Don’t know