The Case Study Method for the Assessment of Student Learning: Using Scientific

Reasoning and Deep Geological Time to Predict Future Environmental Impacts

Chen ZhuGeological Sciences and School of Public and Environmental Affairs

Brooke A. Treadwell Educational Leadership and Policy Studies

Acknowledgements Departmental colleagues, Simon Brassell, Claudia Johnson,

Michael Hamburger, Jim Brophy, Bruce Douglas for sharing interests and discussions

Lilly Freshmen Learning Project leaders Joan Middendorf, David Pace, and Simon Brassell and “cohorts”

Collaborators for this research project: George Rehrey and Claudia Johnson

Often, the immensity of deep geological time (DGT) is very difficult for non-science majors to understand and to use when applying it to geological problems

DGT can be both a bottleneck and a threshold concept for many science classes

DGT is important for student success within Geology and other sciences (Hawkins 1978)

Prepare scientifically literate, informed citizens to vote in elections that will impact environmental policy on high-stakes issues such as climate change/the carbon tax and nuclear waste disposal.

The Teaching and Learning Problem

The 4.6 billion-year history of Earth

Dinosaur extinction65 mya

We are now in Holocene Epoch (11,500 years before present to the present)

Holocene is an interglacial period

Pleistocene “most recent in Greek” (11,500 - 2.5 million years before present)

Last glacial maximum 25,000 years ago

Preconceptions inhibit students from achieving an adequate understanding of DGT Numerical difficulties:

• Millions vs. billions (Trend 2000)

• Dinosaurs died before life began (Libarkin 2007)

Confusion about basic science• Lunar phases (Schoon & Boone 1998)

• Reasons for seasonal change (DeLaughter & Stein 1998)

• Global warming is due to destruction of Ozone layer

Review of Literature

DeLaughter, J. E. and S. Stein (1998). "Preconceptions about earth science among students in an introductory course." EOS 79: 429-432.

Elementary School Ault, C. R. (1982). "Time in geological explanations as perceived by

elementary school students." Journal of Geological Education 30: 304-309. High School

Hidalgo, A. J. and J. Otero (2004). "An analysis of the understanding of geological time by students at secondary and post-secondary level." International Journal of Science Education 26(7): 845-857.

Dodick, J. and N. Orion (2003). "Measuring student understanding of geological time." Science Education 87(5): 708-731.

College Libarkin, J. C., J. P. Kurdziel, et al. (2007). "College student conceptions

of geological time and the disconnect between ordering and scale." Journal of Geoscience Education 55(5): 413-422.

Catley, K. M. and L. R. Novick (2009). "Digging deep: Exploring college students' knowledge of macroevoluntionary time." Journal of Research in Science Teaching 46(3): 311-332.

Review of Literature

Visualize magnitude by analogy or metaphor-based activities (Pyle 2007).

Equate geologic time to physical distance (Richardson 2000, Hemler & Repine 2002)

Translate DGT into a familiar time span (one calendar year) (Everitt, Good & Pankiewicz 1996, Nieto-Obregon 2001).

Create personal metaphor (Ritger & Cummins 1991).

All these activities presuppose that once students compare the geological time scale to a familiar distance, volume, mass or time period, they will be more likely to grasp the brevity of humans’ existence on Earth relative to the enormity of DGT.

Review of Literature – Suggested Learning Activities

Geologic Time Metaphors

Geologic Time Metaphors

Environmental Geology G171 for non-science majors Possibly a gateway course 37 undergraduates in 2009 73% of students - First college science course 49% of students - Self reported knowing “almost

nothing” about DGT Only 8% indicated being highly confident in their

understanding of DGT

Situational Factors

Introductory Geology Courses

G103Earth Science: Materials and Processes Brophy/Dunning Fall/Spring

G104Evolution of the Earth Brassell/Millen Fall/Spring

G105Earth, Our Habitable Planet Pratt/Douglas Fall/Spring

G111Physical Geology Brophy Fall

G112Historical Geology Polly Spring

G114Dinosaurs and Their Relatives Johnson Fall

G116Our Planet and Its Future Dunning Fall/Spring

G171Environmental Geology Zhu/Douglas Fall/Spring

Quantitative skills:

• Large numbers, abstract numbers

• Proportions and scales Lack of scientific background and literacy Relation between geological time scale and

something real (dinosaurs died a long, long time ago, 65 mya)

Why is DGT a bottleneck?

Can participation in distance-metaphor building activities help students visualize the immensity of DGT?

Does learning DGT help students acquire better scientific reasoning skills?

To what extent does the case study method enable students to understand and apply the concept of DGT to problem-solving?

The Research Questions

Decoding Discipline Model and the Freshman Learning Project Where do most students get stuck?

– Pace, D and Middendorf, J. 2004. Decoding the disciplines: Helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning Summer, no. 98.

– Pace, D and Middendorf J. 2007. Easing entry into the Scholarship of Teaching and Learning through focused assessments: The decoding the disciplines approach. In To Improve Academy, (Bolton: Anker Publishing).

Background to Project

Innovation: (a) a real world environmental problem with the need of understanding geological time scale that students can relate to and engage with; (b) future vs. past; (c) apply/outcome driven

Scaffold learning - based upon Anderson’s Revised Taxonomy of Educational Objectives

How this study is different

Cognitive Process Understand >

Apply >

Analyze

Learning Opportunities

Readings3 - LecturesClass activities

2 Labs Homework #2Case-Study in final exam

Anderson, Lorin W., David R. Krathwohl, and Benjamin Samuel Bloom. A Taxonomy for Learning, Teaching, and Assessing : A Revision of Bloom's Taxonomy of Educational Objectives. Abridged ed. New York: Longman 2001.

Geological Repository for High-level Nuclear Wastes

Las Vegas

Nevada Test Site

Nevada

Yucca Mountain

Into the realm of geological time scale

July 9, 2004, the U.S. Court of Appeals of the District of Columbia vacated Environmental Protection Agency’s 10,000-year period for compliance

Now to 1 million years or “within the period of geologic stability”

In the next million years …

All of these orbital changes influence the amount of sunlight hitting the Earth.The recurrence of ice ages is roughly on 100,000 and 40,000 time scales

Milankovitch cycle

Personal engagement Nuclear waste example; requires society’s decision and has

an outcome students may care about; Case Study: local coal burning power plant (part of the

final exam); Lab requires students to create a visual product

representing geological time and apply it to the nuclear waste and global warming problems;

Authentic Assessment – Students are asked to do the “type of thinking” and tasks that the expert in that field would do.

How this study is different

Wiggins, Grant P. 1998. Educative assessment: Designing assessments to inform and improve student performance. San Francisco, Calif.: Jossey-Bass.

Mark the following events on the toilet paper, each sheet = 20 million years

Lab – Part I

Sheets Geological time Events

0.0005 10,000Age of human

0.75 15,000,000Formation of Himalayan Mountains

3.25 65,000,000Dinosaurs became extinct

9.00 180,000,000Early birds and mammals

14.00 280,000,000Dinosaurs appear, final assembly of Pangaea

18.5 363,000,000Early trees, formation of coal deposits

28.5 570,000,000Beginning of Cambrian period, rise of multi-cellular animals

170 3,400,000,000Earliest life-forms (single-celled bacterial)

190 3,800,000,000Oldest known Earth rocks

230 4,600,000,000Origin of Earth

Using the perforations between sheets as a ruler, mark the names of items as listed in the table below. You will have to calculate the number of sheets required to complete each step in the table below. Each sheet of the paper towel is equal to 10,000 years

Lab – Part II Step # Time Events

1 ~ 781,000 BP last magnetic pole reversal

2 ~126,000 BP base of Eemian interglacial phase

3 ~100,000 BP Neanderthal man

4 50,000 BP First Homo sapiens

5 15,000 BP The last glacial maximum

6 13,000 BP Humans first inhabit North America

7 10,000 BP End of last Ice Age, Modern man

8 8,000 BP Founding of Jericho, the first known city

9 ~6000 BP Human written records

10 2,000 BP Roman domination of the world

11 500 BP European rediscovery of the Americas

12 ~ 450 BP scientific revolution

13 ~220 BP First President of United States

14 ~150 BP Industrial revolution began

15 ~ 40 BP Humans first explore the moon

16 0 Now

17 2 AP Congressional election, 2010

32 APKnown crude oil reserves are used up (these numbers are highly contested. This number comes from Wikipedia as of November 14, 2006)

72 APKnown natural gas reserves are used up (these numbers are highly contested. This number comes from Wikipedia as of November 14, 2006)

~96 APGlobal warming effects set in and global mean temperature increase 2 -4 oC. Sea level rises

18 ~100 AP The birth of your 5th generation offspring.

252 AP Known coal reserves are used up (these numbers are highly contested. This number comes from Wikipedia as of November 14, 2006)

19 290 AP The 100th Olympic game

20 328 AP The 100th FIFA World Cup

21 ~ 4000 AP The 1000th President of the United States

22 ~4800 AP 5000th anniversary of the foundation of the United States

23~25,000 AP

Your 1000th great grandchildren

24~50,000 AP

Next ice age

251,000,000 AP

Safety of the Yucca Mountain repository

Given your knowledge of the geological time scale and human history, what seems like a reasonable and feasible time period for the nuclear waste to be stored safely at the Yucca Mountain geological repository? Would 10,000 years be enough? 100,000 years? 1,000,000 years? Why? Be sure to support your answer by using geological numbers and the time scale of events that you identified in the lab. This answer should be about 4-5 sentences long.

Follow up Homework

Write an essay in which you either agree or disagree with the following statement: “We are in the warm period of the glacial and inter-glacial cycle, and are surely heading toward the next glacial period. Therefore, global warming is a good thing because it will delay the coming of the next ice age.”

Follow-up Homework

I. Pre-course knowledge survey at first class

II. Lectures and class discussion about DGT

III. Distance-metaphor building lab

IV. Post lab survey/homework/essay

V. Post-course knowledge survey at last class

VI. Final take home exam using case study method

Class Activities

Pre and Post-Knowledge Survey Post lab survey Final Exam – Case Study

Evidence of Student Learning

Measure students’ perceptions of their ability to solve problems, not their actual ability.

Nuhfer and Knipp (2003) found that very few students display gross overconfidence when self reporting. They concluded that such “aberrations contributed by occasional individuals never affect a class average in a significant way” (p. 66).

We averaged student scores to draw

conclusions about improved DGT understanding and application.

Knowledge Survey

Nuhfer, E. and D. Knipp (2003). "The Knowledge Survey: A Tool for All Reasons." To Improve the Academy 21: 59-78.

Selected Questions Explain the Geological Time Scale & why it is important to Environmental

Geology Describe the age of the earth in geological time and how we know it is an

accurate estimation Estimate the number of years we should guarantee buried high level nuclear

waste will be safe at Yucca Mountain. Explain why. Estimate how far back we need to look into the geological past to determine if

human activity is causing climate change. Describe why this is significant.

Scale: How confident are you that you could answer this question on a graded test:

1 – Not confident 3 – Somewhat confident 5 – Very confident

Knowledge Survey

Pre and Post-Knowledge Survey Results

Post Lab Survey

Post Lab Survey Results

Ratings: 1 = strongly disagree 5 = strongly agree

You just joined a student environmental group. Write the text of a presentation you will give to IU student government defending the belief that anthropogenic activities are a contributor to global warming, and hence an important reason why IU should retire its coal fired power plant. Your presentation should:

Explain the immensity of geological time, variations in CO2 concentrations and glacial & inter-glacial cycles within the Quaternary

Use scientific reasoning (observations, theory, experiments, evidence & facts) to show that the CO2 increase is likely beyond natural variability

Use graphs to support your argument Identify 2 other contemporary environmental issues which require

an understanding of the geological time scale. Explain how.

Case Study

Criteria Description of the immensity of DGT Use of DGT to describe the link between the CO2 rise & global

warming to human activity Use of DGT to argue that global warming won't inhibit next ice

age Explanation of how DGT is key to understanding other

environmental issues (excluding global warming)

Scores 3 – accurate answer, clear & detailed 2 – accurate answer, some detail 1 – vague/unclear/answer indicated misunderstanding

Case Study Rubric

Case Study Results

65% Met 1 or more criteria, demonstrating at least a basic understanding of DGT

56% Demonstrated the ability to use scientific reasoning

45% Met 2 or more criteria, demonstrating a solid understanding of DGT

17.5% Met all 4 criteria, demonstrating an excellent understanding of DGT

7.5% Met and exceeded all 4 criteria, demonstrating an exceptional understanding of DGT

Students perceived the distance-metaphor activities as enabling them to visualize the immensity of DGT

Large increase in students’ confidence in their ability to engage in scientific reasoning using DGT

The majority of students demonstrated the ability to use DGT to engage in scientific reasoning

The majority of students applied the concept of DGT to problem-solve within case-study assignments

Conclusions

Adjustments to the course This Fall we added an additional metaphor building

activity to the lab, asking students to create their own visual depiction of the immensity of DGT, after they have worked in teams using the toilet paper exercise to visualize its immensity.

Further Research Larger sample size Examine students’ ability to transfer problem solving

skills to a different discipline, course, or problem set

Next Steps

Thank you

Case Study Results

65% Met 1 or more criteria, demonstrating at least a basic understanding of DGT

45% Met 2 or more criteria, demonstrating a solid understanding of DGT

17.5% Met all 4 criteria, demonstrating an excellent understanding of DGT

7.5% Met and exceeded all 4 criteria, demonstrating an exceptional understanding of DGT

56% Demonstrated the ability to use scientific reasoning

Ault, C. R. (1982). "Time in geological explanations as perceived by elementary school students." Journal of Geological Education 30: 304-309.

Catley, K. M. and L. R. Novick (2009). "Digging deep: Exploring college students' knowledge of macroevoluntionary time." Journal of Research in Science Teaching 46(3): 311-332.

DeLaughter, J. E. and S. Stein (1998). "Preconceptions about earth science among students in an introductory course." EOS 79: 429-432.

Dodick, J. and N. Orion (2003). "Measuring student understanding of geological time." Science Education 87(5): 708-731.

Everitt, C. L., S. C. Good, P. R. Pankiewicz (1996). "Conceptualizing the inconceivable by depicting the magnitude of geological time with a yearly planning calendar." Journal of Geoscience Education 44: 290-293.

Fink, L. Dee (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, Calif.: Jossey-Bass.

Hawkins, D. (1978). "Critical barriers to science learning." Outlook 29: 3-23. Hemler, D. and T. Repine (2002). "Reconstructing the geologic timeline." The Science Teacher 69(4):

32-35. Hidalgo, A. J. and J. Otero (2004). "An analysis of the understanding of geological time by students at

secondary and post-secondary level." International Journal of Science Education 26(7): 845-857.

Sources

Libarkin, J. C., J. P. Kurdziel, et al. (2007). "College student conceptions of geological time and the disconnect between ordering and scale." Journal of Geoscience Education 55(5): 413-422.

Nieto-Obregon, J. (2001). "Geologic time scales, maps, and the chronoscalimeter." Journal of Geoscience Education 49(1): 25-29.

Novak, J. D. (1988). "Learning science and the science of learning." Studies in Science Education 15: 77-101. Nuhfer, E. and D. Knipp (2003). "The Knowledge Survey: A Tool for All Reasons." To Improve the Academy

21: 59-78. Pace, D and J Middendorf (2004). Decoding the disciplines: Helping students learn disciplinary ways of

thinking. New Directions for Teaching and Learning Summer, no. 98. Pyle, C. (2007). "Teaching the time: Physical geography in four dimensions." Teaching Geography 32(3):

121-123. Richardson, R. M. (2000). "Geologic time (clothes) line." Journal of Geoscience Education 48: 584. Ritger, S. D. and R. H. Cummins (1991). "Using student-created metaphors to comprehend geologic time."

Journal of Geological Education 39: 9-11. Schoon, K. J. (1992). "Students' alternative conceptions of Earth and space." Journal of Geological Education

40: 209-214. Schoon, K. J. and W. J. Boone (1998). "Self-efficacy and alternative conceptions of science preservice

elementary teachers." Science Education 82(5): 553-568. Trend, R. D. (2000). "Conceptions of geological time among primary teacher trainees, with reference to their

engagement with geoscience, history, and science." International Journal of Science Education 22(5): 539-555. Wiggins, Grant P. (1998). Educative assessment: Designing assessments to inform and improve student

performance. San Francisco, Calif.: Jossey-Bass

Sources

Atmospheric CO2 in the past

Yucca Mt. – Potential Repository

Desert climate (170 mm/yr vs. Bloomington ~1200 mm/yr);

On federal land; Away from large population centers; Nevada has only two congress men (and two

senators);

30 years study and 30 billion dollars.

Infiltration rate

At present, evaporation exceeds precipitation (170 mm/yr) - hypothesis;

Water drips down in tunnel – reality check;

When is the next ice age?

0

5

10

15

20

Holocene Late Pleistocene

Re

ch

arg

e, m

m/y

r

Estimated Recharge for Yucca Mountain(Zhu et al. Water Resources Research , 2003)

5±1 mm/yr15±5 mm/yr

U.S. Nuclear Regulatory Commission: Next 10 ky 13-64 mm/yr 10 ky - 1 ma