1

Correlating Knowledge of Landscape Formation Timescales and Geologic Time using a New Validated Concept Test.

~

Dep’t Earth and Ocean Sciences

Alison Jolley, Francis Jones, Sara Harris, Jamil RhajiakEarth and Ocean Sciences, UBC.

2

Outline

1. Context

2. Geotime concepts

3. Landscape Identification and Formation Test (LIFT)

4. Results & implications

5. Lessons learned, and questions

3

Context• Concept tests: used increasingly in several disciplines

– To measure learning gains (in a course or module)– To compare students, pedagogies, etc.

• 2008: GeoTime test developed– Concept test goal: measure learning gains in EOS.– Student attitude survey about Earth & Ocean Sciences

• 2009: L.I.F.T. developed, focusing on geomorphic rates and processes.

4

Geotime concept test development• Experts: Interviewed to identify topics of interest.

– 20 questions produced.

• Students: Validation via iterative think-aloud interviews.• The Test: Range of concepts & Bloom’s level coverage.

Key Concepts GeoT Timescale 3 Relative dating 6 Absolute dating 4 Earth history 6 Uniformitarianism 1 Processes & rates 0

Bloom's level GeoT knowledge 8 comprehension 3 application 5 analysis 4

(Honors thesis, Rhajiak, 2009)

# Qns

• After using GeoTime in a 3rd/4th yr elective course, eight questions were selected for use in L.I.F.T.

341

321

5

Landscapes concept test development

Iterativ

e

Iterativ

e

revision

revision

Iterativ

e

Iterativ

e

revision

revision

(Honors thesis, Jolley, 2010)

• Questions first.

• Student interviews crucial for validation.

• Expert interviews used to generate ranges of acceptable answers.

6

Expert results for answer ranges:

• 12 landscapes: seconds < T < 108 yrs.

• 7 experts;“Correct” selected from the range of consensus.

• “Incorrect” added outside that range.

• Spread of expert answers increased for middle time ranges.

7

LIFT question sequence:1. Image projected for 45 seconds. Students answer the following:

2. Next image is shown (total of 12 images used).

3. Eight Geo-Time questions at the end. Image Copyright © Marli Miller, University of Oregon; Image source: Earth Science World Image Bank; http://www.earthscienceworld.org/images

ID

Confidence

Confidence

Timescale

8

Initial LIFT deployment

• Two geoscience courses: 2nd yr. and 4th yr. (N = 71 and 25) .

• Geoscience majors (9 not geoscientists in 4th yr).

• 30 mins. / paper-based / marked by the researcher.

• Demographics recorded: gender, age, major, prior geology courses taken.

9

LIFT results: by class vs. by experience

• Diverse geoscience classes: class year NOT a good indicator of geoscience ability.• Correlation between knowledge of geologic time and landscape formation times• ALL students: - Good at recognizing landscapes.

- LESS good at estimate formation times.• Lower scores seem “overconfident”. (more later…)

10

LIFT results: right / wrong sorted by timescale

• Time scales not well recognized.• “Locally familiar” landscapes recognized better.• Very short and very long timescales known better.• Experts also were variable with intermediate timescales.• What implications for geo-science classes ???

ID vs. Formation timescales

0% 20% 40% 60% 80% 100%

MountainsVolcanoU-valley

River Hoodoos

Alluvial fanSand dunesMud cracks

Lava flowLandslide

FaultImpact crater right

wrong

0% 20% 40% 60% 80% 100%

MountainsVolcanoU-valley

River Hoodoos

Alluvial fanSand dunesMud cracks

Lava flowLandslide

FaultImpact crater right

wrong

11

Confident NOTadvanced 70% 9%beginners 58% 23%

Confidence in formation times (correct landscape ID).

• Beginners: confidence is “flatter”, regardless of right or wrong.

Do beginners’ metacognitive skills improve simply with more practice?

0%5%

10%15%20%25%30%35%40%45%

10 30 50 70 90

Perc

enta

ge e

ach

grou

p

Confidence

Confidence in time when wrongbeginneradvanced

0%5%

10%15%20%25%30%35%40%45%

10 30 50 70 90

Perc

enta

ge e

ach

grou

p

Confidence

Confidence in time when rightbeginneradvanced

• Advanced: more certain than beginners when right.

12

LIFT results – gender (all students) (preliminary)

• All roughly equal at I.D.I.D. and confidence in ID.

• All have roughly equal timescaletimescale scores.

• MALES seem more confident about timescales.

• FEMALES seem to have alarger std. errors.

30

35

40

45

50

55

60

65

70

75

80

30 35 40 45 50 55 60 65 70 75 80

Scor

e on

test

s %

Corresponding confidence %

ID and Timescale: score vs. confidence

Identification

Timescale

Female (N=52)

Male (N=44)

Male (N=44)

Female (N=52)

13

Geological time scores by gender & prior knowledge:

Evidently, at 4th yr level: • Little effect of gender, • Some effect of prerequisite.

Evidently, at 2nd yr level:• Slight effect of gender.• Nil effect of major.

0123456

F (42) M (29)

2nd yr by gender

0

1

2

3

4

5

6

F (10) M (15)

4th yr by gender

Regarding knowledge of geological time:

0123456

geosci (49)

not geosci (22)

2nd yr by major*

0123456

geosci (16)

gen sci (9)

4th yr by prerequisite*

14

Comments so far:

• These data provide an initial window into …– Comparing types of knowledge about geological time – Degree of agreement among experts about landscapes – Impact of students’ background in diverse courses– Development of metacognition (confidence) in the context

of geological time and processes– Priorities for teaching various types of knowledge & skills

• Lessons: assessing geoscience learning …– Is context dependent – Requires various types of validated instruments– Is challenging

15

Conclusions

• Both theses at : https://circle.ubc.ca/handle/2429/23321

• Questions and discussion

Thanks to: - Students who participated.- Faculty who contributed time, advice and expertise.- Colleagues with the Carl Wieman Science education Initiative.- Colleagues at Colorado University.