How to Make Meaning from your Data:The InTeGrate Attitudinal Instrument

Kim Kastens & Vali Mara InTeGrate External Evaluation Team

For the Materials Development WorkshopCarleton CollegeJune 21, 2016

Inputs Activities OutcomesOutputs

http://www.pbs.org/wgbh/nova/sciencenow/3410/03-ever-01.html

Evaluation examines the effectiveness of InTeGrate in the aggregate, and does not monitor individual students, teachers, or modules. We focus on the flock, not the birds.

InTeGrate Attitudinal

Instrument (IAI)

Geoscience Literacy Exam

(GLE)

Essay questions on

interdisciplinary problem solving

& system thinking

Embedded summative

assessments

Students’ knowledge about the Earth, oceans, atmosphere and climate

Students’ ability to contribute to addressing grand challenges of environmental sustainability

Students’ mastery of the learning goals established for the module or course

Students’ interest in Earth-related careers and college major; also student demographicsStudents’ motivation to contribute to addressing grand challenges of environmental sustainability

This talk

InTeGrate Attitudinal Instrument (IAI)

• Administered online pre- and post-instruction

• Used in pilot tests of new materials, some Implementation Programs, and research project.

• Probes:o Career interests and college major selectiono Concern about environmental issues and motivation towards

sustainability behaviorso Demographics

• New for 2016 Materials Developers’ Meeting:o Toolkit for analyzing IAI data for a subsample you care abouto Built in Microsoft Excelo Pre-populated with a project-wide data set from 1125 students

who used InTeGrate modules from Fall 2012 to Spring 2014o One Excel spreadsheet per IAI itemo Paste in your data, and spreadsheet generates graphs and

tables comparing your data with the project-wide sample.

Go to Excel now

• Demo SusGrph first

• Then demo SusEmp

SusGrph: People differ in how motivated they are to take action in their personal and professional lives to create a more environmentally sustainable society. Which of the following graphs best represents your degree of motivation before and after taking this course?

(2014 data: question asked on post- only: non-respondents omitted)

Shown in Excel Demo

CarGrph: Which of the following graphs most accurately depicts your level of interest in a career in Earth or Environmental Sciences before and after taking this course or studying this module?

CarGrph: Which of the following graphs most accurately depicts your level of interest in a career in Earth or Environmental Sciences before and after taking this course or studying this module?

SusEmp: As you consider employment after graduation, how important is it to you to work in an organization committed to environmentally sustainable practices (independent of the field)? Examples of environmentally sustainable practices would include minimizing energy and water use in the workplace. Please select one number: [Likert scale 1 = not important to 7 = very important]

Shown in Excel Demo

UseKno: As you consider career directions after graduation, how important is it to you to do work in which you use your knowledge of the Earth and environment? Please select one number: [Likert scale 1 = not important to 7 = very important]

4.74.1

4.84.2

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

EnvCrn: Please indicate your level of concern about each of the following potential developments on the Earth. Focus on the impact on your region in your lifetime.

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week:

Not at all

Once a week

2-3 times per week

Daily

Turned off the water while brushing teeth

( ) ( ) ( ) ( )

Recycled paper, glass or aluminum ( ) ( ) ( ) ( )

Washed clothes in cold water ( ) ( ) ( ) ( )Unplugged appliances to eliminate 'ghost' power use

( ) ( ) ( ) ( )

Walked or biked instead of using a car

( ) ( ) ( ) ( )

Turned off lights when leaving a room

( ) ( ) ( ) ( )

Used public transportation instead of a car

( ) ( ) ( ) ( )

Used a 'power saver' scheme for your computer

( ) ( ) ( ) ( )

Purchased locally grown food ( ) ( ) ( ) ( )

Brought a reusable bag to the store ( ) ( ) ( ) ( )

Talked to your friends and family about the ways that human actions affect the environment

( ) ( ) ( ) ( )

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week:

SusAct: Please indicate the extent to which you have engaged in the following activities during the past week:

Index is constructed by summing number of sustainable activities respondent engaged in at least once per week.

6.0 6.5

WhySus:. When you engage in behaviors such as those listed in the previous question, what factors or sources of information influence your decision to do so? (mark all that apply):

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( ) no

If yes, how? If no, why not?

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( X ) no

If yes, how? If no, why not?

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( ) yes ( X ) no

OpResp: As you think about your future, can you envision using what you have learned in this course to help society overcome problems of environmental degradation, natural resources limitations, or other environmental issues? ( X ) yes ( ) no

Questions? Comments?

Kim Kastens kastens@ldeo.columbia.edu

Vali Mara vmara@ciesin.columbia.edu