Assessment II:Moving from Theory to Practice

April 14, 2011

Ken Doxsee, Academic Affairsand

Jason Schreiner, Teaching Effectiveness Program

Overview

• Motivation – The value of assessment• Curriculum mapping – the concept• Samples of curriculum maps• Creation of a curriculum map– Steps and strategies– Establishing outcomes at the Department or

Program level

Resources• University of Oregon Assessment Workshop Slides

http://assessment.uoregon.edu/node/70

• University of Hawaii Manoa Curriculum Mapping/Matrixhttp://manoa.hawaii.edu/assessment/howto/mapping.htm

• University of West Florida Center for Teaching, Learning and Assessmenthttp://uwf.edu/cutla/curriculum_maps.cfm

• Boise State Universityhttp://ctl.boisestate.edu/programs/CurriculumRevision.asp

http://ctl.boisestate.edu/maps/Faculty_Program-Level_Outcomes.pdf

Student Experience at the Research University Survey (SERU)

• Origins in U of California system – UCUES• Extension to AAU institutions– Oregon, Texas, Florida, Pitt, Rutgers, Michigan,

Minnesota• Highlights– Annual analysis of student self-reported observations

and experience– Target population – all undergraduates– Data drillable to individual department level

• 2010 UO SERU – ca. 3,800 student responses

SERU Focus Areas

• Satisfaction with Educational Experience• Current Skills Self-Assessment• Gains in Self-Assessment of Skills• Development of Scholarship• Understanding Other Perspectives• Research Experiences• Quantitative Skills• Use of Time / Academic Disengagement

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Overall Academic Experience

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

Value of Education for Price

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Quality of Advising - Faculty

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Quality of Advising - Student Peers

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Quality of Advising - School Staff

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Quality of Advising - Dept Staff

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied

-10%

0%

10%

20%

30%

40%

50%

60%

Quality of Instruction - Faculty

UO Dept X

Very satisfied Satisfied Somewhat satisfied Somewhat dissatisfied Dissatisfied Very Dissatisfied

-10%

0%

10%

20%

30%

40%

50%

60%

Quality of Instruction - GTF

UO Dept X

Strongly agree Agree Agree somewhat Disagree somewhat Disagree Strongly disagree0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

I feel I belong on this campus

UO Dept X

Strongly agree Agree Agree somewhat Disagree somewhat Disagree Strongly disagree0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Knowing what I know, I would still enroll on this campus

UO Dept X

- 0.50 1.00 1.50 2.00 2.50 3.00 3.50 4.00 4.50 5.00 5.50

4.05 3.98 4.00

3.05 3.48 3.56

4.88

3.73 4.11

When you started here

Mea

n

- 1.00 2.00 3.00 4.00 5.00 4.74 4.66 4.69

3.44

4.69 3.79

5.16 4.58 4.56

Current ability level

Me

an

Self-Reported Gains in Competencies

UO Students Think They Learn A Lot

• Does that make you wonder if it’s true?• If so, when do these gains occur?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

-7.00

-5.00

-3.00

-1.00

1.00

3.00

5.00

7.00

CAS - Group X

* *

* * * * *

* = n < 10

Time UtilizationTime spent in class (blue) and out of class on academic pursuit (red), relative to UO average

1 2 3 4 5 6 7 8 9 10 11 12 13

-7.00

-5.00

-3.00

-1.00

1.00

3.00

5.00

7.00

CAS - Group Y*

* *

* = n < 10

Time UtilizationTime spent in class (blue) and out of class on academic pursuit (red), relative to UO average

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

-7.00

-5.00

-3.00

-1.00

1.00

3.00

5.00

7.00

CAS - Group Z

*

BB

B

* = n < 10B = Bend campus offering

Time UtilizationTime spent in class (blue) and out of class on academic pursuit (red), relative to UO average

Curriculum Mapping

A method to align instruction in courses with desired program-level outcomes and to support decision making about the curriculum.

– Are the objectives aligned with the curriculum?– Is the curriculum aligned with overall objectives?– What is the relationship between what students do in their

courses and the program’s learning objectives?– Identification of gaps (or overemphasis) can lead to

curricular change, offering promise for improved student learning and attenuated faculty workloads.

Benefits of Curriculum Mapping

• Improves program coherence.• Increases the likelihood that students achieve

program level outcomes.• Documents what is taught and when.• Reveals gaps and redundancies in the

curriculum.• Assists the program in making informed

decisions regarding the curriculum.• Helps design an assessment plan.

Types of Curriculum Maps

• “Yes or no”• “Intensity scale”– E.g., Introduced/Developing/Mastering– E.g., Basic/Intermediate/Advanced

• Scales of maps– Alignment of course assignments with course or

program goals– Alignment of course goals with program goals– Curricular alignment with program goals

“Yes or No” Mapping

“Intensity Scale” Mapping

Assignment Alignment with Course(or Program) Objectives

Course Alignment with Program Objectives

Curriculum Alignment with Program Objectives

http://assessment.uconn.edu/docs/Curriculum_Map_Business.pdf

I=Introduce; R=Reinforce; E=Emphasize

Developing and Using aCurriculum Map

• Develop/confirm program-level student learning outcomes.• List recommended and required courses, including Core/LFL courses.• Create the map in the form of a table.• Mark courses that currently address these outcomes indicating the

level at which the outcomes are addressed (and how they are assessed, if you wish).

• Analyze the map, noting gaps, redundancies, and areas where additional information is needed.

• Gather additional information including evidence related to student achievement of the outcomes.

• Use the map to make decisions about the program’s curriculum and assess its effectiveness.

Program Core Outcomes

• Communication: oral (speaking/listening) and written (reading/writing)

• Interpersonal: collaboration, leadership

• Problem-solving: application of content/methods to a variety of contexts

• Critical thinking: application of inquiry/methods to a variety of contexts

• Information literacy: how to use information (computer, library, media, technical, modeling, etc.)

Program Core Outcomes

• Multicultural awareness: respecting others and multiple views

• Intellectual flexibility: open to new ideas and adaptive to changing environments

• Methods: inquiry process, evidence gathering and assessment, statistics, etc.

• Ethics: world, personal, research

• Responsibility: well-rounded character, civic engagement, etc.

Examples

Vague: The student will gain knowledge of automated chemistry tests.

Specific: The student will state the principle for each automated chemistry test listed.

Examples

Vague: The student will be familiar with red blood cell maturation in the bone marrow.

Specific: The student will diagram the maturation of red blood cells.

Examples

Vague: The student will become familiar with theories of population growth.

Specific: The student will compare and contrast neo-Malthusian, modernization, and distributionist theories of population growth.

Examples

Vague: The student will understand the benefits of various exercise modalities for an elderly person.

Specific: The student will determine the most appropriate exercise modality for health maintenance in a patience who is elderly.

Drafting ProcessDraft: Students will be familiar with the major theories of the discipline

Revision 1: Students will be familiar with withdrawal, smoothing, forcing, compromising, and problem solving

Revision 2: Students will summarize the concepts of withdrawal, smoothing, forcing, compromising, and problem solving

Specific: Students will summarize the five major approaches to conflict resolution: withdrawal, smoothing, forcing, compromising, and problem solving

Higher order thinking objective: Students will choose and defend a conflict resolution approach appropriate for a given situation

Course ExampleThis course is designed to facilitate your learning and practice of essential knowledge and skills for engaging in critical social and environmental inquiry or ‘critical environmental studies.’ If you invest your time and effort fully in meeting course expectations and requirements, you should finish the term being able to:

– Recognize and describe environmental problems

– Examine and diagnose social ‘root causes’ of environmental problems

– Appraise potential solutions for addressing environmental problems

– Articulate viable courses of action for addressing environmental problems

– Use your own voice to contribute meaningful ideas to public discourse about environmental issues

– Summarize and critique major social scientific interpretations of environmental change