Assessment For Assessment For Improvement In The Improvement In The ClassroomClassroom

byby

Barry McNeill & Lynn BellamyBarry McNeill & Lynn Bellamy

Arizona State UniversityArizona State University

Slide 2

ASEE Seattle 1998

Need some sort of attention Need some sort of attention getter!getter!

Slide 3

ASEE Seattle 1998

Continuous Improvement Continuous Improvement ProcessProcess

Assess

Define

Modify

Slide 4

ASEE Seattle 1998

Continuous Improvement Continuous Improvement Process For A CourseProcess For A Course Defining A CourseDefining A Course

learning objectiveslearning objectives learning taskslearning tasks

Assessing The CourseAssessing The Course student workstudent work course learning objectivescourse learning objectives course learning environmentcourse learning environment student preparationstudent preparation

Modifying The CourseModifying The Course

Slide 5

ASEE Seattle 1998

Continuous Improvement Continuous Improvement Process Tools Process Tools Defining A CourseDefining A Course

Course Articulation MatrixCourse Articulation Matrix

Assessing The CourseAssessing The Course ChecklistsChecklists Competency MatrixCompetency Matrix Course Activity Impact MatrixCourse Activity Impact Matrix

Slide 6

ASEE Seattle 1998

Presentation Goals Presentation Goals

Awareness of how to create a Awareness of how to create a Course Articulation MatrixCourse Articulation Matrix

Awareness of how to create a Awareness of how to create a Course Activity Impact MatrixCourse Activity Impact Matrix

Awareness of how to Awareness of how to useuse a a Course Activity Impact Matrix Course Activity Impact Matrix to improve a courseto improve a course

Slide 7

ASEE Seattle 1998

Course Articulation MatrixCourse Articulation Matrix

A compact way to relate A compact way to relate course course learning objectiveslearning objectives with course with course learning learning activitiesactivities

Slide 8

ASEE Seattle 1998

Notes On Learning Notes On Learning ObjectivesObjectives There are different There are different levels of levels of

learninglearning The levels of learning are The levels of learning are

observableobservable or or measurablemeasurable The levels are reasonably The levels are reasonably

hierarchicalhierarchical

Slide 9

ASEE Seattle 1998

Skeleton Course Articulation Matrix (CAM)

Learning Objectives Leve

l of L

earn

ing

Cou

rse

Act

iviti

es

Level of Learning Legend K C AKnowledge Comprehension Application

Slide 10

ASEE Seattle 1998

Learning Objectives Lev

el o

f L

earn

ing

Co

urs

e A

ctiv

itie

s

1. Engineering Design Process1.1 formulating the problem1.2 solving a problem1.3 implementing a solution1.4 documenting the process

1.5 using engineering/physical principles

1.6 using quality principles

2 Working in Teams

2.1 team dynamics2.2 team communication

Level of Learning Legend K C AKnowledge Comprehension Application

Partial CAM For ECE 100

Slide 11

ASEE Seattle 1998

Learning Objectives Lev

el o

f L

earn

ing

Co

urs

e A

cti

viti

es

1. Engineering Design Process A

1.1 formulating the problem A1.2 solving a problem A1.3 implementing a solution A1.4 documenting the process A

1.5 using engineering/physical principles

K

1.6 using quality principles A

2 Working in Teams C

2.1 team dynamics C2.2 team communication C

Level of Learning Legend K C AKnowledge Comprehension Application

Partial CAM For ECE 100

Slide 12

ASEE Seattle 1998

Learning Objectives Level o

f L

earn

ing

Co

urs

e A

cti

vit

ies

In C

las

s A

cti

vit

ies

ta

ke q

uiz

zes/

exa

ms

be

fore

cla

ss

act

ive

lea

rnin

g e

xerc

ise

s

co

nst

ruct

ma

the

ma

tica

l mo

de

ls

ora

lly r

ep

ort

to p

ee

rs a

nd

cla

ss

pe

er

ass

ess

de

sig

n n

ote

bo

oks

wo

rk o

n d

esi

gn

pro

ject

s

wa

tch

ma

nu

fact

urin

g/o

the

r vi

de

os

lis

ten

to b

rief l

ect

ure

s

Ou

t o

f C

las

s A

cti

vit

ies

re

ad

an

d s

um

ma

rize

text

bo

oks

co

nst

ruct

mo

de

l ba

sed

on

ge

om

ety

Pro

jec

ts d

isse

ct a

nd

re

ass

em

ble

art

ifact

de

velo

p a

n a

sse

mb

ly p

lan

(p

roce

ss)

de

sig

n, b

uild

, an

d te

st a

de

vice

de

mo

nst

rate

de

sig

n

1. Engineering Design Process A

1.1 formulating the problem A1.2 solving a problem A1.3 implementing a solution A1.4 documenting the process A

1.5 using engineering/physical principles

K

1.6 using quality principles A

2 Working in Teams C

2.1 team dynamics C2.2 team communication C

Level of Learning Legend K C AKnowledge Comprehension Application

Partial CAM For ECE 100

Slide 13

ASEE Seattle 1998

Partial CAM For ECE 100

Learning Objectives Lev

el o

f L

earn

ing

Co

urs

e A

cti

vit

ies

In C

las

s A

cti

vit

ies

ta

ke q

uiz

zes/

exa

ms

be

fore

cla

ss

act

ive

lea

rnin

g e

xerc

ise

s

co

nst

ruct

ma

the

ma

tica

l mo

de

ls

ora

lly r

ep

ort

to p

ee

rs a

nd

cla

ss

pe

er

ass

ess

de

sig

n n

ote

bo

oks

wo

rk o

n d

esi

gn

pro

ject

s

wa

tch

ma

nu

fact

urin

g/o

the

r vi

de

os

lis

ten

to b

rief l

ect

ure

s

Ou

t o

f C

las

s A

cti

vit

ies

re

ad

an

d s

um

ma

rize

text

bo

oks

co

nst

ruct

mo

de

l ba

sed

on

ge

om

ety

Pro

jec

ts d

isse

ct a

nd

re

ass

em

ble

art

ifact

de

velo

p a

n a

sse

mb

ly p

lan

(p

roce

ss)

de

sig

n, b

uild

, an

d te

st a

de

vice

de

mo

nst

rate

de

sig

n

1. Engineering Design Process A1.1 formulating the problem A C1.2 solving a problem A C1.3 implementing a solution A C1.4 documenting the process A C

1.5 using engineering/physical principles

K

1.6 using quality principles A C

2 Working in Teams C

2.1 team dynamics C C2.2 team communication C C

Level of Learning Legend K C AKnowledge Comprehension Application

Slide 14

ASEE Seattle 1998

Part Of CAM For ECE 100

Learning Objectives Level o

f L

earn

ing

Co

urs

e A

cti

vit

ies

In C

las

s A

cti

vit

ies

ta

ke q

uiz

zes/

exa

ms

be

fore

cla

ss

act

ive

lea

rnin

g e

xerc

ise

s

co

nstr

uct

ma

the

ma

tica

l mo

de

ls

ora

lly r

ep

ort

to p

ee

rs a

nd

cla

ss

pe

er

ass

ess

de

sig

n n

ote

bo

oks

wo

rk o

n d

esi

gn

pro

ject

s

wa

tch

ma

nu

fact

urin

g/o

the

r vid

eo

s

lis

ten

to b

rief l

ectu

res

Ou

t o

f C

las

s A

cti

vit

ies

re

ad

an

d s

um

ma

rize

text

bo

oks

co

nstr

uct

mo

de

l ba

sed

on

ge

om

ety

Pro

jec

ts

dis

sect

an

d r

ea

sse

mb

le a

rtifa

ct

de

velo

p a

n a

sse

mb

ly p

lan

(p

roce

ss)

de

sig

n, b

uild

, an

d te

st a

de

vic

e

de

mo

nst

rate

de

sig

n

1. Engineering Design Process A

1.1 formulating the problem A K C C C K K C C A A1.2 solving a problem A K C C C K K C C A A1.3 implementing a solution A K C C C K K C C A A1.4 documenting the process A K C C C K K C C A A

1.5 using engineering/physical principles

K K

1.6 using quality principles A K C C C K K C C A A

2 Working in Teams C

2.1 team dynamics C K C C K K C2.2 team communication C K C C K K C

Level of Learning Legend K C AKnowledge Comprehension Application

Slide 15

ASEE Seattle 1998

Course Activity Impact Course Activity Impact MatrixMatrix

A compact way to relate the A compact way to relate the impact the course impact the course learning learning activitiesactivities (e.g., homework, (e.g., homework, reports) have on the course reports) have on the course learning objectiveslearning objectives

Slide 16

ASEE Seattle 1998

Objectives ##

Weights

Tasks 5 Dot Prod

Su

cces

s in

ac

hie

vin

g t

he

ob

ject

ives

A Skeleton Course Activity Impact Matrix (CAIM)

Slide 17

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights

Tasks 1 2 3 4 5 Dot Prod

Su

cces

s in

ac

hie

vin

g t

he

ob

ject

ives

Start Of CAIM For ECE 300 Fall 97

Slide 18

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights

Tasks 1 2 3 4 5 Dot Prod

Design Project 1

Memos 2U&E Textbook 3

Prepare Reports 4R&P Textbook 5

Assessed Design Notebooks 6

Quizzes 7Presentation

Sandwich 8Assessed Oral & Written Reports 9

Lectures 10

In-Class Activites 11

Su

cc

es

s i

n

ac

hie

vin

g t

he

o

bje

cti

ve

s

Start Of CAIM For ECE 300 Fall 97

Slide 19

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights 0.149 0.064 0.039 0.160

Tasks 1 2 3 4 5 Dot Prod

Design Project 1 9

Memos 2U&E Textbook 3

Prepare Reports 4R&P Textbook 5

Assessed Design Notebooks 6

Quizzes 7Presentation

Sandwich 8Assessed Oral & Written Reports 9

Lectures 10

In-Class Activites 11

high

lowSu

cc

es

s i

n

ac

hie

vin

g t

he

o

bje

cti

ve

s

Start Of CAIM For ECE 300 Fall 97

Slide 20

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights 0.149 0.064 0.039 0.160

Tasks 1 2 3 4 5 Dot Prod

Design Project 1 9

Memos 2 9U&E Textbook 3 1

Prepare Reports 4 3R&P Textbook 5 9

Assessed Design Notebooks 6 0

Quizzes 7 0Presentation

Sandwich 8 0Assessed Oral & Written Reports 9 1

Lectures 10 3

In-Class Activites 11 0

high 4.0

lowSu

cc

es

s i

n

ac

hie

vin

g t

he

o

bje

cti

ve

s

Start Of CAIM For ECE 300 Fall 97

Slide 21

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights 0.149 0.064 0.039 0.160

Tasks 1 2 3 4 5 Dot Prod

Design Project 1 9 9 9 3 1 4.99

Memos 2 9 1 3 1 1 2.98

U&E Textbook 3 1 9 3 9 0 3.34

Prepare Reports 4 3 1 9 1 3 3.49

R&P Textbook 5 9 0 1 3 3 3.63Assessed Design

Notebooks 6 0 3 3 0 0 0.31

Quizzes 7 0 0 0 0 0 0.00Presentation

Sandwich 8 0 0 0 0 1 0.82

Assessed Oral & Written Reports 9 1 0 3 0 1 1.79

Lectures 10 3 0 0 0 3 2.21

In-Class Activites 11 0 1 1 1 1 1.79

25.35

high 4.0 #

3.0 3.0

2.0

lowSu

cc

es

s i

n

ac

hie

vin

g t

he

o

bje

cti

ve

s

Part of One Student’s ECE 300 CAIM, Fall 97

Slide 22

ASEE Seattle 1998

Objectives communication design work with others problem solving ##

Weights 0.149 0.064 0.039 0.160

Tasks 1 2 3 4 5 Dot Prod

Design Project 1 5.40 6.60 7.80 3.80 1 5.30

Memos 2 5.00 0.20 1.20 0.60 1 1.88

U&E Textbook 3 1.00 4.20 0.60 2.80 0 2.16

Prepare Reports 4 3.40 0.60 3.40 0.80 3 2.77

R&P Textbook 5 3.40 0.40 0.80 1.20 3 1.67Assessed Design

Notebooks 6 1.20 3.40 3.40 1.00 0 1.87

Quizzes 7 0.40 0.60 0.40 0.60 0 0.50Presentation

Sandwich 8 3.60 0.20 0.60 0.80 1 2.06

Assessed Oral & Written Reports 9 3.80 1.00 2.60 1.00 1 2.64

Lectures 10 1.80 1.00 1.20 0.40 3 1.22

In-Class Activites 11 1.60 1.40 1.80 1.00 1 1.54

sum 30.6 19.6 23.8 14.0 23.62

high #

2.60 2.40 2.40 2.20lowS

uc

ce

ss

in

a

ch

iev

ing

th

e

ob

jec

tiv

es

Summary Of CAIM For ECE 300 Fall 97

Slide 23

ASEE Seattle 1998

Improvements To ECE 300Improvements To ECE 300

Use What Is WorkingUse What Is Working more use of more use of Design ProjectDesign Project more oral more oral ReportsReports

Reduce What Is Not WorkingReduce What Is Not Working fewer fewer QuizzesQuizzes fewer fewer MemosMemos fewer fewer Notebook AssessmentsNotebook Assessments

Slide 24

ASEE Seattle 1998

Comparison of CAIM Dot Products For ECE 300Task Fall 1997 Spring 1998

Design Project 5.30 6.13

Memos 1.88 3.86

U & E Textbook 2.16 1.97

Prepare Reports 2.77

Oral Reports 5.46

R & P Textbook 1.67 2.02

Assessed Design Notebooks 1.87 4.22

Quizzes 0.50 0.81

Presentation Sandwich 2.06

Written Reports 4.81

Assessed Oral & Written Reports 2.64 1.61

Lectures 1.22

Experts 2.39

In-Class Activities 1.54 1.22

Total 23.62 34.50

Slide 25

ASEE Seattle 1998

Summary Summary

Showed how to create a Showed how to create a Course Articulation MatrixCourse Articulation Matrix

Showed how to create a Showed how to create a Course Activity Impact MatrixCourse Activity Impact Matrix

Showed how to Showed how to useuse a Course a Course Activity Impact Matrix to Activity Impact Matrix to improve a courseimprove a course