Developing Formative AssessmentsSarah Ades

Penn State University

At the end of this workshop scholars will be able to:• Describe the role of formative assessment in learning.• Explain why assessments should be aligned to learning

objectives and outcomes.• Evaluate different types of assessment activities to find

those that are aligned best with the learner’s abilities and level of understanding to be assessed.

• Develop aligned assessment activities to provide evidence of learning to both the instructor and learner.

workshop learning objectives (outcomes):

I am a junior majoring in biology. I usually get A’s in my courses; only a few B’s so far in college. I totally breezed through high school. It was so easy. I’m think that I might go to graduate school or medical school.

This semester, I enrolled in introductory microbiology. I approach this class like most others—I attend lecture (have only missed two this semester!), read the textbook (usually before class, if I have time), and turn in the homework if it’s going to be graded. Prof. Lopez is great; he’s really well organized and follows the book closely. The homework has been helpful for learning the terms and information.

The first midterm exam in this course was NOT what I expected. None of the questions were multiple choice. Instead, we had to write out short (and sometimes LONG) answers. I barely finished it in the 2-hour exam period. Plus, three of the questions tested us on things we never learned and skipped stuff we had covered in class. For example, we learned about a specific example of the lac operon in class last week. I worked really hard to memorize all the important steps in the process, and then it wasn’t even on the test. But there was this question about asking us to “describe a strategy that bacteria use to regulate gene expression and explain why such a strategy might have been selected for over time.” How am I supposed to know about that? I got a 72% on that test. What a crock!

Handelsman, Miller & Pfund (2007) Scientific Teaching

What went wrong?

go to: b.socrative.comenter code: GX2MXSME

link

Instruction without goals….

Ready?

Fire!

Aim.

©2010 Michael Palmer

aligned courseWhat knowledge/skills do your

students already have?

What experiences do your students bring to

the course?

Why are the students taking the course?

What misconceptions do your students have ?

change throughout

course

Assessment

What is assessment?

A method to determine whether of not students learned (or accomplished) the learning objective.

Evaluate and determine if the students are learning what they are supposed to learn.Measuring student learning. Assessment is how we monitor student's ability to learn.No real idea to be honest. A method or means of measuring learning.

A way of determining what students actually understand and at what level they understand.To find out if what you think you are teaching them is what they are learning.

Assessment is a process of measuring how effectively a course or program accomplishes its goals.

A tool that is used to measure how well students have met the learning goals the educator has defined. It can also provide information to the educator about the effectiveness of their teaching methods.Tools used to measure student understanding and inform teaching practices.A way of evaluating effectiveness of teaching and learning.

Student-focusedTeacher-focusedOther Just in Time

AssessmentWhat is role of assessment in teaching and learning?

It gives instructors a way to monitor student learning.Helps us to understand where the problem is and figure out how to improve teaching and learning.To assure that students are learning what we intend them to be learning in the course. Assessment is used to plan, implement, and modify courses.

It is how I know that students have learned a concept or idea. Not only how they have learned but at what level they are learning it.

To gauge student learning for give formative feedback to help students in the learning process or to determine final learning gains using a summative instrument. If done correctly it should guide teaching and learning both.

To find out if you are being successful in your teaching, and to show the student where their misconceptions are.

Teaching strategies can be modified based on the results of assessment to improve learning outcomes.

A consistent and defined way of improve teaching (iterative process of assessment, adaptation of practices, repeat).

Assessment should be aligned with learning outcomes and should help us identify whether or not we've taught the materials effectively. It should include lots of low-stakes practice (formative assessment) and culminate in summative assessment(s).

In the formative form it is to inform teaching and learning. What do I (my students) know and what do we need to work on. In the summative form it informs instructors and students about level of knowledge.

Student-focusedTeacher-focusedOther Just in Time

Assessment? Assessment a way of providing students and teachers with

information regarding students' progress towards achieving the learning objectives. Evidence.

The role of assessment in teaching and learning is to provide feedback to both the teacher and the student. It allows teachers to tell whether their teaching methods have been effective or whether they need to re-teach a particular concept. On the other hand, it allows students to identify misconceptions and gaps relating to the new content being learned. It also allows students to identify if they are studying effectively.

Lets student interact with the material and practice skills

Why do we assess / question?

•Gather evidence on student learning (evaluation)•Improve a course•Improve our teaching•Improve society (?)•Get feedback on student understanding•Elicit misconceptions•Guide your own instructional decisions•Make expectations clear to students•Provide feedback to students•Give students an opportunity to gauge their progress•Help guide student studying and learning behavior

Guide students

Guide improvements

Guide teaching

Learner

assessments

instructional activtiies

learning objectives

(outcomes)

• What evidence can show that students have achieved the desired results? • What assessment tasks and other evidence will anchor our curricular units and

thus guide our instruction? • What should we look for, to determine the extent of student understanding?

Wiggins and McTighe (2013) Understanding by Design

Novice to Expert• According to Donald Rumsfeld: “There are known knowns. There are

things we know that we know. There are known unknowns. That is to say, there are things that we now know we don't know. But there are also unknown unknowns. There are things we do not know we don't know.”

unconscious unknowing:don’t know what you

don’t know

unconscious knowing:don’t know how you know

what you know

conscious unknowing:know that you don’t

know

conscious knowing:know how you know

what you know

Novice Expert• Content: coherent structure of

concepts.• Describes nature, established by

experiment.• Prob. Solving: Systematic concept-

based strategies. Widely applicable.

GOAL: use assessments to guide learning, build skills, and measureprogression from novice to expert

• Content: isolated pieces of information to be memorized.

• Handed down by an authority. Unrelated to world.

• Problem solving: pattern matching to memorized recipes.

Learner

Ambrose 2010 How Learning Workshttp://www.cwsei.ubc.ca/resources/tools.htm

formative • Adjective – of and relating to a

person’s development• of and relating to a pedevelopment• Synonyms

– Developmental– Growing– Malleable– Impressible– Determining– Influential

• Purpose – “brief, low-stakes activities that students do in order to give both themselves and the teacher feedback about their level of understanding.”

Stephen Chew in Chronicle of Higher Education

summative • Adjective – of and relating to a

summation• of and relating to a pedevelopment• Synonyms

– Additive– Cumulative

• Purpose – higher-stakes activities that are “cumulative in nature and is utilized to determine whether students have met the goals or student learning outcomes at the end of a course, unit, or program.”

what types of activities can you use for formative assessments?

http://padlet.com/sea10/formative

what types of activities can you use for formative assessments?

• One minute paper• Muddiest point• Clicker questions- ConcepTest• Empty Outlines• One sentence summary• Productive Study-Time Logs• Misconception/Presumption Check• Compare/contrast• Concept maps• Concept sketches• Examine lab notebooks• Course discussion board/blogging

see also engaugements file on wiki

Endpoint: learning objectives and outcomes – summative assessmentswhat students will know upon completion of this course/unit/class

Starting point: prior knowledge and misconceptionswhat students know

Roadmap: formative assessments and learning activitieshow students will reach goal- What gear is needed?- What are the intermediate

steps along the way?- How will instructor and

students know they are making progress?

Designing Aligned Assessments

Where’s the start?Assessing prior knowledge, gaps in knowledge, and misconceptions

Self-assessment:• How familiar are you with ___?

a) never heard of it.b) heard of it but don’t

remember much.c) know what it is and can

explain what it does.d) know what it is, when and

how to use it.

Remember:• Recall does not mean students can apply.• Look for misconceptions as well as concepts.• What is missing can be as informative as what is reported.

?

Multiple Choice Quiz: • include common

misconceptions as options• use both information

retrieval and application questions

Brainstorm:• ask leading question or pose

a scenario, have students generate answers

Concept maps:• get an overview of what

key ideas students know and how they think ideas are connected.

traditional approach to teaching – Prof. Lopezyou’ve read the book and sat through the

lectures

now you’re the expert

ready, set, go!!

developing understanding through formative assessments- add some steps along the way -

practice, practice, practice (reinforcement)increase complexity over time (scaffolding)

To develop mastery, students must acquire component skills and knowledge, practice integrating them, and know when to apply them.

acquire component skills and knowledge

practice skills and using knowledge

know when and how to apply skills and knowledge

mastery

Ambrose 2010 How Learning Works

Jigsaw activity-• Formative assessments Dr. Lopez could have

used in his class addressing the following course level learning objective.

Learning Objective: students will understand that bacteria are able to sense the environment around them and integrate that information to alter gene expression.

Course Snap Shot

Micrb202: Introductory Microbiology Laboratory (IQB)

3 sections, up to 36 students per section2nd semester freshmen – first bio lab class

taught by a faculty member with 2 learning assistantsWed/Fri 2 hours

Goal(s)By the end of this course • you will have a strong foundation in scientific inquiry and be prepared to apply

these skills in future science courses.• you will be able to evaluate issues in “real-life” making you a life-long educated

consumer of science• you will have the technical skills and fundamental concepts to address important

questions confronting microbiologists in a variety of professions from medicine to environmental microbiology.

Inquiry-Based Intro Micro Lab

Inquiry Based Traditional

basics provided, students seek additional information as needed

background information generally all provided

array of materials provided students decide what to use materials students told what

materials to use

protocols providedtechniques

protocols provided

used to address scientific question focal point of experiment

student-directed experimental designteacher-directed, step-by-step instructions

driven by scientific question driven by technique

open-ended outcomes known in advance

active role as researchers role of students passive role following directions

help students formulate experiments role of teacher tell students what to do

contrasting lab approaches

scientific method

concepts of microbiologyexperimental techniques

formulate a clear questiondevelop a testable hypothesisplan an experimentevaluate and synthesize datacommunicate findings

Utilize:sterile techniquekey microbiological

methodslaboratory safety

Discuss:microbial evolutionenvironmental influence on microbesmicrobial interactionsidentify an unknown microbe

learning objectives/outcomes

IQB approach

Students will beable to:

Course Design• seven modules• lab guides - background, experimental challenge,

key questions, relevant protocols- online quiz on major concepts taken before each

module starts• design experiment - hypothesis, predictions,

controls- brainstorming, group problem solving, peer and

instructor review• perform experiment• evaluate data and draw conclusions- discuss with peers and instructors- compare results with other groups

• keep lab notebook• make classroom a scientific community

- discuss have periodic class mini-conferences• communicate findings - written lab report or oral

presentation

Experiment 3: Bacterial Behavior- Motility and Chemotaxis One of the distinguishing characteristics of animals is their ability to move in response to stimuli that originate from within their own bodies or from the outside world. Many of us are attracted to the smell of fresh-baked chocolate chip cookies and repelled by the aroma of a recently antagonized skunk. An animal moving rapidly toward an object it recognizes as food or fleeing a harmful chemical are examples from the repertoire of responses known as animal behavior. Even the simplest and smallest animals exhibit behaviors within their own sensory capabilities…

ResourcesYou can find basic information about flagella and chemotaxis in any microbiology textbook.Online resource with an overview of bacterial motility, flagella, and taxis:http://textbookofbacteriology.net/structure_2.html Key conceptBacteria behave in response to stimuli.

ChallengeWe will use two different assays to determine if your pets are motile and chemotactic . We will also provide you with a plate of swimming E. coli and an assay to determine if a substance is an attractant or a repellent. Develop a hypothesis about E. coli’s responses to different compounds- we will provide you with some sugars and amino acids. You are also welcome to bring in other substances that you would like to test.

Key Questions• What advantages does chemotaxis provide for a bacterium?• High concentrations of repellents increase the tumbling time of E. coli. How does this help the bacteria move

away from the repellent?• Different bacteria chemotax towards different chemicals. Why might this be?• Some bacteria are not motile. What factors in their environment might allow them to flourish without self-

propelled motility?• Penicillium notatum is not motile. However, it spreads over the surface of a petri dish and can even spread from

one piece of bread to another. How might either of these behaviors occur in the absence of self-propelled motility?

conce

pts

tech

niq

ues

sim

ple

com

ple

x

pet microbe

environmental influences on

microbial growth

microbial metabolism

microscopy

selective and differential

growth media

purification of bacteria

count bacteria

identification of unknown

antibiotic resistance

measurements

motility assays

bacterial motility and chemotaxis

environmental influences on

microbial behavior

motility assays

pathogenesis

host-pathogen interactions

purification of bacteria/fungi

microscopy

identification of unknown

skin microbiome

host-pathogen interactions

evolution

environmental microbiology

antibiotic resistance

measurements

identification of unknown

count viruses

microscopy

purification of bacteria/virus

es

random vs. adaptive mutation

host-pathogen interactions

evolution

count viruses

isolate mutants

bread, mold, and the

environment

environmental influences on

microbial growth

host-pathogen interactions

microscopy

bioenergy

environmental microbiology

microbial metabolism

microbial fuel cell

course modules:reinforcement and scaffolding

Day 1intromini-

lecture-

background micro-exp to design

- overview of

procedure

reading:basic micro

concepts

P. acnes/ph

age

WedMon Tu FriThSa/Su

preclass online quiz on readingsbasic knowledge

think-pair-share-brainstorm • list 3 factors that could influence antibiotic resistance of P. acnes Analyze Data Table on

host range of P. acnes & phage•describe, interpret, apply

•students explain what they might find

Objective: Students will grasp basic concepts of microbiome.Students will be familiar with overview of P. acnes and that there are phage that infect P. acnes

Objective: Students will begin to design experiment.

in class

out of class

1

2 3

1

2

3

Mini-Lecture• skin microbiome

• sebaceous gland

•overview of unit

• start thinking abt experiments to design

•overview of procedures

Pre-class• skin microbiome- textbook

•Ted Talk podcast

•pop. press article about P. acnes/phage

Week 1

activity assessment

notebook check

Objectives: Students will understand (how & why) procedures to be followed.Students will make connections to previous units.Students will understand the experimental strategy for purification of bacteriophage (Day 2).

preclass online quiz •purpose of main steps

•sterile techniqueknowledge level

Week 2

Day 2Initial

Isolation

Day 3phage

purification

reading:lab manual

phage morpholog

y

WedMon Tu FriThSa/Su

in class

out of class

3

reading:lab manual

Fill-in worksheet/flowchartdo as teams - go over as class•why purify phage?•2nd approach to get phage

1

3

Lab work

Pre-class• lab manual - isolation steps

notebook checkpreclass online quiz•purpose of each step in phage purif.

• i.d. of phage- plaque and capsid morph.

knowledge/apply level

2Pre-class

• lab manual - phage purification

21

Plaques?

yes no

Starter activity- worksheet/flowchartLab work

Purify!how?why?

Find those phage!how?

notebook check

notebook check

activity assessment

Motivational methods: • Relate experiments to real world• Give students ownership of experiments• Treat students like scientists

Formative assessments:• Pre-class quizzes (3 tries)• In class activities - design experiments, analyze data (own and from liter-

ature)• Notebook checks• Informal questions while circulating in class• Students can repeat some activities until they master them

Summative assessments:• Lab reports (graded by rubric), rewriting allowed• Key questions – apply learning• No lab practicums – must master methods to identify and keep pet alive• Oral reports

Does it work?

informal observations• students get microscopes themselves to examine a

sample• students ask for materials for controls• students discuss amongst themselves experimental

design and results• lab notebook checks show learning - improved

hypotheses, controls• self-reporting on SRTEs

- ...involved thought and understanding. It was very helpful having the instructors come around and asking questions to be sure we understood why and what we were doing not just to make sure we followed a procedure correctly.

- This course built on the material we learned in the beginning and continued to build on it, which personally helped me retain the information more than just about every course I have ever taken.

- The student query aspect of the course allowed me to learn that most that I possibly could have from MICRO 202. I really enjoyed asking my own questions for each experiment and feeling as though I had autonomy to set up the experiment as I saw fit to answer my questions. I believe that the set-up of this course allowed me to become a more confident science student and that I now have more of a sense of how to work in lab setting--it is now common sense to me to start with a hypothesis, to figure out what controls to set up to make sure that I can distinguish true results from inconclusive data...I don''t think I could have done that nearly as well without this course.

• applications from departmental majors for independent research - 60% of IQB section, 17% of all other 11 sections

Learning objectives

1. Studentprior knowledgemisconceptionsskills

2. ContentIdentify components of concept.How are concepts integrated?How are concepts applied?How are concepts used in real world?

3. DeliverableChoose learning activity and concept/skill to be assessed.Decide level of learning to assess.Ensure assessment is appropriate for desired learning outcome. Ensure assessment will give desired evidence of learning.Consider how assessment will be evaluated.

Designing Aligned Assessments

Your turn• Develop formative assessment aligned to your

learning objective/outcome.• Pair with another scholar to share your assessments• Give feedback!

Learning Objective

Taxonomy Level/Categor

yFormative

AssessmentsSummative Assessment

Alignment Grid

https://www.flickr.com/photos/gforsythe/8204963410/sizes/z/in/set-72157626965187420/

The most important method of education . . . always has consisted of that in which the pupil was urged to actual performance. —Albert Einstein, Ideas and Opinions, 1954/1982, p. 60

References• Connecting Learning Goals and Assessments Workshop by Stephanie

Chasteen (CU-SEI), 2011 http://www.cwsei.ubc.ca/resources/workshops.htm

• Assessment strategies, Assessments ASM 2013_Obom_6.13.13.pptx Biology Scholars 2013 Assessment Residency

• Ambrose, S. et al. How Learning Works, 2010, Jossey-Bass a Wiley Imprint, San Francisco

• Handelsman, J. et al. Scientific Teaching, 2007, WH Freeman and Company, New York

• Carl Wieman Science Education Initiative at UBC http://www.cwsei.ubc.ca/resources/index.html

• Wiggins and McTighe, 2013, Understanding by Design Expanded 2nd Edition, ASCD