Exploring design features for effective teaching cases

Matthew J. Koehler

Michigan State University

Context for my interest

A circuitous route to this research area

Mathematics Computer Science Artificial Intelligence Cognitive Psychology Educational Psychology

Work in mathematics and science classrooms exposed me to research projects interested in conveying models of student thinking to teachers so that they could become better teachers (Carpenter & Fennema; Lehrer; Lehrer & Schauble).

Each of these projects wrestled with how to develop materials and professional development experiences that helped teachers accomplish this.

Context for my interest

Some common elements across efforts:

Complex, but structured models to be conveyed.

A belief that pieces of video of student reasoning, or classroom practices were the fertile ground for this learning to take place.

What could I offer?

Thinking about the ways that recent advances in technology (e.g. video and hypermedia) might be designed to scaffold this process of teacher development.

Ancient History (e.g., 1998)

Koehler, M.J., & Lehrer, R. (1998). Designing a hypermedia tool for learning about children’s mathematical cognition. Journal of Educational Computing Research, 18(2), pp. 123-145.

My first work in this area was developing teacher resources for Cognitively Guided Instruction (Carpenter & Fennema, 1992)

They had a handbook for teachers that described a model of children’s thinking about simple arithmetic, rich with text-based examples of children’s solution strategies

Could materials adapted to a hypermedia format, using video vignettes of children’s reasoning show any advantage in teacher learning?

Ancient History (e.g., 1998)

ANSWER: Yes

The study used pre-service teachers studying with the text-only materials and hypermedia, video-rich materials.

Results showed that the hypermedia / video materials were more effective in two areas of teacher learning:

A taxonomy of word problems for addition and subtraction that had implications for instruction and the development of children’s reasoning

Identifying the developmental sequence of solution strategies that children bring to bear for each of the problem types.

Ancient History -- Summary

At the very least, well-designed hypermedia that used video to exemplify children’s thinking has something to offer in the realm of teacher learning.

Important here, is that the way in which video was used was only to exemplify children’s thinking. Exemplification, is of course, important for understanding in general, but seemed to be particularly useful here for developing teachers knowledge of children’s thinking (even though text described the same “episodes” of children’s thinking)

Broadening the questions

Given that the approach of using video and hypermedia showed some potential, what were the next steps?

Moved to a consideration of what else you could do with video and hypermedia. What about teacher performances? What about classroom video? What to put in and what to leave out?

Enter :: “Case” and “Case-methods”

“One common definition suggests that a case is a descriptive research document, often presented in narrative form, that is based on a real-life situation or event. It attempts to convey a balanced, multidimensional representation of the context, participants, and reality of the situation” (Merseth, 1994).

Broadening the questions

Also enter a research project by Lehrer (geometry); and Lehrer & Schauble (modeling in mathematics and science).

Up to 40 practicing teachers at one point.

Regular teacher meetings that use text, classroom video, examples of children’s work, and teacher writings.

New curriculum, being design and implemented by teacher-researchers

A *LOT* of video from a *LOT* of classrooms.

QUESTION: Given the complexity in these projects (student models, teaching models, curriculum, examples of student work, teacher writings) …

HOW to develop a case that would be effective at helping learning teachers “see” and understand all of this.

Recent History (2002)

Koehler, M.J. (2002). Designing case-based hypermedia for developing understanding of children’s mathematical reasoning. Cognition and Instruction, 20(2), 151-195.

Most of the remaining talk explains this paper in more detail.

High level overview: Tests the effectiveness of a number of decisions about how to use video (and cases) effectively.

Core Research Questions

What makes for a good case?

What elements should be present?

How should elements be sequenced?

Do you follow the teacher or students?

How do you design a case that meets your goals?

When to link to cases?

When (and how) to link *from* cases?

How to help teachers “see” what is important in the cases.

Given that there are different ways to accomplish the same goals, which ways are most effective for teacher learning?

What makes for a good case?

Some caveats

Primarily talking about what makes a good case in K-6 mathematics teaching

Keeping in mind the kind of materials that are used in the professional development communities that I described (video,children’s work, texts, etc.)

Five general principles to guide development. Good cases are:

Situated in Practice

Layered with Annotation

Annotated with Big Ideas

“Criss-cross” the domain

Anchor exploration

Elements of good casesSituated in practice

Since teaching is situated in classroom practice, cases of teaching should also be situated in classroom practices.

Advocate use of classroom video

Video is more engaging and facilitates remembering (CTGV).

Video is more like “being there” than text

Written accounts of a classroom assume that textual expression can completely express the dynamics of classroom activity.

Elements of good casesLayered with annotation

Video does not speak for itself.

Any two viewers of a classroom video are likely to see different things, especially if they differ in experience, perspective, or expertise.

Classroom events are often subtle and difficult to interpret.

Therefore, video cases should be layered with annotation that helps teachers interpret classroom situations, so that teachers understand what the video is “a case of.”

Elements of good casesAnnotated with “Big Ideas”

Big ideas in mathematics are important landmarks in teaching based on models of student thinking (Schifter, 1996; Lehrer & Schauble, in press).

Accordingly, annotation should help teachers “lift out” and interpret the “big ideas” of the domain as they occur in the case.

Like big mathematical ideas

norms for argument (Yackel & Cobb, 1996)

general trajectories of student thinking (Carpenter & Fennema, 1992).

Elements of good cases“Criss-cross” the domain

Teaching and learning comprises a complex, ill-structured domain, cases often embody more than one “big idea.”

The same episode can be related to the “big ideas” in mathematics, children’s thinking, the use of tools and notations, and the classroom norms of teaching.

Good teaching requires not only understanding these ideas in isolation, but also orchestrating them to design effective classroom environments.

Cognitive Flexibility Theory (Spiro, Coulson, Feltovich, & Anderson, 1988) suggests that cases should “criss-cross” the conceptual landscape.

Elements of good casesAnchor Exploration

Cases that portray complex, ill-structured classroom situations often raise several important issues

For example, the same episode bring up “big ideas” in mathematics, children’s thinking, the use of tools and notations, and the classroom norms of teaching.

Cases should situate, or anchor (CTGV, 1990), explorations into these important ideas by providing access to further information (e.g., text, interpretation, related case, etc.) as issues arise in the case.

In contrast, if cases only represent the main story line, teachers may come to understand children’s development as a fixed progression through stages.

The Domain of Measurement

Tried to use these ideas in the development of a case-based tool for teachers about length and area Measurement (based on the work of Lehrer et. al).

Often taught and understood procedurally

Instead, instruction should help children to understand the mathematical ideas that underlie measurement (e.g., all the units are the same size)

Goal was to build a case-based hypermedia that emphasized 6 strands of teaching and learning:

Key Mathematical Ideas (e.g., Identical Units)

Classroom Norms (e.g., make thinking visible)

Children’s thinking (e.g., measurement = rulers)

Connections to other Math ideas (e.g., fractions)

Classroom activities (e.g., building tape measures)

Tools and Notations (e.g., graph paper)

Candidates for good cases

Given:

the domain to cover (The six strands of measurement)

A set of design principles for good cases

There a still a lot of possible designs to explore.

This study focused on two possibilities …

Two types of casesExemplification

One emphasized exemplification (like in the previous work)

Mini-Demo of this type of case

Uses all five principles

Situated in practice

Layered with Annotation

Annotated with “Big Ideas”

“Criss-crosses” the domain

Anchors exploration

Two types of casesNarrative

The other type of case emphasizes narrative structure

Cognitive reasons in general -

• Narrative structure facilitates remembering and structures knowledge organization (Mandler, 1984)

Teacher learning reasons

• Has the same footage as the examples do, but shows how they are structured over time.

• How students’ ideas are initially raised, developed, and recalled in later classroom episodes.

• Shows how episodes of student thinking are sequences to form a lesson, how lessons are used to design activities, and how activities and lessons are sequence to develop big ideas in the curriculum.

Two types of casesNarrative

Mini-Demo of this type of case

Uses all five principles

Situated in practice

Layered with Annotation

Annotated with “Big Ideas”

“Criss-crosses” the domain

Anchors exploration

An ExperimentRationale

Wanted to contrast the type of learning afforded by these two types of cases

Believed that the advantage of narrative cases lies in the causal structure that ties stories together (van den Broek & Trabasso, 1998).

The ability to apply knowledge relies, in part, on understanding the causal relationships between situations and actions that need to be taken (Eylon & Reif, 1984).

Therefore, I expected that narrative cases would be more likely to lead to knowledge that could be applied.

An ExperimentProcedure

Made two versions of the hypermedia tool

One version had exemplification cases only.

The other version had exemplification AND narrative cases.

Twenty-four pre-service teachers, randomly assigned to study with one version of the hypermedia tool.

Measures before study, after study, and 6 weeks after study

Speak aloud to video - participants saw short classroom segments. Following each clip, participants were asked to identify any important elements of teaching or learning about measurement that they saw. This was used to track the type of knowledge that participants acquired.

Analysis of student work - Participants were asked to apply their knowledge to an analysis of student work. Interviews addressed what the sample student understood (or did not), what the student needed to understand, and what classroom activities would most likely help this student gain understanding. This measured the ability to apply knowledge.

An ExperimentResults

Speak aloud video interviews showed that:

Both groups gained knowledge about the mathematics of measurement and about the teaching norms in place in the classes illustrated in the hypermedia.

Before, after, and six-weeks after instruction, both groups were equally able to name the big ideas (e.g., “identical units”) when they saw them in the videos

Conclusion: There were no difference in knowledge acquisition

Analyses of student work showed that:

The group who had access to narrative cases did better at applying their knowledge to their analyses of student work

More about this ...

An ExperimentAnalysis of student work

Measuring Application of Knowledge …

What does the following student understand about measurement?

Before instructionBoth groups tended to give procedural explanations

After instructionBoth groups improved … more so for the narrative group

Procedural Measurement Other

61% 20% 19%

Procedural Measurement Other

Exemplification only 18 % 33% 41 %

Exemp with Narrative 17 % 73 % 10 %

An ExperimentOther findings

This trend towards better application of knowledge by the narrative group shows up in other questions of the student work interview

Better at listing all the requisite knowledge a student would need to understand the problem

Better at suggesting appropriate follow-up activities

Have better memory for the classroom activities outlined in the hypermedia tool

Analysis of their time allocation during study supports the view that the narrative cases were responsible for these differences

Tended to read less text than their exemplar-only counterparts

Tended to watch less exemplar cases

The more time spent watching narrative cases was predictive of better analysis of student work (up to a point).

Take-away message

The nature of cases, how they should be crafted, and the consequences of different knowledge structuring are all important questions to investigate.

This work shows that even given some guiding principles for design (the five), competing designs have different affordances for learning

Cases used for exemplification are pretty good at helping students acquire declarative knowledge.

Cases organized around narrative of classroom events has some potential for fostering the application of that knowledge.

More questions … / Future work ..

Since then, my work has opened up considerably …

Most broadly, I’m interested in how the many design decisions that go into making and delivering cases impact learners.

Here are few specifics of things that I’m working on …

Cases for Different Domains

Science Learning (with Lehrer and Schauble)

Different way to highlight ideas in the video cases

Different subject matter (science)

http://mkoehler.educ.msu.edu/growth/

Early literacy teaching (K-2) (Hopefully w/ Nell Duke)

What elements are needed in a case about literacy instruction

Overall, interested in learning what elements of effective design are domain dependent and which are general purpose.

Future work - Student-authored cases

Working with a team at MSU on how to design a system that scaffolds student-authored cases of classroom video

What a student might learn from crafting a case

What kinds of video do they use as evidence for their claims?

What types of sequences would students build?

What would they learn?

Interface design by Charlie Ruggiero

Future Work - When is video advantageous? And for what?

Aimed at figuring out what the cognitive affordances are for video vs. text treatments for various types of information/stories/genres in the teacher ed population (those 18-21 yr olds)

Currently collecting data

Looking at text and video versions of the same material

• Video lectures vs. text transcripts

• Poetry vs. Poetry readings

• Stories from the nightly news.

Trying to figure out what the effect of video treatments are in terms of motivation, interest, affect, fact learning, persuasion, recall, and “gist” learning.

Measures

Future Work - Video vs. Text

Preliminary results are “interesting”

Poetry reading vs. text of poem

• Video has, if anything, a negative impact on students interest level

• Doesn’t seem to help their understanding of poem at all (and perhaps hurt it)

News story on college drinking (fact-laden story, very serious)

• Seems to be no difference in text vs. video treatments on students’ interest, their ability to recall facts from the story, or a change in their beliefs about the problems of student drinking.

Story about AIBO, the robotic dog.

• “Medium-size effects” for people’s interest level, change in emotional state, change in belief that robotic dogs could make a good companion, and ability to recall information

• BIG differences in the quality of students’ written summary of the story.

• WHY???

QuickTime™ and aYUV420 codec decompressorare needed to see this picture.

Future work - Genres differences

Looking at the “same classroom video” footage rendered in different story or TV genres

Case-format (from before)

News-story

Documentary

Drama

Reality TV (e.g., MTV’s “The Real World” … no narrator)

Looking for the different learning affordances of each for the teacher candidate population (e.g., 18-21 yr olds). Looking at factual recall, interest levels, “gist learning,” opinion changes, etc.

Hopefully some correspondance to previous slide

QuickTime™ and aYUV420 codec decompressorare needed to see this picture.

Conclusions and Questions

In general, I’m keenly interested in unpacking the learning affordances of the elements of effective case design.

Questions? Comments? Critiques?

Lesson has good use of visual manipulativesTeaches about the need for same-size unitsTeaches about units that are not wholeTeaches that things that look different may be the same sizeTeaches about breaking big measurements into smaller onesTeaches about reusing unitsActivity is hands onTeacher uses a variety of unitsGood introduction to tools such as rulersTeacher uses open-ended questionsStudents lead class discussions

Teacher takes input from the studentsUsing more than just standard unitsTeachers build understanding by restating students ideasTeachers bring up historyChildren understand that you cannot have gaps between unitsChildren understand the need for identical unitsChildren account for fractional unitsTeachers help students see consequences of their ideasChildren understand that different objects can have the same measureChildren understand that units can be reused

Episode Only

(Post-test)

Episode Plus

Narrative(Post-test)

Sample Data from speak aloud video

Not using the diagram and 1x1 squares correctly

Doesn’t understand all the units have to be the same size

Student would need to know the diff between square and rectangle

All the units of a measurement need to be the same size

There can be no spaces between the units

Episode Only

(6-weeks later)

Episode Plus

Narrative(6-weeks

later)

What would a student need to know to understand the problem

Follow up activities suggested

Time Procedural Conceptual Other

Before Instruction

52% 43% 4%

After

Episode 44% 56% 0%

Episode + Narrative

32% 67% 0%

Very Early ResultsTEXT VIDEO

College Drinking

Summary 18.0 21.4

Facts 36% 40%

Engaged 47.4 56.3

Argument 35% 58%

Prior beliefs 0.3 0.4

AIBO the dog

Summary 25.3 15.9

Facts 29% 22%

Engaged 45.1 53.1

Argument 88% 91%

Prior beliefs 0.82 0.55

Talking Head

Summary 19.3 20.9

Facts 30% 22%

Engaged 18.2 18.7