Scaling up our instruction

• Our prior work demonstrates an effective method for teaching the Control of Variables Strategy (CVS), a means of designing simple experiments, for students as young as third grade (Strand Cary & Klahr, to appear in special issue of Cognitive Development),

• Though promising, this brief, one-size-fits-all instruction has some disadvantages:

• It does not reach all students. Particularly students in low-income schools with large minority populations require explicit scaffolding, repetition, and instruction in multiple domains to master the procedures and concepts of CVS (Li, Klahr, & Siler, 2006). Such adaptations are labor intensive and still fail to reach some students.

• Many students have beliefs and misconceptions that are very resistant to change. Even when they seem to thoroughly understand CVS, some fail to transfer to new domains or question types.

• Thus we are in the process of creating the TED computer tutor (for Training in Experimental Design) to provide individualized instruction.

Working toward robust knowledge of experimental design: Development of the TED TutorMari Strand Cary, Stephanie Siler, Cressida Magaro & David Klahr

Carnegie Mellon University

A need for instruction

• Experimental design and inference are critical skills for scientific thinking. Teachers emphasize this by asking students to complete science fair projects and standardized tests include related items.

• Although the ability to infer causal mechanisms is present to some degree in infancy (Schultz & Sommerville, 2006), the ability to design experiments from which to draw unambiguous causal inferences is lacking for many middle school children (Chen & Klahr, 1999; Klahr & Nigam, 2004).

• Most students do not attain experimental design and inference skills on their own or in their K-8th grade classes (see Klahr & Li, 2005 for review)

• Prior research (Strand Cary & Klahr, to appear in special issue of Cognitive Development; Chen & Klahr, 1999) has resulted in a promising technique for teaching experimental design skills. However, some students are left behind.

This research was supported by the Department of Education, IES (# R305H060034)Questions? Comments? Please contact MariStrandCary@cmu.edu

CDS 2007

Training in Experimental Design (TED) Tutor

What is TED?

• An experimental design computer tutor used individually by 4th-8th grade students in full-class setting

• Used before, during, or after teacher-instruction

• Individualized, adaptive instruction about simple experimental design, evaluation, and interpretation.

• Efficient instruction that will be effective for all students

• Multifaceted instruction leading to long-lasting, generalized knowledge demonstrable in new domains and question formats

How will it help all students learn?

• Instruction, practice, and assessment will be based on individuals’ knowledge and mastery in real time.

• Ongoing recording and analysis of student actions and formative assessments will enable tutor to do “knowledge tracing” and “model tracing” and adapt to the current “student model” (Anderson, Boyle, Corbett & Lewis, 1990; Corbett & Anderson, 1995) while selecting instruction.

• Teachers can use student performance information to decide which students to spend more time with or to guide additional full-class instruction (e.g., if common errors arise).

How will the tutor adapt to each student?

• Build on student strengths and current knowledge

• Focus on student’s weak or missing knowledge

• Provide “just-in-time” and “just enough” feedback

• Ask a range of question types

• Balance domain specific and domain general instruction

• Adjust pacing and coverage

• “Fade” scaffolding

• Introduce new domains (to renew motivation, test understanding, and promote generalization)

• Require forward and backward reasoning (e.g., design an experiment to answer a specific question; determine the experimental question by looking at a well-designed experiment)

How do students incorrectly design simple experiments?

Do you think this is a good way to find out whether the age of the child (Older or Younger) m akes a difference in how much they sell?

(a) If you think it is a good way, then circle the word "Good" below. If you think it is a bad way, circle "Bad".

Set-up A Set-up B

Do you think this is a good way to find out whether the number of windows (One or Four) makes a difference in how high the rockets fly? (a) If you think it is a good way, then circle the word "Good" below. If you think it is a bad way, circle "Bad".

Rocket A Rocket B

Variable Ramp 1 Ramp 2

Slope Steep Not steep

Ball Red Yellow

Surface Smooth Rough

Starting position Top Middle

Variable Group 1 Group 2

Studying location Desk Desk

Lighting Well lit Dim

Noise level Loud Quiet

Does WHERE YOU STUDY affect your grades?

What is behind these errors?

• Guessing

• Carelessness

• Different goals

• Different or incorrect experimental logic

• Misuse of visual representations

• Completely confounded experiment

• Two identical set-ups

• Hold the target variable constant and vary the other variables

• Partially-confounded experiment

• Choose combinations of variables such that each set-up is optimally designed.

• Unconfounded experiment for wrong target variable

Some fourth-grade students were doing a project for their science class. They were trying to find the answer to the question “Do beetles choose to live in bright light or in the shade?” The picture below shows how a student set up the experiment to find out if beetles choose to live in bright light or in the shade. Is this a good way to set up the experiment? Why or why not?

For example, identifying and fixing problematic experiments (as often required by confusing standardized test items like the one on the right) seems to be more difficult than designing experiments from scratch.

Does the SURFACE affect how far balls roll?

To find out whether seeds grow better in the light or dark, you could put some seeds on pieces of damp paper and

A. keep them in a warm, dark place B. keep one group in a light place and another in a dark

place C. keep them in a warm, light place

D. put them in a light or dark place that is cool

TED’s Iterative Design Process

Version n Pilot testing and

Classroom implementation(+ pre, post, and formative

assessments)

Human tutoring (+ delayed post

assessment)

Revise model of student thinking, instruction, and interface

Teaching CVS with ramps(Screen shots taken from current version which involves small-group instruction by researchers)

Ramp familiarization

Reviewing CVS

Seeking reminder during experiment design

Explaining the visual representations

Good / Bad

Good / Bad

ReferencesAnderson, J.R., Boyle, C.F., Corbett, A.T. & Lewis, M.W. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42

Corbett, A.T. & Anderson, J.R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted Interaction

Chen, Z. & Klahr, D. (1999). All other things being equal: Children’s acquisition of the control of variables strategy. Child Development, 70(5),

Klahr, D. & Li, J. (2005). Cognitive research and elementary science instruction: From the laboratory, to the classroom, and back. Journal of science and educational technology, 4.

Klahr, D. & Nigam, M (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological science, 15(10)

Li, J., Klahr, D. & Siler, S. (2006). What Lies Beneath the Science Achievement Gap? The Challenges of Aligning Science Instruction with Standards and Tests. Science Educator, 15, 1-12

Schulz, L. & Sommerville, J. (2006). God does not play dice: Causal determinism and preschoolers’ causal inferences. Child Development, 77(2)

Strand Cary, M. & Klahr, D. (to appear in special issue of Cognitive Development)