Physics Concept Surveys TDSB physics team May 28, 2004.

Physics Concept Surveys

TDSB physics team

May 28, 2004

Problem

Students do not learn physics concepts very well under traditional classroom instruction

Problem


They memorize “plug and chug” techniques, get 80% on a physics exam, but only score 50% on a multiple choice concept survey

Problem


They memorize “plug and chug” techniques, get 80% on a physics exam, but only score 50% on a multiple choice concept survey

In fact, when concept surveys are applied pre-post, gains are only .286 (30% to 50%)

Purpose

2003-2004: develop concept surveys for Ontario SPH 3U teachers to use as pre-post diagnostics to measure conceptual gains

Purpose

2003-2004: develop concept surveys for Ontario SPH 3U teachers to use as pre-post diagnostics to measure conceptual gains

2004-2005: encourage teachers to try new interactive teaching strategies to see if they lead to increased conceptual gains

History of Project I

August, 2003 – team of physics teachers developed a plan: Shawn Bullock, Peter Carlow, Doug Hayhoe, Jason Law



Sept. 2003 – we studied existing concept surveys, such as Force Concept Inventory, as well as the Ontario curriculum



Sept. 2003 – we studied existing concept surveys, such as Force Concept Inventory, as well as the Ontario curriculum

Oct. to Dec. 2003 – drafted and revised five surveys of approx. 20 MC questions each (Each item is referenced to the curriculum)

Force and Motion: Unit 1 Survey

12. A large truck collides head on with a small compact car. During the collision …

A. The truck exerts a greater amount of force on the car than the car exerts on the truck

B. The car exerts a greater amount of force on the truck than the truck exerts on the car

C. Neither exerts a force on each other. The car gets smashed because it is in the way

D. The truck exerts a force on the car but the car does not exert a force on the truck

E. The truck exerts the same amount of force on the car as the car exerts on the truck

Curriculum References for Unit 1, #12

Specific Curricular Expectation state Newton’s laws, and apply them to

explain the motion of objects in a variety of contexts


Specific Curricular Expectation state Newton’s laws, and apply them to

explain the motion of objects in a variety of contexts

Achievement Chart Criteria demonstrate understanding of concepts,

laws, and theories

Light and Geometric Optics: Unit 4

17. In the following diagram, two different lenses with different focal lengths are placed in sequence in an optical device. An object is located where the arrow is in the diagram.

Left Right

The second image, after light from the object has passed through both lenses, would appear to be:A. to the left of the converging lens and right side upB. between the two lenses and right side upC. between the two lenses and upside downD. to the right of the diverging lens and right side upE. to the right of the diverging lens and upside down


Specific Curricular Expectation describe and explain, with the aid of light-

ray diagrams, the characteristics and positions of the images formed by lenses


Specific Curricular Expectation describe and explain, with the aid of light-

ray diagrams, the characteristics and positions of the images formed by lenses

Achievement Chart Criteria understand relationships among concepts

History of Project II

January to May, 2004 – Field-tested concept surveys in 20 schools across the TDSB



April to June, 2004 – conducted item analysis of questions on concept surveys



April to June, 2004 – conducted item analysis of questions on concept surveys

May 7, 2004 – physics team met to examine each question on surveys and recommend for inclusion, revision or deletion (including Dianne Ness and Tasha Richardson)

Field-test Results Overall

Unit 1: 9 schools, 306 students, 47% average


Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%)





Units 4 & 5: results are still being collected

Force and Motion: Unit 1 Survey


A. The truck exerts a greater amount of force on the car than the car exerts on the truck

B. The car exerts a greater amount of force on the truck than the truck exerts on the car

C. Neither exerts a force on each other. The car gets smashed because it is in the way

D. The truck exerts a force on the car but the car does not exert a force on the truck

E. The truck exerts the same amount of force on the car as the car exerts on the truck

Item Assessment Criteria

Difficulty level – What % of students answered the question correctly?



Discrimination coefficient – Did a higher % of better students answer correctly?



Discrimination coefficient – Did a higher % of better students answer correctly?

Pre-post gain – Did more students answer correctly after studying the topic?

Difficulty Level – 51% Correct


Students writing post-test 306A. The truck exerts a greater amount of force on the car than the car exerts on the truck 135B. The car exerts a greater amount of force on the truck than the truck exerts on the car 6C. Neither exerts a force on each other. The car gets

smashed because it is in the way 1D. The truck exerts a force on the car but the car does not exert a force on the truck 8E. The truck exerts the same amount of force on the car as the car exerts on the truck 156

Discrimination Coefficient = 0.31


306 students divided into higher 153 & lower 153 A. The truck exerts a greater amount of force on the car than the car exerts on the truck 45 90B. The car exerts a greater amount of force on the truck than the truck exerts on the car 3 3C. Neither exerts a force on each other. The car gets

smashed because it is in the way 0 1

D. The truck exerts a force on the car but the car does not exert a force on the truck 3

5E. The truck exerts the same amount of force on the car as the car exerts on the truck 102

54

Pre-post Gain = 0.54 (21% to 64%)

12. A large truck collides head on with a small compact car. During the collision … Students writing both pre and post 127 Pre PostA. The truck exerts a greater amount of force on the car than the car exerts on the truck 79 41B. The car exerts a greater amount of force on the truck than the truck exerts on the car 5 1C. Neither exerts a force on each other. The car gets

smashed because it is in the way 3 0D. The truck exerts a force on the car but the car does not

exert a force on the truck 8 4E. The truck exerts the same amount of force on the car as

the car exerts on the truck 27 81

Field-test Results for Unit 3, #14

14. Consider the following wave properties:1. frequency

2. wavelength 3. Amplitude

Which of these properties listed above never changes as a wave travels along one rope?

A. 1 onlyB. 2 onlyC. 3 only D. 1 and 2 onlyE. 1 and 3 only

Difficulty Level – 21% Correct



Students writing post-test 258Which of these properties listed above never changes as a wave travels along one rope?

A. 1 only 91 B. 2 only 47 C. 3 only 48 D. 1 and 2 only 55 E. 1 and 3 only 17

Discrimination Coefficient = 0.19



258 students divided into higher 129 & lower 129 Which of these properties listed above never changes as a wave travels along one rope?

A. 1 only 57 34B. 2 only 14 33C. 3 only 14 34D. 1 and 2 only 40 15E. 1 and 3 only 5 12

Pre-post Gain = 0 (21.7% to 21.1%)

14. Consider the following wave properties:1. frequency 2. wavelength 3. amplitudeStudents writing both pre and post 166 pre postWhich of these properties listed above never changes as a wave travels along one rope?

A. 1 only 40 58B. 2 only 35 29

C. 3 only 45 34D. 1 and 2 only 36 35E. 1 and 3 only 12 10

Future of Project I

June to August, 2004 – preparation of “final” surveys for pre-post use in 2004-2005

Future of Project I

June to August, 2004 – preparation of “final” surveys for pre-post use in 2004-2005

They will have this formatPart 1. Units 1-2, Mechanics, 35 itemsPart 2. Units 3-5, Waves, Light, EM, 35 items

Future of Project II

Sept-Jan, 2005: collection of baseline data (volunteer teachers will use revised concept surveys to collect pre-post course data without changing their teaching strategies)

Future of Project II

Sept-Jan, 2005: collection of baseline data (volunteer teachers will use revised concept surveys to collect pre-post course data without changing their teaching strategies)

Feb-June 2005: collection of experimental data (the same teachers will collect pre-post course data on a second course, in which they use substantially different teaching strategies)

Physics Concept Surveys TDSB physics team May 28, 2004.

Documents

Transcript of Physics Concept Surveys TDSB physics team May 28, 2004.