Can student attitudesbe used as predictors of success in problem solving?

Karen CummingsSouthern Connecticut State UniversityNew Haven, CT

Outline

Existing work on measuring student attitudes/beliefs/expectations in physics

My recent work in measuring student attitudes/beliefs/expectations in regard to problem solving

Teaching problem solving in student centered, conceptually focused courses.

Problem Solving Attitudes Survey

Maryland Physics Expectations Survey (MPEX) like scoring philosophy and administration.

Some questions were taken from the MPEX (and modified or not).

Maryland Physics Expectation Survey Work done by Richard Steinberg, Joe

Redish and Jeff Saul (U Maryland) Likert scale survey. Statements are made.

Respondents “strong agree” to “strongly disagree” with the statement.

Several “clusters” of questions that probe student attitudes, beliefs and expectations in regard to learning physics.

Beliefs about learning physics: " Understanding" physics basically means being able to recall something you've read or been shown.

Beliefs about the content of physics knowledge: The most crucial thing in solving a physics problem is finding the right equation to use.

Beliefs about the structure of physics knowledge: A significant problem in this course will be being able to memorize all the information I need to know.

Beliefs about the connection of physics to reality: To understand physics, I expect to think about my personal experiences and relate them to the topic being analyzed

Beliefs about the role of mathematics in physics: All I learn from a derivation is that the formula obtained is valid and that it is OK to use it in problems.

“Testing” procedure

“Right” or “Favorable” answers are determined by giving a survey to a group of “experts”

Students are given the survey pre and/or post-instruction

Motion of student responses toward those given by the experts is seen as progress.

Beliefs about learning physics: " Understanding" physics basically means being able to recall something you've read or been shown.

Strongly agree

Agree

Neutral

Disagree

Strongly Disagree

General Trend:

The Problem Solving Attitudes Survey Part of a project to develop a measure of

student quantitative problem solving ability Based on the assumption that “expert”

introductory physics problem solvers share characteristic attitudes.

Markers of Expertise in Quantitative Problem Solving

Expertise is characterized by more than just the ability to generate a correct answer.

Confidence-even in the face of uncertainty Ability/tendency to evaluate progress. Well established and acknowledged processes

for solving standard types of problems. Conceptual basis overrides distraction of various

surface features or mathematical tasks.

Problem Solving Attitudes Survey

MPEX like scoring philosophy and administration.

Some questions were taken from the MPEX (and modified or not).

23 total questions. Given to PER experts, RPI students (pre-

post), Southern CT and McDaniel students (post).

Example Expert Response Distributions (n=25)

Question # 48 49 50 51 52 53 56 58

Agree 23 3 23 2 23 12 1 14

Neutral 0 1 0 1 1 4 4 2

Disagree 1 21 2 22 1 7 19 9

# of comments 5 3 1 4 4 9 6 11

The three discarded questions: 53.In solving textbook problems in introductory physics,

one should NOT start out by ignoring the details presented in the problem statement.

56. In solving textbook problems in introductory physics, identifying the correct equation or equations to use is the most important part of the process.

58. In solving textbook problems in introductory physics, I often find it helpful if I can find a similar problem that is worked out in the textbook. I can then use that problem solution as a pattern for use in the solving the new problem.

Scoring the Survey Agree and Strongly agree answers are

given a value of +1 Disagree and Strongly disagree answers

are given a value of -1 All expert answers are taken to be -1 or

+1.

The Remaining 20 questions are Very Good Discriminators Between Top 1/3 and Bottom

1/3 of Students at RPI

Breaking by FMCE score, the top third was more “expert like” than the bottom third on every question.

Breaking by numerical course grade, the top third was more “expert like” on 18/20 questions.

The progression toward expert was bottom, middle, top in 13/20 questions with grouping by FMCE score and 10/20 questions with grouping by course grade.

The 10 are a subset of the 13, so this information may be used help to weed out questions.

Question # 67 64 51 70

Expert -1.00 -1.00 -1.00 -1.00

Top -0.28 -0.55 0.00 -0.27

middle 0.02 -0.28 0.00 -0.09

bottom 0.25 -0.10 0.45 0.11

Difference top/bottom -0.53 -0.45 -0.45 -0.38

# 67. If I came up with two different approaches to a problem and they gave different answers, I would not worry about it; I would just choose the answer that seemed most reasonable. (Assuming the answer is not in the back of the book.) (MPEX)

#64. Equations are not things that one understands in an intuitive sense; they are just givens that one can use to calculate numerical answers. (Modified MPEX)

Question # 67 64 51 70

Expert -1.00 -1.00 -1.00 -1.00

Top -0.28 -0.55 0.00 -0.27

Middle 0.02 -0.28 0.00 -0.09

Bottom 0.25 -0.10 0.45 0.11

Difference top/bottom -0.53 -0.45 -0.45 -0.38

# 51. In solving problems in physics, being able to handle the mathematics is the most important part of the process.

#70. If I’m not sure about the right way to start a problem, I’m stuck. There is nothing I can do with that problem except to go see the teacher or a friend for help.

Question # 61 69 55 48

Expert 1.00 1.00 1.00 1.00

Top 0.62 0.60 0.53 0.88

middle 0.57 0.70 0.47 0.79

bottom 0.43 0.40 0.21 0.53

Difference top/bottom 0.20 0.20 0.32 0.35

#61. I have a general approach that I apply in solving all problems that are solvable using conservation of linear momentum.

#69. To be able to use an equation in a problem (particularly in a problem that I haven't seen before), I need to know more than what each term in the equation represents. (MPEX)

Question # 61 69 55 48

Expert 1.00 1.00 1.00 1.00

Top 0.62 0.60 0.53 0.88

middle 0.57 0.70 0.47 0.79

bottom 0.43 0.40 0.21 0.53

Difference top/bottom 0.20 0.20 0.32 0.35

55. In solving textbook problems in introductory physics, I can often tell when my work and/or answer is wrong, even without looking at the answer in the back of the book or talking to someone else about it.

# 48. Suppose you are given two problems. One is about a box sliding down an inclined plane. There is friction between the incline and the box. The other is about a person swinging on a rope. There is air resistance between the person and the air molecules. You are told that both problems can be solved using the concept of conservation of energy. Which of the following statement do you MOST agree with? Chose only one answer.A) The two problems can be solved using very similar methodsB) The two problems can be solved using somewhat similar methodsC) The two problems must be solved using quite different methodsD) The two problems must be solved using very different methodsE) There is not enough information given to know how the problems will be solved

Question # 61 69 55 48

Expert 1.00 1.00 1.00 1.00

Top 0.62 0.60 0.53 0.88

middle 0.57 0.70 0.47 0.79

bottom 0.43 0.40 0.21 0.53

Difference top/bottom 0.20 0.20 0.32 0.35

55. In solving textbook problems in introductory physics, I can often tell when my work and/or answer is wrong, even without looking at the answer in the back of the book or talking to someone else about it.

Markers of Expertise in Quantitative Problem Solving

Confidence-even in the face of uncertainty Ability/tendency to evaluate progress. Well established and acknowledged processes

for solving standard types of problems. Conceptual basis overrides distraction of various

surface features or mathematical tasks.

Scoring the Survey Agree and Strongly agree answers are

given a value of +1 Disagree and Strongly disagree answers

are given a value of -1 All expert answers are taken to be -1 or

+1. Then student answers which agree with

experts are awarded one point.

Correlation Between Survey Score and N RExam Average (RPI) 272 0.28

FMCE Post-test (RPI) 272 0.35

Instructor Assessment of Ability (McDaniel) 17 0.34

Instructor Assessment of Ability (SCSU) 20 0.48

Correlation Between Final Grade N R

and FMCE Post-test (RPI) for these students 272 0.44

0102030405060708090

100

Experts McDanielPost

SCSU Post RPI Pre RPI Post

% A

gre

em

en

t with

Exp

ert

s

Conclusions and Future Tasks:

These questions show promise as a diagnostic tool for evaluation of intro. courses.

Expert pool should be broadened to improve face validity. Number of questions can be further reduced. Survey will be combined with other types of questions

already developed to produce a preliminary problem solving assessment.

Then re-evaluate correlations and perform validation studies.

Work funded in part by an NSF-POWRE grant.

Teaching Problem Solving in Student Centered Classrooms.

As a man asked last night:

Where does one find the time to teach problem solving when instructional time is shifted to building conceptual understanding through hands-on activities?

Spent time wisely-Do What Helps

Teach and Model an “expert-like” approach to solving problems.

Heller’s-University of Minnesota

Reif’s- Carnegie Mellon

GOAL (Beichner)

Do What Helps Teach and Model an “expert-like” approach to solving

problems. In generalA) Draw a PictureB) Identify what is given/ unknown, assign variable names

and coordinate systemC) Identify guiding conceptD) Draw Diagrams/Graphs/PicturesE) Chose equationsF) Solve algebraically-Use dimensional analysis to check

answerG) Plug in numbers, check to reasonableness.

Do What Helps Teach and Model an “expert-like” approach to

solving problems. Model the problem solving approach in doing 1-2

“touch stone” problems before the homework is assigned.

Give the students 1-2 similar problems to do in class as a group, before homework is assigned. Teach is there as resource for individual interventions.

Collect the homework, or use a computer based homework system. Force students to do the work. “You are what you grade.”

Don’t Waste Time

Don’t assign problems that require “tricks”. Challenge students with a few, but not all

of the assigned problems. Don’t go over all the homework after it is

due. If less then 25% of the class want to see a problem done, don’t do it.

I don’t post solutions and the students have never complained.

Thank You For Coming