Benefits of Early "Do-It-Yourself" Teaching

Evaluation

Ever seen this in your classroom?

Dr. David McConnell MEAS

Dr. Meg BlanchardScience Ed

Dr. Chris Osburn MEAS

Why evaluate teaching?

Formative evaluations Provide feedback to improve

teaching effectiveness

Summative evaluation Provide information to assist

administration in making personnel decisions

Fact or FictionThe average student‘s attention span is

between 10 and 20 minutes

Fact(Penner, 1984). But the average college

class lasts for 60-90 minutes. Deal with drifting attention by a lively

lecturing style, and by breaking up longer lecture into 10-15 minute

chunks (Atherton, 2001).

Fact or Fiction?Active learning is almost impossible in

large lecture sections

FictionActive learning formats that work well

are cooperative group work (Johnson, Johnson & Smith, 1991), class-wide discussions (Gullette, 1992), and

interactive lectures (Mazur, 1993; Sokoloff, 1994; Van Heuvelen, 1991).

Fact or Fiction?Current trends point to more and

more large class sizes –small classes may be a thing of the past

Fact(McKinnon, 1998). You can exploit advantages of size and diversity with many activities that work

better with more students (Wolfman, 2002).

For example: You can do this with a large class http://www.youtube.com/watch?v

=8O4A903Thlw

Pair Share: Confer with a neighbor for 1 minute to try to come up with something you can do or do better with a large class.

Fact or FictionLecture is as effective as other methods

in promoting independent thought or developing students' thinking skills

FictionHowever, research shows that lecturing is as effective as other

instructional methods in transmitting information (Bligh,

1971).

Fact or Fiction

Research has shown that large class sizes are not necessarily detrimental to

education.

FactIt is the quality of the instructon

that has the greater effect (Godfrey, 1998).

Students’ complaints about large class instruction

Lack of interaction

Little attention to personal needs

Little variation (Boring)

Afraid to ask a question

Education is not the filling of a pail but the lighting of a fire.

About student evaluations

Reliability, validity of student ratings is generally robust

Moderate to strong correlations between student ratings and student performance on final exams

Students well prepared to assess Clarity of presentation Instructor-student rapport Concern for students’ progress

Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

About student evaluations

Some Caveats Electives get higher ratings than required

courses Instructors in graduate and upper division

courses receive higher ratings than undergraduate and lower division courses

Students ratings are higher when Instructor is present Students know evaluations are used for

personal decisions They are not anonymous

Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

Instructor Behaviors and Student Evaluations

Student observers visited classrooms of 18 instructors with high teacher evaluation ratings, 18 with low ratings, and 18 with intermediate ratings.

All professors had been teaching for at least three years and had a class with at least 30 students.

All were in the social sciences and represented six departments.

Each teacher was observed at least three times by 6-8 different student observers using a 60-item Teacher Behavior Index (TBI).

Murray, H., Journal of Educational Psychology, 1983. 75(1): p. 138-149.

Instructor Behaviors and Student Evaluations

Which 6 behaviors showed significant differences between low- and high-rated instructors and which 6 behaviors were scored evenly between instructors with different ratings.

• Speaks expressively or emphatically

• Puts outline of lecture on board

• States teaching objectives

• Shows strong interest in subject

• Advises students about tests

• Moves about while lecturing

• Uses humor

• Smiles or laughs

• Uses audiovisual aids

• Uses graphs and diagrams

• Addresses students by name

• Asks questions of individual students

Murray, H., Journal of Educational Psychology, 1983. 75(1): p. 138-149.

How Can We Evaluate our Teaching?

• Self-review of actual teaching (video)

• Student review using Group Instructional Feedback Technique

• Peer review using Reformed Teaching Observation Protocol

The Blanchard Challenge:

Experience your class as your students do.

Videotape one class

Watch the tapeReflect:• How did it go?• What did I want my

students to learn?• What would I do

differently next time?

“Dr. McConnell, may I be excused? My brain is full.”

Pedagogical Discontentment

Recent study: 2-years with 28 middle school

mathematics and science teachers Teachers who were less satisfied with what they

were doing were 8 times more likely to change their practices to more reform-based (with PD) than teachers who were more satisfied with their teaching (Blanchard, Osborne, and Albert, 2010).

Watching a video of your teaching can help you to explore aspects of your teaching that you may find less satisfying.

Good Practices for Undergraduate Teaching

Don’t have a video camera handy? Try these:• Encourage student-faculty contact• Develop cooperation among students• Encourage active learning• Provide prompt feedback• Emphasize time on task• Communicate high expectations• Respect diverse talents and ways of learning

Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7

Review handout checklists for 70 potential practices

How Can We Evaluate our Teaching?

• Self-review of actual teaching (video)

• Student review using Group Instructional Feedback Technique

• Peer review using Reformed Teaching Observation Protocol

What is working?What is going well in this class? What is helping you learn?

What isn't?Give an example of one or two things that make it more difficult for you to learn in this course.

What can be improved?What could use improvement in this course, and what specific suggestions would you make for change?

Angelo, T. A. & Cross, K. P. 1993. Classroom Assessment Techniques: Jossey-Bass.)

Group Instructional Feedback Technique

Case Study

Dr. Christopher Osburn, MEAS

Group Instructional Feedback Technique

How Can We Evaluate our Teaching?

• Self-review of actual teaching (video)

• Student review using Group Instructional Feedback Technique

• Peer review using Reformed Teaching Observation Protocol

About peer evaluations

Reliability, validity of peer evaluations not as well established as student evaluations

Peers may be better than students in evaluating Content mastery Course design Curriculum development

Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

Reformed Teaching Observation Protocol RTOP: Cliffs Notes Version

• Range from 0 “Never Occurred” to 5 “Very descriptive”

• Use to see how teacher (0-2) to Learner (3-5) centered a lesson is across 25 items on 5 subscales

• Score ranges form 0 – 100, with less than 50 being more traditional and above 50 is more reform-based

• Descriptive, not judgmental—if it occurred, you code it, if not, you don’t.

http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/

RTOP Data

RTOP range, 20s-70s for Physical Sciences classes

Higher RTOP scores correlate with higher learning gains

RTOP Data

Higher RTOP scores correlate with higher learning gains

RTOP scores range from high teens to 80s in introductory geology classes

Who determines the direction of the lesson, the teacher or the student?

How strong is teacher’s content knowledge, connections to real world & conceptual understanding?

How much ownership do students take over representing, testing, and reflecting on ideas?

How much do students communicate with the teacher, and each other and what is the quality of those communications?

How much listening and support and patience does the teacher have for the students’ ideas?

Lesson Design

Content Knowledge

Procedural Knowledge

Communicative Interactions

Student/Teacher Relationships

Reformed Teaching Observation Protocol RTOP: 5 Subscales

Group RTOP Activity

Pick an RTOP Subscale

Your Task: Review the 5 items in the subscale and discuss in your group:

1) Which item(s) do you most want to reform?

2) How might you do that (strategies)?

3) Present to the whole groupReview RTOP rubric handout for tips on what to look

for when using scoring scales

Benefits of Peer/Self-Evaluation

√ Help with end-of-semester evaluations

√ Part of faculty development efforts

√ Personal reflection on teaching and

learning

√ Increase faculty credentials for internal

and external teaching awards

√ Data for use in program review and

accreditation

√ More enjoyment

√ More student learning

Fact or Fiction?

When students have copies of the lecture notes or a text they are less likely to come to a traditional lecture class

FactA significant percentage of students

prefer reading notes rather than attending classes that offer little or

no interaction (Edlich 1993; Sullivan & McIntosh, 2002).