Improving impacts of classrooms: Professional Development and Classroom Observation Robert C....

Improving impacts of classrooms:Professional Development and

Classroom Observation

Robert C. Pianta, Ph.D.University of Virginia

CASTL

CASTLCenter for Advanced Studyof Teaching and Learning

Questions and challenges for policy, research, and training in education

What are offered to students in classrooms?

Do interactions with teachers and experiences in classrooms matter for students?

Can observation leverage improvements quality, and effectiveness of teachers/teaching?

Measure, validate, improve teacher quality at scale through standardized observation

Results of large-scale observational studies

National-level studies National Center for Early Development and

Learning (NCEDL)

NICHD Study of Early Child Care

Up to 1,000 settings observed at preschool, K, 1, 3, 5 ― more than 4,000 classrooms

Largest set of systematic standardized classroom observation in U.S. schools

All teachers credentialed/certified

Observations and large-scale applications

Trade-offs and decisions Multiple versus single occasions

Length of the “window”

Time of day / content of instruction

Unit of analysis – global or micro

Classroom-level or child-level

Training demands and reliability

Applicable across diverse settings

No system can address every concern

Ultimate criterion is link to child outcomes

Describing opportunities to learn: Counting behaviors, activities, practices

Vast majority of interaction/activity is whole group or individual seatwork

Few, if any, social or instructional interactions between teacher and individual child

Mostly literacy activities

Exceptional variation within and across grades

Consistent patterns from pre-k to 5th grade

High levels (30%) of “business/routine” activity pk-5: managing materials, routines

High levels of “basic skills” focus 7:1 in pk-1; 14:1 in 3-5

Ratio of listening, sitting, watching: Doing 10: pk-1 1,3,5

(NICHD ECCRN 2002, 2004)

How do students spend time?

Rating interactions: What is the CLASS?

Derived from developmentally-informed analysis of settings and putative impacts on broad-based outcomes.

Focus on dimensions of teacher-student interaction in 3 domains - a theoretical claim about latent structure of classroom settings

Emotional Support

Organization / Management

Instructional Support

DO

MA

INS

DIM

EN

SIO

NS

IND

ICA

TO

RS

InstructionalSupport

EmotionalSupport

Classroom Organization

Relationships, Affect, Respect, Communication

Punitive, Sarcasm/ disrespect, Negativity

Aware, Responsive, Address problem,

Comfort

Flexibility, Autonomy, Student expression

Clear expectation, Proactive, Redirection

Maximize time, Efficient routines and transitions

Variety, Promote student interest, Clarity, Engaging

Analysis/reasoning, Creativity, Integration

Feedback loops, Encourage responses, Expand performance

Conversation, Open-ended, Repeat/extend,

Advanced language

Classroom ratings: CLASS PK-5

Positive climate Negative climate Teacher sensitivity Regard for student perspectives

Effective behavior management Learning formats/engagement Productivity

Concept development Evaluative feedback Language modeling

EmotionalSupport

InstructionalSupport

Organization/Management

Positive emotional climate

Productivity

Quality of feedback

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

What is the quality of the classroom setting?

Profiles of classroom quality: First grade

Qu

alit

y

Positive emotionalclimate

Low academicdemand

Very positiveemotional climateHigh instructional

quality

Mediocre,Low academic

demand

Negative emotionalclimate,

Low instructionalquality

31% 23% 29% 17%

0

1

2

3

4

5

6

7

Emotional

Instructional

Exceptional variability within and across grades, including across the same grade/curriculum. Stability across grades is low – 15%

Little to no association of observed interactions with:

Teacher experience or training

Teacher salary

Small associations with structure and selection (.10 - .20)

Class size: larger classes more structured; smaller classes more social and higher instructional quality

Family income/education related to more positive ratings

Students needing access to stable high-quality instruction do not typically receive it – 10% rate for low achievers

Associations with structural/selection factors

Designs that isolate effects for instructional and emotional inputs controlling for other influences – in growth models Family and demographic factors Child’s prior performance Structural features of schooling

Primarily small main effects (+/- .10) Instructional and emotional quality predict more positive

achievement and social outcomes Larger effects on more proximal outcomes (e.g., child engagement) More instruction in literacy and math also predict to those

outcomes

Stronger effects for different groups of children Low maternal education Adjustment problems in K Poor

Classroom interactions and children’ssocial and academic performance

Pre-k quality and growth in child outcomes

Where should we focus attention in policy, program development, and teacher preparation?

Predicting achievement growth during preK from:

Structural features (teacher ed., curriculum, etc.)

Observed interactions (ECERS, CLASS)

No association of structure with outcome, singly or in combination (e.g., NIEER index)

Instructional and Emotional Supports (CLASS) predict positive changes in literacy, language, and math skills – small effect sizes – persist into kindergarten

Gains in Gr. 1 achievement in instructionally supportive classrooms

Standardized tests of

achievement adjusted

Low Moderate High

1st Grade Instructional Support

98

99

100

101

102

103

104

105

106

107

High educ.

Low educ.

Gains in Gr. 1 achievement in emotionally supportive classrooms

Standardized tests of

achievement adjusted

Low Moderate High

1st Grade Emotional Support

Kindergartenadjustmentproblems

98

99

100

101

102

103

104

105

106

107

No problems

Multiple problems

Implications of our work P-5

Redefine “teacher quality” in terms of performance/interactions in classroom

Strive toward moving instructional dimensions and implementation “up” the scale of quality interactions

Develop training and support approaches that address teachers’ interactions with children

Approach these goals systematically and scientifically with standardized, validated observations at core

A science of teaching and teacher-training that relies on direct and validated observation

Measurement issues/directions

Develop extension of CLASS for grades 6-12

Ecometrics – Partitioning variance related to rater, time, day, season, window, consider alternative units of analysis.

Global features more stable, valid. Reliability improved through adding raters in the system. Implications for scalability and for research on intervention effects, etc.

Informant-report version; empirical tests of content-knowledge hypotheses

Standardized observation: Support for high-quality interactions

Professional development/ training

Resource allocation

Evaluation

Curriculum

Social and academic outcomes for children

Improved teacher outcomes

CLASS InstructionalOrganization

Social

Observational Assessment

myteachingpartner.net Summer/Fall 2005


Focus on interactions in classrooms

Teacher-child interactions and relationships are the means by which the curricula are implemented

MTP uses CLASS as the basis for defining high-quality implementation and as the target/focus of professional development

Goals of MTP are to:

Increase teachers’ observation skills in identifying interactive behaviors and cues related to CLASS

Identify children’s differential responses to teacher behaviors

Increase teachers’ skills in identifying alternative responses to children's’ cues – create opportunity


MTP support for teachers

Student growth in language, literacy and social relationships

Teacher-child interactions and relationships: CLASS

Two forms of web-based support for quality implementation

MTP Activities


Project design

240 Participating PreK teachers statewide - VPI

3 Conditions of participation / levels of support

Consultancy plus website plus curriculum

Website access plus curriculum

Materials/curriculum only

All conditions

Receive iBook laptops

Receive MTP activities

Complete surveys and assess 4 randomly selected children


Quality teaching videos: PK-3


CLASS examples: PK-3


MTP consultation cycle

Classroom video recordingat an established time

1

Teacher reviews clipsand reflects on practice

3

Consultant reviewsand edits video clips

2

Teacher and consultantmeet and discussteaching practices

4


Prompts

“This clip is a nice demonstration of concept development. You ask the children why the girl in the book is happy. You receive one answer and then go on and get ideas from two other children. What other strategies do you use to promote concept development?”

Intended to identify positive aspect of teacher’s instruction in relation to a class dimension


Prompts

“Here you quickly and effectively redirect the children back to the story. As you watch this clip, tell me what you are doing to help the children remember the rules.”

Intended to help a teacher identify CLASS dimensions and examples in her teaching interactions


Prompts

“Some of the other children are

not engaged in this lesson.

What kinds of learning formats

might you have used to draw

all of the children in?”

Feedback on how the teacher implements lessons


MTP-S Prompts: Feedback for teachers

Effects of MTP support on interactions

Examine both effects of condition on outcome (web vs consultation), treatment on treated, and moderation with regard to classroom demands.

Teachers receiving consultation show greater increases in quality of instructional interactions; early career teachers who view CLASS videos show gains in interactions with children; effects seem attributable to video review

Consultation moderates poverty effect

Changes in sensitivity for teachers in the MTP consultation and web only study conditions

3.5

4

4.5

5

Septe

mber

Octobe

r

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Mar

ch

April

May

June

Te

ac

he

r S

en

sit

ivit

y

Consultancy

Web Only

Changes in language stimulation for teachers in the MTP consultation and web conditions

3.5

4

4.5

5

Septe

mbe

r

Octobe

r

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Mar

ch

April

May

June

Lan

gu

age

Mo

del

ing

Consultancy

Web Only

Moderating effects of study condition on the association between classroom poverty and changes in teacher sensitivity

3.5

4

4.5

5

5.5

Septe

mbe

r

Octo

ber

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Mar

ch

April

May

June

Te

ac

he

r S

en

sit

ivit

y

Consultancy--50% Poor

Consultancy--100% Poor

Web Only--50% Poor

Web Only--100% Poor

Moderating effects of study condition on the association between visiting video pages and changes in teacher sensitivity

3.5

4

4.5

5

Septem

ber

Octob

er

Novem

ber

Decem

ber

Janu

ary

Febru

ary

Marc

h April

May

June

Tea

cher

Se

nsi

tivi

ty

Consultancy--No Videos

Consultancy--Average Videos

Web Only--No Videos

Web Only--Average Videos

Effects of MTP support on child outcomes

Examine effects of condition and treatment on treated with Consultancy, Web, and Activities groups.

When teachers participate in consultation, children show greater gains in tests of early literacy

Definitional Vocabulary

Phonological Awareness

Print Knowledge

PALS Total

Level-1 B SE B SE B SE B SE

Child Characteristics

Pre-test 0.45*** 0.03 0.53*** 0.03 0.48*** 0.02 0.49*** 0.02

Boy -0.06 0.17 -0.92** 0.27 -1.19** 0.39 -1.33* 0.64

Non-English -0.66 0.34 -0.26 0.52 0.46 0.70 0.51 1.18

Maternal Education (years) 0.11* 0.05 0.30*** 0.08 0.13 0.11 0.12 0.18

Poverty -0.24 0.24 -0.07 0.40 -0.21 0.47 -0.58 0.83

Second Year of Intervention 0.55** 0.17 0.62* 0.28 0.50 0.35 1.62** 0.59

Associations between Teachers’ Exposure to the MyTeachingPartner Consultancy, Language and Literacy Activities and Web-Site, and Children’s Development of

Language and Literacy Skills during Pre-K

*p≤.05. **p≤.01. ***p≤.001.

DefinitionalVocabulary

PhonologicalAwareness

PrintKnowledge

PALSTotal

Level-2 B SE B SE B SE B SE

Classroom Characteristics

Mean Pre-test Score 0.17** 0.06 0.07 0.08 0.12 0.08 -0.11 0.06

Proportion in Poverty 0.04 0.60 0.44 1.06 0.42 1.72 0.53 2.80

Proportion Non-English -0.35 0.63 -0.90 1.01 -3.00 1.67 -10.2*** 2.70

Mean Mother’s Education -0.05 0.16 0.30 0.28 -0.04 0.47 -0.27 0.79

Teacher Characteristics

Advanced Degree 0.07 0.22 -0.44 0.36 -0.32 0.62 0.03 1.05

Major in ECE 0.33 0.21 0.59 0.35 0.59 0.60 2.69** 0.99

Years Teaching PK -0.00 0.01 0.02 0.02 0.01 0.04 0.06 0.06

Intervention Components

GT 20 hours/0 hours 1.03** 0.33 -0.12 0.54 0.88 0.95 2.45 1.50

GT 20 hours/LT 20 hours 0.76* 0.31 -0.22 0.57 1.40 0.90 2.23 1.54

Hours per week MTP-LL Activities

0.22 0.14 0.67** 0.24 0.44 0.38 1.41* 0.64

Low Web Use 0.42 0.32 -0.33 0.53 1.09 0.94 1.98 1.52

Medium Web Use 0.08 0.29 -0.54 0.48 0.41 0.82 1.53 1.34

*p≤.05. **p≤.01. ***p≤.001.

Associations between Teachers’ Exposure to the MyTeachingPartner Consultancy, Language and Literacy Activities and Web-Site, and Children’s Development of

Language and Literacy Skills during Pre-K (continued)

Moderating Effects of Teachers Years of Experience on the Association Between Exposure to the Consultancy

and Children’s Development of Print Knowledge

24

26

28

30

32

2 years 8 years 14 years

Years Teaching PK

Sp

rin

g P

rin

t K

no

wle

dg

e

0 hoursConsultancyGT 20 hoursConsultancy

Standardized observation of interactions

Is feasible, reliable and valid. A scalable language and lens for classroom settings

Three domains: Emotional, Organizational, Instructional appear valid across grades

A lever for research on teacher professional development and preparation to increase setting quality and child outcomes

Implications for accountability systems, teacher quality, research on teacher ed.

Improving impacts of classrooms: Professional Development and Classroom Observation Robert C....

Documents

Transcript of Improving impacts of classrooms: Professional Development and Classroom Observation Robert C....