Overview

OverviewPaper #1 - An efficacy trial of Word Generation: Results from the first year of a

randomized trialJoshua F Lawrence (University of California, Irvine), E. Juliana Paré-Blagoev (Strategic Education Research

Partnership), Amy Crosson (LRDC, University of Pittsburgh), David Francis (University of Houston), Catherine E. Snow (Harvard University)

Paper #2 - Patterns of Students’ Vocabulary Improvement from One-time Instruction and Review Instruction

Wenliang He (University of California, Irvine), Emily Galloway, Claire White, Catherine Snow (Harvard University), Judy Hsu (University of Illinois at Urbana-Champaign)

Paper #3 Engaging Middle School Students in Classroom Discussion of Controversial Issues

Alex Lin (University of California, Irvine), Joshua F Lawrence (University of California, Irvine), Patrick Hurley (Strategic Education Research Partnership)

Paper #4 - Heterogeneous Treatment Effects for Redesignated Fluent English Proficient Middle School Students

Jin Kyoung Hwang (University of California, Irvine), Joshua F Lawrence (University of California, Irvine), Elaine Mo (University of the Pacific), Patrick Hurley (Strategic Education Research Partnership)

Academic Vocabulary Instruction Across the Content Areas: Results from a Randomized Trial

of the Word Generation Program

Word Generation: Weekly Schedule

MondayParagraphintroduces

words

Tuesday-ThursdayMath-Science-Social Studies

FridayWriting withfocus words

Day 1 - Launch Introduction to weekly passage, containing academic vocabulary, built around a question that can support discussion and debate, (comprehension questions, student friendly definitions included)

Day 2 - Science Thinking experiments to promote discussion and scientific reasoning

Topic: not directly related to stem cell research but clearly a link could be madeTarget Words: investigate, theory, obtain Background Information: Countries have different views about citizens carrying guns. In some countries the import and export of guns is illegal. Subsequently, no citizen can obtain a gun in those countries (text continues).Questions: Are people more aggressive in countries that allow handguns?Hypothesis: Citizens of countries that allow handguns are more aggressive than citizens of countries that do not.Materials: Procedure: Data: Conclusion:What evidence do you have that supports your conclusion?

Day 3 - MathMathematics problems using some of the target words:a) Students can

work in pairsb) Whole class

discussionc) Open-response

(show/explain how you got your answer)

1. Some people believe that embryonic stem cell research is important. They think this because scientists use these cells to investigate diseases. Scientists try to find cures for these diseases, and for conditions like paralysis. Other people believe that embryonic stem cell research is wrong. They think this because scientists must destroy embryos to obtain these cells. In a recent poll, 40.75% of people said that the government should not pay for embryonic stem cell research. Which decimal is equivalent to 40.75%? A) 4.075B) .4075 *C) .04075D) .02

Day 4- Social StudiesDeveloping positions on the issue set out in the passage, to help the class frame the debate.

Positions:1. Scientists should not be

allowed to investigate cures for disease using stem cells from embryos. This is trying to “play God”.

2. Destroying an embryo to get the stem cells is murder.

3. The government should pay for embryonic stem cell research. This could lead to cures for many injuries and diseases.

4. Scientists should be allowed to do research on embryonic stem cells, but the government should not pay for it because many taxpayers oppose it.

Note: these are optional. The class may want to develop its own positions!

Day 5 - ELA

Writing Activity:

Should the government pay for stem cell research?

Give evidence to support your position.

Mon – Introduce words in ELA class

Tues – Math activity with target words

Wed – Social studies debate

Thurs – Science activity with target words

Friday – Writing activity

Focus on classroom discussion

• There are some studies on talk exposure and peer interactions and vocabulary for young children

• Studies on discussion of older children have examined outcomes like reading comprehension, math, science, and philosophy content.

• Connections to ELA and SS student outcomes is strong.

Research QuestionsHow engaging were classroom discussions in treatment and

control schools? Were there differences in quality across content areas? Did schools that participated in the Word Generation program demonstrate improved classroom discussion?

Did participating in the Word Generation program impact students’ knowledge of the academic words taught? Did participation effect students’ general vocabulary knowledge?

Did improved classroom discussion mediate the impact of the Word Generation program on students’ academic vocabulary knowledge?

District / School School Code District Grade 6

ContributionGrade 7

ContributionGrade 8

ContributionTotal

Contribution

Control Schools 1 1 11 17 0 283 1 24 15 0 395 1 24 19 0 436 1 38 17 0 5511 1 17 25 15 5713 1 27 43 59 12920 1 11 12 0 2335 1 21 14 17 5236 1 69 74 0 14330 2 7 0 0 731 2 6 4 0 1032 2 12 0 0 1233 2 14 10 0 24

Total 281 250 91 622

Word Generation Schools 8 1 0 29 0 29

9 1 39 28 11 7810 1 41 41 26 10812 1 32 33 23 8815 1 0 60 0 6016 1 54 87 115 25617 1 5 0 0 518 1 43 29 0 7221 2 0 5 0 522 2 0 0 12 1223 2 18 13 17 4824 2 0 0 15 1525 2 0 45 0 4526 2 0 60 0 6027 2 0 0 51 51

Total 232 430 270 932

Support for Participation (1 – 3)

the teacher created a well-ordered and respectful environment that enabled engagement with lesson content and participation in the discussion

• 1 was reserved for classrooms that were chaotic or there was student hostility or lack of participation

• 3 points were scored if nearly all students appeared consistently engaged with minimal side talk and distractions

Student Engagement (1 – 3)

percent of students participating or attending to the classroom discussion.

• 1 was awarded if around a quarter of the students participated in discussion during the observation period

• 3 was awarded when 50% to 100% of students participated in discussion

Teacher Talk Moves (1 – 5)

Teachers’ use of open-ended questions and follow-up questions asking students to explain their thinking or provide evidence for their ideas.

• 1 was given to classroom discussions in which all teacher questions had single, known answers (closed questions).

• 5 reserved for classrooms where the teacher initiated a range of question types including open-ended questions and also asked students to provide evidence or explain their ideas more clearly.

Substantive contributions (1 – 5)

The level of students’ contributions to the discussion was rated on a five point scale.

• 1 perfunctory answers were given the lowest ratings• 5 multiple students elaborated ideas and explaining

their thinking while providing evidence. (In these classrooms students also asked each other to explain their thinking or explicitly link their own to others’ contributions.)

Composite Scores

Composite Discussion Quality Rating (1 – 4) – Overall quality score for each class.

Weighted School Level Discussion Quality Ratings (z score

0

10

20

30

Feb Mar Apr May Jun Feb Mar Apr May Jun

Control Schools Word Generation SchoolsN

umbe

r of O

bser

vatio

ns

Observation Date

Academic Vocabulary Knowledge

• 36-item multiple-choice test

General Vocabulary Knowledge

Participants completed level 6 or level 7/9 of the Gates-MacGinitie vocabulary assessment (depending on their grade level)..

Covariates

Grade-level proficiency scores

School percent free and reduced lunch

School percent special education

Student grade level

RQ1

How engaging were classroom discussions in treatment and control schools? Were there differences in quality across content areas? Did schools that participated in the Word Generation program demonstrate improved classroom discussion?

0

10

20

30

1 2 3 4 1 2 3 4

Control Schools Word Generation SchoolsN

umbe

r of O

bser

vatio

ns

Composite Discussion Quality Rating

Content-Area Observed

Control Schools

Word Generation

SchoolsTotal

Difference (WG - Control

Schools)

Effect size

(Cohen's d)

Mean n Mean n Mean n

Math (n = 48) 1.85 25 2.90 23 2.35 48 + 1.05 1.08

(0.80) (0.84) (0.97)Science (n = 47) 1.85 27 2.28 20 2.03 47 + 0.43 0.47

(0.85) (0.93) (0.90)Social Studies(n = 54) 2.24 24 2.59 30 2.43 54 + 0.35 0.41

(0.83) (0.86) (0.86)ELA (n = 63) 2.26 36 2.55 27 2.39 64 + 0.29 0.35

(0.89) (0.73) (0.82)Total (n = 213 ) 2.07 113 2.59 100 2.31 213 + 0.52 0.58 (0.86) (0.85) (0.89)

RQ2

Did participating in the Word Generation program impact students’ knowledge of the academic words taught? Did participation affect students’ general vocabulary knowledge?

Treatment Effect

=

Effect Sizes (Student Level)

Outcome Measure

Overall Sample Mean Control Word Generation

Pre Post n Pre Post n δC Pre Post n δT δT - δC

Effect Size

Academic Vocabulary 18.58 19.94 1540 18.04 18.8 618 0.76 18.94 20.7 922 1.76 1.00 0.16

(6.21) (7.08) (6.02) (6.83) (6.21) (7.14)

Gates-MacGinitie Vocabulary 505.26 510.9 1400 501.54 508.44 570 6.9 507.5 512.59 830 5.05 -1.85 -0.06

(32.46) (35.0) (33.49) (36.37) (31.6) (33.87)

Basic Model

+ + + +

HLM Predicting Student

Knowledge of

Academic Words

Model A Model B With co variates

Outcome Academic Vocabulary

Academic Vocabulary

Academic Vocabulary Teaching Team Mean

0.837*** 0.840***

(0.097) (0.145)

Academic Vocabulary Teaching Team Mean Centered

0.697*** 0.697***

(0.024) (0.024)Treatment 1.264* 1.551**

(0.500) (0.501)Intercept 3.397 4.581

(1.757) (4.342)

Level 2 Variance (Teaching Team) 28.29*** 28.24***

(0.517) (0.516)Residual 1.775 1.555

(0.313) (0.283)N 1554 1554-2LL 9654.227 9647.524Standard errors in parentheses* p < 0.05, ** p < 0.01, *** p < 0.001

Model C Model DWith Covariates

Outcome Gates MacGinitie Vocabulary

Gates MacGinitie Vocabulary

Gates MacGinitie Vocabulary Teaching Team Mean

0.691*** 0.580***

(0.070) (0.086)

Gates MacGinitie Vocabulary Teaching Team Mean Centered

0.798*** 0.798***

(0.022) (0.022)Treatment 0.0151 -0.373

(2.323) (2.167)Intercept 160.9*** 235.2***

(34.920) (43.480)

Level 2 Variance (Teaching Team) 530.7*** 530.7***

(10.180) (10.180)Residual 40.88*** 27.25***

(6.785) (5.360)N 1416 1416-2LL 12956.545 12944.513

Standard errors in parentheses

* p < 0.05, ** p < 0.01, *** p < 0.001

HLM Predicting Student

knowledge of Gates Vocab

RQ3

Did improved classroom discussion mediate the impact of the Word Generation program on students’ academic vocabulary knowledge?

c prime path a path b path

Outcome Academic

Vocabulary (Post)

Composite Discussion

Quality Rating

Academic Vocabulary

(Post)Academic Vocabulary (Pre) 0.700*** 0.138*** 0.696***

(0.024) (0.031) (0.024)Treatment 1.633** 0.396* 1.271*

(0.530) (0.167) (0.560)Composite Discussion Score 0.533

(0.376)Intercept 5.812*** -2.789*** 6.090***

(0.567) (0.567) (0.586)Level 2 Variance (Teaching Team) 2.108* 1.775

(0.375) (0.334)Residual 28.38*** 28.38***

(0.538) (0.538)N 1446 1446 1446

𝑻𝑹𝑬𝑨𝑻 𝒋

𝑫𝒊𝒔𝒄𝒖𝒔𝒔𝒊𝒐𝒏 𝒋0.396*

Multilevel Mediation Model


𝑫𝒊𝒔𝒄𝒖𝒔𝒔𝒊𝒐𝒏 𝒋

𝑽𝑶𝑪𝑨𝑩𝒊𝒋

0.396*


1.633**




0.396*


0.533

1.633**




0.396*


0.533

1.633**

indirect effect = .396*.533 = .211total effect = indirect effect + direct effect = 0.211 + 1.28 = 1.48

Bootstrapping

Coefficient Standard Error z p value 95% Confidence Interval

Indirect Effect 0.21 0.10 2.22 0.03 0.02 0.40

Direct Effect 1.28 0.31 4.06 > 0.001 0.66 1.90

Total Effect 1.49 0.30 4.99 > 0.001 0.90 2.07

command: ml_mediation, dv(WGV_TCTR_W2) iv(TREAT) mv(wrub_tot_SM_Y1) cv(WGV_TCTR) l2id(unit)

Conclusions

Changes in curricular materials can improve discussion.

A free, low “implementation cost” program can have credible effects.

The proportion of total effect of Word Generation mediated through improved discussion is 0.14.

Limitation and questions

How can we further improve discussion?

Does improved discussion transfer?

What are the other effective pathways for student learning from WG?

Patterns of Students’ Vocabulary Improvement from One-time Instruction and Review Instruction

Wenliang He, Joshua Lawrence (University of California, Irvine)

Emily Galloway, Claire White, Catherine Snow (Harvard University)

Judy Hsu (University of Illinois at Urbana-Champaign)

Background

Vocabulary knowledge is positively correlated with reading skills.

Prior research has focused on studies of factors that impact the learning of academic vocabulary (e.g. frequency, exposure, prior knowledge) and effective vocabulary intervention techniques (e.g. multimodal enhancement, active processing, mnemonics).

Regardless of the importance and the prevalence of vocabulary instruction in schools, few classroom-based intervention studies have been conducted on learning the effect of review (second-time) instruction.

Research Questions

What factors predict students’ improvement in vocabulary knowledge from first-time instruction in the WG program?

Can review (second-time instruction) significantly contribute to additional improvement in vocabulary knowledge to first-time instruction?

Research Design

Group A151 students

Group B159 students

Pre-test Right before program started

Review

5 words/weekfor 7 weeks


7 Yellow Words 7 Red Words5 words/weekfor 4 weeks


Post-test By the end of the 12th week

Research DesignOverview of Yellow, Red and Core Instructional Words

Yellow Words Red Words Core Instruction Words

Count 7 7 34 adequate

(1)formulate

(8)recite (15)

conclusion (16)

invasion (17)

components (18)

critical (19)

relevant (2)

retain (9)

cohesion (20)

equity (21)

documented (22)

interaction (23)

altered (24)

capacity (3)

distribute (10)

constrain (25)

contaminated (26)

eligible (27)

complex (28)

distinct (29)

Word List

Contrast(4)

logical (11)

amended (30)

apathy (31)

maintained (32)

Concept (33)

Attributes (34)

assess (5)

disproportionate (12)

generate (35)

enforced (36)

perceived (37)

perspective (38)

conserve (39)

paralyzed (6)

obtain (13)

incentives (40)

amnesty (41)

acquired (42)

prescribed (43)

proceeded (44)

emphasize

(7)restrict

(14) enable (45)

excluded (46)

aptitude (47)

assumed (48)

Analysis & Results (Q1) What factors predict students’ improvement in vocabulary knowledge

from first-time instruction in the WG program?

1 2 3 4 51. Percent Correct in Pretest 1 2. Percent Correct in Posttest 0.90*** 13. Word Frequency 0.34* 0.30* 14. Week of Instruction -0.02 -0.04 0.09 1

5. Improvement -0.23(p=0.11)

0.2(p=0.17)

-0.09(p=0.53)

-0.03(p=0.86) 1

Note. * p < .05. *** p < .001. (two-tailed t-tests)Percent Correct is the percentage of students who are correct in a multiple choice question, testing on a specific word in either pre or post test.

Correlations between Measures from One-time Instruction Data

Analysis & Results (Q1)

Percent Correct at Pretest (%)

Word Count

Mean Percent Improvement (%)

SD (%)

Min (%)

Max (%)

0 P+ 100≦ ≦ 48 10.59 8.56 -4.77 46.00Below Average 26 11.06 10.61 -4.77 46.00Above Average 22 10.04 5.43 0.59 20.16

Detailed Breakdown 10-20 4 18.37 19.35 0.89 46.0020-30 5 11.60 13.43 -4.77 30.8530-40 7 7.11 3.14 3.22 12.5640-50 11 11.49 8.17 -0.16 22.6550-60 9 12.40 3.92 5.01 17.2160-70 4 8.63 6.55 1.10 16.2970-80 5 8.83 3.74 5.76 14.7080-90 3 3.46 2.57 0.59 5.56


1520

25

30

354045

16

21

26

31

36

41

46

17 22

27 32

37 4247

18

23

28

3338

4348

19

24

29

34

3944

assessrelevant

capacities

paralyzed

emphasized

contrast

adequatedistribute

obtain

disproportionately

formulated

restricted

logical

retain

amnesty0

1020

3040

50Im

prov

emen

t fro

m In

stru

ctio

n

10 20 30 40 50 60 70 80 90Percentage of Students Correct at Pretest

. Yellow Words Red Words Fitted values

Analysis & Results (Q1)Improvement for words with above average difficulties

relevant

capacities

distribute

obtain

enable

conclusion

formulated

contaminated

logical

retain

documented

maintained

perceivedacquired

perspective

prescribedassumed

attributes

conserve

excluded

eligible

proceeded

0

5

10

15

20

Impr

ovem

ent f

rom

Inst

ruct

ion

50 60 70 80 90Percentage of Students Correct at Pretest

Fitted values

β = -0.61, p < 0.01

Analysis & Results (Q1)Improvement for words with below average difficulties

recitecohesion

constrain

amended

assess

paralyzed

emphasized

contrast

disproportionately

equityrestricted

apathy

enforced

amnesty

invasionaptitude

components

interaction

complex

concept

criticaldistinct

generateincentives adequate

altered

0

10

20

30

40

50Im

prov

emen

t fro

m In

stru

ctio

n

10 15 20 25 30 35 40 45 50Percentage of Students Correct at Pretest

Fitted values

β = -0.20, p = 0.32

Analysis & Results (Q2) Can review significantly contribute to additional improvement in

vocabulary knowledge to first-time instruction?

Instruction

Review

Instruction

Review

0

5

10

15

20

25

30

35

Impr

ovem

ent i

n Pe

rcen

t Cor

rect

Red Words Yellow Words

Group A Group B


Overview of Review Impacts by Word Groups

Word Groups Word Count Review Impact (%) S.E.(%) p (two-tailed)All Words 14 2.58 1.71 0.16

Hard Words 7 -1.36 2.32 0.58Easy Words 7 6.51 1.47 0.004

Words below the 48% cut-off were recoded as Hard Words, and those above were recoded as Easy Words.

Conclusion

The effects of one-time instruction and review instruction depend on students’ prior word knowledge, reflected by the percentage of student correct at pretest.

For moderately difficult words, i.e. percent correct values greater than 48% in this study, improvement from one-time instruction is stronger for harder words than for easier words. However, if words are exceedingly difficulty, none of the measure in this study can effectively predict improvement from one-time instruction.

Conclusion

Review instruction is most effective (p = 0.004) when students are reasonably familiar with the words, i.e. percent correct values greater than 48%. In this case, review contributes an extra 6.51% improvement compared with the 10.04% improvement from first-time instruction.

Learning overly unfamiliar words twice, however, does not have any statistically significant effect over one-time instruction (p = 0.58).

Implications

This study indicates that

(a) effects of vocabulary instruction is most predictable with Goldilocks words, as students do not benefit as much with either overly simple or exceedingly hard words;

(b) prior knowledge is crucial in explicit vocabulary instruction

(c) teaching difficult words may necessitate more effective instruction techniques.

Thank You

Engaging Middle School Students in Classroom Discussion of Controversial Issues

Alex Lin Dr. Joshua Lawrence

Civic engagement

Declining voting turnout

Declining rate of civic knowledge

Importance of stimulating news media use and political discussion at home

Discussion of Controversial Issues

One way of improving civic knowledge is using controversial issues as topics for classroom discussion

McDevitt and Chaffee (2000) found that enrollment in Kids Vote was positively related to students’ habit of initiating political discussion at home

Hess and Posselt (2002) found that enrollment in the Public Issues was related to students making 70% more contributions in classroom discussion.

What about middle school students?

Program Features

In the Word Generation program:

Students are exposed to daily discussions of controversial topics

Different topic each week

Students take a position on an issueTeacher facilitates classroom debate

Grade 6 Grade 7 Grade 8

YEAR 1 TAUGHT

- Should colleges use Affirmative Action?

- Should the government fund stem cell research?

- Should Creation be taught in school?

- Should English be the official language of the United States?

- Is nuclear power a danger to society?

YEAR 2 TAUGHT - Should there be more strict dress codes at schools?- Is the death penalty justified?- Does rap music have a negative impact on students?- What should be done about global warming?- Should the government allow animal testing?

NON-TAUGHT TOPICS

- Should schools protect students from cyberbullying?

- Should schools be a place for debates?

- Should secret wiretapping be legal?

- Should schools have a vocational track?

Breakdown of Controversial Topics (Year 1 and Year 2) in Two Years of WG Program

Year 1 (Fall 2010 – Spring 2011)

Year 2 (Fall 2011 – Spring 2012)

1 1 & 2 1 & 2

Research Questions:

(1) In the First Year Sample (6th Graders only)How do students enrolled in Word Generation compare with control students, in terms of their confidence to participate in a discussion of controversial topics?

(2) In the Second Year Sample (7th and 8th Graders)How do students enrolled in Word Generation compare with control students, in terms of their confidence to participate in a discussion of controversial topics?

MethodsSurveys administered in May 2012 (2nd year of the study)Students were assessed the question:

How confident are you in being able to participate in a discussion about the following topics?

- 14 topic questions from the year 1, year 2 and non-taught topics

- Students responded about their discussion confidence level from (1) “not at all” to (5) “extremely”

Methods: Participants Grade Level Contributions by Schools

6 7 8 TOTALWord Generation 1,163 1,167 1,121 3,430

Control

617 482 694 1,793

TOTAL 1,780 1,649 1,815 5,223

Analysis Plan

For each sample (6th) and (7th + 8th) graders: Compare discussion confidence levels on the 14 controversial topics between

students enrolled in the WG program and control groups

Compare discussion confidence levels between students enrolled in the WG program with control groups on TAUGHT and NON-TAUGHT topics

Results

Should

there

be m

ore st

rict d

ress c

odes

at sc

hools

?

Is th

e dea

th pe

nalty

justi

fied?

Does r

ap m

usic

have

a ne

gativ

e impa

ct on

stud

ents?

Wha

t sho

uld be

done

abou

t glob

al warm

ing?

Should

the g

overn

ment a

llow an

imal

testin

g?

Should

colle

ges u

se A

ffirmati

ve A

ction

?

Should

the g

overn

ment fu

nd st

em ce

ll res

earch

?

Should

Crea

tion b

e tau

ght in

scho

ol?

Should

Eng

lish b

e the

offic

ial la

ngua

ge of

the U

nited

Stat

es?

Is nu

clear

power

a dan

ger to

socie

ty?

Should

scho

ols pr

otect

stude

nts fro

m cybe

rbully

ing?

Should

scho

ols be

a pla

ce fo

r deb

ates?

Should

secre

t wire

tappin

g be l

egal?

Should

scho

ols ha

ve a

voca

tiona

l trac

k?1.50

2.00

2.50

3.00

3.50

4.00

1.992.06

2.622.80

2.402.56

3.67

3.88

2.282.36

2.732.682.762.69

3.313.26

2.953.153.193.27

4.124.10

2.532.562.402.35

2.862.84

Confidence to Discuss Comparison by Schools (6th Grade Sample)

Confidence to Partic-ipate in Discussion

Control SchoolWG School

TAUGHT Control SchoolWG School

NON-TAUGHT

Results



NON-TAUGHT

CONT

ROL

WO

RD

GENE

R-AT

ION

CONT

ROL

WO

RD

GENE

R-AT

ION

Taught Non-taught

2.00

2.25

2.50

2.75

3.00

2.59

2.72

2.94 2.95

1st Year Students: Comparison of Students’ Self-Reports on Discussion Confidence Level Between Schools

Confi

denc

e to

Par

ticip

ate

in D

iscus

sion

0.13 ***t(1850)= -3.41, p < 0.001

0.01t(1857)= -0.41, ns

Effect size: (sd=.82) = 0.16

Results

Should th

ere be more st

rict d

ress co

des at s

chools?

Is the d

eath penalty ju

stified

?

Does rap m

usic have a

negati

ve im

pact on stu

dents?

What should be done a

bout global w

arming?

Should th

e government a

llow anim

al testi

ng?

Should co

lleges u

se Affirm

ative Acti

on?

Should th

e government fu

nd stem ce

ll research

?

Should Crea

tion be taught in

school?

Should English

be the officia

l language

of the Unite

d States?

Is nucle

ar power a

danger to so

ciety?

Should sc

hools pro

tect stu

dents fro

m cyberbully

ing?

Should sc

hools be a place

for d

ebates?

Should se

cret w

iretapping be legal?

Should sc

hools hav

e a vocational

track

?1.50

2.00

2.50

3.00

3.50

4.00

2.032.21

2.642.72

2.362.47

3.663.69

2.302.46

2.692.712.762.98

3.133.01

3.113.033.093.18

3.823.82

2.552.68

2.332.37

2.792.70

Confidence to Participate in Discussion Comparison (7th & 8th Graders)

Confidence to Partic-ipate in Discussion



NON-TAUGHT

Results



NON-TAUGHT

CONTROL WORD GENERATION CONTROL WORD GENERATIONTaught Non-taught

2.00

2.25

2.50

2.75

3.00

2.76

2.842.88 2.90

2nd Year Students: Comparison of Students’ Self-Reports on Discussion Confidence Level Between Schools

Confi

denc

e to

Par

ticip

ate

in D

iscus

sion

0.08 ***t(3615)= -3.47, p < 0.001

0.02t(1200)= -10.93, ns

Effect size: (sd=0.70) 0.11

Results

How do students enrolled in Word Generation compare with control students, in terms of their confidence to participate in classroom discussion of controversial topics?

In both (6th grade) and (7th + 8th grade) samples, students in the Word Generation program report more confidence in being able to have a classroom discussion of controversial topics that are covered in class

Discussion

Consistent with past studies that curricular exposure to controversial topics can improve students’ confidence to participate in discussion

Students’ abilities to participate effectively in discussion of controversial issues can improve as a result of being enrolled in a course that places primacy on such discussions (Hess & Posselt, 2002)

Young people draw more attention to news information when that information is perceived to be useful for school assignments and peer conversations (Atkin, 1981; Kanihan & Chaffee, 1996; McDevitt and Chaffee (2000).

Future Work

- Examine students’ perception of openness in classroom discussion- Freedom to disagree in class- Teacher’s skill in facilitating discussion

- Compare students’ perception of openness in classroom discussion between WG and CO schools- Student level - Classroom level

- Examine students’ habit of extending discussion of controversial issues outside of home and their news media use (newspaper and Internet)

.

Works Cited

Atkin, C. (1972). Anticipated communication and mass media information seeking. Public Opinion Quarterly, 36, 188-199.

Hess, D., & Posselt, J. (2002). How high school students experience and learn from the discussion of controversial public issues. Journal of

Curriculum and Supervision, 17(4), 283-314.

Kanihan, S., & Chaffee, S. H. (1996). Situational influence of politicalinvolvement on information seeking: A field experiment. Paper

presented at the annual meeting of the Association for Education in Journalism and Mass Communication, Anaheim, CA

McDevitt, M., & Chaffee, S. (2000). Closing gaps in political communication and knowledge effects of a school intervention. Communication Research, 27(3), 259-292.

Appendix

Taught

Non-taught

Taught

Non-taught

2.00 2.25 2.50 2.75 3.00

2.59

2.94

2.72

2.95

1ST Year Students: Comparison of Students' Discussion Ability Between Schools (6th Graders)

Confidence to Participate in Classroom Discussion

CONTROL

CONTROL

WORD GENERATION

WORD GENERATION

-0.35 ***t(650)= -12.22, p < 0.001

-0.23 ***t(1200)= -10.93, p < 0.001

Results

Taught

Non-taught

Taught

Non-taught

2.00 2.25 2.50 2.75 3.00

2.76

2.88

2.84

2.90

2ND Year Students: Comparison of Students' Discussion Ability Between Schools (7th & 8th Graders)

Confidence to Participate in Classroom Discussion

WORD GENERATION

WORD GENERATION

CONTROL

CONTROL

-0.06 ***t(2319)= -15.20 p < 0.001

-0.12 ***t(1283)= -15.20, p < 0.001

Heterogeneous Effects of Word Generation on

Students with Differing Levels of English Proficiency

Jin Kyoung Hwang, Joshua Lawrence (University of California, Irvine)

Elaine Mo (University of the Pacific)

Patrick Hurley (Strategic Education Research Partnership)

Language Minority (LM) Learners

• School-aged students in the US who hear and/or speak a language other than English at home (August & Shanahan, 2006)

• Large population• Over 11 million students, about 5 million

are classified as limited English proficient (Aud et al., 2012)

• Largest growth• One of the fastest growing groups among the

school-aged population in the U.S. (Aud et al., 2011)

Language Minority (LM) Learners

• Differing Levels of English Proficiency• Initially fluent English proficient (IFEP)

• Students with full English proficiency by the time they enter school

• Limited English proficient (LEP) • Students who still need English language support

• Redesignated fluent English proficient (RFEP)• Students who used to be LEP but attained sufficient

English proficiency to be reclassified

Vocabulary Intervention & LM Learners

• Quasi-experiments of WG (Lawrence, Capotosto, Branum-Martin, White, & Snow, 2012; Snow, Lawrence, & White, 2009)

• LM students with full English proficiency benefitted more • LM students with limited English proficiency did not

• Vocabulary Improvement Program (Carlo et al., 2004) • ELLs in treatment schools improved more than EO students on

polysemy task • Improving Comprehension Online (Proctor et al., 2009/2011)

• No language by treatment interaction• Quality English and Science Teaching (August et al., 2009)

• No language by treatment interaction• Language Workshop (Townsend & Collins, 2009)

• All participants were Spanish-English speakers

Research Questions

• Is there a heterogeneous effect of Word Generation for EO, IFEP, and RFEP students?

• Is there a heterogeneous effect of Word Generation for RFEP students according to the number of years they have been redesignated?

Methods: Participants

• 13 middle schools: 7 treatment & 6 control• N = 6,193 students• Language Status

• Initially Fluent English Proficient (IFEP, n = 632)• Redesignated Fluent English Proficient (RFEP, n = 2,515) • Limited English Proficient (LEP, n = 1,011) • English only (EO, n = 2,035)

• Years since Redesignation• 0-1 year: n = 419• 1-2 years: n = 754• 2-3 years: n = 419• > 3 years: n = 661

Methods: Measures• Dependent Variable:

• Academic Vocabulary• Word Generation Vocabulary Posttest (50 items total; 40 items were taught throughout the

program)

• Independent Variables:• Intervention• Language Proficiency Groups (EO, IFEP, RFEP; RQ1)• Years since Redesignation (for RFEP students only; RQ2)• Academic Vocabulary

• Word Generation Vocabulary Pretest (School mean & School centered mean)

• Reading Comprehension• Gates-MacGinitie Comprehension Test (School mean & School centered mean)

• Grade Level• School of Attendance• Gifted and Talented & Special Education

Methods: Analysis Plan

RQ 1: Heterogeneous Effect of WG for EO, IFEP, and RFEP students?

RQ1: Heterogeneous Effect of WG for EO, IFEP, and RFEP students?

EO IFEP RFEP24

24.525

25.526

26.527

27.528

28.529

TreatmentControl

Language Proficiency Groups

Aca

dem

ic V

ocab

ular

y: W

G P

ostte

st

RQ2: Heterogeneous Effects of WG within RFEP students?

RQ2: RFEP Students with Different Years since Redesignation

0-1 years 1-2 years 2-3 years >3 years2021222324252627282930

TreatmentControl

Years since Redesignation

Aca

dem

ic V

ocab

ular

y:W

G P

ostte

st

Conclusion

• Heterogeneous effects of Word Generation• Replicated and extended findings from previous

Word Generation studies

• RFEP students are a heterogenous group

ReferencesAud, S., Hussar, W., Kena, G., Bianco, K., Frohlich, L., Kemp, J., Tahan, K. (2011). The

Condition of Education 2011 (NCES 2011-033). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Aud, S., Hussar, W., Johnson, F., Kana, G., Roth, E., Manning, E., . . . Zhang, J. (2012). The Condition of Education 2012 (NCES 2012-045). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

August, D., & Shanahan, T. (2006). Developing literacy in second-language learners: Report of the National Literacy Panel on language minority chidren and youth. Mahwah, NJ: Erlbaum.

August, D., Branum-Martin, L., Cardenas-Hagan, E., & Francis, D.J. (2009). The impact of an instructional intervention on the science and language learning of middle grade English language learners. Journal of Research on Educational Effectiveness, 2(4), 345-376.

ReferencesCarlo, M.S., August, D., McLaughlin, B., Snow, C.E., Dressler, C., Lippman, D.N., . . . White,

C.E. (2004). Closing the gap: Addressing the vocabulary needs of English language learners in bilingual and mainstream classrooms. Reading Research Quarterly, 39(2), 188-215.

Kieffer, M.J. (2008). Catching up or falling behind? Initial English proficiency, concentrated poverty, and the reading growth of language minority learners in the United States. Journal of Educational Psychology, 100(4), 851-868. doi: 10.1037/0022-0663.100.4.851

Lawrence, J., Capotosto, L., Branum-Martin, L., White, C., & Snow, C. (2012). Language proficiency, home-language status, and English vocabulary development: A longitudinal follow-up of the Word Generation program. Unpublished manuscript, Harvard University.

Proctor, C.P., Dalton, B., Uccelli, P., Biancarosa, G., Mo, E., Snow, C., & Neugebauer, S. (2011). Improving comprehension online: Effects of deep vocabulary instruction with bilingual and monolingual fifth graders. Reading and Writing, 24(5), 517-544.

Townsend, D., & Collins, P. (2009). Academic vocabulary and middle school English learners: An intervention study. Reading and Writing, 22(9), 993-1019.

Overview

Documents

Transcript of Overview