ERP Investigation of Prosodic and Semantic

Focus

Shawn Johnson

Charles Clifton, Jr.

Mara Breen

Andrea Eileen Martin

Joanna Morris Florack

Birch & Clifton, JML 1995, 2002

Effects of pitch accent appropriateness on discourse comprehension

Follow-up to Bock & Mazalla 1983 Evelyn kissed Jeremy. RHONDA kissed Jeremy too. ? Ronda kissed Jason. RHONDA kissed Jeremy too. Faster auditory sentence comprehension times when

pitch accent fell on the NEW item.

Focus Projection: Birch & Clifton, 1995

Question: Isn’t Kerry pretty smart? Answers

A: Yes, she TEACHES MATH B: Yes, she teaches MATH C: Yes, she TEACHES math

A and B more acceptable than C Focus projects from argument “math”

Birch & Clifton, 2002

Focus does not project from adjuncts Question: How did Ted get to Minnesota? Answers

A: He DROVE SPEEDILY B: He drove SPEEDILY C: He DROVE speedily

A acceptable, B and C not 1995: B was acceptable when the final word was an

argument rather than an adjunct

Experiment 1

Materials: 2-speaker dialogs Setting: Rhonda kissed Jason. (active) Question: Who else was kissed by Rhonda?

always passive; half theme question, half agent question

Answer: JEREMY was kissed by Rhonda, too. Always passive; half appropriate pitch accent, half

inappropriate pitch accent

Theme/Theme

Rhonda kissed Jason. Who else was kissed by Rhonda? JEREMY was kissed by Rhonda, too.

Agent/Theme

Evelyn kissed Jeremy. Who else was Jeremy kissed by? JEREMY was kissed by Rhonda, too.

Agent/Agent

Evelyn kissed Jeremy. Who else was Jeremy kissed by? Jeremy was kissed by RHONDA, too.

Theme/Agent

Rhonda kissed Jason. Who else was kissed by Rhonda? Jeremy was kissed by RHONDA, too.

Focused material is Focused material is underlinedunderlined,, pitch accented material is in pitch accented material is in BOLD CAPS, BOLD CAPS, inappropriate inappropriate responses are in responses are in redred, and appropriate responses are in , and appropriate responses are in blueblue..

Appropriate (agent/agent)

Inappropriate (agent/theme)

Details EEG’s were sampled at 500 Hz using a 32-channel

Neuroscan system.

Participants judged whether dialogs ‘sounded acceptable’.

ERP's collected for the first and second noun phrase of the answer (200 ms before onset, 1200ms after onset)

2 x 2 x 2 design (Presence/absence of pitch accent X Appropriate vs. inappropriate accenting X Early vs. late noun phrase)

Focus vs. Non-Focus in answer appeared as interaction between presence/absence of pitch accent X appropriateness of accenting

Electrodes were combined into 2 groups. Parietal electrodes (P3, PZ, P4, CP3, CPZ, CP4, C3, CZ, and C4). Frontal electrodes (FC3, FCZ, FC4, F3, FZ and F4). Samples from these two electrode groups were averaged into 100 ms

bins for statistical analysis.

Time fromWord Onset (ms)

-200 0 200 400 600 800 1000 1200A

pp-N

onF

oc

-4

-3

-2

-1

0

1

2

3

4

Focus vs NonFocus (Appropriate vs. Inappropriate) Posterior 9, Experiment 1

App-NonFoc

App-Foc

Inapp-NonFoc

Inapp-Foc

Time from Word Onset (ms)

-200 0 200 400 600 800 1000 1200

volt

age,

uv

-4

-3

-2

-1

0

1

2

3

4

Focus vs Nonfocus (Appropriate vs. Inappropriate) Anterior 6, Experiment 1

App-NonFoc

App-Foc

Inapp-Nonfoc

Inapp-Foc

Time fromWord Onset (ms)

-200 0 200 400 600 800 1000 1200vo

ltag

e, u

v

-4

-3

-2

-1

0

1

2

3

4

Focus vs NonFocus (Early vs. Late) Posterior 9, Experiment 1

E-NonFoc

E-Foc

L-NonFoc

L-Foc

Time from Word Onset (ms)

-200 0 200 400 600 800 1000 1200

volt

age,

uv

-4

-3

-2

-1

0

1

2

3

4

Focus vs Nonfocus (Early vs. Late) Anterior 6, Experiment 1

E-NonFoc

E-Foc

L-NonFoc

L-Foc

Experiment 1 Conclusions

Effects of Semantic Focus – A phrase that presents queried (focused) information elicits a prolonged late positivity. Doesn’t rely on focused phrase having a pitch accent Like Cutler & Fodor (1979) phoneme monitoring

Extremely similar to Hruska, Steinhauer, Alter, Strube (2001) finding

Effects of Word Position – This positivity was larger and appeared earlier when the focused word was late in the sentence than when it was early, especially for the posterior electrodes.

Conclusions about Pitch Accent

Effects of Pitch Accent - An early negativity appeared in the Focus/Inappropriate condition Negativity was elicited by a ‘missing’ pitch accent (see

Hruska, Alter, Steinhauer & Steube, 2001, for a similar effect).

Extra pitch accents did not trigger any ERP activity but linguists have noted that early pitch accents can be added

quite freely in English.

Why is there a late positivity?

The focus-elicited waveform could reflect some sort of gross integration process. Kaan, Harris, Gibson and Holcomb (2000)

found a similar positive deflection under conditions of long distance syntactic integration

Steinhauer, Alter and Friederici (1999) found similar positive deflections at intonational phrase boundaries (where integration effects might conceivably occur).

Why did pitch accent have so little effect?

Semantic focus effects dominated our data Is it because all our target sentences were passives? The

listener did not need to hear a pitch accent to know when the focused phrase was going to occur

If we make the location of focused information more unpredictable, will listeners rely more on prosodic information and exhibit ERP effects related to prosodic appropriateness?

Experiment 2 (preliminary)

Similar to Experiment 1, except: Target sentence was active or passive 16 conditions:

Active/Passive X Early/Late X Pitch Accent/No Pitch Accent X Appropriate/Inappropriate

All questions were passives Only 10 subjects so far…

Active Appropriate

Passive Sentence Data

E.g. - Who else was kissed by Rhonda? Jeremy was kissed by Rhonda.

Data quite similar to Experiment 1, despite presence of active sentences in Experiment 2 Clear late positivity to semantically focused word Bigger, faster to second than to first word A suggestion of early negativity to missing pitch accent

Passive, focus vs nonfocus

Passive, appropriate vs inappropriate

Active Sentences

E.g. Who else was kissed by Rhonda? Rhonda kissed Jeremy. Patient question, “Jeremy” the focus

E.g., Who else was Jeremy kissed by? Rhonda kissed Jeremy. Agent question, “Rhonda” the focus

Some results similar to passives E.g., possible positivity to focused word,

beginning 400+ ms (parietal electrodes) No strong evidence for appropriateness or PA effects Anterior electrodes, large persistent positivity

to last word in sentence One disconcertingly different result

Strong early negativity to focused words; 100-200 ms after start of word

Active Sentence Data

Active, focus vs nonfocus

Active, appropriate vs inappropriate

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volt

age,

uv

-4-3-2-101234

Appropriate vs Inappropriate (Early vs Late) Expt 2, Posterior 9, Actives

E-App

E-Inapp

L-App

L-Inapp

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volta

ge, u

v

-4

-2

0

2

4

Focus vs NonFocus (Early vs Late) Expt 2, Anterior 6, Actives

E-NonFoc

E-Foc

L-NonFoc

L-Foc

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volta

ge, u

v

-4-3-2-101234

Focus vs Nonfocus (Early vs Late) Expt 2, Posterior 9, Actives

E-NonFoc

E-Foc

L-NonFoc

L-Foc

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volta

ge, u

v

-4

-2

0

2

4

Appropriate vs Inappropriate (Early vs Late) Expt 2, Anterior 6, Actives

E-App

E-Inapp

L-App

L-Inapp

Early negativity to focused words

Perhaps related to nonparallel question-answer structure Who else was Jeremy kissed by? Rhonda

kissed Jeremy. Who else was kissed by Rhonda? Rhonda

kissed Jeremy. But shows up early as well as late

And answer structure is not evident early

Conclusions

Remarkably persistent late positivity (widespread, bilateral) to semantically focused words in answer to question

Rather little ERP response to prosody Early negativity to missing pitch accent Responsiveness to prosody does not increase

when prosody is made (somewhat) more informative (by mixing actives and passives)

Conclusions (cont.)

Large positive shift to last word in sentence Not the same kind of suggested “integration”

signaled by positivity to focus; different scalp distribution

Puzzled by early negativity to focused words in active sentences Would appreciate suggestions for how to

interpret…

REFERENCES

Birch, S., & Clifton, C., Jr. (1995). Focus, accent, and argument structure. Language and speech, 33, 365-391.

Birch, S., & Clifton, C., Jr. (2002). Effects of varying focus and accenting of adjuncts on the comprehension of utterances. Journal of Memory and Language, 47, 571-588.

Bock, K., & Mazella, J. R. (1983). Intonational marking of given and new information: Some consequences for comprehension. Memory & Cognition, 11, 64-76.

Hruska, C., Alter, K., Steinhauer, K., & Steube, A. (2001, June, 2001). Misleading dialogues: Human brain's reaction to prosodic information. Paper presented at the Oralite et Gestualite, Aix en Provence, France.

Kaan, E., Harris, A., Gibson, E., & Holcomb, P. (2000). The P600 as an index of syntactic integration difficulty. Language and Cognitive Processes, 15, 159-201.

Selkirk, E. (1984). Phonology and syntax: The relation between sound and structure. Cambridge, MA: MIT Press.

Steinhauer, K., Altern, K., & Friederici, A. D. (1999). Brain potentials indicate immediate use of prosodic cues in natural speech. Nature Neuroscience, 2, 191-196.

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volt

age,

uv

-4

-3

-2

-1

01

2

3

4

Appropriate vs Inappropriate (Early vs Late) Expt 2, Posterior 9, Passives

E-App

E-Inapp

L-App

L-Inapp

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volt

age,

uv

-4-3-2-101234

Focus vs NonFocus (Early vs Late) Expt 2, Posterior 9, Passive

E-NonFoc

E-Foc

L-NonFoc

L-Foc

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volta

ge, u

v

-4-3-2-101234

Appropriate vs Inappropriate (Early vs Late) Expt 2, Anterior 6, Passives

E-App

E-Inapp

L-App

L-Inapp

Time from Word Onset (ms)-200 0 200 400 600 800 1000

volta

ge, u

v

-4-3-2-101234

Focus vs NonFocus (Early vs. Late) Expt 2, Anterior 6, Passives

E-NonFoc

E-Foc

L-NonFoc

L-Foc