The Horse Raced Past: Gardenpath Processing in Dynamical...

44
The Horse Raced Past: Gardenpath Processing in Dynamical Systems 28.09.2011 Ninth International Tbilisi Symposium on Language, Logic and Computation Kutaisi, Georgia Peter beim Graben Dept. of German Language and Linguistics Humboldt-Universität zu Berlin [email protected]

Transcript of The Horse Raced Past: Gardenpath Processing in Dynamical...

Page 1: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The Horse Raced Past: Gardenpath Processing in

Dynamical Systems 28.09.2011

Ninth International Tbilisi Symposium on Language, Logic and Computation

Kutaisi, Georgia

Peter beim Graben Dept. of German Language and Linguistics

Humboldt-Universität zu Berlin [email protected]

Page 2: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

+

Page 3: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

the

Page 4: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

horse

Page 5: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

raced

Page 6: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

past

Page 7: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

the

Page 8: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

barn

Page 9: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

fell.

Page 10: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Gardenpath Sentences

Bever (1970)

Page 11: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Gardenpath Theory

Frazier (1987)

late closure

Page 12: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Experimental Findings

Self-paced reading: slowing down

Eye-tracking: regressions, longer fixations

Event-related brain potentials: P600

beim Graben et al. (2008)

Page 13: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Hale (2003)

Reduced Relative

Page 14: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Simplified

Page 15: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Context-Free Grammars

Page 16: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 17: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 18: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 19: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 20: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 21: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 22: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 23: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

Page 24: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

The horse raced past the barn fell.

Generalized Left-Corner Parser

symptom

Fodor & Inoue (1994)

Page 25: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Diagnosis

Fodor & Inoue (1994)

The horse raced past the barn fell.

The horse raced past. The barn fell. ? Lexicon: raced = (1) finite verb, past tense raced = (2) non-finite verb, participle past perfect

Page 26: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Repair

Fodor & Inoue (1994) Lewis (1998)

The horse raced past the barn fell.

Page 27: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Repair

Fodor & Inoue (1994) Lewis (1998)

The horse raced past the barn fell.

Page 28: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Repair

Fodor & Inoue (1994) Lewis (1998)

The horse raced past the barn fell.

Page 29: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Reanalysis

The horse raced past the barn fell.

Page 30: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Reanalysis

The horse raced past the barn fell.

Page 31: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Search Space 1

2 3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

Hale (2011)

A* search: arrive rapidly!

default path

repair

decision point

6 7 weighted directed graph

Page 32: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Geometric Representation

embedding into Euclidian space

1

2 3

4

10

11

6 7

Page 33: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Weights

in Euclidian geometry, arc weights could become 1. distances 2. transition times

only for constant velocity, 1. and 2. are equivalent

Page 34: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Dynamics

fixed point attractor saddle node

identify parser states with equilibrium points in geometric space

stable directions

unstable directions

Page 35: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Example logistic map

fixed point attractor

control parameter r

Page 36: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Sequential Dynamics 2

3

4 connecting stable and unstable directions of successive saddle nodes

stable heteroclinic sequence (SHS) Afraimovich et al. (2004)

Rabinovich et al. (2008)

Page 37: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Sequential Decision Problems

1

2 3

10 6

7

saddle node with more than one unstable direction (d > 2)

stable direction

unstable directions

arrow lengths = escape velocities = weights

A* search = high risk decision for large velocities

Rabinovich et al. (2006)

Page 38: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Gardenpath States

unwanted attractor

Page 39: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Bifurcations

fixed point attractor unstable fixed point

increasing

Page 40: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Repair by Bifurcation

unwanted attractor unwanted attractor

beim Graben et al. (2004)

Page 41: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Search Space 1

2 3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

Hale (2011)

default path

repair

decision point

6 7

Page 42: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Dynamic Field Parser

The horse raced past the barn fell.

Page 43: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Conclusion Syntactic parsing represented by sequential dynamics in geometric space

Intermediate representational states as saddle nodes connected through stable heteroclinic sequences

Gardenpath states as unwanted attractors

A* search as high risk sequential decision strategy

Syntactic repair as bifurcation of SHS

Page 44: The Horse Raced Past: Gardenpath Processing in Dynamical ...events.illc.uva.nl/Tbilisi/Tbilisi2011/uploaded... · The Horse Raced Past: Gardenpath Processing in Dynamical Systems

Thank you for your attention!

Funding: DFG Heisenberg-Stipendium GR 3711/1-1

Acknowledgements