Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient...
description
Transcript of Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient...
![Page 1: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/1.jpg)
Back to the future: Where we’re going, we don’t need phonemes.
Implications of a gradient lexicon.
Bob McMurrayUniversity of Iowa
Dept. of Psychology
![Page 2: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/2.jpg)
Collaborators
Richard AslinMichael TanenhausDavid Gow
Joe ToscanoCheyenne MunsonMeghan ClayardsDana Subik
The students of the MACLab
![Page 3: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/3.jpg)
In language, information arrives sequentially.
• Partial syntactic and semantic representations are formed as words arrive.
The cowboy chased by the
• Words are identified over sequential phonemes.
Linguistics department yodeled
![Page 4: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/4.jpg)
Spoken Word Recognition is an ideal arena in which to study these issues because:
• Speech production gives us a lot of rich temporal information to use in this way.
• We have a clear understanding of the input (from phonetics).
• The output is easy to measure online
![Page 5: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/5.jpg)
Online Comprehension
• Listeners form hypotheses as the input unfolds.
• Need measurements of how listeners interpret speech, moment-by-moment.
• May reveal how information is integrated:
- Discreteness vs. Gradiency- Combinatorial Units
![Page 6: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/6.jpg)
Mechanisms of Temporal Integration
Stimuli do not change arbitrarily.
Perceptual cues reveal something about the change itself.
Active integration:• Anticipating future events• Retain current partial representations.• Resolve prior ambiguity.
![Page 7: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/7.jpg)
Overview
1) Speech perception and Spoken Word Recognition.
time
Input: b... u… tt… e… r
beach
bumpputter
dog
butter
time
Input: b... u… tt… e… r
beach
bumpputter
dog
butter
2) Lexical activation is sensitive to fine-grained detail in speech.
4) Back in time: staying off the garden-path.
3) Where we’re going, we don’t need phonemes: evidence for continuous information integration.
Lexicon
Cue 1 Cue 2 time
Lexicon
Cue 1 Cue 2 timeCue 1 Cue 2 time
5) Forward to the future: coping with (and benefiting from) with phonological modification.
![Page 8: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/8.jpg)
bakeryba…
basic
barrier
barricade bait
baby
Xkery
bakery
X
XXX
Online Word Recognition
• Information arrives sequentially• At early points in time, signal is temporarily ambiguous.
• Later arriving information disambiguates the word.
![Page 9: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/9.jpg)
Current models of spoken word recognition
• Immediacy: Hypotheses formed from the earliest moments of input.
• Activation Based: Lexical candidates (words) receive activation to the degree they match the input.
• Parallel Processing: Multiple items are active in parallel.
• Competition: Items compete with each other for recognition.
![Page 10: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/10.jpg)
time
Input: b... u… tt… e… r
beach
bump putter
dog
butter
![Page 11: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/11.jpg)
These processes have been well defined for a phonemic representation of the input.
But considerably less ambiguity if we consider subphonemic information.
Example: subphonemic effects of motor processes.
![Page 12: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/12.jpg)
Coarticulation
Sensitivity to these perceptual details might yield earlier disambiguation.
Example: CoarticulationArticulation (lips, tongue…) reflects current, future and past events.
Subtle subphonemic variation in speech reflects temporal organization.
n ne et c
k
Any action reflects future actions as it unfolds.
![Page 13: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/13.jpg)
These processes have largely been ignored because of a history of evidence that perceptual variability gets discarded.
Example: Categorical Perception
![Page 14: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/14.jpg)
Categorical Perception
B
P
Subphonemic variation in VOT is discarded in favor of a discrete symbol (phoneme).
• Sharp identification of tokens on a continuum.
VOT
0
100
PB
% /p
/
ID (%/pa/)0
100Discrim
ination
Discrimination
• Discrimination poor within a phonetic category.
![Page 15: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/15.jpg)
Categorical Perception (CP)
Defined fundamental computational problems.
CP is output of • Speech perception
Input to • Phonology• Word recognition.
Phon
olog
y
Words
Phonemes
Sound
Sense
![Page 16: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/16.jpg)
Categorical Perception (CP)
Enables a divide-and-conquer approach.
Phon
olog
y
Words
Phonemes
Sound
But, assumes that
1) Speech tasks tap phonemes (or something like them)
2) Phonemes (or something like them) are legitimate processing units.
Sense
![Page 17: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/17.jpg)
Evidence against the strong form of Categorical Perception from psychophysical-type tasks:
Discrimination Tasks Pisoni and Tash (1974) Pisoni & Lazarus (1974)Carney, Widin & Viemeister (1977)
Training Samuel (1977)Pisoni, Aslin, Perey & Hennessy (1982)
Goodness Ratings Miller (1997)Massaro & Cohen (1983)
Classic explanation:
Auditory tasks: non-categoricalPhonological tasks: categorical
Paradigmatic CP: within-category variation is noise.Not important to higher language.
![Page 18: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/18.jpg)
Categorical Perception (CP)
“Phonological/auditory” task distinction
• validates standard paradigm?
or
• assume standard paradigm?Ph
onol
ogy
Words
Phonemes
Sound
Sense
Cont. cues
(non-CP)
![Page 19: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/19.jpg)
Minimal computational problem:
Computing meaning.
Phon
olog
y
Words
Phonemes
Sound
![Page 20: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/20.jpg)
Minimal computational problem:
Computing meaning.
Phon
olog
y
Words
Phonemes
Sound
CP tasks don’t necessarily tap a stage of this problem. ?CP
![Page 21: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/21.jpg)
Minimal computational problem:
Computing meaning.
Phon
olog
y
Words
Phonemes
Sound
CP tasks don’t necessarily tap a stage of this problem.
Lexical representation: clearly a component.
Goal: Reassess continuous sensitivity (non-CP) w.r.t. words
![Page 22: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/22.jpg)
?Does within-category acoustic detail
systematically affect higher level language?
Is there a gradient effect of continusuous acoustic detail on lexical activation?
Experiment 1
![Page 23: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/23.jpg)
A gradient relationship would yield systematic effects of subphonemic information on lexical activation.
If this gradiency is useful for temporal integration, it must be preserved over time.
Need a design sensitive to both acoustic detail and detailed temporal dynamics of lexical activation.
Experiment 1
McMurray, Aslin & Tanenhaus (2002)
![Page 24: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/24.jpg)
Use a speech continuum—more steps yields a better picture acoustic mapping.
KlattWorks: generate synthetic continua from natural speech.
Acoustic Detail
9-step VOT continua (0-40 ms)
6 pairs of words.beach/peach bale/pale bear/pearbump/pump bomb/palm butter/putter
6 fillers.lamp leg lock ladder lip leafshark shell shoe ship sheep shirt
![Page 25: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/25.jpg)
![Page 26: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/26.jpg)
How do we tap on-line recognition?With an on-line task: Eye-movements
Subjects hear spoken language and manipulate objects in a visual world.
Visual world includes set of objects with interesting linguistic properties.
a beach, a peach and some unrelated items.
Eye-movements to each object are monitored throughout the task.
Temporal Dynamics
Tanenhaus, Spivey-Knowlton, Eberhart & Sedivy, 1995
![Page 27: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/27.jpg)
• Relatively natural task.
• Eye-movements generated very fast (within 200ms of first bit of information).
• Eye movements time-locked to speech.
• Subjects aren’t aware of eye-movements.
• Fixation probability maps onto lexical activation..
Why use eye-movements and visual world paradigm?
![Page 28: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/28.jpg)
A moment to view the items
Task
![Page 29: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/29.jpg)
![Page 30: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/30.jpg)
Task
Bear
Repeat 1080 times
![Page 31: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/31.jpg)
By subject: 17.25 +/- 1.33ms By item: 17.24 +/- 1.24ms
High agreement across subjects and items for category boundary.
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
prop
ortio
n /p
/
B P
Identification Results
![Page 32: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/32.jpg)
Task
Target = Bear
Competitor = Pear
Unrelated = Lamp, Ship
200 ms
1
2
3
4
5
Trials
Time
% fi
xatio
ns
![Page 33: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/33.jpg)
Task
00.10.20.30.40.50.60.70.80.9
0 400 800 1200 1600 0 400 800 1200 1600 2000
Time (ms)
More looks to competitor than unrelated items.
VOT=0 Response= VOT=40 Response=Fi
xatio
n p
ropo
rtio
n
![Page 34: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/34.jpg)
Task
Given that • the subject heard bear• clicked on “bear”…
How often was the subject looking at the “pear”?
Categorical Results Gradient Effect
target
competitortime
Fixa
tion
prop
ortio
n target
competitor competitorcompetitortime
Fixa
tion
prop
ortio
n target
![Page 35: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/35.jpg)
Results
0 400 800 1200 16000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 ms5 ms10 ms15 ms
VOT
0 400 800 1200 1600 2000
20 ms25 ms30 ms35 ms40 ms
VOT
Com
petit
or F
ixat
ions
Time since word onset (ms)
Response= Response=
Long-lasting gradient effect: seen throughout the timecourse of processing.
![Page 36: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/36.jpg)
0 5 10 15 20 25 30 35 400.02
0.03
0.04
0.05
0.06
0.07
0.08
VOT (ms)
CategoryBoundary
Response= Response=
Looks to
Looks to C
ompe
titor
Fix
atio
ns
B: p=.017* P: p<.001***Clear effects of VOTLinear Trend B: p=.023* P: p=.002***
Area under the curve:
![Page 37: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/37.jpg)
0 5 10 15 20 25 30 35 400.02
0.03
0.04
0.05
0.06
0.07
0.08
VOT (ms)
Response= Response=
Looks to
Looks to
B: p=.014* P: p=.001***Clear effects of VOTLinear Trend B: p=.009** P: p=.007**
Unambiguous Stimuli Only
CategoryBoundaryC
ompe
titor
Fix
atio
ns
![Page 38: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/38.jpg)
Summary
Subphonemic acoustic differences in VOT have gradient effect on lexical activation.
• Gradient effect of VOT on looks to the competitor.
• Seems to be long-lasting.• Effect holds even for unambiguous stimuli.
Consistent with growing body of work using priming (Andruski, Blumstein & Burton, 1994; Utman, Blumstein & Burton, 2000; Gow, 2001, 2002).
![Page 39: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/39.jpg)
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
2) Continuous acoustic detail is represented as gradations in activation across the lexicon.
4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration.
An alternative framework
3) This can do the work of sublexical units like phonemes.
![Page 40: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/40.jpg)
Lexical Sensitivity
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
Voicing Laterality, Manner, Place Natural Speech Vowel Quality Infant voicing categories
![Page 41: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/41.jpg)
Voicing Laterality, Manner, Place Natural Speech Vowel Quality Infant voicing categories
Metalinguistic Tasks
B
ShL
P
Extensions
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
![Page 42: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/42.jpg)
Voicing Laterality, Manner, Place Natural Speech Vowel Quality Infant voicing categories
Metalinguistic Tasks
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
0
0.02
0.04
0.06
0.08
0.1
Response=BLooks to B
Response=PLooks to B
Com
petit
or F
ixat
ions
Extensions
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
![Page 43: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/43.jpg)
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
0
0.02
0.04
0.06
0.08
0.1
Response=BLooks to B
Response=PLooks to B
Com
petit
or F
ixat
ions
Voicing Laterality, Manner, Place Natural Speech Vowel Quality Infant voicing categories
Metalinguistic Tasks
Extensions
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
![Page 44: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/44.jpg)
Lexical Sensitivity
1) Word recognition is systematically sensitive to subphonemic acoustic detail.
Voicing Laterality, Manner, Place Natural Speech Vowel Quality Infant voicing categories
Metalinguistic Tasks
? Non minimal pairs? Duration of effect
(Exp 3-4)
![Page 45: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/45.jpg)
time
Input: b... u… m… p…
bun
bumper
pump
dump
bump
bomb
2) Continuous acoustic detail is represented as gradations in activation across the lexicon.
![Page 46: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/46.jpg)
If lexical processes can represent speech detail, do we need sublexical processes?
Perhaps:How are multiple cues (to the same phoneme) integrated? (Exp 2)
3) This can do the work of sublexical units like phonemes.
![Page 47: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/47.jpg)
Regressive ambiguity resolution (exp 3-5):• Ambiguity retained until more information arrives.
Progressive expectation building (exp 5-6):• Phonetic distinctions are spread over time• Anticipate upcoming material.
Temporal Integration
4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration.
![Page 48: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/48.jpg)
Overview
1) Speech perception and Spoken Word Recognition.
2) Lexical activation is sensitive to fine-grained detail in speech.
4) Back in time: staying off the garden-path.
3) Where we’re going, we don’t need phonemes: evidence for continuous information integration.
Lexicon
Cue 1 Cue 2 time
Lexicon
Cue 1 Cue 2 timeCue 1 Cue 2 time
5) Forward in time: coping with (and benefiting from) with phonological modification.
![Page 49: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/49.jpg)
Traditional speech chain: signal-> phonemes -> words
phonemes
words
Substitute your favorite sublexical unit here (syllables, diphones, etc)…
ə
![Page 50: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/50.jpg)
Measuring phonemes
What do phonemes do?
phonemes
words
phonemes
words
1) Categorize continuous acoustic detail?
2) Integrate multiple cues?
3) Generalize phonological information during development?
4) Learning new words?
5) Speech production?
6) Reading?
![Page 51: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/51.jpg)
Measuring phonemes
phonemes
words
phonemes
words
1) Categorize continuous acoustic detail? (categorizer)
2) Integrate multiple cues? (buffer)
What do you do with phonemes?
We have: an extremely sensitive measure of:- lexical activation
- temporal dynamicsCan we use it to assess sublexical processes?
- Categorization - Integration
Occam’s razor: if this stuff doesn’t happen until the information meets the lexicon then phonemes are
- not adding anything computationally.- not theoretically necessary
![Page 52: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/52.jpg)
Measuring phonemes
No.
Computational Necessity of the phoneme.
Logic:Take the phoneme seriously.What does it do (computationally) (in a specific task)?Does that actually get done (during comprehension)?
The psychological reality of the phoneme?
May still “exist”…May still have computational necessity for other tasks…
![Page 53: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/53.jpg)
1) Categorizing continuous detail
1) Categorize continuous acoustic detail. (phoneme as “categorizer”
Categorical perception: continuous detail discarded.- only in metalinguistic tasks (what are these
tapping?)
Gradient lexical activation: continuous detail systematically affects lexical activation.
Phonemesə
No phonemes required
![Page 54: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/54.jpg)
2) Integrating multiple cues
2) Integrating multiple cues. (phoneme as “buffer”)
Phonetic cues for a given phoneme are spread out over time.
• Combined at phoneme level prior to accessing lexicon?
or
• Direct access to lexicon. Lexical integration?
![Page 55: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/55.jpg)
Integrating multiple cues
Lexicon
time
Phoneme
Cue 1 Cue 2 time
Cue 2Cue 1Phoneme Story
• Integration before lexical access
• “Buffer”
The Alternative
• Integration at the Lexicon
![Page 56: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/56.jpg)
The Logic
The Logic:
1) Assess Temporal Integration.
- Use two asynchronous cues to single phoneme.- Assess lexical activation over time.
Simultaneous effects: phonemic integration.Asynchronous effects: lexical integration.
![Page 57: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/57.jpg)
The Logic
The Logic:
1) Assess Temporal Integration.
- Use two asynchronous cues to single phoneme.- Assess lexical activation over time.
Simultaneous effects: phonemic integration.Asynchronous effects: lexical integration.Which cues?
![Page 58: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/58.jpg)
VOT Vowel Length
Asynchronous cues
Both covary with speaking rate: rate normalization
Asynchronous cues to voicing: VOT Vowel Length
![Page 59: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/59.jpg)
VOT Vowel LengthVOT Vowel Length
Phonetic Context
Asynchronous cues to voicing: VOT Vowel Length
Both covary with speaking rate: rate normalization
![Page 60: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/60.jpg)
Manner of Articulation Formant Transition Slope (FTSlope): Temporal cue like VOT: covaries with vowel length.
belt
welt
![Page 61: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/61.jpg)
9-step VOT continua (0-40 ms) beach/peachbeak/peakbees/peas
9-step formant transition slopebench/wenchbelt/weltbell/well
2 Vowel Lengths x
The usual task
1080 Trials
![Page 62: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/62.jpg)
Experiment 2
Results
Step 1: Assess gradiency
Step 2:Assess temporal integration
![Page 63: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/63.jpg)
9-step b/w continuaVOT varied.
bench/wenchbelt/weltbell/well
Manner Continua
00.10.20.30.40.50.60.70.80.91
1 2 3 4 5 6 7 8 9
Step
% W
LongShort
N=36
![Page 64: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/64.jpg)
Manner Continua
Experiment 1
Looks to competitor
Clicked on Clicked on
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Look
s to
B
12345
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Look
s to
W
-5-4-3-2-1
All ConsB .0001 .003W .0001 .01
![Page 65: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/65.jpg)
Gradiency Results
Exp 2
All ConsManner B
W
Voicing BP
![Page 66: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/66.jpg)
Gradiency Results
All ConsManner B
W
Voicing B P
![Page 67: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/67.jpg)
Gradiency Results
All Cons All ConsManner B
W
Voicing B P
Replication
• Replication of gradiency with Manner + Voicing.
• Phoneme not required as “categorizer” (categorization isn’t happening)
![Page 68: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/68.jpg)
Integrating multiple cues
Lexicon
time
Phoneme
Cue 1 Cue 2 time
Cue 2Cue 1Phoneme Story
• Integration before the Lexicon
• “Buffer”
The Alternative
• Integration at the Lexicon
![Page 69: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/69.jpg)
?Results: Temporal Dynamics
When do effects on lexical activation occur?
VOT / FTStep effects cooccurs with vowel length.(Phonemic Integration)
VOT / FTStep precedes vowel length.(Lexical integration)
![Page 70: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/70.jpg)
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
-30 -25 -20 -15 -10 -5 0
Distance from Boundary (VOT)
Com
petit
or F
ixat
ions
Y = M320x + B M320 = 0
• VOT / FTStep: Regression slope of competitor fixations as a function of VOT.
Compute 2 effect sizes at each 20 ms time slice.
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Loo
ks to
P
-5-4-3-2-1
t
Time = 320 ms…
![Page 71: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/71.jpg)
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
-30 -25 -20 -15 -10 -5 0
Distance from Boundary (VOT)
Com
petit
or F
ixat
ions
Y = M720x + B
• VOT / FTStep: Regression slope of competitor fixations as a function of VOT.
Compute 2 effect sizes at each 20 ms time slice.
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Loo
ks to
P
-5-4-3-2-1
t
Time = 720 ms…
![Page 72: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/72.jpg)
VOT = 30
0
0.05
0.1
0.15
0.2
0.25
0 500 1000 1500 2000 2500
Time (ms)
Com
petit
or F
ixat
ions
LongShort
Compute 2 effect sizes at each 20 ms time slice.
• Vowel Length: Difference (D) between fixations after hearing long vs. short vowel.
• Repeat for each time slice, subject.
L-S = VL
![Page 73: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/73.jpg)
Resulting dataset…
Subject Time VOT (M) Vowel (D)1 20 -0.0023 0.0094
40 -0.0016 0.009560 -0.0008 0.0108…2000 0.06021 0.123
2 20 0.0014 0.009140 0.0018 0.008860 0.0029 0.0104…2000 0.0604 0.1223
…
Compute average effect size at each time slice.
When does it (statistically) depart from 0?
![Page 74: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/74.jpg)
Voiced Sounds OnlyVOT: 660 ms
Vowel: 820 ms
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Effe
ct S
ize
VOTVowel
t
Voicing Continua: Temporal Dynamics
![Page 75: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/75.jpg)
Voiceless Sounds Only
Voicing Continua: Temporal Dynamics
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 500 1000 1500 2000
Time (ms)
Effe
ct S
ize
VOTVowel
VOT: 640 msVowel: 780 ms
![Page 76: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/76.jpg)
Combined
Voicing Continua: Temporal Dynamics
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Effe
ct S
ize
VOTVowel
VOT: 560 msVowel: 800 ms
![Page 77: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/77.jpg)
Integration Results
VOT VowelVoicing B 660 820
P 640 780 All 560 800
Manner BWAll
![Page 78: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/78.jpg)
Stops only
Manner Continua: Temporal Dynamics
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Effe
ct S
ize
FTStepVowel
FTStep: 840 msVowel: 1340 ms
![Page 79: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/79.jpg)
Approximants only
Manner Continua: Temporal Dynamics
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Effe
ct S
ize
FTStepVowel
FTStep: 280 msVowel: 860 ms
![Page 80: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/80.jpg)
Combined
Manner Continua: Temporal Dynamics
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Effe
ct S
ize
FTStepVowel
FTStep: 620 msVowel: 880 ms
![Page 81: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/81.jpg)
Integration Results
VOT/FT VowelVoicing B 660 820
P 640 780 All 560 800
Manner B 840 1340 W 280 860 All 620 820
![Page 82: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/82.jpg)
Replication Underway
VOT/FT VowelVoicing B 660 820
P 640 780 All 560 800
Voicing II All 560 800 B 560 800 P 660 860
Manner B 840 1340 W 280 860 All 620 820
Manner II All 620 740 B 440 680 W 700 880
![Page 83: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/83.jpg)
Exp 2: Summary
Phonemes not computationally necessary for:• Phonetic categorization.• Asynchronous cue integration.
words
ə
Still may play a role in • Development• Reading• Production…
Between signal and words there doesn’t appear to be any
• Complex computation• Nonlinearities
What role do intermediate units play?
![Page 84: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/84.jpg)
Exp 2: Summary
Phonemes not computationally necessary for:• Phonetic categorization.• Asynchronous cue integration.
words
ə
What is doing the work?
Lexical activation processes.
![Page 85: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/85.jpg)
Overview
1) Speech perception and Spoken Word Recognition.
2) Lexical activation is sensitive to fine-grained detail in speech.
4) Back in time: staying off the garden-path.
3) Where we’re going, we don’t need phonemes: evidence for continuous information integration.
5) Forward in time: coping with (and benefiting from) with phonological modification.
![Page 86: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/86.jpg)
Competitor still active - easy to activate it rest of the way.
Competitor completely inactive- system will “garden-path”.
P ( misperception ) distance from boundary.
Gradient activation allows the system to hedge its bets.
What if initial portion of a stimulus was misperceived?
Misperception
![Page 87: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/87.jpg)
time
Input: p/b eI r ə k i t…
parakeetbarricade
Categorical Lexicon
barricade vs. parakeet
parakeet
barricade
Gradient Sensitivity
/ beIrəkeId / vs. / peIrəkit /
![Page 88: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/88.jpg)
Experiment 3
Can gradient maintenance of lexical alternatives prevent system from wandering down “garden-path”?
By avoiding commitment to a discrete phoneme, can the system elegantlyrecover from earlymisperception?
Misperception
come on, gz.lt’s go!
əə
![Page 89: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/89.jpg)
10 Pairs of b/p items.
Voiced Voiceless OverlapBumpercar Pumpernickel 6Barricade Parakeet 5Bassinet Passenger 5Blanket Plankton 5Beachball Peachpit 4Billboard Pillbox 4Drain Pipes Train Tracks 4Dreadlocks Treadmill 4Delaware Telephone 4Delicatessen Television 4
Experiment 3 Methods
![Page 90: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/90.jpg)
X
![Page 91: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/91.jpg)
05101520253035
0
0.2
0.4
0.6
0.8
1
300 600 900
Time (ms)
Fixa
tions
to T
arge
t
VOT
Barricade -> Parricade
Experiment 3: Results
Faster activation of target as VOTs near lexical endpoint.
—Even within the non-word range.
![Page 92: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/92.jpg)
05101520253035
0
0.2
0.4
0.6
0.8
1
300 600 900
Time (ms)
Fixa
tions
to T
arge
t
VOT
Barricade -> Parricade Parakeet -> Barakeet
300 600 900 1200
Time (ms)
Faster activation of target as VOTs near lexical endpoint.
—Even within the non-word range.
Experiment 3: Results
![Page 93: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/93.jpg)
Target: p=.001VOT: p = .0001T x V: p=.002
Voiceless: No sudden, categorical shift—just smooth gradiency.
0.4
0.45
0.5
0.55
0.6
0.65
0 5 10 15 20 25 30 35
Distance from Prototype (VOT, ms)
Fixa
tions
to T
arge
t
B
P
![Page 94: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/94.jpg)
Target: p=.001VOT: p = .0001T x V: p=.002
Voiced:Categorical???
0.4
0.45
0.5
0.55
0.6
0.65
0 5 10 15 20 25 30 35
Distance from Prototype (VOT, ms)
Fixa
tions
to T
arge
t
B
P
0.520.530.540.550.560.570.580.590.6
0.610.62
0 5 10 15 20 25 30 35
Distance from Prototype (VOT, ms)
Fixa
tions
to T
arge
t
B
![Page 95: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/95.jpg)
Experiment 3: Garden path
XXXb/parri…
Garden-path analysis:
Identify trials in which
Time FixationPre POD Competitor
![Page 96: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/96.jpg)
XXXb/parricade
Garden-path analysis:
Identify trials in which
Time FixationPre POD CompetitorPost POD Target
Is the latency to switch to the target related to VOT?• Accelerated latency = more (residual) target activation
![Page 97: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/97.jpg)
XXXb/parricade
Is the latency to switch to the target related to VOT?
• Accelerated latency = more (residual) target activation
Sparse databut:
100
120
140
160
180
200
220
240
260
0 5 10 15 20 25 30
Distance from Prototype (VOT, ms)
Tim
e to
Tar
get
Voiced
Voiceless
B: p=.002P: p=.007
![Page 98: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/98.jpg)
XXXb/parricade
Is the latency to switch to the target related to VOT?
• Accelerated latency = more (residual) target activation
VOT: p=.0001Targ: p=.001V x T: p>.1
120
140
160
180
200
220
240
260
280
0 5 10 15 20 25 30 35
Distance from Prototype (VOT, ms)
Tim
e to
Tar
get
Voiced
Voiceless
More data…
![Page 99: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/99.jpg)
Experiment 3: Summary
?
Same gradiency seen in McMurray et al (2002)
Facilitates ambiguity resolution (time-to-target)• 240 ms after VOT
Gradiency lasts…… and is useful
?????Idiosyncracies:
• Attenuated gradiency for B• Interactions with target (voiced/voiceless).
Effect of reduced VOT range?
![Page 100: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/100.jpg)
Barricade -> Parricade
00.10.20.30.40.50.60.70.80.9
300 500 700 900 1100 1300Time (ms)
Fixa
tions
to T
arge
t
0.4
0.45
0.5
0.55
0.6
0.65
0 10 20 30 40
Distance from Prototype (VOT, ms)
Fixa
tions
to T
arge
t
VoicedVoiceless
Target fixations.
Gradient VOT: - both targetsReduced target effectNo interaction
p=.0001
Replication: longer continua (0-45 ms).
![Page 101: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/101.jpg)
Experiment 4
Exp 4Does the presence of the visual competitor (parakeet)
artificially heighten competitor activation (and cause the effect)?
XXX0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
300 500 700 900 1100 1300
Time (ms)
Fixa
tions
to T
arge
t
0
5
10
15
20
25
30
35
40
450
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
300 500 700 900 1100 1300
Time (ms)
Fixa
tions
to T
arge
t
0
5
10
15
20
25
30
35
40
45
![Page 102: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/102.jpg)
XXXb/parricade
Is the latency to switch to the target related to VOT?
• Accelerated latency = more (residual) target activation
p=.0001
Experiment 4: Garden Path
100
150
200
250
0 10 20 30 40
Distance from Prototype (VOT, ms)
Tim
e to
Tar
get
VoicedVoiceless
No Competitor
![Page 103: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/103.jpg)
Gradiency:Gradient effect of within-category variation without minimal-pairs.
Regressive Ambiguity Resolution
Gradient effect long-lasting: mean POD = 240 ms.
![Page 104: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/104.jpg)
Gradiency:Gradient effect of within-category variation without minimal-pairs.
Regressive Ambiguity Resolution
Regressive ambiguity resolution:• Subphonemic gradations maintained until more
information arrives.• Improves (or hinder) recovery from garden path.• Gradient lexical sensitivity prevents over-
committing to a garden-path interpretation.• Likely lexical locus: echoic memory decays
around 400 ms.
Gradient effect long-lasting: mean POD = 240 ms.
![Page 105: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/105.jpg)
Overview
1) Speech perception and Spoken Word Recognition.
2) Lexical activation is sensitive to fine-grained detail in speech.
4) Back in time: staying off the garden-path.
3) Where we’re going, we don’t need phonemes: evidence for continuous information integration.
5) Forward in time: coping with (and benefiting from) with phonological modification.
![Page 106: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/106.jpg)
Progressive Expectation Formation
Can within-category detail be used to predict future acoustic/phonetic events?
Yes: Phonological regularities create systematic within-category variation.
• Predicts future events.
![Page 107: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/107.jpg)
time
Input: m… a… rr… oo… ng… g… oo… s…
maroongoose
goatduck
Word-final coronal consonants (n, t, d) assimilate the place of the following segment.
Place assimilation -> ambiguous segments —anticipate upcoming material.
Experiment 5: Anticipation
Maroong Goose Maroon Duck
![Page 108: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/108.jpg)
Assimilation is subphonemic, continuous, not discrete.
Experiment 5: Anticipation
F2 Transitions in /æC/ Contexts
Pitch Period155016001650
1700175018001850
Freq
uenc
y (H
z)
F3 Transitions in /æC/ Contexts
Pitch Period2550
2600
2650
2700
2750
2800
Freq
uenc
y (H
z)coronalassimilatedlabial
![Page 109: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/109.jpg)
Subject hears “select the maroon duck”“select the maroon goose”“select the maroong goose”“select the maroong duck” *
We should see faster eye-movements to “goose” after assimilated consonants.
![Page 110: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/110.jpg)
Results
Looks to “goose“ as a function of time
00.10.20.30.40.50.60.70.80.9
0 200 400 600Time (ms)
Fixa
tion
Prop
ortio
n
AssimilatedNon Assimilated
Onset of “goose” + oculomotor delay
Anticipatory effect on looks to non-coronal.
![Page 111: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/111.jpg)
Inhibitory effect on looks to coronal (duck, p=.024)
0
0.05
0.1
0.15
0.2
0.25
0.3
0 200 400 600Time (ms)
Fixa
tion
Prop
ortio
n
AssimilatedNon Assimilated
Looks to “duck” as a function of time
Onset of “goose” + oculomotor delay
![Page 112: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/112.jpg)
a quick runm picks you up.
a quick runm takes you down ***
When /p/ is heard, the bilabial feature can be assumed to come from assimilation (not an underlying /m/).
When /t/ is heard, the bilabial feature is likely to be from an underlying /m/.
![Page 113: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/113.jpg)
Subject hears“select the mud drinker”“select the mudg gear” “select the mudg drinker
Critical Pair
![Page 114: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/114.jpg)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 200 400 600 800 1000 1200 1400 1600 1800 2000Time (ms)
Fixa
tion
Prop
ortio
n
Initial Coronal:Mud Gear
Initial Non-Coronal:Mug Gear
Onset of “gear” Avg. offset of “gear” (402 ms)
Mudg Gear is initially ambiguous with a late bias towards “Mud”.
![Page 115: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/115.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Fixa
tion
Prop
ortio
n
Initial Coronal: Mud Drinker
Initial Non-Coronal: Mug Drinker
Onset of “drinker” Avg. offset of “drinker (408 ms)
Mudg Drinker is also ambiguous with a late bias towards “Mug” (the /g/ has to come from somewhere).
![Page 116: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/116.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 200 400 600Time (ms)
Fixa
tion
Prop
ortio
n
AssimilatedNon Assimilated
Onset of “gear”
Looks to non-coronal (gear) following assimilated or non-assimilated consonant.
In the same stimuli/experiment there is also a progressive effect!
![Page 117: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/117.jpg)
Assimilated(ambiguous)
Segment
SubsequentConsonant
predicts
resolves ambiguity
Phonological modification has a benefit and a cost:
• Creates predictive regularities.
• Creates ambiguity.
Gradiency:• Enables anticipation.• Retain both items until ambiguity resolution.
What is the mechanism?
![Page 118: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/118.jpg)
What is the mechanism? Assimilated(ambiguous)
Segment
SubsequentConsonant
predicts
resolves ambiguity
Assimilated(ambiguous)
Segment
SubsequentConsonant
predicts
resolves ambiguity
Lexical activation• Activation/competition processes retain partial
representations.• Lexical processes are predictive by nature.
Prediction:• Lexical competition ought to inhibit these
processes.
![Page 119: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/119.jpg)
Experiment 3: Extensions
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
200 300 400 500 600 700 800
Time (ms)
Loo
ks to
Lab
ial
Assim-LabialsLabials
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
200 300 400 500 600 700 800
Time (ms)
Loo
ks to
Lab
ial
AssimilatedNeutral
Compare progressive effect as a function of competition
Green/m Boat
Eight/Ape BabiesCompetition:100 ms delay in effect.
Reduction inmagnitude.
![Page 120: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/120.jpg)
Sensitivity to subphonemic detail allows the system to simultaneously cope with and harness graded phonological modification.
• Increase priors on likely upcoming events.• Decrease priors on unlikely upcoming events.• Retain ambiguity until resolution occurs.
Lexical processes may play a pivotal role.
![Page 121: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/121.jpg)
Conclusions
Lexical activation is exquisitely sensitiveto within-category detail.
3) Phonological (maroong goose): progressive facilitation, regressive ambiguity resolution.
This sensitivity enables integration over time at multiple levels and time-scales using normal word recognition mechanisms
2) Lexical (e.g. barricade/parakeet): regressive ambiguity resolution.
Lexicon
Cue 1 Cue 2 time
Lexicon
Cue 1 Cue 2 timeCue 1 Cue 2 time
1) Phonetic (e.g. VOT / Rate): No need for “buffer”.
ə
![Page 122: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/122.jpg)
Summary
Words
Phonemes
Sound
SenseThe standard paradigm
• Does not capture data.• Limits our thinking in
terms of how continuous detail might be used.
![Page 123: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/123.jpg)
Summary
Words
Sound
SenseThe standard paradigm
• Does not capture data.• Limits our thinking in
terms of how continuous detail might be used.
The Alternative
• Continuous phonetic cues.• Integrated directly by
normal lexical processes.• Rich temporal integration.• Computationally simple.
![Page 124: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/124.jpg)
Integration over time at multiple levels.1) Phonetic (e.g. VOT / Rate)2) Lexical (e.g. barricade/parakeet) 3) Phonological (maroong goose)
Conclusions
My lab
These are rich, general processes:• Sentential garden-paths.• V-to-V coarticulation and harmony.• Anticipatory R-coloration• Vowel Nasalization
• Prosodic domain (Keating)
• Reduction (Manuel)• Misproduction (Goldrick)• Bilingual categories (Ju)• Allophonic variation
(Samuel, McLennan)• “Parsing” (Gow, Fowler)
• V-to-V coarticulation (Beddor, Fowler, Cole).
• Assimilation (Byrd, Mitterer, Gow)
• Nasalization (Dahan, Fowler)• R-Coloration (Hillenbrand)• Vowel length/embedded words
(Crosswhite, Salverda)
Other folks
![Page 125: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/125.jpg)
Take home message
Spoken language is defined by change.
But the information to cope with it is in the signal—if we look online.
Within-category acoustic variation is signal, not noise.
![Page 126: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/126.jpg)
Back to the future: Where we’re going, we don’t need phonemes.
Implications of a gradient lexicon.
Bob McMurrayUniversity of Iowa
Dept. of Psychology
![Page 127: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/127.jpg)
Head-Tracker Cam Monitor
IR Head-Tracker Emitters
EyetrackerComputer
SubjectComputer
Computers connected via Ethernet
Head
2 Eye cameras
![Page 128: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/128.jpg)
Misperception: Additional Results
![Page 129: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/129.jpg)
10 Pairs of b/p items.• 0 – 35 ms VOT continua.
20 Filler items (lemonade, restaurant, saxophone…)
Option to click “X” (Mispronounced).
26 Subjects
1240 Trials over two days.
![Page 130: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/130.jpg)
0.000.100.200.300.400.500.600.700.800.901.00
0 5 10 15 20 25 30 35
Barricade
Res
pons
e R
ate
VoicedVoicelessNW
Identification Results
Parricade
0.000.100.200.300.400.500.600.700.800.901.00
0 5 10 15 20 25 30 35
VoicedVoicelessNW
Barakeet Parakeet
Res
pons
e R
ate
Significant target responses even at extreme.
Graded effects of VOT on correct response rate.
![Page 131: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/131.jpg)
“Garden-path” effect:Difference between looks to each target (b
vs. p) at same VOT.
VOT = 0 (/b/)
0
0.2
0.4
0.6
0.8
1
0 500 1000
Time (ms)
Fixa
tions
to T
arge
t
BarricadeParakeet
VOT = 35 (/p/)
0 500 1000 1500
Time (ms)
Phonetic “Garden-Path”
![Page 132: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/132.jpg)
-0.1
-0.05
0
0.05
0.1
0.15
0 5 10 15 20 25 30 35
VOT (ms)
Gar
den-
Path
Eff
ect
( Bar
rica
de -
Para
keet
)
-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0 5 10 15 20 25 30 35
VOT (ms)
Gar
den-
Path
Eff
ect
( Bar
rica
de -
Para
keet
)
Target
Competitor
GP Effect:Gradient effect of VOT.
Target: p<.0001Competitor: p<.0001
![Page 133: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/133.jpg)
Assimilation: Additional Results
![Page 134: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/134.jpg)
Within-category detail used in recovering from assimilation: temporal integration.
• Anticipate upcoming material• Bias activations based on context
- Like barricade/parakeet: within-category detail retained to resolve ambiguity..
Phonological variation is a source of information.
Exp 3 & 4: Conclusions
![Page 135: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/135.jpg)
• Similar properties in terms of starting and sparseness.
VOT
Categories• Competitive Hebbian Learning
(Rumelhart & Zipser, 1986).• Not constrained by a particular
equation—can fill space better.
Non-parametric approach?
![Page 136: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/136.jpg)
9-step b/p continua VOT varied.
beach/peachbees/peasbeak/peak
Voicing Continua
00.10.20.30.40.50.60.70.80.91
0 5 10 15 20 25 30 35 40
VOT
% P
LongShort
N=29
![Page 137: Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.](https://reader036.fdocuments.us/reader036/viewer/2022070420/56815db7550346895dcbe69a/html5/thumbnails/137.jpg)
Voicing Continua
Looks to competitor
Clicked on Clicked on
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)Lo
oks t
o B
+1+2+3+4
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Time (ms)
Loo
ks to
P
-5-4-3-2-1
t
All ProtoB .0001 .0001P .008 >.1