The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of...
-
Upload
ashley-golden -
Category
Documents
-
view
212 -
download
0
Transcript of The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of...
![Page 1: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/1.jpg)
The Neural Basis of Speech Perception – a view from
functional imaging
Sophie Scott
Institute of Cognitive Neuroscience,
University College London
![Page 2: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/2.jpg)
This approach to speech perception
• Speech is an auditory signal• It is possible to address the neural processing
of speech within the framework of auditory cortical processing.
• This is not synonymous with the entire language system.
• If one is a skilled speaker of a language, then speech perception is obligatory.
![Page 3: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/3.jpg)
Functional imaging
• Where neural activity occurs, blood is directed.
• Measure neural activity by tracking these changes in local blood flow.
• Thus measuring mass synaptic activity• Poor temporal resolution• Essentially a comparison of blood flow
changes across conditions - so the baseline comparisons are critical
![Page 4: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/4.jpg)
Listening
Wise et al, Lancet, 2001
![Page 5: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/5.jpg)
Neuroanatomy of speechSpeech production
Speech perception
![Page 6: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/6.jpg)
sts
Ins
TS2
TS1
TS3
A1Tpt
paAlt
Pro
caudal
CORE
BELT
PARABELT
rostral
medial
lateralsts dorsal
RTM
RM
CM
R
RT
A1
RTL
AL
ML
MC
RP
CP
Scott and Johnsrude, 2003, from Romanski et al, 1999
![Page 7: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/7.jpg)
AI
R
RT
CL
ML
AL
STGc
CBP
RBP
STGr
Dorsal prearcuate (8a)
Dorsal principal sulcus
(46)
Inferior convexity
(12)
Orbitalpolar
From Kaas and Hackett, 1999
Core Belt ParabeltPrefrontal
cortex
![Page 8: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/8.jpg)
![Page 9: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/9.jpg)
tonotopy
bandwidth
Conspecific vocalisations
Spatial representations
![Page 10: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/10.jpg)
HG PT
CB
PB
Assoc
Assoc
Tpt
Anterior Posterior
Ven
tral
STS
STP
STS
STP
Human
Monkey
![Page 11: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/11.jpg)
medial
Scott and Johnsrude, 2003
anterior
posterior
lateral
AAMALAA1
PASTAALALP
![Page 12: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/12.jpg)
medial
Scott and Johnsrude, 2003
anterior
posterior
lateral
Sounds with harmonic structure against pure tones: Hall, Johnsrude et al., 2002
Frequency modulated tones against unmodulated tones: Hall, Johnsrude et al., 2002
Amplitude modulated noise against unmodulated noise: Giraud et al, 1999
Spectral change against steady state sounds: Thivard et al, 2000
![Page 13: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/13.jpg)
Hierarchical processing
• Structure in sound is computed beyond primary auditory cortex
• More complex structure (e.g. spectral change) processed further from PAC
• How does this relate to speech processing?
![Page 14: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/14.jpg)
speech
noise vocoded speech
rotated speech
rotated noise vocoded speech
![Page 15: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/15.jpg)
Sp VCo RSp RVCo
Sp VCo RSp RVCo Sp VCo RSp RVCo
Sp VCo RSp RVCo
-2
1
00
00
1
11
-2-2
-1
2
-1
-1-1
Anterior
-60 -4 -10 Z = 6.6
-54 +6 -16 Z = 4.7 -62 -12 -12 Z = 5.5
-64 -38 0 Z = 5.7
(Sp + VCo + RSp) - RVCo (Sp + VCo + RSp) - RVCo
(Sp + VCo) - (RSp + RVCo) (Sp + VCo) - (RSp + RVCo)
Left hemisphere
Scott, Blank, Rosen and Wise, 2000
![Page 16: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/16.jpg)
1
0
-1
2
Sp VCo RSp RVCo
Right hemisphere
Anterior
+66 -12 0 Z = 6.7
(Sp + RSp) - (VCo + RVCo)
Scott, Blank, Rosen and Wise, 2000
![Page 17: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/17.jpg)
Intelligibility
![Page 18: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/18.jpg)
Plasticity within this system
Naïve subjects were scanned before they could understand noise vocoded speech, then they were trained, then scanned again.
![Page 19: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/19.jpg)
Activity to noise vocoded speech after a training period, relative to prior activity to NVC before the training period. Narain, Wise, Rosen, Matthews, Scott, under review.
Flexibility in speech perception: learning to understand noise vocoded speech
As well as left lateralised STS, there is involvement of left premotor cortex and the left anterior thalamus (which receive projections from the belt and parabelt).
![Page 20: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/20.jpg)
Spectrograms of the stimuli
(speech)
16
8
4
3
2
1
(rotated speech)
16R
3R
![Page 21: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/21.jpg)
Intelligibility - behavioural data
![Page 22: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/22.jpg)
Right
Scott, Rosen, Lang and Wise, 2006
Z=5.96 x=64 y=-4 z=-2Z=4.73 x=-48 y=-16 z=-16
Z=4.52 x=-64 y=-28 z=8Z=5.6 x=-62 y=-10 z=80 Left
1 2 3 4 8 16 3R 16R
1 2 3 4 8 16 3R 16R
1 2 3 4 8 16 3R 16R
1 2 3 4 8 16 3R 16R
![Page 23: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/23.jpg)
medial
Scott and Johnsrude, 2003
anterior
posterior
lateral
Sounds with harmonic structure against pure tones: Hall, Johnsrude et al., 2002
Frequency modulated tones against unmodulated tones: Hall, Johnsrude et al., 2002
Amplitude modulated noise against unmodulated noise: Giraud et al, 1999
Spectral change against steady state sounds: Thivard et al, 2000
Peak responses to Intelligibility (Scott et al, 2006)
![Page 24: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/24.jpg)
Speech specific processing
• Does not occur in primary auditory cortexd
• Begins early in auditory cortex - in areas that also respond to AM
• As we move forward down the STS, the responses become less sensitive to acoustic structure - resembles behavioural profile
![Page 25: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/25.jpg)
Speech comprehension - The role of context
• e.g., words recognised more easily in sentences
• “The ship sailed the sea” > “Paul discussed the dive”.
• Can we identify the neural basis of this contextual modulation of speech comprehension?
(Miller et al., 1951; Boothroyd and Nittrouer, 1988; Grant and Seitz, 2000;
Stickney and Assmann, 2001; Davis et al., 2005)
![Page 26: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/26.jpg)
(noise vocoding:Shannon et al., 1995predictability:Kalikow et al., 1977)
![Page 27: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/27.jpg)
Low predictability:log increase with more channels
…‘Sue was interestedin the bruise’…
jonas obleser 27
![Page 28: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/28.jpg)
Behav 2 low+high
High predictability:influence at intermediate number of channels
…‘He caught thefish in his net’…
…‘Sue was interestedin the bruise’…
jonas obleser 28
![Page 29: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/29.jpg)
(cf. e.g. Binder et al. 2000; Scott et al., 2000; Davis & Johnsrude 2003; Zekveld et al., 2006)
Bottom-up processes:correlations with number of channels
RFX p<0.005 uncorrected, k>30 Obleser, Wise, Dresner, & Scott, 2007
![Page 30: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/30.jpg)
Left-hemispheric array of brain regions when context affects comprehension
Lateral Prefrontal (BA 8)
Posterior Cingulate (BA 30)
Medial Prefrontal (BA 9)
Angular Gyrus (BA 39)
Ventral IFG (BA 47)
RFX p<0.005 uncorrected, k>30Obleser, Wise, Dresner, & Scott, 2007
![Page 31: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/31.jpg)
findings
• A range of brain areas outwith auditory cortex contribute to ‘top down’ semantic influences on speech perception
• Further studies will be able to dissociate the contributions of different linguistic factors
![Page 32: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/32.jpg)
Words are not the only things we say
![Page 33: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/33.jpg)
Non speech sounds?
x=54
Regions in red respond to noises and rotated noises
Regions in yellow respond to noises and rotated noises
![Page 34: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/34.jpg)
1
0
-1
2
Sp VCo RSp RVCo
Right hemisphere
Anterior
+66 -12 0 Z = 6.7
(Sp + RSp) - (VCo + RVCo)
![Page 35: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/35.jpg)
What drives lateral asymmetry?
• Previous studies have not generally used ‘speech-like’ acoustic modulations
• We aimed to manipulate speech stimuli to vary the amplitude and spectral properties of speech independently
• Control for intelligibility• Do we see additive effects of amplitude and
spectral modulations?• Are these left lateralised?
![Page 36: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/36.jpg)
Steady spectrum, steady amplitude
Steady spectrum, varying amplitude
Varying spectrum, steady amplitude
Varying spectrum, varying amplitude
![Page 37: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/37.jpg)
Ideal additive effectsE
ffec
t si
ze
Down for flat amplitude and spectrum
Similar response to AM and SpM
Significantly more activated by stimuli with both AM and SpM
![Page 38: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/38.jpg)
Additive effects
Flat AM SpM SpMAM
Flat AM SpM SpMAM
PET scanning, 16 runs, N=13, thresholded at p<0.0001, 40 voxels
![Page 39: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/39.jpg)
Additive effects
Flat AM SpM SpMAM
Flat AM SpM SpMAM
PET scanning, 16 runs, N=13, thresholded at p<0.0001, 40 voxels
![Page 40: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/40.jpg)
But…
• Is there a problem - were these stimuli really processed as speech?
• To address this, 6 of the 13 subjects were pretrained on speech exemplars, and the speech stimuli were included as a 5th condition.
![Page 41: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/41.jpg)
A
B
C
D
E speech
![Page 42: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/42.jpg)
A
B
C
D
E speech
![Page 43: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/43.jpg)
Speech conditions
Flat AM SpM SpMAM
Flat AM SpM SpMAM
Flat AM SpM SpMAM
N=6, thresholded at p<0.0001, 40 voxels
![Page 44: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/44.jpg)
Speech conditions
Flat AM SpM SpMAM
Flat AM SpM SpMAM N=6, thresholded at p<0.0001, 40 voxels
![Page 45: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/45.jpg)
Asymmetries in speech perception
• Exist!• Are not driven by simple properties of
the speech signal• Right - preferentially processes speech-
like sounds - voices?• Left - processes linguistically relevant
information
![Page 46: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/46.jpg)
Posterior auditory areas
• In primates, medial posterior areas show auditory and tactile responses
• What do these areas do in speech processing in humans?
![Page 47: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/47.jpg)
Speaking and mouthing
This region, in the left posterior temporal-parietal junction, responds when subject repeat a phrase, mouth the phrase silently, or go ‘uh uh’, over mentally rehearsing the phrase
Wise, Scott, Blank, Murphy, Mummery and Warburton, 2001
Wise et al, 2001, Brain
![Page 48: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/48.jpg)
QuickTime™ and aDV - PAL decompressor
are needed to see this picture.
![Page 49: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/49.jpg)
Listening over silence
Amount of DAF (0, 50, 125, 200ms)
![Page 50: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/50.jpg)
DAF peak on right
0 50 125 200
![Page 51: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/51.jpg)
Neural basis of speech perception
• Hierarchical processing of sound in auditory cortex• The anterior ‘what’ pathway is important in the
perceptual processing of speech• Activity in this system can be modulated by top
down linguistic factors• There are hemispheric asymmetries in speech
perception - the left is driven by phonetic, lexical and linguistic properties: the right is driven by pitch variation, emotion and indexical properties
• There are sensory motor links in posterior auditory areas - part of a ‘how’ pathway?
![Page 52: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/52.jpg)
what
where
what
how
where
Scott, in press
Scott, Current Opinions in Neurobiology, 2005
![Page 53: The Neural Basis of Speech Perception – a view from functional imaging Sophie Scott Institute of Cognitive Neuroscience, University College London.](https://reader035.fdocuments.us/reader035/viewer/2022070408/56649e5d5503460f94b56444/html5/thumbnails/53.jpg)
Charlotte Jacquemot
Frank Eisner
Disa Sauter
Carolyn McGettigan
Narly Golestani
Jonas Obleser
Sophie ScottStuart Rosen
Richard WiseCharvy NarainAndrew FaulknerHideki Takaso