Post on 02-Feb-2016
description
Generative Models of M/EEG:
Group inversion and MEG+EEG+fMRI multimodal integration
Rik Henson
(with much input from Karl Friston)
Overview
1. A Generative Model of M/EEG
2. Group inversion (optimising priors across subjects)
3. Multimodal integration:
3.1 Symmetric integration (fusion) of MEG + EEG
3.2 Asymmetric integration of MEG + fMRI
3.3 Full fusion of MEG/EEG + fMRI?
1. A PEB Framework for MEG/EEG(Generative Model)
Phillips et al (2005), Neuroimage
Y = LJ +EY = Data n sensorsJ = Sources p>>n sourcesL = Leadfields n sensors x p sourcesE = Error n sensors
(Linear) Forward Model for MEG/EEG (for one timepoint):
| ( | ) ( )p( ) p pJ Y Y J J
(Gaussian) Likelihood:
( )( | ) ( , )ep NY J LJ C
( ) (0, )jp N ( )J C
C(e) = n x n Sensor (error) covariance
Prior:
C(j) = p x p Source (prior) covariance
Posterior:
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
1. A PEB Framework for MEG/EEG(Generative Model)
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
1. A PEB Framework for MEG/EEG(Generative Model)
Friston et al (2008) Neuroimage
Fixed
Variable
Data
( )1jQ ( )
2jQ
1. A PEB Framework for MEG/EEG(Inversion)
Friston et al (2002) Neuroimage
ˆ max ( | ) maxp F
λ Y λ
ˆˆln ( | ) ln ( , | ) ( , )p m p J m dJ F Y Y J λ
ˆˆ max ( | , ) maxj jp F J J Y λ
ln ( | , ) ( | ) lnq
F p p q Y J λ J λ
1. Obtain Restricted Maximum Likelihood (ReML) estimates of the hyperparameters (λ) by maximising the variational “free energy” (F):
2. Obtain Maximum A Posteriori (MAP) estimates of parameters (sources, J):( ) ( ) ( ) -1ˆ ˆ ˆ( )j T j T eC L LC L +C Y cf. Tikhonov
…and an estimate of their posterior covariance (inverse precision):
( ) ( ) ( )ˆ ˆ ˆ ˆˆ -j j j T -1Σ C C L C LC
3. Maximal F approximates Bayesian (log) “model evidence” for a model, m:
(relevant to MEG+EEG integration)
(relevant to MEG+fMRI integration)
( )ˆ ˆ ˆj e T ( )C LC L C
ˆ{ , , }m L Q λ
Summary:
1. A PEB Framework for MEG/EEG
• Automatically “regularises” in principled fashion…
• …allows for multiple constraints (priors)…
• …to the extent that multiple (100’s) of sparse priors possible…
• …(or multiple error components or multiple fMRI priors)…
• …furnishes estimates of source precisions and model evidence
2. Group Inversion
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
( ) ( , )p NX m C
2. Optimise Multiple Sparse Priors by pooling across participants
2. Group Inversion
Litvak & Friston (2008) Neuroimage
# sources
# s
ou
rce
s
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
2. Group Inversion (single subject)(Generative Model)
Litvak & Friston (2008) Neuroimage
( )1jQ ( )
2jQ
ΕJ
1Y
iL
( )jC ( )eC
( , )N 0 C ( , )N 0 C
...
( )jk
( )21eQ
( )eik
2. Group Inversion (multiple subjects)(Generative Model)
Litvak & Friston (2008) Neuroimage
( )1jQ ( )
2jQ
2Y NY
( )12eQ ( )
1eNQ( )
11eQ ...
)()()( jk
jk
j QC
Ti
eki
eik
ei AQAC )()()(
)(0
)(0
eTj CLCLC
)(
)(2
)(1
)(
00
0
0
00
eN
e
e
e
C
C
C
C
NNNE
E
E
J
L
L
L
Y
Y
Y
2
1
2
1
2
1
~
~
~
iii YAY ~
…projecting data and leadfields to a reference subject (0):
Common source-level priors:
Subject-specific sensor-level priors:
10 )( T
iiTii LLLLA
2. Group Inversion(Generative Model)
Litvak & Friston (2008) Neuroimage
2. Group Inversion(Generative Model)
Litvak & Friston (2008) Neuroimage
MMN MSP MSP (Group)
fMRI MEG ? (future)Data:
Causes (hidden):
Generative (Forward)Models:
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
3. Types of Multimodal Integration
(inversion)
AsymmetricIntegration
fMRI MEG ? (future)Data:
Causes (hidden):
Generative (Forward)Models:
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
SymmetricIntegration(Fusion)
3. Types of Multimodal Integration
Daunizeau et al (2007), Neuroimage
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
3.1 Fusion of MEG+EEG(Theory)
( )eijQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
Ci(e) = Sensor error covariance for ith modality
Qij = jth component for ith modalityλij = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
# sources
# s
ou
rce
s
( ) ( ) ( )e e ei ji ij
j
C Q
# sensors
# s
en
sors
# sensors
# s
en
sors
E.g, white noise for 2 modalities:( )11eQ ( )
21eQ
Henson et al (2009) Neuroimage
3.1 Fusion of MEG+EEG(Theory)
ΕJ
MEGY
MEGL
( )jC( )eC
( , )N 0 C ( , )N 0 C
( )1jQ ( )
2jQ
( )ji
( )1eQ ( )
2eQ
( )ei
3.1 Fusion of MEG+EEG(Generative Model)
Henson et al (2009) Neuroimage
ΕJ
MEGY
MEGL
( )jC( )1eC
( , )N 0 C ( , )N 0 C
( )1jQ ( )
2jQ
( )ji
( )11eQ ( )
12eQ
( )eij
3.1 Fusion of MEG+EEG(Generative Model)
EEGY
EEGL
( )2eC
( )21eQ ( )
22eQ
Henson et al (2009) Neuroimage
3.1 Fusion of MEG+EEG(Theory)
Henson et al (2009) Neuroimage
(1)11 1(1)22 2
(1)dd d
EY L
EY LJ
EY L
• Stack data and leadfields for d modalities:
1 ( )i
ii T
i im
YY
tr YY
• Where data / leadfields scaled to have same average / predicted variance:
)(
)(2
)(1
)(
00
0
0
00
ed
e
e
e
C
C
C
C
mi = Number of spatial modes (e.g, channels)1 ( )
i
ii T
i im
LL
tr L L
(note: common sources and source priors, but separate error components)
ERs from 12 subjects for 3 simultaneously-acquired Neuromag sensor-types:
RM
S f
T/m V
FacesScrambled
fT
3.1 Fusion of MEG+EEG(Application)
Magnetometers (MEG, 102)
(Planar) Gradiometers (MEG, 204)
Electrodes (EEG, 70)
Henson et al (2009) Neuroimage
150-190ms
Faces - Scrambled
ms ms ms
MEG mags MEG grads
EEG
FUSED
+31 -51 -15 +19 -48 -6
+43 -67 -11 +44 -64 -4
3.1 Fusion of MEG+EEG
Henson et al (2009) Neuroimage
ˆ1/ 59ii ˆ1/ 76ii
IID noise for each modality; common MSP for sources
ˆ1/ 95ii ˆ1/ 127ii
(fixed number of spatial+temporal modes)
Scrambled
150-190msFaces – Scrambled,
Faces
3.1 Fusion of MEG+EEG(Conclusions)
Henson et al (2009) Neuroimage
• Fusing magnetometers, gradiometers and EEG increased the conditional precision of the source estimates relative to inverting any one modality alone (when equating number of spatial+temporal modes)
• The maximal sources recovered from fusion were a plausible combination of the ventral temporal sources recovered by MEG and the lateral temporal sources recovered by EEG
• (Simulations show the relative scaling of mags and grads agrees with empty-room data)
3.2 Integration of M/EEG+fMRI
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
Henson et al (in press) Human Brain Mapping
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
fMRI Priors:
( ) ( , )p NX m C
# sources
# s
ou
rce
s
2. Each suprathreshold fMRI cluster becomes a separate prior
3.2 Integration of M/EEG+fMRI
3.2 Integration of M/EEG+fMRI(Generative Model)
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
( )1jQ ( )
2jQ
3.2 Integration of M/EEG+fMRI(Generative Model)
ΕJ
L
( )jC( )eC
( , )N 0 C ( , )N 0 C
MEGY
fMRIY
( )1jQ ( )
2jQ
( )ji
( )ei
( )1eQ ( )
2eQ( )
3jQ ( )
4jQ
Henson et al (in press) Human Brain Mapping
T1-weighted MRI
Anatomical data
{T,F,Z}-SPM
Gray matter segmentation
Cortical surfaceextraction
3D geodesicVoronoï diagram
Functional data
…
1. Thresholding and connected component labelling
…
2. Projection onto the cortical surface using the Voronoï diagram
…
3. Prior covariance components ( )jiQ
3.2 Integration of M/EEG+fMRI (Priors)
3.2 Integration of M/EEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
SPM{F} for faces versus scrambled faces,
15 voxels, p<.05 FWE
5 clusters from SPM of fMRI data from separate group of (18) subjects in MNI space
1 2
3 4 5
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
Gradiometers (MEG)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
fMRI priors counteract superficial bias of L2-norm
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
fMRI priors counteract superficial bias of L2-norm
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
NB: Priors affect variance, not precise timecourse…
R
L
Gradiometers (MEG)
(5 Local Valid Priors)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
Right Posterior Fusiform (rPF) Right Medial Fusiform (rMF) Right Lateral Fusiform (rLF)
Left occipital pole (lOP)
-27 -93 0
+26 -76 -11 +41 -43 -24 +32 -45 -12
-43 -47 -21
Left Lateral Fusiform (lLF)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
• Adding a single, global fMRI prior increases model evidence
• Adding multiple valid priors increases model evidence furtherHelpful if some fMRI regions produce no MEG/EEG signal (or arise from neural activity at different times)
• Adding invalid priors rarely increases model evidence, particularly in conjunction with valid priors
• Can counteract superficial bias of, e.g, minimum-norm
• Affects variance but not not precise timecourse
• (Adding fMRI priors to MSP has less effect)
3.2 Fusion of MEG+fMRI (Conclusions)
Henson et al (in press) Human Brain Mapping
3.3 Fusion of fMRI and MEG/EEG?
fMRI MEG ? (future)Data:
Causes (hidden):
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
Fusion of fMRI + MEG/EEG?
Henson (2010) Biomag
J
MEGY
( )sMEGL
( )jC
( )1jQ ( )
2jQ
MEGΕ
( )eC
( )1eQ ( )
2eQ
3.3 Fusion of fMRI and MEG/EEG?
Henson (2010) Biomag
J
MEGY
( )sMEGL
( )jC
( )1jQ ( )
2jQ
( )tfMRIH
MEGΕ
( )eC
( )1eQ ( )
2eQ
fMRIΕ
fMRIY
( )1tA ( )
2sQ
time (t)?space (s)
3.3 Fusion of fMRI and MEG/EEG?
Henson (2010) Biomag
Overall Conclusions
1. The PEB (in SPM8) framework is advantageous
2. Group optimisation of MSPs can be advantageous
3. Full fusion of MEG and EEG is advantageous
4. Using fMRI as (spatial) priors on MEG is advantageous
5. Unclear that fusion of fMRI and M/EEG is advantageous
The End
3. Fusion of MEG+EEG
Henson et al (2009) Neuroimage
Participant
Grads
Mags
log
(λx10
6 )
Participant
EE
G
Grads
Mags
log
(λx10
6 )
3. Fusion of MEG+EEGH
yper
pa
ram
ete
rs
Henson et al (2009) Neuroimage
4. Fusion of MEG+fMRI
Henson et al (in press) Human Brain Mapping
fMR
I hyp
erp
ara
me
ters
ln(λ
)+32
ln(λ
)+32
Participant
Participant
Magnetometers (MEG) Gradiometers (MEG) Electrodes (EEG)
Local Valid
Local Invalid