Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with...
-
Upload
austen-clark -
Category
Documents
-
view
217 -
download
0
Transcript of Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with...
![Page 1: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/1.jpg)
Generative Models of M/EEG:
Group inversion and MEG+EEG+fMRI multimodal integration
Rik Henson
(with much input from Karl Friston)
![Page 2: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/2.jpg)
Overview
1. A Generative Model of M/EEG
2. Group inversion (optimising priors across subjects)
3. Multimodal integration:
3.1 Symmetric integration (fusion) of MEG + EEG
3.2 Asymmetric integration of MEG + fMRI
3.3 Full fusion of MEG/EEG + fMRI?
![Page 3: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/3.jpg)
1. A PEB Framework for MEG/EEG(Generative Model)
Phillips et al (2005), Neuroimage
Y = LJ +EY = Data n sensorsJ = Sources p>>n sourcesL = Leadfields n sensors x p sourcesE = Error n sensors
(Linear) Forward Model for MEG/EEG (for one timepoint):
| ( | ) ( )p( ) p pJ Y Y J J
(Gaussian) Likelihood:
( )( | ) ( , )ep NY J LJ C
( ) (0, )jp N ( )J C
C(e) = n x n Sensor (error) covariance
Prior:
C(j) = p x p Source (prior) covariance
Posterior:
![Page 4: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/4.jpg)
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
1. A PEB Framework for MEG/EEG(Generative Model)
![Page 5: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/5.jpg)
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
1. A PEB Framework for MEG/EEG(Generative Model)
Friston et al (2008) Neuroimage
Fixed
Variable
Data
( )1jQ ( )
2jQ
![Page 6: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/6.jpg)
1. A PEB Framework for MEG/EEG(Inversion)
Friston et al (2002) Neuroimage
ˆ max ( | ) maxp F
λ Y λ
ˆˆln ( | ) ln ( , | ) ( , )p m p J m dJ F Y Y J λ
ˆˆ max ( | , ) maxj jp F J J Y λ
ln ( | , ) ( | ) lnq
F p p q Y J λ J λ
1. Obtain Restricted Maximum Likelihood (ReML) estimates of the hyperparameters (λ) by maximising the variational “free energy” (F):
2. Obtain Maximum A Posteriori (MAP) estimates of parameters (sources, J):( ) ( ) ( ) -1ˆ ˆ ˆ( )j T j T eC L LC L +C Y cf. Tikhonov
…and an estimate of their posterior covariance (inverse precision):
( ) ( ) ( )ˆ ˆ ˆ ˆˆ -j j j T -1Σ C C L C LC
3. Maximal F approximates Bayesian (log) “model evidence” for a model, m:
(relevant to MEG+EEG integration)
(relevant to MEG+fMRI integration)
( )ˆ ˆ ˆj e T ( )C LC L C
ˆ{ , , }m L Q λ
![Page 7: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/7.jpg)
Summary:
1. A PEB Framework for MEG/EEG
• Automatically “regularises” in principled fashion…
• …allows for multiple constraints (priors)…
• …to the extent that multiple (100’s) of sparse priors possible…
• …(or multiple error components or multiple fMRI priors)…
• …furnishes estimates of source precisions and model evidence
![Page 8: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/8.jpg)
2. Group Inversion
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
![Page 9: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/9.jpg)
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
( ) ( , )p NX m C
2. Optimise Multiple Sparse Priors by pooling across participants
2. Group Inversion
Litvak & Friston (2008) Neuroimage
# sources
# s
ou
rce
s
![Page 10: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/10.jpg)
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
2. Group Inversion (single subject)(Generative Model)
Litvak & Friston (2008) Neuroimage
( )1jQ ( )
2jQ
![Page 11: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/11.jpg)
ΕJ
1Y
iL
( )jC ( )eC
( , )N 0 C ( , )N 0 C
...
( )jk
( )21eQ
( )eik
2. Group Inversion (multiple subjects)(Generative Model)
Litvak & Friston (2008) Neuroimage
( )1jQ ( )
2jQ
2Y NY
( )12eQ ( )
1eNQ( )
11eQ ...
![Page 12: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/12.jpg)
)()()( jk
jk
j QC
Ti
eki
eik
ei AQAC )()()(
)(0
)(0
eTj CLCLC
)(
)(2
)(1
)(
00
0
0
00
eN
e
e
e
C
C
C
C
NNNE
E
E
J
L
L
L
Y
Y
Y
2
1
2
1
2
1
~
~
~
iii YAY ~
…projecting data and leadfields to a reference subject (0):
Common source-level priors:
Subject-specific sensor-level priors:
10 )( T
iiTii LLLLA
2. Group Inversion(Generative Model)
Litvak & Friston (2008) Neuroimage
![Page 13: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/13.jpg)
2. Group Inversion(Generative Model)
Litvak & Friston (2008) Neuroimage
MMN MSP MSP (Group)
![Page 14: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/14.jpg)
fMRI MEG ? (future)Data:
Causes (hidden):
Generative (Forward)Models:
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
3. Types of Multimodal Integration
(inversion)
![Page 15: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/15.jpg)
AsymmetricIntegration
fMRI MEG ? (future)Data:
Causes (hidden):
Generative (Forward)Models:
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
SymmetricIntegration(Fusion)
3. Types of Multimodal Integration
Daunizeau et al (2007), Neuroimage
![Page 16: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/16.jpg)
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
3.1 Fusion of MEG+EEG(Theory)
![Page 17: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/17.jpg)
( )eijQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
Ci(e) = Sensor error covariance for ith modality
Qij = jth component for ith modalityλij = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
# sources
# s
ou
rce
s
( ) ( ) ( )e e ei ji ij
j
C Q
# sensors
# s
en
sors
# sensors
# s
en
sors
E.g, white noise for 2 modalities:( )11eQ ( )
21eQ
Henson et al (2009) Neuroimage
3.1 Fusion of MEG+EEG(Theory)
![Page 18: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/18.jpg)
ΕJ
MEGY
MEGL
( )jC( )eC
( , )N 0 C ( , )N 0 C
( )1jQ ( )
2jQ
( )ji
( )1eQ ( )
2eQ
( )ei
3.1 Fusion of MEG+EEG(Generative Model)
Henson et al (2009) Neuroimage
![Page 19: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/19.jpg)
ΕJ
MEGY
MEGL
( )jC( )1eC
( , )N 0 C ( , )N 0 C
( )1jQ ( )
2jQ
( )ji
( )11eQ ( )
12eQ
( )eij
3.1 Fusion of MEG+EEG(Generative Model)
EEGY
EEGL
( )2eC
( )21eQ ( )
22eQ
Henson et al (2009) Neuroimage
![Page 20: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/20.jpg)
3.1 Fusion of MEG+EEG(Theory)
Henson et al (2009) Neuroimage
(1)11 1(1)22 2
(1)dd d
EY L
EY LJ
EY L
• Stack data and leadfields for d modalities:
1 ( )i
ii T
i im
YY
tr YY
• Where data / leadfields scaled to have same average / predicted variance:
)(
)(2
)(1
)(
00
0
0
00
ed
e
e
e
C
C
C
C
mi = Number of spatial modes (e.g, channels)1 ( )
i
ii T
i im
LL
tr L L
(note: common sources and source priors, but separate error components)
![Page 21: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/21.jpg)
ERs from 12 subjects for 3 simultaneously-acquired Neuromag sensor-types:
RM
S f
T/m V
FacesScrambled
fT
3.1 Fusion of MEG+EEG(Application)
Magnetometers (MEG, 102)
(Planar) Gradiometers (MEG, 204)
Electrodes (EEG, 70)
Henson et al (2009) Neuroimage
150-190ms
Faces - Scrambled
ms ms ms
![Page 22: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/22.jpg)
MEG mags MEG grads
EEG
FUSED
+31 -51 -15 +19 -48 -6
+43 -67 -11 +44 -64 -4
3.1 Fusion of MEG+EEG
Henson et al (2009) Neuroimage
ˆ1/ 59ii ˆ1/ 76ii
IID noise for each modality; common MSP for sources
ˆ1/ 95ii ˆ1/ 127ii
(fixed number of spatial+temporal modes)
Scrambled
150-190msFaces – Scrambled,
Faces
![Page 23: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/23.jpg)
3.1 Fusion of MEG+EEG(Conclusions)
Henson et al (2009) Neuroimage
• Fusing magnetometers, gradiometers and EEG increased the conditional precision of the source estimates relative to inverting any one modality alone (when equating number of spatial+temporal modes)
• The maximal sources recovered from fusion were a plausible combination of the ventral temporal sources recovered by MEG and the lateral temporal sources recovered by EEG
• (Simulations show the relative scaling of mags and grads agrees with empty-room data)
![Page 24: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/24.jpg)
3.2 Integration of M/EEG+fMRI
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ2. Source components,
(priors/regularisation):
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
Multiple Sparse Priors (MSP):
( ) ( , )p NX m C
# sources
# s
ou
rce
sFriston et al (2008) Neuroimage
![Page 25: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/25.jpg)
Henson et al (in press) Human Brain Mapping
( )eiQ
Specifying (co)variance components (priors/regularisation):
1. Sensor components, (error):
i ii
C Q C = Sensor/Source covarianceQ = Covariance componentsλ = Hyper-parameters
( )jiQ
“IID” (white noise):
# sensors
# s
en
sors
Empty-room:
# sensors
# s
en
sors
“IID” (min norm):
# sources
# s
ou
rce
s
fMRI Priors:
( ) ( , )p NX m C
# sources
# s
ou
rce
s
2. Each suprathreshold fMRI cluster becomes a separate prior
3.2 Integration of M/EEG+fMRI
![Page 26: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/26.jpg)
3.2 Integration of M/EEG+fMRI(Generative Model)
ΕJ
YL
( )jC( )eC
( , )N 0 C ( , )N 0 C
...
( )ji
( )1eQ ( )
2eQ ...
( )ei
( )1jQ ( )
2jQ
![Page 27: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/27.jpg)
3.2 Integration of M/EEG+fMRI(Generative Model)
ΕJ
L
( )jC( )eC
( , )N 0 C ( , )N 0 C
MEGY
fMRIY
( )1jQ ( )
2jQ
( )ji
( )ei
( )1eQ ( )
2eQ( )
3jQ ( )
4jQ
![Page 28: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/28.jpg)
Henson et al (in press) Human Brain Mapping
T1-weighted MRI
Anatomical data
{T,F,Z}-SPM
Gray matter segmentation
Cortical surfaceextraction
3D geodesicVoronoï diagram
Functional data
…
1. Thresholding and connected component labelling
…
2. Projection onto the cortical surface using the Voronoï diagram
…
3. Prior covariance components ( )jiQ
3.2 Integration of M/EEG+fMRI (Priors)
![Page 29: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/29.jpg)
3.2 Integration of M/EEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
SPM{F} for faces versus scrambled faces,
15 voxels, p<.05 FWE
5 clusters from SPM of fMRI data from separate group of (18) subjects in MNI space
1 2
3 4 5
![Page 30: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/30.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
Gradiometers (MEG)
![Page 31: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/31.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
![Page 32: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/32.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
![Page 33: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/33.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
![Page 34: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/34.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
(binarised, variance priors)
Magnetometers (MEG)
* *
* *
Gradiometers (MEG)
None Global Local (Valid) Local (Invalid) Valid+Invalid
Electrodes (EEG)
Neg
ativ
e F
ree
Ene
rgy
(a.u
.)(m
odel
evi
denc
e)
*
*
**
*
*
*
![Page 35: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/35.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
![Page 36: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/36.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
![Page 37: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/37.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
fMRI priors counteract superficial bias of L2-norm
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
![Page 38: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/38.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
fMRI priors counteract superficial bias of L2-norm
None Global Local (Valid) Local (Invalid)
Magnetometers (MEG)
Gradiometers (MEG)
Electrodes (EEG)
IID sources and IID noise (L2 MNM)
![Page 39: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/39.jpg)
3.2 Fusion of MEG+fMRI (Application)
Henson et al (in press) Human Brain Mapping
NB: Priors affect variance, not precise timecourse…
R
L
Gradiometers (MEG)
(5 Local Valid Priors)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
Right Posterior Fusiform (rPF) Right Medial Fusiform (rMF) Right Lateral Fusiform (rLF)
Left occipital pole (lOP)
-27 -93 0
+26 -76 -11 +41 -43 -24 +32 -45 -12
-43 -47 -21
Left Lateral Fusiform (lLF)
Diff
eren
tial R
espo
nse
(Fac
es v
s S
cram
bled
)
![Page 40: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/40.jpg)
• Adding a single, global fMRI prior increases model evidence
• Adding multiple valid priors increases model evidence furtherHelpful if some fMRI regions produce no MEG/EEG signal (or arise from neural activity at different times)
• Adding invalid priors rarely increases model evidence, particularly in conjunction with valid priors
• Can counteract superficial bias of, e.g, minimum-norm
• Affects variance but not not precise timecourse
• (Adding fMRI priors to MSP has less effect)
3.2 Fusion of MEG+fMRI (Conclusions)
Henson et al (in press) Human Brain Mapping
![Page 41: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/41.jpg)
3.3 Fusion of fMRI and MEG/EEG?
fMRI MEG ? (future)Data:
Causes (hidden):
BalloonModel
HeadModel
?
EEG
HeadModel
“Neural”Activity
Fusion of fMRI + MEG/EEG?
Henson (2010) Biomag
![Page 42: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/42.jpg)
J
MEGY
( )sMEGL
( )jC
( )1jQ ( )
2jQ
MEGΕ
( )eC
( )1eQ ( )
2eQ
3.3 Fusion of fMRI and MEG/EEG?
Henson (2010) Biomag
![Page 43: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/43.jpg)
J
MEGY
( )sMEGL
( )jC
( )1jQ ( )
2jQ
( )tfMRIH
MEGΕ
( )eC
( )1eQ ( )
2eQ
fMRIΕ
fMRIY
( )1tA ( )
2sQ
time (t)?space (s)
3.3 Fusion of fMRI and MEG/EEG?
Henson (2010) Biomag
![Page 44: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/44.jpg)
Overall Conclusions
1. The PEB (in SPM8) framework is advantageous
2. Group optimisation of MSPs can be advantageous
3. Full fusion of MEG and EEG is advantageous
4. Using fMRI as (spatial) priors on MEG is advantageous
5. Unclear that fusion of fMRI and M/EEG is advantageous
![Page 45: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/45.jpg)
The End
![Page 46: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/46.jpg)
3. Fusion of MEG+EEG
Henson et al (2009) Neuroimage
![Page 47: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/47.jpg)
Participant
Grads
Mags
log
(λx10
6 )
Participant
EE
G
Grads
Mags
log
(λx10
6 )
3. Fusion of MEG+EEGH
yper
pa
ram
ete
rs
Henson et al (2009) Neuroimage
![Page 48: Generative Models of M/EEG: Group inversion and MEG+EEG+fMRI multimodal integration Rik Henson (with much input from Karl Friston)](https://reader036.fdocuments.us/reader036/viewer/2022062519/56649eb15503460f94bb79b6/html5/thumbnails/48.jpg)
4. Fusion of MEG+fMRI
Henson et al (in press) Human Brain Mapping
fMR
I hyp
erp
ara
me
ters
ln(λ
)+32
ln(λ
)+32
Participant
Participant
Magnetometers (MEG) Gradiometers (MEG) Electrodes (EEG)
Local Valid
Local Invalid