8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
1/106
Machine Learning and Signal Processing Tools for BCI
Klaus-Robert Mller, Benjamin Blankertz, Gabriel Curio et al.
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
2/106
BBCI team:
Gabriel Curio
Florian LoschVolker KunzmannFrederike Holefeld
Vadim Nikulin@Charite
Andreas ZieheFlorin Popescu
Christian GrozeaSteven LemmMotoaki Kawanabe
Guido Nolte@FIRST
Yakob Badower@Pico Imaging
Marton Danozcy
Benjamin Blankertz
Michael TangermannClaudia Sannelli
Carmen VidaurreBastian VenthurSiamac FazliMartijn Schreuder
Matthias TrederStefan Haufe
Thorsten DickhausFrank MeineckePaul von Bnau
Marton DanozcyFelix BiessmannKlaus-Robert Mller@TUB
Matthias KrauledatGuido Dornhege
Roman Krepki@industry
Collaboration with: U Tbingen, Bremen, Albany, TU Graz, EPFL, Daimler, Siemens, MES, MPIs,U Tokyo, TIT, RIKEN, Bernstein Focus Neurotechnology, Bernstein Center for ComputationalNeuroscience Berlin, picoimaging, Columbia, CUNY
Funding by: EU, BMBF and DFG
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
3/106
Noninvasive Brain-Computer Interface
DECODING
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
4/106
Brain Pong with BBCI
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
5/106
[From Birbaumer et al.] [From Pfurtscheller et al.]
Noninvasive BCI: clinical applications
Brain- Computer Interface
Signal Processing
EEGAcquisition
EEGAcquisition
ApplicationInterface
ApplicationInterface
FES DeviceGrasp-Pattern
3 channelStimulation
BBCI: Leitmotiv: let the machines learn
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
6/106
EEG based noninvasive BCI
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
7/106
The cerebral cocktail party problem
use ICA/NGCAprojections for artifactand noise removal
feature extraction andselection
[cf. Ziehe et al. 2000, Blanchard et al. 2006]
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
8/106
BBCI paradigms
- healthy subjects (BCI untrained) perform "imaginary movements (ERD/ERS)
- instruction: imagine
- squezzing a ball,
- kicking a ball,
- feel touch
Leitmotiv: let the machines learn
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
9/106
Towards imaginations: Modulation of Brain Rhythms
IMAGINATION of left arm
Single channel
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
10/106
Variance I: Single-trial vs. Averaging
Single channel
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
11/106
Variance II: Trial to trial variability
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
12/106
Variance III: inter subject variability [l vs r]
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
13/106
BCI with machine learning: training
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
14/106
BBCI paradigms
- healthy subjects untrainedfor BCI
A: training 20min: right/left hand imagined movements
infer the respective brain acivities (ML & SP)
B: online feedback session
Leitmotiv: let the machines learn
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
15/106
Playing with BCI: training session (20 min)
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
16/106
Machine learning approach to BCI: infer prototypical pattern
Inference by CSP Algorithm
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
17/106
BCI with machine learning: feedback
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
18/106
Lecture Blankertz here
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
19/106
BBCI Set-up
Artifact removal
[cf. Mller et al. 2001, 2007, 2008, Dornhege et al. 2003, 2007, Blankertz et al. 2004, 2005, 2006, 2007, 2008]
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
20/106
What can Machine Learning tell us about physiology?
[cf. Blankertz et al. 2001, 2006]
11
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
21/106
ML for knowledge discovery
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
22/106
Results: Exploring the limits of untrained users
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
23/106
BCI: 1st session performance for novices
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
24/106
Spelling with BBCI: a communication for the disabled I
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
25/106
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
26/106
Variance IV: covariate shift: from training to feedback
Need for adaptation !
[cf. Sugiyama & Mller 2005, Shenoy et al. 2005,Sugiyama et al. 2007]
N h i l i l l i
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
27/106
Neurophysiological analysis
[cf. Krauledat et al. submitted]
W i ht d Li R i f i t hift ti
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
28/106
Weighted Linear Regression for covariate shift compensation
yields unbiased estimator even underconvariate shift
, choosing
[cf. Sugiyama & Mller 2005, Sugiyama et al. 2007]
BCI Illiteracy
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
29/106
y
Screening Study (N=80):
Cat I: good calibration (cb), good feedback (fb)
Cat II: good cb, no good fb
Cat III: no good cb
Percentage of ~20% of nave users:BCI accuracy does not reach level criterion,i.e., control not accurate enough to control applications
design a predictor !
SMR Predictor
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
30/106
SMR-Predictor
Calculate the power spectral density (PSD) in three Laplacian channels C3, Cz,C4 under rest cond.
Model each resulting curve by g = g1 + g2, with
g1 = g1 (x,, k) =k1 +k2/ x (estimated noise)
g2 = g2(x, , , k) =k3(x ; 1 , 1) + k4 (x ; 2, 2) (2 peaks)
Proposed predictor: Average height of the larger peak
SMR Predictor (Results)
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
31/106
SMR-Predictor (Results)
As much as R
2
= 30% (calibration) and R
2
= 26% (feedback) of variability in BCI accuracycan be explained by the SMR predictor in our study sample!
Pearson R = 0.55 Pearson R = 0.51
[cf. Blankertz, Kbler, Mller et al. 2009]
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
32/106
Approach to Cure BCI Illiteracy
Runs1-
3
Runs4
-6
Runs
7-
8
fixed
Laplace
CSP +sel.Lap.
CSP
supervised
unsupervis
ed
Direct feedback -> Unspecific LDA classifier. Each trial, perform adaptation of the cls.
Features: log band power (alpha and beta).
Laplacian channels C3, C4 and Cz.
Compute CSP and sel. Laps. from runs 1-3. Fixed CSP filters, automated laps. selection. Each trial retrain the classifier.
Compute CSP from runs 4-6.
Perform unsupervised adaptation of pooled mean.
Update the bias of the classifier.
[cf. Vidaurre, Blankertz, Mller et al. in preparation]
Results (Grand Averages)
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
33/106
Results (Grand Averages)
C
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
34/106
Example: one subject of Cat. III
!Runs 1 and 2 Runs 7 and 8
[cf. Vidaurre, Blankertz, Mller et al. 2009]
Real Man Machine Interaction
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
35/106
Real Man Machine Interaction
[Tangermann, Mller et al 2009]
Harvest from BCI-gaming
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
36/106
Harvest from BCI-gaming
Research platform:BCI gaming +
Machine Learning +
Single Trial Analysis (MSM)
Limits of BCI-reaction time?
Mental statesJUST BEFORE
success or failure?
Dynamics:limits of temporal
control precision?
Before-after
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
37/106
Future issues: sensors
Popescu et al 2007
Future Issues: Shifting distributions within experiment
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
38/106
g p
Conclusion
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
39/106
Conclusion
BBCI: non-invasive with high Information transfer rates for the Untrained
BBCI: Untrained, Calibration < 20min, data analysis
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
40/106
BBCI team:
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
41/106
Gabriel Curio
Florian Losch
Volker KunzmannFrederike Holefeld
Vadim Nikulin@Charite
Andreas ZieheFlorin Popescu
Christian GrozeaSteven LemmMotoaki Kawanabe
Guido Nolte@FIRST
Yakob Badower@Pico Imaging
Marton Danozcy
Benjamin Blankertz
Michael TangermannClaudia SannelliCarmen Vidaurre
Bastian VenthurSiamac FazliMartijn Schreuder
Matthias TrederStefan Haufe
Thorsten DickhausFrank Meinecke
Paul von BnauMarton DanozcyFelix BiessmannKlaus-Robert Mller@TUB
Matthias KrauledatGuido Dornhege
Roman Krepki@industry
Collaboration with: U Tbingen, Bremen, Albany, TU Graz, EPFL, Daimler, Siemens, MES, MPIs,U Tokyo, TIT, RIKEN, Bernstein Focus Neurotechnology, Bernstein Center for ComputationalNeuroscience Berlin, picoimaging, Columbia, CUNY
Funding by: EU, BMBF and DFG
BCI Competitions
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
42/106
For BCI IV Competition see www.bbci.de
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
43/106
FOR INFORMATION SEE: www.bbci.de
Machine Learning opensource software initiative:MLOSS see
www.jmlr.org
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
44/106
M a c h i n e L e a r n i n g a n d S i g n a l P r o c e s s i n g T o o l s f o r
B r a i n - C o m p u t e r I n t e r f a c i n g
B e n j a m i n B l a n k e r t z
1,2
K l a u s - R o b e r t M l l e r
1
1M a c h i n e L e a r n i n g L a b o r a t o r y , B e r l i n I n s t i t u t e o f T e c h n o l o g y
2F r a u n h o f e r F I R S T ( I D A )
b l a n k e r @ c s . t u - b e r l i n . d e
0 8 - J u l - 2 0 0 9
T o p i c o f T h i s S e c t i o n
http://[email protected]/8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
45/106
M e t h o d :
c l a s s i c a t i o n o f s p a t i o - t e m p o r a l f e a t u r e s ;
s h r i n k a g e o f t h e s a m p l e c o v a r i a n c e m a t r i x t o c o u n t e r b a l a n c e
t h e e s t i m a t i o n b i a s
A p p l i c a t i o n :
c l a s s i c a t i o n o f s i n g l e - t r i a l E R P s i n a n a t t e n t i o n - b a s e d s p e l l e r
S o m e N e u r o p h y s i o l o g i c a l B a c k g r o u n d
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
46/106
A n i n f r e q u e n t s t i m u l u s i n a s e r i e s o f s t a n d a r d s t i m u l i e v o k e s a P 3 0 0
c o m p o n e n t a t c e n t r a l s c a l p p o s i t i o n i f a t t e n d e d :
Cz
N1
P3
Cz
T h e p r e s e n t a t i o n o f a v i s u a l s t i m u l u s e l i c i t s a V i s u a l E v o k e d
P o t e n t i a l ( V E P ) i n v i s u a l c o r t e x i f f o c u s e d :
Oz
Oz
N1
P1
E x p e r i m e n t a l D e s i g n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
47/106
C l a s s i c M a t r i x S p e l l e r A t t e n t i o n - b a s e d H e x - o - S p e l l
S e e P o s t e r W 0 7 ( T r e d e r e t a l . ) f o r a i n v e s t i g a t i o n o f o v e r t v s .
c o v e r t a t t e n t i o n a n d a c o m p a r i s o n o f t h o s e t w o s p e l l e r d e s i g n s .
E x p e r i m e n t a l D e s i g n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
48/106
C l a s s i c M a t r i x S p e l l e r A t t e n t i o n - b a s e d H e x - o - S p e l l
S e e P o s t e r W 0 7 ( T r e d e r e t a l . ) f o r a i n v e s t i g a t i o n o f o v e r t v s .
c o v e r t a t t e n t i o n a n d a c o m p a r i s o n o f t h o s e t w o s p e l l e r d e s i g n s .
S i n g l e - s u b j e c t E R P s i n H e x - o - S p e l l
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
49/106
D a t a s e t f o r i l l u s t r a t i o n o f c l a s s i c a t i o n m e t h o d s :
AF3
Target
Nontarget
AF4
F7
Fz
F8
FT7
FC3 FC4
FT8
C5 Cz C6
TP7
CP3 CP4
TP8Pz
PO7 PO8
Oz5 V
+
200 ms
T o p o g r a p h i e s o f E R P C o m p o n e n t s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
50/106
T h e r e a r e s e v e r a l E R P c o m p o n e n t s t h a t c a n b e u s e d t o d e t e r m i n e
t h e a t t e n d e d s y m b o l :
100 0 100 200 300 400 500
5
0
5
Cz () PO7+PO8 ()
ms
[V]
Target
Nontarget
5
0
5
5
0
5
Target
Nontarget
vP1
115135 ms
N1
135155 ms
vN1
155195 ms
P250
205235 ms
vN2
285325 ms
P400
335395 ms
N500
495535 ms
C l a s s i c a t i o n o f T e m p o r a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
51/106
A s a r s t s t e p : c l a s s i c a t i o n o n r a w t i m e c o u r s e s ( 1 1 5 5 3 5 m s ) i n
s i n g l e c h a n n e l s . T h e r e s u l t i s d i s p l a y e d a s s c a l p m a p :
error[%]
15
20
25
30
35
40
45
50
E x t r a c t i o n o f S p a t i a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
52/106
22 22.2 22.4
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
dev
23.323.423.5 23.623.7
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
24.524.6 24.724.8 24.9
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
25.8 26 26.2
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
dev
27 27.2 27.4
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
E x t r a c t i o n o f S p a t i a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
53/106
22 22.2 22.4
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
dev
23.323.4 23.523.6 23.7
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
24.5 24.6 24.7 24.8 24.9
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
25.8 26 26.2
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
dev
27 27.2 27.4
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
std
T h e r2
- M a t r i x o f D i e r e n c e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
54/106
T h e t e m p o r a l a n d s p a t i a l s t r u c t u r e o f t h e d i e r e n c e b e t w e e n E R P s
o f d i e r e n t c o n d i t i o n s c a n b e i n v e s t i g a t e d b y t h e s i g n e d r2 - m a t r i x :
[ms]
c
anne
s
100 0 100 200 300 400 500
Fz
FCz
Cz
CPz
Pz
Oz0.15
0.1
0.05
0
0.05
0.1
0.15
115 135 ms 135 155 ms 155 195 ms 205 235 ms 285 325 ms 335 395 ms 495 535 ms
r(x) :=
N1 N2
N1 + N2
mean{xi | yi = 1} mean{xi | yi = 2}std{xi}
S p a t i a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
55/106
#01: std #02: std #03: std #04: std #05: dev #06: std
#07: std #08: dev #09: std #10: std #11: std #12: dev
#13: std #14: std #15: std #16: std #17: dev #18: std
#19: std #20: std #21: std #22: std #23: dev #24: dev
L i n e a r C l a s s i e r a s S p a t i a l F i l t e r
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
56/106
A l i n e a r c l a s s i e r t h a t w a s t r a i n e d o n s p a t i a l f e a t u r e s c a n a l s o b e
r e g a r d e d a s a s p a t i a l l t e r .
L e t w b e t h e L D A w e i g h t v e c t o r a n d X R# c h a n s # t i m e p o i n t s b ec o n t i n u o u s E E G s i g n a l s . T h e n
Xf := wX R1#t i m e p o i n t s
i s t h e r e s u l t o f s p a t i a l l t e r i n g : e a c h c h a n n e l o f
Xi s w e i g h t e d w i t h
t h e c o r r e s p o n d i n g c o m p o n e n t o f w
a n d s u m m e d u p .
T h e w e i g h t v e c t o r o f t h e
c l a s s i e r c a n b e d i s p l a y a s
s c a l p m a p :
0.1
0.08
0.06
0.04
0.02
0
0.02
0.04
0.06
0.08
0.1
C l a s s i c a t i o n R e s u l t s f o r S p a t i a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
57/106
5
0
5
vP1 N1 vN1 P250 vN2 P400 N500
115135 ms 135155 ms 155195 ms 205235 ms 285325 ms 335395 ms 495535 ms
10
15
20
25
30
35
error[%]
E x t r a c t i o n o f S p a t i o - T e m p o r a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
58/106
22 22.1 22.2 22.3 22.4 22.5 22.6
FC3
FCz
FC4
C3
Cz
C4
CP3
CPz
CP4
Pz
[s]
dev
S p a t i o - T e m p o r a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
59/106
S p a t i o - t e m p o r a l f e a t u r e s a r e t y p i c a l l y h i g h - d i m e n s i o n a l ( h e r e
5 9 E E G c h a n n e l s
7 t i m e i n t e r v a l s = 4 1 3 d i m e n s i o n a l f e a t u r e s ) :
mean of time interval [ms]
space
[channe
ls]
125 145 175 220 305 365 515
Fz
FCz
Cz
CPz
Pz
Oz
[V]
15
10
5
0
5
10
15
C l a s s i c a t i o n R e s u l t f o r S p a t i o - T e m p o r a l F e a t u r e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
60/106
5
0
5
vP1 N1 vN1 P250 vN2 P400 N500
115135 ms 135155 ms 155195 ms 205235 ms 285325 ms 335395 ms 495535 ms
10
15
20
25
30
35
error[%]
Combined
A l t h o u g h i n f o r m a t i o n w a s a d d e d , c l a s s i c a t i o n o n t h e c o n c a t e n a t e d
f e a t u r e b e c o m e s w o r s e : o v e r t t i n g .
B i a s i n E s t i m a t i n g C o v a r i a n c e s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
61/106
L e t x1, . . . ,xn Rd b e n v e c t o r s d r a w n f r o m a d - d i m e n s i o n a l G a u s s i a n d i s t r i b u t i o n
N(,)
.
F o r c l a s s i c a t i o n a n d h a v e t o b e e s t i m a t e d f r o m t h e d a t a :
= 1n
nk=1 xk
= 1n1
nk=1(xk )(xk )
B u t , i f t h e n u m b e r o f s a m p l e s n i s n o t l a r g e r e l a t i v e t o t h e d i m e n s i o n d, t h e e s t i m a t i o n i s e r r o r - p r o n e . T h e r e i s a s y s t e m a t i c a l b i a s :
L a r g e E i g e n v a l u e s o f a r e t o o l a r g e
S m a l l E i g e n v a l u e s o f a r e t o o s m a l l
T h i s a e c t s , e . g . , c l a s s i c a t i o n w i t h L D A :
N o r m a l v e c t o r o f L D A : w = 1(1 2) .
B i a s i n E s t i m a t i n g C o v a r i a n c e s ( 2 )
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
62/106
2 1 0 1 2
1.5
1
0.5
0
0.5
1
1.5 true covempirical cov
0 50 100 150 200
0
50
100
150
200
250
index of Eigenvector
Eigen
value
true
N=500
N=200
N=100
N= 50
A R e m e d y f o r C l a s s i c a t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
63/106
A s i m p l e w a y t h a t c a n p a r t l y x t h e b i a s i s s h r i n k a g e : t h e
e m p i r i c a l c o v a r i a n c e m a t r i x i s m o d i e d t o b e m o r e s p h e r i c a l .
I n L D A t h e e m p i r i c a l c o v a r i a n c e m a t r i x i s r e p l a c e d b y
() = (1 )+ If o r a [0, 1] a n d d e n e d a s a v e r a g e E i g e n v a l u e trace(Si)/d.
A R e m e d y f o r C l a s s i c a t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
64/106
A s i m p l e w a y t h a t c a n p a r t l y x t h e b i a s i s s h r i n k a g e : t h e
e m p i r i c a l c o v a r i a n c e m a t r i x i s m o d i e d t o b e m o r e s p h e r i c a l .
I n L D A t h e e m p i r i c a l c o v a r i a n c e m a t r i x i s r e p l a c e d b y
() = (1 )+ If o r a [0, 1] a n d d e n e d a s a v e r a g e E i g e n v a l u e trace(Si)/d.S i n c e i s p o s i t i v e s e m i - d e n i t e w e c a n h a v e a n E i g e n v a l u e
d e c o m p o s i t i o n
= VDVw i t h o r t h o n o r m a l
Va n d d i a g o n a l
D.
F r o m
= (1 )VDV + I = V ((1 )D+ I)V
w e s e e t h a t
() a n d h a v e t h e s a m e E i g e n v e c t o r s ( c o l u m n s o f V )
e x t r e m e E i g e n v a l u e s ( l a r g e / s m a l l ) a r e s h r u n k / e x t e n d e d
t o w a r d s t h e a v e r a g e .
= 0y i e l d s L D A w i t h o u t s h r i n k a g e ,
= 1a s s u m e s s p h e r i c a l
c o v a r i a n c e m a t r i c e s .
M o d e l s e l e c t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
65/106
L D A w i t h s h r i n k a g e o f t h e e m p i r i c a l c o v a r i a n c e m a t r i x h a s o n e f r e e
p a r a m e t e r ( ) , a l s o c a l l e d h y p e r p a r a m e t e r , t h a t n e e d s t o b e s e l e c t e d . T h e r e i s n o g e n e r a l w a y t o d o i t .
N u m e r o u s s t r a t e g i e s w i t h d i e r e n t p r o p e r t i e s e x i s t , e . g .
e m p i r i c a l B a y e s s h r i n k a g e e s t i m a t o r
M D L : M i n i m u m D e s c r i p t i o n L e n g t h
M o d e l - s e l e c t i o n b a s e d o n c r o s s - v a l i d a t i o n .
. . .
A n e a s y ( a n d a l s o t i m e - c o n s u m i n g ) w a y i s m o d e l - s e l e c t i o n b a s e d o n
c r o s s - v a l i d a t i o n .
R e g u l a r i z e d L D A a t W o r k
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
66/106
C r o s s - v a l i d a t i o n r e s u l t s f o r d i e r e n t s i z e s o f t r a i n i n g d a t a ( 2 5 0 ,
5 0 0 , 2 0 0 0 ) f o r d i e r e n t v a l u e s o f t h e r e g u l a r i z a t i o n p a r a m e t e r
( x - a x i s ) . F e a t u r e s v e c t o r s h a v e 2 5 0 d i m e n s i o n s .
0 0.2 0.4 0.6 0.8 15
10
15
20
25
regularization parameter
crossvalidationerror[%]
250
0 0.2 0.4 0.6 0.8 15
10
15
20
25
regularization parameter
crossvalidationerror[%]
500
0 0.2 0.4 0.6 0.8 15
10
15
20
25
regularization parameter
crossvalidationerror[%]
2000
I n v e s t i g a t i n g t h e I m p a c t o f S h r i n k a g e
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
67/106
L D A : w = 1(1
2) ; s h r i n k a g e : () = (1
)+ I
= 1
w 1 2
I n v e s t i g a t i n g t h e I m p a c t o f S h r i n k a g e
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
68/106
L D A : w = 1(1
2) ; s h r i n k a g e : () = (1
)+ I
= 0
w 1(1 2) = 1
w 1 2
I n v e s t i g a t i n g t h e I m p a c t o f S h r i n k a g e
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
69/106
L D A : w = 1(1
2) ; s h r i n k a g e : () = (1
)+ I
= 0
w 1(1 2)
a c c o u n t i n g f o r
s p a t i a l s t r u c t u r e o f
t h e n o i s e
= 1
w 1 2
E R P a n d N o i s e
S i m p l e a s s u m p t i o n f o r E R P s : s i n g l e t r i a l xk(t) i s c o m p o s e d o f a n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
70/106
( )E R P s(t) a n d G a u s s i a n ` n o i s e ' nk(t) :
xk(t) = s(t) + nk(t) f o r a l l t r i a l s k = 1, . . . , K
10 5 0 5 108
6
4
2
0
2
4
6
8
10
potential at FCz [V]
potentialatCPz
[V
]
mean: phaselocked ERP
cov: nonphaselocked noise
S p a t i a l S t r u c t u r e o f t h e N o i s e
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
71/106
T h e t w o s t r o n g e s t p r i n c i p a l c o m p o n e n t s o f t h e n o i s e ( c o v a r i a n c e
m a t r i x ) i n t h i s d a t a s e t :
T r i a l - t o - t r i a l v a r i a t i o n o f P 3
V i s u a l a l p h a
U n d e r s t a n d i n g S p a t i a l F i l t e r s
( a )
ith littl di t b
( b )
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
72/106
10 5 0 5 108
6
4
2
0
2
4
6
8
10
potential at FCz [V]
potentialatCPz
[V]
with little disturbance
0.2
0.4 CPz
FCz
Oz
U n d e r s t a n d i n g S p a t i a l F i l t e r s
( a )
i h li l di b
( b )
ith di t b f O
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
73/106
10 5 0 5 108
6
4
2
0
2
4
6
8
10
potential at FCz [V]
potentialatCPz
[V]
with little disturbance
10 5 0 5 108
6
4
2
0
2
4
6
8
10
potential at FCz [V]
potentialatCPz
[V]
with disturbance from Oz
T w o c h a n n e l c l a s s i c a t i o n o f ( a ) : 1 5 % e r r o r , ( b ) : 3 7 % e r r o r
W h e n d i s t u r b i n g c h a n n e l O z i s a d d e d t o t h e d a t a ( 3 D ) : 1 6 % e r r o r .
H e r e , c h a n n e l O z i s r e q u i r e d f o r g o o d c l a s s i c a t i o n a l t h o u g h i t s e l f i s
n o t d i s c r i m i n a t i v e .
I m p a c t o f S h r i n k a g e o n t h e S p a t i a l F i l t e r s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
74/106
W i t h i n c r e a s i n g s h r i n k a g e , t h e s p a t i a l l t e r s ( c l a s s i e r ) l o o k
s m o o t h e r , b u t c l a s s i c a t i o n m a y d e g r a d e w i t h t o o m u c h s h r i n k a g e .
= 0 = 0.001 = 0.005 = 0.05 = 0.5 = 1
15
20
25
30
35
40
er
ror[%]
M a p s o f s p a t i a l l t e r s f o r d i e r e n t v a l u e s o f .
A N o v e l A n a l y t i c a l M e t h o d
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
75/106
R e c e n t l y , a m e t h o d t o a n a l y t i c a l l y c a l c u l a t e t h e o p t i m a l s h r i n k a g e
p a r a m e t e r w a s p u b l i s h e d ( [ 1 ] ) .
T h a n k s t o N i c o l e K r m e r f o r p o i n t i n g t h e B B C I g r o u p t o t h i s
m e t h o d .
O p t i m a l S e l e c t i o n o f S h r i n k a g e P a r a m e t e r
L e t x1, . . . ,xn Rd b e n f e a t u r e v e c t o r s a n d l e t = 1nn
k=1 xk
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
76/106
b e t h e e m p i r i c a l m e a n .
A i m : g e t a b e t t e r e s t i m a t e o f t h e t r u e c o v a r i a n c e m a t r i x
( e s p e c i a l l y i n c a s e n < d ) t h a n t h e s a m p l e c o v a r i a n c e m a t r i x = 1
n1
nk=1(xk )(xk ) b y s e l e c t i n g a i n
() := (1 )+ I.
O p t i m a l S e l e c t i o n o f S h r i n k a g e P a r a m e t e r
L e t x1, . . . ,xn Rd b e n f e a t u r e v e c t o r s a n d l e t = 1nn
k=1 xk
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
77/106
b e t h e e m p i r i c a l m e a n .
A i m : g e t a b e t t e r e s t i m a t e o f t h e t r u e c o v a r i a n c e m a t r i x
( e s p e c i a l l y i n c a s e n < d ) t h a n t h e s a m p l e c o v a r i a n c e m a t r i x = 1
n1
nk=1(xk )(xk ) b y s e l e c t i n g a i n
() := (1 )+ I.W e d e n o t e b y (xk)i r e s p . ()i t h e i - t h e l e m e n t o f t h e v e c t o r xkr e s p .
. F u r t h e r m o r e w e d e n o t e b y
sij t h e e l e m e n t i n t h e i- t h r o w
a n d j - t h c o l u m n o f . W e d e n e
zij(k) = ((xk)i ()i) ((xk)j ()j)T h e n t h e o p t i m a l s h r i n k a g e p a r a m e t e r f o r w h i c h () = argminS||S||2F c a n b e a n a l y t i c a l l y c a l c u l a t e d ( [ 2 ] ) a s
=n
(n 1)2
di,j=1 vark(zij(k))
i=j s2ij +
i(sii )2
R e s u l t o f C l a s s i c a t i o n w i t h S h r i n k a g e
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
78/106
5
0
5
vP1 N1 vN1 P250 vN2 P400 N500
115135 ms 135155 ms 155195 ms 205235 ms 285325 ms 335395 ms 495535 ms
10
15
20
25
30
35
error[%]
Combined
LDA
with
shrinkage 5%
U s i n g s h r i n k a g e t h e c l a s s i c a t i o n e r r o r c o u l d b e d r a s t i c a l l y r e d u c e d
t o 4 % .
R e s u l t s f o r t h e C l a s s i c M a t r i x S p e l l e r
100bae
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
79/106
1 2 3 4 5 6 7 8 9 100
20
40
60
80
acc
uracy[%]
intensification [#]
sab
sac
sad
sae
saf
sag
sah
saisaj
sak
daj
sal
mean
A c c u r a c y i n l e t t e r s e l e c t i o n , c h a n c e l e v e l : 3 . 3 3 % .
R e s u l t s f o r t h e C l a s s i c M a t r i x S p e l l e r
100bae
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
80/106
1 2 3 4 5 6 7 8 9 100
20
40
60
80
acc
uracy[%]
intensification [#]
sab
sac
sad
sae
saf
sag
sah
saisaj
sak
daj
sal
median
S e e P o s t e r W 0 7 ( T r e d e r e t a l ) , a n d f o r o t h e r a p p l i c a t i o n s o f t h i s
t e c h n i q u e A 0 2 ( T h u r l i n g s e t a l ) , W 1 0 ( H h n e e t a l ) .
S u m m a r y o f S p a t i o - T e m p o r a l C l a s s i c a t i o n
L i n e a r c l a s s i c a t i o n w i t h s h r i n k a g e i s a p o w e r f u l m e t h o d .
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
81/106
C o m p l e t e s h r i n k a g e ( = 1) m e a n s n e g l e c t i n g t h e s t r u c t u r e o f
t h e n o i s e . I n t h i s c a s e t h e c l a s s i e r i s t h e d i e r e n c e o f t h e
E R P s .
T h e a p p r o p r i a t e n e s s o f a l i n e a r s e p a r a t i o n d e p e n d s o n t h e w a y
f e a t u r e s a r e e x t r a c t e d a n d t r a n s f o r m e d .
I n c o n t r a s t t o n o n - l i n e a r c l a s s i e r s , t h e w e i g h t s o f a l i n e a r
c l a s s i e r a r e i n f o r m a t i v e .
T h e w e i g h t s o f t h e t r a i n e d c l a s s i e r c a n b e v i s u a l i z e d a s a s e q u e n c e
o f s c a l p t o p o g r a p h i e s :
[250 ms] [150 ms] [50 ms] [50 ms] [150 ms] [250 ms]
0.02
0
0.02
T o p i c o f T h i s S e c t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
82/106
M e t h o d :
C l a s s i c a t i o n o f s p e c t r a l f e a t u r e s , n a m e l y m o d u l a t i o n s o f t h e
a m p l i t u d e i n s p e c i c f r e q u e n c y b a n d s .
I n p a r t i c u l a r , C o m m o n S p a t i a l P a t t e r n ( C S P ) a n a l y s i s t o
c l a s s i f y d i e r e n t c o n d i t i o n s t h a t a r e c h a r a c t e r i z e d b y a
m o d u l a t i o n o f t h e a m p l i t u d e o f b r a i n r h y t h m s ( [ 3 , 4 ] ) .
A p p l i c a t i o n :
C l a s s i c a t i o n o f m o t o r i m a g e r y c o n d i t i o n s i n a B C I p a r a d i g m .
N e u r o p h y s i o l o g y : S e n s o r i m o t o r R h y t h m s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
83/106
right hemisphereleft hemisphere
right
hand hand
left
20 30 4010 50SpectralPo
wer[dB]
Frequency [Hz]
~ 1/f
Spectrum over left hand area
hand at rest
C3 C4Cz
N e u r o p h y s i o l o g y : S e n s o r i m o t o r R h y t h m s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
84/106
right hemisphereleft hemisphere
right
hand hand
left
20 30 4010 50
Spectrum over left hand area
imagined hand movement
SpectralPo
wer[dB]
Frequency [Hz]
~ 1/f
C3 C4Cz
I m a g i n i n g a m o v e m e n t o f a l i m b c a u s e s a l o c a l b l o c k i n g o f t h e
c o r r e s p o n d i n g s e n s o r i m o t o r r h y t h m ( S M R ) , s e e [ 5 , 6 , 7 ] .
A v e r a g e T o p o g r a p h y o f I d l e S M R
F o r e a c h L a p l a c e l t e r e d c h a n n e l i n a r e l a x r e c o r d i n g , t h e s t r e n g t h
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
85/106
o f t h e l o c a l r h y t h m w a s e s t i m a t e d . T h e g r a n d a v e r a g e o v e r 8 0
p a r t i c i p a n t s i s d i s p l a y e d a s t o p o g r a p h i c m a p p i n g :
[dB]
3.5
4
4.5
5
5.5
6
6.5
C o n c l u s i o n
L o c a t i o n s C 3 a n d C 4 a r e g o o d c a n d i d a t e s t o o b s e r v e S M R
m o d u l a t i o n s . T h e s e c o v e r t h e s e n s o r i m o t o r a r e a s o f t h e r i g h t a n d
t h e l e f t h a n d .
S p a t i a l S m e a r i n g
R a w E E G s c a l p p o t e n t i a l s a r e k n o w n t o b e a s s o c i a t e d w i t h a
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
86/106
l a r g e s p a t i a l s c a l e o w i n g t o v o l u m n e c o n d u c t i o n .
I n a s i m u l a t i o n o f N u n e z e t a l [ 8 ] o n l y h a l f t h e c o n t r i b u t i o n t o
o n e s c a l p e l e c t r o d e c o m e s f r o m s o u r c e s w i t h i n a 3 c m r a d i u s .
0
0.1667
0.3333
0.5
0.6667
0.8333
1
T h e N e e d f o r S p a t i a l F i l t e r i n g
raw: CP4 bipolar: FC4CP4 CAR: CP4
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
87/106
5 10 15 20 25 30
15
20
25
30
[Hz]
[dB]
5 10 15 20 25 30
10
15
20
25
[Hz]
[dB]
5 10 15 20 25 30
10
15
20
25
[Hz]
[dB]
5 10 15 20 25 30
0
5
10
15
[Hz]
[dB]
Laplace: CP4
5 10 15 20 25 30
0
5
10
[Hz]
[dB]
CSP
leftright
0
0.2
0.4
0.6r2
A n a l y s i s o f M o t o r I m a g e r y C o n d i t i o n s : S p e c t r a
VPk 08 08 07/i VPk l ft / i ht N 66/63 [5 40] H [ 15 25] dB
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
88/106
left
right
10 dB+
10 Hz
P6 larP4 lar
10 20 30 40
Pz larP3 larP5 lar
CP4 larCP2 larCP1 larCP3 lar
C4 larC2 larCz larC1 larC3 lar
F4 larFz larF3 lar
VPkg_08_08_07/imag_arrowVPkg: left / right, N= 66/63, [5 40] Hz [15 25] dB
0.2
0
0.2
sgn r2
F i r s t s t e p : d e t e r m i n e a s u i t a b l e f r e q u e n c y b a n d t h a t s h o w s g o o d
d i s c r i m i n a t i o n b e t w e e n t h e c o n d i t i o n s .
E R D C u r v e s o f M o t o r I m a g e r y
F4 larFz larF3 lar
VPkg_08_08_07/imag_arrowVPkg: left / right, N= 66/63, [500 6000] ms [2 1] V
sgn r2
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
89/106
left
right1 V
+
2000 ms
P6 larP4 larPz larP3 larP5 lar
CP4 larCP2 larCP1 larCP3 lar
C4 larC2 larCz larC1 larC3 lar
0.20.10
0.10.2
sgn r
S e c o n d s t e p : d e t e r m i n e a s u i t a b l e t i m e i n t e r v a l d u r i n g w h i c h
d i s c r i m i n a t i o n i s m o s t p r o m i n e n t .
R e m a r k : S i m u l t a n e o u s s e l e c t i o n o f f r e q u e n c y b a n d a n d i n t e r v a l i s
m o r e a p p r o p r i a t e .
C o m m o n S p a t i a l P a t t e r n ( C S P ) A n a l y s i s
G o a l : F i n d s p a t i a l l t e r s t h a t o p t i m a l l y c a p t u r e m o d u l a t i o n s o f
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
90/106
b r a i n r h y t h m s
O b s e r v a t i o n : p o w e r o f a b r a i n r h y t h m v a r i a n c e o f b a n d - p a s s l t e r e d s i g n a l .
min variancefor
min variancefor
unknownsources
discriminativesignals
V1
left
right
EEGno classspecificinfluenceon variance
csp:L1,2,...
csp:R1,2,...
V
projection filter
forward
model model
backward
observedsignals
C S P A n a l y s i s
T h e g o a l o f C S P :
right left right
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
91/106
2425 2430 2435 [s]
csp:R
csp:L
right left right
C S P a n a l y s i s y i e l d s s p a t i a l l t e r s t h a t c a n b e v i s u a l i z e d :
CSP
filter
right
CSP
filter
left
C S P M o r e P r a c t i c a l
E E G - s i g n a l s d u r i n g m o t o r i m a g e r y , b a n d - p a s s l t e r e d ( h e r e 9 1 3 H z ) :
left rightright
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
92/106
C4
C3
728 732 734 736 738730
left rightright
[s]
L
:= XL
XL
R
:= XR
XR
X L
( T
c h a n s ) ( T
c h a n s )
X R
r i g h t d a t a l e f t d a t a
VL
V = D & V(L
+R
)V = I
1 ) c h o o s e e i g e n v e c t o r vi f r o m V h a v i n g a l a r g e e i g e n v a l u e di w . r . t . L .
CSP
728 730 732 734 736 738 [s]
right rightleft
var(XL
vi) = dil a r g e
var(XR
vi) = 1di s m a l l
I n M a t l a b : [ V , D ] = e i g ( S i g m a 1 , S i g m a 1 + S i g m a 2 ) .
C S P M o r e P r a c t i c a l
E E G - s i g n a l s d u r i n g m o t o r i m a g e r y , b a n d - p a s s l t e r e d ( h e r e 9 1 3 H z ) :
left rightright
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
93/106
C4
C3
728 732 734 736 738730
left rightright
[s]
L
:= XL
XL
R
:= XR
XR
XL
( T
c h a n s ) ( T
c h a n s )
XR
r i g h t d a t a l e f t d a t a
VL
V = D & V(L
+R
)V = I
2 ) c h o o s e e i g e n v e c t o r vi f r o m V h a v i n g a s m a l l e i g e n v a l u e di w . r . t . L .
CSP
728 730 732 734 736 738 [s]
left right right
var(XL
vi) = dis m a l l
var(XR
vi) = 1di l a r g e
I n M a t l a b : [ V , D ] = e i g ( S i g m a 1 , S i g m a 1 + S i g m a 2 ) .
T r a i n i n g C S P - b a s e d C l a s s i c a t i o n
T o o b t a i n f e a t u r e s f r o m t h e C S P l t e r e d E E G , i n e a c h c h a n n e l a n d
t r i a l , t h e v a r i a n c e a c r o s s t i m e i s c a l c u l a t e d a n d t h e l o g a r i t h m i s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
94/106
a p p l i e d . T h i s i s a s c a t t e r p l o t o f t h e r e s u l t i n g C S P f e a t u r e s :
3 2 1 0 1
3
2
1
0
1
log(var(CSPL))
log(var(CSP
R))
leftright
H e r e , o n l y t w o d i m e n s i o n s a r e s h o w n . N o t e , t h a t a p p l y i n g t h e
l o g a r i t h m t o t h e b a n d p o w e r f e a t u r e s m a k e s t h e d i s t r i b u t i o n m o r e
G a u s s i a n a n d t h e r e f o r e e n h a n c e s l i n e a r s e p a r a b i l i t y .
T r a i n i n g C S P - b a s e d C l a s s i c a t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
95/106
D e t e r m i n e m o s t d i s c r i m i n a t i v e f r e q u e n c y b a n d ,
b a n d - p a s s l t e r E E G i n t h a t b a n d ,
e x t r a c t s i n g l e t r i a l s u s i n g a n a p p r o p r i a t e t i m e i n t e r v a l ,
c a l c u l a t e a n d s e l e c t C S P l t e r s ,
a n d a p p l y t h e m t o E E G s i n g l e t r i a l s ,
c a l c u l a t e t h e l o g v a r i a n c e w i t h i n t r i a l s .
T h i s r e s u l t s i n a l o w d i m e n s i o n a l f e a t u r e v e c t o r f o r e a c h t r i a l
( d i m e n s i o n a l i t y e q u a l s n u m b e r o f s e l e c t e d C S P l t e r s ) .
T r a i n a l i n e a r c l a s s i e r l i k e L D A o n t h e f e a t u r e s .
( S i n c e t h e s e f e a t u r e s a r e l o w - d i m e n s i o n a l , s h r i n k a g e i s t y p i c a l l y
n o t n e c e s s a r y . )
S u m m a r y : T r a i n i n g C S P - b a s e d C l a s s i c a t i o n
12
C4 lap
0.4
C4 lap
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
96/106
( 1 )
5 10 15 20 25 30 35 40 45
2
4
6
8
10
12
[Hz]
[dB]
left
right
( 2 )
0 1000 2000 3000 4000 5000
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
[ms]
[V]
left
right
( 4 )
3 2 1 0 1
3
2
1
0
1
log(var(CSPL))
log(var(CSPR
)
)
leftright
( 3 )
csp1 [0.68] csp2 [0.64] csp3 [0.60] csp4 [0.71] csp5 [0.68] csp6 [0.61]
28.2 17.0 39.3 15.5
A p p l y i n g C S P - b a s e d C l a s s i c a t i o n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
97/106
P r o j e c t E E G w i t h s p a t i a l C S P l t e r s a n d a p p l y b a n d - p a s s l t e r ,
c a l c u l a t e t h e v a r i a n c e i n s h o r t w i n d o w s ( e . g . l a s t 5 0 0 m s ) ,
t a k e t h e l o g a r i t h m ,
a n d a p p l y t h e c l a s s i e r w e i g h t i n g .
R e m a r k : O n e n i c e f e a t u r e o f C S P i s t h a t t h e l e n g t h o f t h e
c l a s s i c a t i o n w i n d o w c a n b e c h a n g e d a t r u n t i m e ( i . e . d u r i n g
f e e d b a c k ) .
F o r m o r e t h e o r e t i c a l c o n s i d e r a t i o n s a s w e l l a s p r a c t i c a l h i n t s s e e [ 3 ] .
S e c t i o n : C a v e a t s i n V a l i d a t i o n
W h e n m a c h i n e l e a r n i n g t e c h n i q u e s a r e u s e d f o r c l a s s i c a t i o n o f
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
98/106
E E G s i n g l e - t r i a l s , t h e e x p e c t e d p e r f o r m a n c e o f a m e t h o d h a s t o b e
e v a l u a t e d c a r e f u l l y , a n d t h e r e a r e s e v e r a l p o s s i b l e p i t f a l l s .
T h e e s t i m a t i o n o f g e n e r a l i z a t i o n p e r f o r m a n c e r e q u i r e s a t r a i n i n g
a n d a t e s t s e t . T h e e s t i m a t i o n i s o n l y p r o p e r
i f t h e t e s t s e t w a s n o t u s e d i n a n y w a y t o d e t e r m i n e
p a r a m e t e r s o f t h e m e t h o d , a n d
i f t h e s a m p l e s i n t h e t e s t s e t a r e i n d e p e n d e n t f r o m t h e s a m p l e s
i n t h e t r a i n i n g s e t .
A l t h o u g h t h e s e p r i n c i p l e s a r e q u i t e o b v i o u s , i t h a p p e n s t h a t t h e y
a r e v i o l a t e d .
U n f o r t u n a t e l y , e v e n s o m e p u b l i s h e d j o u r n a l a r t i c l e s l a c k a p r o p e r
v a l i d a t i o n o f t h e p r o p o s e d m e t h o d s .
H a l l o f P i t f a l l s i n S i n g l e - T r i a l E E G A n a l y s i s
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
99/106
p r e p r o c e s s i n g m e t h o d s t h a t u s e s t a t i s t i c s o f t h e w h o l e d a t a s e t
l i k e I C A , o r n o r m a l i z a t i o n o f f e a t u r e s
( p a r t i c u l a r l y s e v e r e f o r m e t h o d s t h a t u s e l a b e l i n f o r m a t i o n )
f e a t u r e s a r e s e l e c t e d o n t h e w h o l e d a t a s e t , i n c l u d i n g t r i a l s
t h a t a r e l a t e r i n t h e t e s t s e t
s e l e c t p a r a m e t e r s b y c r o s s v a l i d a t i o n o n t h e w h o l e d a t a s e t
a n d r e p o r t t h e p e r f o r m a n c e f o r t h e s e l e c t e d v a l u e s
a r t i f a c t s / o u t l i e r s a r e r e j e c t e d f r o m t h e w h o l e d a t a s e t
( r e s u l t i n g i n a s i m p l i e d t e s t s e t )
u n s u c i e n t v a l i d a t i o n f o r p a r a d i g m s w i t h b l o c k d e s i g n
I n t h i s p r e s e n t a t i o n w e h i g h l i g h t t h e l a s t i s s u e .
B l o c k D e s i g n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
100/106
A s s u m e t h e t a s k i s t o d i s c r i m i n a t e b e t w e e n m e n t a l s t a t e s i n
d i e r e n t c o n d i t i o n s .
W e s a y t h a t a n e x p e r i m e n t h a s a b l o c k d e s i g n , i f t h e p e r i o d s f o r
w h i c h t h e r e i s n o a l t e r n a t i o n b e t w e e n c o n d i t i o n s a r e l o n g e r t h a n
t h e i n t e n d e d c h a n g e o f s t a t e s i n o n l i n e o p e r a t i o n .
cond #1 cond #1cond #2 cond #2cond #1 cond #2 cond #1
block #1 block #2 block #3 block #4 block #5 block #6 block #7
A p r o b l e m a r i s e s , i f t h e p e r f o r m a n c e i s e s t i m a t e d f o r s u c h a d a t a
s e t b y c r o s s v a l i d a t i o n .
S l o w l y C h a n g i n g V a r i a b l e s
Training set
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
101/106
Test set
I n E E G t h e r e a r e m a n y s l o w l y c h a n g i n g v a r i a b l e s o f b a c k g r o u n d
a c t i v i t y , t h e r e f o r e t h e s i n g l e - t r i a l s a r e n o t i n d e p e n d e n t . F o r a n
o r d i n a r y c r o s s v a l i d a t i o n i n a b l o c k d e s i g n d a t a s e t , t h e r e q u i r e m e n t
o f i n d e p e n d e n c e b e t w e e n t r a i n i n g a n d t e s t s e t i s v i o l a t e d .
A V a l i d a t i o n T e s t
T o d e m o n s t r a t e i m p a c t o f b l o c k d e s i g n i n c r o s s v a l i d a t i o n , w e
p e r f o r m c r o s s v a l i d a t i o n i n t h e f o l l o w i n g s e t t i n g . T a k i n g a n
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
102/106
a r b i t r a r y E E G d a t a s e t , w e a s s i g n f a k e l a b e l s ( r e g a r d l e s s o f w h a t
h a p p e n e d d u r i n g t h e r e c o r d i n g ) l i k e t h i s :
n B l o c k s P e r C l a s s = 1 :
class #2class #1
n B l o c k s P e r C l a s s = 2 :
class #1 class #2class #1class #2
n B l o c k s P e r C l a s s = 3 :
class #1 class #2 class #1 class #2class #2 class #1
a n d s o o n .
R e s u l t s o f t h e V a l i d a t i o n T e s t
F r o m e a c h b l o c k s i n g l e - t r i a l s a r e e x t r a c t e d o f l e n g t h 1 s . T h i s
p r o c e d u r e w a s p e r f o r m e d f o r 8 0 E E G d a t a s e t s . B l u e b o x p l o t s s h o w
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
103/106
t h e r e s u l t s o f c r o s s - v a l i d a t i o n :
1 2 3 4 5 10
0
10
20
30
40
50
60
70
testerror
[%]
number of blocks per class
R e s u l t s o f t h e V a l i d a t i o n T e s t
F r o m e a c h b l o c k s i n g l e - t r i a l s a r e e x t r a c t e d o f l e n g t h 1 s . T h i s
p r o c e d u r e w a s p e r f o r m e d f o r 8 0 E E G d a t a s e t s . B l u e b o x p l o t s s h o w
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
104/106
t h e r e s u l t s o f c r o s s - v a l i d a t i o n :
1 2 3 4 5 10
0
10
20
30
40
50
60
70
testerror
[%]
number of blocks per class
F o r c o m p a r i s o n , r e s u l t s f o r l e a v e - o n e - b l o c k - o u t v a l i d a t i o n a r e
s h o w n i n g r e e n .
F u r t h e r C o m m e n t s a n d S u m m a r y
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
105/106
T h e s e v e r e n e s s o f t h e u n d e r e s t i m a t i o n o f t h e t r u e e r r o r
d e p e n d s o n t h e c o m p l e x i t y o f t h e f e a t u r e s a n d t h e c l a s s i e r .
C r o s s v a l i d a t i o n i n b l o c k d e s i g n d a t a m i g h t a l s o g i v e t h e
c o r r e c t r e s u l t b u t a l t e r n a t i v e e v a l u a t i o n i s r e q u i r e d .
T h e s i t u a t i o n g e t s w o r s e i f t r i a l s a r e e x t r a c t e d f r o m
o v e r l a p p i n g s e g m e n t s .
T h e m o s t r e a l i s t i c v a l i d a t i o n i s t o t r a i n t h e m e t h o d s o n t h e
r s t N 1 r u n s a n d t o e v a l u a t e o n t h e l a s t r u n . L e a v e - o n e - b l o c k - o u t a n d l e a v e - o n e - r u n - o u t h a v e l a r g e r
s t a n d a r d e r r o r s t h a n c r o s s v a l i d a t i o n .
R e f e r e n c e s
O . L e d o i t a n d M . W o l f , A w e l l - c o n d i t i o n e d e s t i m a t o r f o r l a r g e - d i m e n s i o n a l c o v a r i a n c e m a t r i c e s ,
J M u l t i v a r A n a l , 8 8 ( 2 ) : 3 6 5 4 1 1 , 2 0 0 4 .
J . S c h f e r a n d K . S t r i m m e r , A S h r i n k a g e A p p r o a c h t o L a r g e - S c a l e C o v a r i a n c e M a t r i x E s t i m a t i o n
a n d I m p l i c a t i o n s f o r F u n c t i o n a l G e n o m i c s , S t a t i s t i c a l A p p l i c a t i o n s i n G e n e t i c s a n d M o l e c u l a r
8/2/2019 Tutorial_Machine Learning and Signal Processing Tools for BCI
106/106
B i o l o g y , 4 ( 1 ) , 2 0 0 5 .
B . B l a n k e r t z , R . T o m i o k a , S . L e m m , M . K a w a n a b e , a n d K . - R . M l l e r , O p t i m i z i n g S p a t i a l F i l t e r s
f o r R o b u s t E E G S i n g l e - T r i a l A n a l y s i s , I E E E S i g n a l P r o c M a g a z i n e , 2 5 ( 1 ) : 4 1 5 6 , 2 0 0 8 , U R L
h t t p : / / d x . d o i . o r g / 1 0 . 1 1 0 9 / M S P . 2 0 0 8 . 4 4 0 8 4 4 1 .
H . R a m o s e r , J . M l l e r - G e r k i n g , a n d G . P f u r t s c h e l l e r , O p t i m a l s p a t i a l l t e r i n g o f s i n g l e t r i a l E E G
d u r i n g i m a g i n e d h a n d m o v e m e n t , I E E E T r a n s R e h a b E n g , 8 ( 4 ) : 4 4 1 4 4 6 , 2 0 0 0 .
C . N e u p e r a n d W . K l i m e s c h , e d s . , E v e n t - r e l a t e d D y n a m i c s o f B r a i n O s c i l l a t i o n s , E l s e v i e r , 2 0 0 6 .
G . P f u r t s c h e l l e r , C . B r u n n e r , A . S c h l g l , a n d F . L . d a S i l v a , M u r h y t h m ( d e ) s y n c h r o n i z a t i o n a n d
E E G s i n g l e - t r i a l c l a s s i c a t i o n o f d i e r e n t m o t o r i m a g e r y t a s k s , N e u r o I m a g e , 3 1 ( 1 ) : 1 5 3 1 5 9 ,
2 0 0 6 .
G . P f u r t s c h e l l e r a n d F . L o p e s d a S i l v a , E v e n t - r e l a t e d E E G / M E G s y n c h r o n i z a t i o n a n d
d e s y n c h r o n i z a t i o n : b a s i c p r i n c i p l e s , C l i n N e u r o p h y s i o l , 1 1 0 ( 1 1 ) : 1 8 4 2 1 8 5 7 , 1 9 9 9 .
P . L . N u n e z , R . S r i n i v a s a n , A . F . W e s t d o r p , R . S . W i j e s i n g h e , D . M . T u c k e r , R . B . S i l b e r s t e i n ,
a n d P . J . C a d u s c h , E E G c o h e r e n c y I : s t a t i s t i c s , r e f e r e n c e e l e c t r o d e , v o l u m e c o n d u c t i o n ,
L a p l a c i a n s , c o r t i c a l i m a g i n g , a n d i n t e r p r e t a t i o n a t m u l t i p l e s c a l e s , E l e c t r o e n c e p h a l o g r C l i n
N e u r o p h y s i o l , 1 0 3 ( 5 ) : 4 9 9 5 1 5 , 1 9 9 7 .
http://dx.doi.org/10.1109/MSP.2008.4408441Top Related