Dynamics of Attention Shifting
Transcript of Dynamics of Attention Shifting
![Page 1: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/1.jpg)
These slides are at:
ftp://sccn.ucsd.edu/pub/bcilab/mallorca
![Page 2: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/2.jpg)
BCILAB Briefing
http://sccn.ucsd.edu/wiki/BCILAB
![Page 3: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/3.jpg)
Idea & Purpose
• Like EEGLAB, but for BCI (and/or cognitive state assessment)
– Seeding a community
– Strengthening links between BCI and Neuroscience
• SCCN’s in-house tool for BCI problems
– Main focus: Advanced cognitive monitoring
– Part of a large US research program (CaN CTA)
– Funded by ARL (and ONR, Swartz Foundation, …)
![Page 4: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/4.jpg)
BCILAB Specialty
• State of the art • Largest collection of machine learning & signal processing
components in any open-source BCI package – Many standard components (CSP, LDA, SVM, …) – Many modern components (SBL, SSA, AMICA, HKL, DPGMM, LR-
DAL, …) – Some novel components (OSR, RSSD, SSB, …)
• Next-generation framework – Fully probabilistic – Model inference from data corpora* – Anatomical priors, other neuroscience-aware features – Processing of parallel streams
(*: not yet in the current release)
![Page 5: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/5.jpg)
BCILAB Components
Dependencies CVX BNT GUI utils
Driver I/O
EEGLAB LIBSVM GLMNET …
Infrastructure GUI
generation cluster
computing disk
caching helper
functions environment
services
Signal Processing Machine Learning BCI Paradigms Devices
Plugins
ICA SSA FIR
IIR FFT …
LDA QDA
GMM SVM …
DAL CSP Spec-CSP
ERP RSSD …
TCP
BCI2000 …
OSC
Framework
Approach Definition
Offline Evaluation
Visualization Online
Execution
GUI / Scripting Interfaces
![Page 6: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/6.jpg)
BCI Behavior
• BCIs in BCILAB are acting as an oracle that consumes one or more biosignals and can respond to (pre-defined) queries about cognitive state
BCI
EEG
EMG
Q?
A!
![Page 7: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/7.jpg)
Signal Processing?
• Some signal-level computations can be done more efficiently than window-by-window (esp. when successive windows overlap a lot)
![Page 8: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/8.jpg)
Signal Processing?
• Some signal-level computations can be done more efficiently than window-by-window (esp. when successive windows overlap a lot)
• Room for good DSP use (e.g., frequency filter, spatial filter, …) before actual prediction
• Also, can assemble approaches from existing components
Inference/
Estimation
Biosignal Predictions Signal
Processing
![Page 9: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/9.jpg)
Online Data Flow
• A filter graph receives all input samples and produces pre-filtered data (signal flows through it)
• The prediction function may be queried on demand on the filter graph’s outputs
Filter Filter
Filter
Filter
Filter Graph
Prediction Function
Extract Features
EEG
EMG
filtered data
Pre-dict
tPred
![Page 10: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/10.jpg)
BCI Models
• BCIs are described by “BCI models” that specify both the filter graph and the prediction function (incl. parameters)
Filter Filter
Filter
Filter
Filter Graph Prediction Function
…
BCI Model
![Page 11: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/11.jpg)
BCI Paradigms
• BCI paradigms are the coarsest plugin type in BCILAB and tie all parts of a BCI approach together
• They are seeds for new BCI designs and cornerstones of BCILAB usage
Calibration recording
Calibrate
Filter Graph Pre-
dict
BCI Model
![Page 12: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/12.jpg)
Offline Evaluation
• Given calibration data
• Estimate model parameters (spatial filters, statistics)
• Apply the model to new data (online / single-trial)
• Optionally: compare outputs with known state, compute loss statistics for the model / approach (e.g., mis-classification rate)
Model
Calibration recording Future data…
![Page 13: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/13.jpg)
Offline Evaluation
• Evaluation of computational approaches on a single data set?
Calibration recording
?
![Page 14: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/14.jpg)
Offline Evaluation
• Evaluation of computational approaches on a single data set?
– Can not test on the training data! (always on separate data)
– Instead can split data set repeatedly into training/test blocks systematically, a.k.a. cross-validation
Training part
Test part
Model
![Page 15: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/15.jpg)
Resolving Free Parameters
• Can be done using cross-validation in a grid search (try all values of free parameters)
• Caveat: Resulting “optimal” numbers are non-reportable (cherry-picked!)
Best Model
Training Test
For all param. values…
![Page 16: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/16.jpg)
Resolving Free Parameters
• Can be done using cross-validation in a grid search (try all values of free parameters)
• Caveat: Resulting “optimal” numbers are non-reportable (cherry-picked!)
• But may test resulting best model on separate data
Best Model
Training Test
For all param. values… Future data…
![Page 17: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/17.jpg)
Resolving Free Parameters
• Can be done using cross-validation in a grid search (try all values of free parameters)
• Caveat: Resulting “optimal” numbers are non-reportable (cherry-picked!)
• But may test resulting best model on separate data
• Or run grid search within an outer cross-validation (“nested cross-validation”)
Test part
Best Model
Training Test
For all param. values…
![Page 18: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/18.jpg)
System requirements
• MATLAB 2008a
• 1GB+ RAM (better: 2GB+)
• Windows, Linux, or Mac
• For smooth workshop: No toolboxes in MATLAB path other
than Mathworks toolboxes (or EEGLAB)
• To use certain additional features (not covered today):
Signal Processing Toolbox, Statistics Toolbox, Real-time
experimentation environment (DataRiver, BCI2000,
OpenViBE or your own)
• To use certain advanced features (also not covered today):
Correct MEX compiler setting (this requires Microsoft Visual
C++ Express under Win64 and Xcode/gcc under Mac)
![Page 19: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/19.jpg)
When Processing your own Data
• Note the following requirements:
– You need proper channel labels (usually the 10-20 labels); 3d locations not necessary
– You need event markers in your data for the time points with known target condition
– BCILAB needs raw (unprocessed) data
– Make sure you have a file format supported by EEGLAB
Rawr!
![Page 20: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/20.jpg)
Clearing the Path
![Page 21: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/21.jpg)
Clearing the Path
![Page 22: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/22.jpg)
Clearing the Path
![Page 23: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/23.jpg)
Clearing the Path
![Page 24: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/24.jpg)
Clearing the Path
![Page 25: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/25.jpg)
Clearing the Path
![Page 26: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/26.jpg)
Starting the Toolbox
• Type: cd C:\your\path\to\bcilab; bcilab
![Page 27: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/27.jpg)
Starting the Toolbox
• Type: cd C:\your\path\to\bcilab; bcilab
![Page 28: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/28.jpg)
Starting the Toolbox
• You should see the welcome message
![Page 29: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/29.jpg)
Starting the Toolbox
• … and the main menu
![Page 30: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/30.jpg)
The Data
• Recorded at the famous BCI Lab in Graz, Austria
• Part of the BCI competition IV (“dataset 2a”)
• More info: http://www.bbci.de/competition/iv/desc_2a.pdf
• Contains imagined movements of 4 classes: – Left hand
– Right hand
– Feet
– Tongue
• 2 Training sets, 2 evaluation sets
![Page 31: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/31.jpg)
The Data
• Timing overview
![Page 32: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/32.jpg)
Fixing the Data
• Execute the following line:
• You can copy & paste this from the .pdf:
pop_saveset(rmfield(exp_eval(io_loadset('data:/workshop/A01T.set')),'tracking'),'filepath','A01T_fixed.set')
![Page 33: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/33.jpg)
Fixing the Data
• … and the same for the three other data sets:
pop_saveset(rmfield(exp_eval(io_loadset('data:/workshop/A01E.set')),'tracking'),'filepath','A01E_fixed.set')
pop_saveset(rmfield(exp_eval(io_loadset('data:/workshop/A03T.set')),'tracking'),'filepath','A03T_fixed.set')
pop_saveset(rmfield(exp_eval(io_loadset('data:/workshop/A03E.set')),'tracking'),'filepath','A03E_fixed.set')
![Page 34: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/34.jpg)
Getting help (if needed)
![Page 35: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/35.jpg)
Getting help (if needed)
![Page 36: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/36.jpg)
Getting help (if needed)
![Page 37: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/37.jpg)
Loading the Data
![Page 38: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/38.jpg)
Loading the Data
![Page 39: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/39.jpg)
Loading the Data
![Page 40: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/40.jpg)
Loading the Data
• Please load A01T_fixed.set (training set #1)
![Page 41: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/41.jpg)
Defining a new Approach
![Page 42: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/42.jpg)
Defining a new Approach
• Let’s start with Common Spatial Patterns:
![Page 43: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/43.jpg)
Defining a new Approach
![Page 44: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/44.jpg)
Adapting an Approach
• Marker names are {‘769’,’770’} in this set:
![Page 45: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/45.jpg)
Further Editing an Approach
![Page 46: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/46.jpg)
Saving the Approach
![Page 47: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/47.jpg)
See also: MATLAB’s Workspace
• Can be enabled via “Desktop” menu
![Page 48: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/48.jpg)
Now Hitting the Data…
![Page 49: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/49.jpg)
Now Hitting the Data…
![Page 50: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/50.jpg)
Watching the Computation…
![Page 51: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/51.jpg)
Cross-validation Results:
![Page 52: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/52.jpg)
Visualizing a Trained Model
![Page 53: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/53.jpg)
Visualizing a Trained Model
Left hand motor cortex
Right hand motor cortex
![Page 54: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/54.jpg)
Applying a Model to New Data
![Page 55: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/55.jpg)
Applying a Model to New Data
![Page 56: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/56.jpg)
Results: Training-Set Error
![Page 57: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/57.jpg)
Online Processing
• For lack of EEG hardware, play back a data set in real time:
![Page 58: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/58.jpg)
Online Processing
• For lack of EEG hardware, play back a data set in real time:
![Page 59: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/59.jpg)
Online Processing
• Choosing a destination for outputs:
![Page 60: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/60.jpg)
Setting up a Visualization
![Page 61: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/61.jpg)
Real-time Class Probabilities
![Page 62: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/62.jpg)
Clearing the Online Processes
![Page 63: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/63.jpg)
Fancier Visualization
![Page 64: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/64.jpg)
Online Processing
• For lack of EEG hardware, play back a data set in real time:
![Page 65: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/65.jpg)
Online Processing
• For lack of EEG hardware, play back a data set in real time:
![Page 66: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/66.jpg)
Online Processing
• Choosing a destination for outputs:
![Page 67: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/67.jpg)
Setting up a Visualization
• The command is: plot(y,0,'o','MarkerSize',100); xlim([1 2])
![Page 68: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/68.jpg)
Real-time Class Probabilities
• The sphere indicates the imagined movement side
![Page 69: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/69.jpg)
Looking at ERP Data
![Page 70: Dynamics of Attention Shifting](https://reader030.fdocuments.us/reader030/viewer/2022012707/61a7cceccfdda800cc49ba70/html5/thumbnails/70.jpg)
Setting up
• Markers: {{'S101','S102'},{'S201','S202'}}
• Time Windows: [0.25 0.3;0.3 0.35;0.35 0.4; 0.4 0.45;0.45 0.5;0.5 0.55;0.55 0.6]
• Freq flt: [0.1 15]
• Epoch: [0 0.8]