Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview....
Transcript of Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview....
![Page 1: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/1.jpg)
Bimodal Brain!Machine Interfac" for Motor Control of Robotic
Prosthetic
![Page 2: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/2.jpg)
Four!university effort #Duke, UF, MIT, SUNY$Funded by DARPAProject goal: develop direct brain!machine interfacesApplication: intelligent prosthetic devices for handicapped peopleUF’s role: develop mapping between motor!cortex neural activity and arm movements #in monkeys$
Brain!Machine Interface: Project Overview
![Page 3: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/3.jpg)
Brain!Machine Interface: System Diagram
Super Monkey
![Page 4: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/4.jpg)
Sample neural data
Input:104!channel neural spike data
Output: hand trajectory
Output: stationary/moving
![Page 5: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/5.jpg)
Global models: FIR, recurrent neural networks, etc.
Multiple local models: two!step approach
Partition neural input space to motion primitives
Train model for each motion primitive partitio#
Benefit: reduce noise when arm is at res$
Mapping neural activity: two approaches
![Page 6: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/6.jpg)
Initial approach
time
104
neur
al ch
anne
ls
time
less neural activity more neural activity
movementno movement
Classifying neural activity to arm movement
Multi!channel neural data
1st!level partitioning
3d arm movement
movemen!
non"movemen!model #1
model #2
Hidden Markov Models %HMMs&
![Page 7: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/7.jpg)
Two partitions: movement vs. non!movement:Classification of data into two classes
VQ 104!channel neural spike data Train two HMMs on quantized input data
3d!trajectory modelingFIR filter for each class
Bimodal system details
![Page 8: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/8.jpg)
87%/90% correct classification on test dataTracking error of arm movement comparable to RNN #global model$Better end!point reaching than RNN
Results & analysis
RNN Bimodal syste'
![Page 9: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/9.jpg)
Different neurons encode different things
VQ introduces substantial loss of informatio#
Insights...
![Page 10: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/10.jpg)
Train HMMs on individual neurons #data is already discrete$.Similar to mixture!of!experts approach.How to combine observation probabilities?
...therefore...
![Page 11: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/11.jpg)
Mean of probability ratiosVariance of probability ratioHand!segmented movement class
Let’s see how this works...
![Page 12: Bimodal Brain Machine Interfac for Motor Control of ......Brain!Machine Interface: Project Overview. Brain!Machine Interface: System Diagram Super Monkey. Sample neural data Input:104!channel](https://reader033.fdocuments.us/reader033/viewer/2022050502/5f94cc05d3f8f6703e6d6d58/html5/thumbnails/12.jpg)
Improved classification performance from 87%/90% to 93%/93%.Biased classifier to equalize classification performance for two classes.
Results
0.99 1.0380(
100(
93(
movemen$
stationary