http://nrg.mbi.ufl.eduhttp://nrg.mbi.ufl.edu

Symbiotic Brain-Machine Interfaces

Justin C. Sanchez, Ph.D.

Assistant ProfessorNeuroprosthetics Research Group (NRG)

University of Floridahttp://nrg.mbi.ufl.edu

jcs77@ufl.edu

http://nrg.mbi.ufl.edu

Enabling Neurotechnologies for Overcoming Paralysis

Develop direct neural interfaces to bypass injury. Communicate and control (closed-loop, real-time) directly via the interface.

Leuthardt

http://nrg.mbi.ufl.edu

Vision for BMI in Daily Life

Lebedev

http://nrg.mbi.ufl.edu

What are the Building Blocks?

Signal SensingAmplification

Pre-Processing

Telemetry

Interpret Neural Activity

Control

SchemeFeedback

Closed Loop BMI

Provide neurophysiologic basis and engineering theory for a fully implantable neuralInterface for restoring communication and control

http://nrg.mbi.ufl.edu

BMI lessons learnedRelationship between user

and BMI is inherently lopsided. Users are intelligent and can use dynamic brain organization and specialization while BMIs are passive devices that enact commands

I/O models have difficulty contending with new environments without retraining

Laboratory BMIs need to be better prepared for ADL

http://nrg.mbi.ufl.edu

Translating Thoughts into Action: The Neural Code

StimulusNeuralSystem Neural Response

Stimulus Neural Response

Coding Given To determine

Decoding To determine Given

http://nrg.mbi.ufl.edu

Vision for Next Generation Brain-Machine Interaction

Intelligent behavior arises from the actions of an individual seeking to maximize received reward in a complex and changing world.

Perception-Action Cycle: Adaptive, continuous process of using sensory information to guide a series of goal-directed actions.

http://nrg.mbi.ufl.edu

Co-Adaptive BMIs using Reinforcement Learning

http://nrg.mbi.ufl.edu

Prerequesites for Symbiosis

http://nrg.mbi.ufl.edu

Co-Adaptive BMI involves TWO intelligent agents involved in a continuous dialogue!!!

ROBOT

action

s

rew

ard

s

bra

in s

tate

s

RAT’S BRAIN

environment

RAT’S BRAIN

COMPUTER AGENT

http://nrg.mbi.ufl.edu

Decoding using Reinforcement Learning

Rather than knowledge of the kinematic hand trajectory only a performance score is supplied. The score could represent reward or penalty, but does not directly provide information about how to correct for the error.

Reward based learning - try to choose strategy to maximize rewards.

RL originated from optimal control theory in Markov Decision Processes.

€

Rt = γ n−t +1rn

n= t +1

∞

∑

€

Q st ,at( )*

= E Rt | st ,at{ }

http://nrg.mbi.ufl.edu

Experimental Co-Adaptive BMI Paradigm

-3 -2 -1 0 1 2 3

0

1

2

3

0

1

2

3

IncorrectTarget

CorrectTarget

StartingPosition

Match LEDs

Grid-space

Match LEDs

Rat’s Perspective

Water Reward

Map workspace to grid

Rat

Robot Arm

Left Lever Right Lever

27 discrete actions 26 movements

1 stationary

http://nrg.mbi.ufl.edu

Agent - Value function estimation

€

Qk (v s t ) = tanh si,t

i

∑ wij

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟

⎛

⎝ ⎜ ⎜

⎞

⎠ ⎟ ⎟w jk

j

∑

€

δt = rt +1 +γQ(st +1,at +1) −Q(st ,at )

http://nrg.mbi.ufl.edu

Evidence for Symbiosis

Valuation Change in Computer Agent

Brain Reorganization

Overall Performance

http://nrg.mbi.ufl.edu

Key Concepts for the Future

Fully implantable interfaces are only half of the story.

Sharing of goals enables brain-computer dialogue and symbiosis

Need for intelligent decoders that assist and co-adapt with the user.

http://nrg.mbi.ufl.edu

History of Man-Machine Interaction

“Implanting tiny machines into the nerves of the heart would make us less human”

Today, over half a million pacemakers are implanted annually!

We are at the frontier for integrating machines with the nervous system to restore and enhance function.

Nicolelis

http://nrg.mbi.ufl.edu

Tremendous team effort!

Jack DiGiovanna - BME

Babak Mahmoudi - BME

This work is supported by NSF project No. CNS-0540304

Jose Principe - ECE

Jose Fortes - ECE

http://nrg.mbi.ufl.edu

Please visit the lab website for publications and additional information.

Neuroprosthetics Research Group http://nrg.mbi.ufl.edu