Optimal Therapy After Stroke: Insights from a Computational Model
-
Upload
roanna-mclaughlin -
Category
Documents
-
view
30 -
download
0
description
Transcript of Optimal Therapy After Stroke: Insights from a Computational Model
Optimal Therapy After Stroke:
Insights from a Computational Model
Cheol HanJune 12, 2007
What is rehabilitation?
Behavioral Compensation Use the un-paralyzed arm instead of using the par
alyzed arm Develop the alternative strategy
Neural Recovery Use the paralyzed arm in order to be as same as t
he normal person Change neuro-plasticity to use the peri-lesion neur
ons
Learned non-useDr Taub: (1966)
"Right after a stroke, a limb is paralyzed,"
“Whenever the person tries to move an arm, it simply doesn't work."
“Even when all the cells that represent the arm in the brain are not dead, the patient, expecting failure, stops trying to move it.
"We call it learned non-use,"
(from http://www.mult-sclerosis.org/news/Aug2001/RehabTherapy.html)
The first question: Is “Learned non-use” myth? or reality?
Hypothesis (Learned non-use is myth): Less use of the arm due to lower performance after stroke, but the use is more or less proportional to performance.
Alternative hypothesis (Leaned non-use is reality): % of spontaneous hand use is very small even with non zero performance
Learned non-use"
Motor Performance% o
f sp
on
taneous
hand u
se
Learned Non-use
How to define or measureMotor performance?
Another possible explanationFrom Dr Gordon and Dr Winstein
The second question: How to find optimal schedule of rehabilitation?
Rehabilitation program is expensive Optimal duration of rehabilitation may
be different, when Speed of learning is different Size of stroke is different And so on.
One-fits-all rehabilitation is not cost-efficient. -> Optimal therapy fitting to individuals.
Approach
Find “ optimal therapy schedule” using a SIMPLE computational model that has TWO components:
1. Motor cortex for arm reaching
Motor learning and re-learning Motor lesion due to stroke Error-based learning
2. Adaptive spontaneous arm use “Action chooser” Reward-based learning
ActionChoiceModule
LeftMotorCortex
RightMotorCortex
DesiredInitial
Direction
RewardFunction
ExecutedInitial
Direction
+ -
Error-basedLearning
Reward-basedLearning
Error-driven learning vs. Reward-driven learning
Error-driven learning(Supervised learning)
Reward-driven learning(Reinforcement learning)
“Therapist”: Your initial direction is off 20 degree leftward and your final hand position is 5 cm far from the target in the left.
“Therapist”: Your movement was better than what it was before! Great, your are making progress
Tell patient whether the movement was good or not.
Overall
Grade
only
(Rewar
d)
Specify how much and which direction patient should update
Specific
Error
Experimental Setup for simulation
Each hand starts at the same position.
Reach to the randomly selected target (equal distance)
Two conditions after stroke Free choice (no rehabilitation) Rehabilitation: force to
use the affected arm in all directions. “constraint induce therapy”
Motor cortex model: simplifying assumptions
Assumption 1: The motor cortex has directional coding neurons with signal dependent noise. (Georgopoulos et al., 1982 and Reinkensmeyer, 2003) Todorov (2000) showed with a simple model that directional
coding is correlated with muscle movements. Assumption 2: Stroke lesions part of preferred direction coding.
Based on Beer et al.’s Assumption 3: Rehabilitation retunes preferred directions of remai
ning cells. Li et al.(2001)’s data showed that directional tuning of the muscl
e EMG is retuned during motor training. Based on Todorov (2000)’s idea above, retuning in directional tun
ing of muscle EMG (Li et al., 2001) may be interpreted as retuning in directional tuning of the motor cortex neurons.
Each neuron in the motor cortex has directional coding
Georgopoulos et al, 1982
cos( )
: desired direction
: preferred direction of a neuron
d p
d
p
a b k
Population coding is a vector sum of each neuron’s activation
Georgopoulos et al, 1986
Stroke deteriorates part of movements
Thin line: unaffected arm, Solid line: affected armRF Beer, JPA Dewald, ML Dawson, WZ Rymer (2004, Exp Brain Res)
Motor Learning induces change in directional tuning of muscle EMG
Li et al, Neuron, 2001
Motor Cortex model
Cosine coding extended with signal dependent noise (Reinkensmeyer, 2003) Each cell has its own preferred direction. Same activation rule with Georgopoulos et al.’s. Stroke lesions preferred direction with equal distribution. # of cell surviving affects the motor variance.
cos( )
: desired direction
: preferred direction of a neuron
d p
d
p
a b k
Supervised learning in the motor cortex
We extended the model with different simulation of stroke and learning process. Stroke lesions
preferred direction with unequal distribution
Rehabilitation retunes preferred directions of remaining cells
How to retune the preferred direction?
cos( )
: desired direction
: preferred direction of a neuron
d p
d
p
a b k
( )
: preferred direction of a neuron
: desired direction
: executed movement direction
p p d r
p
d
r
a
Error-driven (Supervised) Learning
Activation Rule
Action Chooser: Action valueAction value “Action value” is an expected
cumulative sum of rewards by performing a specific action
Here, for each target, we have two action values: one for the left arm VL(theta), and one for the right arm Vr(theta).
The arm selected will be the arm that correspond to the higher value.
Three types of rewards 1. Directional Reward
(transformation from directional error)
2. Reward for workspace efficiency Right arm uses for right hand
side workspace is rewarded Left arm uses for left hand side
workspace is rewarded 3. Possible learned non-use negative
rewards (punishments).
-20 -15 -10 -5 0 5 10 15 20-0.5
0
0.5
1
1.5
Directional error (degree)R
ewar
d
Action Chooser: Probabilistic selection
Based on the action value, probabilistically select which arm will be used to generate movement.
Probabilistic formulation implies competition between two arms
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Action value(R) - Action Value (L)
Spo
ntan
eous
Han
d-us
e of
the
rig
ht h
and
Spontaneous hand use (probability to choose the right hand)
1( )
1 exp( ( ( ) ( )))
( ) : action value of the right arm
( ) : action value of the left arm
: a parameter for sharpness
P RV R V L
V R
V L
g
Results
CIT retunes preferred directions
Spontaneous hand use improves
Preferred direction redistribution
Free Choice Condition Rehabilitation condition
Aff
ect
ed
range
Initial
Afterstroke
Afterrehabilitation
Efficacy and Efficiency
Future work
Model “learned non-use” by modeling “expected failures” (add negative rewards).
Motor cortex model More realistic lesions Unsupervised learning to account for spontaneous
recovery Mapping the direction coding to the muscle coding
Experiments with stroke subjects using the new VR system Updating the model parameters based on real data
Acknowledgements
Dr Arbib
Dr Schweighofer
Dr Winstein
Jimmy Bonaiuto