Presented by Jan Nabers (0824712) Symposium Studiereis Radboud University Nijmegen
Eye-Hand coordination Stan Gielen Radboud University Nijmegen.
-
Upload
derek-bruce -
Category
Documents
-
view
218 -
download
3
Transcript of Eye-Hand coordination Stan Gielen Radboud University Nijmegen.
![Page 1: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/1.jpg)
Eye-Hand coordination
Stan GielenRadboud University Nijmegen
![Page 2: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/2.jpg)
Why eye-hand coordination ?
Visual feedbackguidanceExploratory behaviour
![Page 3: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/3.jpg)
Some scientific problems
1. The dynamics of eye and hand movements are very different2. Dynamics for the eye are constant; inertia of the arm varies3. Different neuronal structures are involved how to synchronize ?4. Different frames of reference:
• retinal coordinate system for visual perception• Body-reference system for coordination of arm movements• World coordinate system for planning actions
5. Synchronisation of visual (afferent) and motor (efferent) activity due to time delays in efferent and afferent pathways
6. ……..
![Page 4: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/4.jpg)
Questions
1. Is there a common drive for eye- and hand in aiming movements ?2. How does the CNS deal with visuo-motor interaction if the
dynamics of the eye and hand are very different ?3. How is information about self-initiated movements used in gaze
control ?4. How does the CNS deal with the fact that dynamics of the arm are
different due to variable inertia (arm posture, objects) ?5. Which coordinate system is used in visuo-motor control ?6. How does the CNS deal with synchronisation of visual (afferent)
and motor (efferent) activity due to time delays in efferent and afferent pathways ?
7. If gaze is used to guide the hand along a path (tracking), does gaze lead the hand by a constant time or a constant distance ?
![Page 5: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/5.jpg)
Dynamics of eye and arm movements
![Page 6: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/6.jpg)
Introduction in saccadic system
van Gisbergen, Robinson, S.Gielen. J. Neurophysiol 45, 417-442, 1981.
![Page 7: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/7.jpg)
Position and velocity dependent force components
EEEkF )( 0
![Page 8: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/8.jpg)
Burster cel
EEEkF )( 0
![Page 9: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/9.jpg)
Motoneuron behavior
EEEkF )( 0
![Page 10: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/10.jpg)
Neuronal wiring diagram
Saccadic RT
![Page 11: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/11.jpg)
Dynamics for limb movements
+β+k(Θ-Θ0)
Agonist EMG
AntagonistEMG
![Page 12: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/12.jpg)
Smooth pursuit : driven by retinal slip
Carpenter, Movement of the Eyes, PION, 1977
Solid : predictable target
Open: unpredictable target
![Page 13: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/13.jpg)
Eye-Hand coordinationin aiming movements
![Page 14: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/14.jpg)
Eye-hand coordination in aiming movements
time
posi
tion
![Page 15: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/15.jpg)
Coordination of eye and hand movements
Gielen et al. Exp Brain Res, 56, 154-161, 1984.
eye
hand
![Page 16: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/16.jpg)
Hand and eye positions as a function of RT
![Page 17: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/17.jpg)
Conclusion
• When the eye and hand have to track an object, they are driven by a common drive.
• The common drive changes continuously as a function of time
Gielen et al. Exp Brain Res, 56, 154-161, 1984.
![Page 18: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/18.jpg)
Action and perception are coupled:Action-perception cycle
Active versus Passive perception
![Page 19: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/19.jpg)
Smooth pursuit : anticipation/prediction
Carpenter, Movement of the Eyes, PION, 1977
Solid : predictable target
Open: unpredictable target
Tracking of passively moving target and of target motion due to active head movements
![Page 20: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/20.jpg)
Gaze in passive and active conditions
![Page 21: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/21.jpg)
Gaze in passive conditions
Mean gain for the passive condition for 3 distances(25, 50 and 75 cm), 3 frequencies (0.5, 1 and 1.5 Hz)and 3 stimulus diameters
![Page 22: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/22.jpg)
Frequency-dependent gain for active (closed) and passive (open) conditions
Gielen et al. Exp Brain Res (2004) 155: 211–219
![Page 23: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/23.jpg)
Summary
Gaze control is significantly better in active conditions than in passive conditions due to anticipation and/or efference copies
![Page 24: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/24.jpg)
Frames of reference in eye-hand coordination
![Page 25: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/25.jpg)
Frames of reference viewer centered: relative to the eye, head, or shoulder ?
![Page 26: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/26.jpg)
Frames of reference for pointing
Desmurget et al., J.Neurophysiol
In darkness In light environment
![Page 27: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/27.jpg)
Potential error sources in pointing
• Error in sensory signal (visual, auditory)• Error in storing information in memory• Error in coordinate transformation from sensory to motor• Error in motor command
![Page 28: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/28.jpg)
Constant error
Variable error
![Page 29: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/29.jpg)
Pointing to a remembered target
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
![Page 30: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/30.jpg)
Pointing to a remembered target
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
![Page 31: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/31.jpg)
Subjects were standing,in a completely dark room
Targets were removed during pointing
![Page 32: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/32.jpg)
FINGER: no frame, LED on finger
DARK: no frame, no LED on finger
FRAME: frame and LED on finger
0 1 2 3 4 5 6 7s
targetcuepointingframe
delay
![Page 33: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/33.jpg)
DarkFinger Frame
Pointing results
1. Errors in perception small for azimuth and elevation, but large for depth direction
2. Errors in motor programming relatively small !
![Page 34: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/34.jpg)
Gaze and pointing to remembered visual targets
Fixation is on target after target presentation in all conditions
![Page 35: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/35.jpg)
Gaze and pointing to remembered visual targets
Fixation drifts away from the subject in radial direction in the delay period.
The effect is larger in the FRAME condition.
![Page 36: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/36.jpg)
Gaze and pointing to remembered visual targets
Fixation and pointing do not overlap,
but orientation tuning is similar
Variable error reflects eye-centered fram of reference
![Page 37: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/37.jpg)
Have a cup of coffee and relax !
![Page 38: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/38.jpg)
Pointing to a remembered target after egomotion
gaze
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
![Page 39: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/39.jpg)
DARK FINGERPointing position
FRAME
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
![Page 40: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/40.jpg)
DARK FINGERPointing position
FRAME
Fixation during pointing
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
![Page 41: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/41.jpg)
DARK FINGERPointing position
FRAME
Fixation at the end of step
Fixation during pointing
![Page 42: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/42.jpg)
DARK FINGER
Average over all subjects
Admiraal, Gielen et al. J Neurophysiol. 2003, 2004
For discrete changes in relative target position:
Covariation between gaze and hand position:
Hand position is biased to gaze position
![Page 43: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/43.jpg)
Conclusions
1. Errors in visual perception are small in azimuth and elevation, but large depth
2. An external frame of reference reduces pointing errors3. Errors in motor programming are relatively small; main contribution to
variable error in pointing is due to errors in visual perception.4. Updating for ego-motion is incomplete
![Page 44: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/44.jpg)
Pointing accuracy in pathology: Parkinson Disease
![Page 45: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/45.jpg)
Pointing in Parkinsons Disease
Parkinson Disease is primarily a sensory-motor disorder
Keijsers, Gielen et al., European Journal of Neuroscience, 21, 239–248, 2005
![Page 46: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/46.jpg)
Reference frames for eye-hand coordination
For a retinal frame of reference pointing errors should depend on gaze position relative to target position
?
![Page 47: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/47.jpg)
Pointing errors in retinal frame of reference !
Beurze, van Pelt en Medendorp J Neurophysiol 96: 352–362, 2006.
H, initial hand position; F, fixation point; T, target for movement. B: schematic representation of the paradigm. Horizontal position of the eyes (- - -) and the hand’s pointing direction (—). Boxes indicate location and duration of initial hand position (H, 0–2500 ms), fixation (F, 2500–4500 ms), and target (T, 3500–4500 ms) . Visual feedback about index finger position was available until start of movement.
![Page 48: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/48.jpg)
Spatial updating operates in gaze-centered coordinates.
Medendorp et al. J Neurophysiol 93: 954–962, 2005.
• Occipital, posterior parietal, and premotor cortex are active for saccades and during pointing movements,
• retIPS is activated more strongly for saccades than for pointing. • The activation associated with pointing was significantly greater when pointing with the unseen hand
to targets ipsilateral to the hand. Although there was activation in the left retIPS when pointing to targets on the right with the left hand, the activation was significantly greater when using the right hand (mirror symmetric effect for right retIPS).
• Each hand is more effective in directing movements to targets in ipsilateral visual space.
![Page 49: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/49.jpg)
Hypothetical flow of eye- and hand-related signals to Posterior Parietal Cortex (IPS)
A posteriorly directed stream flows to the retIPS, where relative joint angles are converted into a map with the position of possible effectors coded with respect to the eye. In retIPS, this map, with the effectors initially coded in the contralateral cortex having the weaker representation, is integrated with the map of target location.
Visual information about target location, travels along an anteriorly directed stream into retIPS, which encodes a map of target location in the contralateral visual field.
Proprioceptive information of each effector resides within the primary somatosensory cortex, in the postcentral sulcus, which somatotopically represents the contralateral body parts.
![Page 50: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/50.jpg)
Eye-Hand coordination in tracking and tracing
![Page 51: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/51.jpg)
Eye and hand tracking in 3D
azimuth
ele
vati
on
depth
![Page 52: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/52.jpg)
Eye and hand in 3D
azimuth
ele
vati
on
depth azimuth
ele
vati
on
depth
tracking tracing
![Page 53: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/53.jpg)
Interpretation of crosscorrelation : optimal overlap of eye and hand traces
T
T
xy dttytxT
R )()(2
1)(
Obviously gaze leads hand position.How to quantify the delay between saccadic eye position and hand position ?
![Page 54: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/54.jpg)
How to quantify the delay between saccadic eye position and hand position
Tramper & Gielen The Journal of Neuroscience 31 (21): 7857-7866, 2011
![Page 55: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/55.jpg)
Tracking
Gielen et al. Cortex 45: 340 – 355, 2009
![Page 56: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/56.jpg)
Tracing
Tramper & Gielen The Journal of Neuroscience 31 (21): 7857-7866, 2011
![Page 57: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/57.jpg)
Lead time of gaze relative to finger is not constant !
Tramper & Gielen The Journal of Neuroscience 31 (21): 7857-7866, 2011
After correction for saccadic lead time
![Page 58: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/58.jpg)
Gaze leads the finger by a constant distance, not a constant lead time !
Δs : 2.2 – 3.7 cm
Tramper & Gielen The Journal of Neuroscience 31 (21): 7857-7866, 2011
![Page 59: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/59.jpg)
Mean correlation between hand velocity and saccade interval -0.45 (SD 0.10)*
* p<0.01
Eye fixations lead hand position by a constant displacement
Gielen et al., Cortex 45: 340 – 355, 2009
![Page 60: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/60.jpg)
Simplified model for trajectory generation for discrete aiming movements
![Page 61: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/61.jpg)
Simplified model for trajectory generation for discrete aiming movements
Taken from Wolpert, Kawato et al.
![Page 62: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/62.jpg)
Time delays should be incorporated for moving targets !
Δt1
Δt2
Comparing signals at different positions in time !
![Page 63: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/63.jpg)
Sensory prediction
Courtesy Daniel Wolpert
![Page 64: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/64.jpg)
Courtesy Daniel Wolpert
![Page 65: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/65.jpg)
Courtesy Daniel Wolpert
![Page 66: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/66.jpg)
Virtual reality
![Page 67: Eye-Hand coordination Stan Gielen Radboud University Nijmegen.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649f205503460f94c38a55/html5/thumbnails/67.jpg)
Thanks for your attention !