Vision-Based Reach-To-Grasp Movements
From the Human Example to an Autonomous Robotic System
Alexa Hauck
Context
Special Research Program “Sensorimotor”
C1: Human and Robotic Hand-Eye Coordination
• Neurological Clinic (Großhadern), LMU München
• Institute for Real-Time Computer Systems, TU München
MODEL
ofHand-Eye
CoordinationAN
ALY
SIS
of
hum
an r
each
ing m
ovem
ents
SYN
TH
ESIS
of
a r
oboti
c sy
stem
The Question is ...
How to use which visual information for motion control?
control strategy representation catching reaching
State-of-the-art Robotics
)(),(),( txtxtxxT
+ easy integration with path planning
+ only little visual information needed– sensitive against model errors
)())(( txtxxT
+ model errors can be compensated
– convergence not assured
– high-rate vision needed)())(( txtffT
Impressive results
... but nowhere near human performance!
Visual Servoing: (visual feedback control)
Look-then-move: (visual feedforward control)
The Human Example
Separately controlled hand transport:• almost straight path• bell-shaped velocity profile
Experiments with target jump:• smooth on-line correction of the trajectory
Experiments with prism glasses:• on-line correction using visual feedback • off-line recalibration of internal models
Use of visual information in spatial representation Combination of visual feedforward and feedback
... but how ?
New Control Strategy
1
1
))()(()())(()()(n
iniiiTnnn tgtgteDtxxtgttx
Example: Point-to-point
Example: Target Jump
Example: Target Jump
Example: Target Jump
Example: Multiple Jumps
Example: Multiple Jumps
Example: Double Jump
Hand-Eye System
Robotimages
ImageProcessing
features
ImageInterpretation
position target & hand
MotionPlanning
trajectory
RobotControl
commands
Models
Hand-EyeSystem
&Objects
objectmodel
sensormodel
armmodel
objectmodel
The Robot: MinERVA
manipulator with 6 joints
CCD cameras
pan-tilt head
Robot Vision
3D
Bin. Stereo
Target
correspondingpoints
Hand
correspondingpoints
Example: Reaching
Example: Reaching
Example: Reaching
Model Parameters
Arm:• geometry, kinematics• 3 parameters
Arm-Head Relation:• coordinate transformation• 3 parameters
Head-Camera Relations:• coordinate transformations• 4 parameters
Cameras:• pinhole camera model• 4 parameters (+ rad. distortion)
Calibration
manufacturer
measuring tape
HALCON
HALCON
Use of Visual Feedback
mean maxcorr0 8.9cm 20cm
1 Hz 0.4cm 1cm
Example: Vergence Error
Example: Compensation
Summary
• New control strategy for hand-eye coordination
• Extension of a biological model
• Unification of look-then-move & visual servoing
• Flexible, economic use of visual information
• Validation in simulation
• Implementation on a real hand-eye system
Top Related