Uncalibrated Image-Based Robotic Visual Servoing (knowdiff.net)
-
Upload
knowdiff -
Category
Technology
-
view
1.104 -
download
4
description
Transcript of Uncalibrated Image-Based Robotic Visual Servoing (knowdiff.net)
Uncalibrated Image-based Uncalibrated Image-based Control of RobotsControl of Robots
Azad ShademanPhD Candidate
Computing Science, University of AlbertaEdmonton, Alberta, CANADA
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Vision-Based Control
current
desired
Left Image Right Image
A
A
A
B
BB
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Vision-Based Control
Left Image Right Image
B
BB
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Where is the camera located? Eye-to-Hand
e.g.,hand/eye coordination
Eye-in-Hand
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Vision-Based Control Feedback from visual sensor (camera) to
control a robot Also called visual servoing Visual servoing is the task of minimizing a
visually specified objective by giving appropriate control commands to a robot
Is it any difficult?Images are 2D, the robot workspace is 3D 2D data 3D geometry
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual Servo Control law Position-Based:
Robust and real-time pose estimation + robot’s world-space (Cartesian) controller
Image-Based:Desired image features seen from cameraControl law entirely based on image features
Hybrid:Depth information is added to image data to
increase stability
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Position-Based Robust and real-time relative pose
estimation Extended Kalman Filter to solve the
nonlinear relative pose equations. Cons:
EKF is not the optimal estimator.Performance and the convergence of pose
estimates are highly sensitive to EKF parameters.
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Position-Based
Desired pose
Estimated pose
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Position-Based
Measurement noise
State variable
Process noise
yaw pitch roll
Measurement equation (projection) is nonlinear and must be linearized.
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
K
xk-1,k-1
zk
Rk
Pk,k
Pk,k-1
Ck
Qk-1
xk,k
xk,k-1
Pk-1,k-1
Kalman Gain
Measurement noise covariance
A priori error cov. @ k-1
Process noise covariance
Initial/previous state
Linearization
Measurement
State update
State prediction Error cov. prediction
Error cov. update
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Image-Based
Desired Image feature
Extracted image feature
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual-motor Equation
x1
x2
x3
x4
q=[q1 … q6]
Visual-Motor Equation
This Jacobian is important for motion control.
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual-motor JacobianImage spacevelocity
Joint spacevelocity
A
A
B
B
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Image-Based Control Law
1. Measure the error in image space
2. Calculate/Estimate the inverse Jacobian
3. Update new joint values
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Image-Based Control Law
Desired Image feature
Extracted image feature
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Jacobian calculation Analytic form available if model is known.
Known model Calibrated
Must be estimated if model is not known
Unknown model Uncalibrated
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Calibrated: Interaction Matrix
Analytic form depends on depth estimates.
Camera/Robot transform required. No flexibility.
CameraVelocity
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Uncalibrated: Visual-Motor Jacobian
A naïve method: Orthogonal projections
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Uncalibrated: Visual-Motor Jacobian
A naïve method: Orthogonal projections
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Uncalibrated: Visual-Motor Jacobian
A naïve method: Orthogonal projections
…
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Uncalibrated: Visual-Motor Jacobian
A popular local estimator:
Recursive secant method (Broyden update):
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Relaxed model assumptions
Traditionally: Local methods No global planning (-) Difficult to show
asymptotic stability condition is ensured (-)
Main problem of traditional methods is the locality.
Calibrated vs. Uncalibrated
Model derived analytically Global asymptotic
stability (+) Optimal planning is
possible (+) A lot of prior knowledge
on the model (-)
Global Model Estimation (Research result)
Optimal trajectory planning (+)
Global stability guarantee (+)
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model
Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation
Our contributions:K-NN Regression-Based EstimationLocally Least Squares Estimation
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Local vs. Global
Key idea: using only the previous estimation to estimate the Jacobian
1. RLS with forgetting factor Hosoda and Asada ’94
2. 1st Rank Broyden update: Jägersand et al. ’97
3. Exploratory motion: Sutanto et al. ‘98
4. Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04
Key idea: using all of the interaction history to estimate the Jacobian
Globally-Stable controller design
Optimal path planning Local methods don’t!
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
3 NN
K-NN Regression-based Method
q1
q2
x1
q1
q2
?
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
q1
q2
x1 ?
K-neighbour(q)
(X,q)
Locally Least Squares Method
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-to-hand Experiments
Puma 560 Stereo vision Features: projection of the end-effector’s
position on image planes (4-dim) 3 DOF for control
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Measuring the Estimation Error
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Global Estimation Error
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual Task Specification Image Features:
Geometric primitives (points, lines, etc.) Higher order image moments Shape parameters …
Visual Tasks Point-to-point (point alignment) Point-to-line (colinearity) Point-to-plane (coplanarity) …
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-in-hand
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-in-Hand Experiments
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-in-Hand Experiments
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-in-Hand Experiments
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Eye-in-Hand Experiments
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Mean-Squared-Error
Task 1
Task 2
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Task Errors
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Conclusions Reviewed position-based and image-based visual
servoing schemes. Presented two global methods to learn the visual-
motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and
local updates. The Jacobian of more complex visual tasks can also
be learned using LLS method.
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Thank you!
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual Ambiguity: Single Camera
Azad Shademan, Uncalibrated image-based control of robots Nov. 5, 2008
Visual Ambiguity: Stereo