Post on 18-Jul-2020
Physics-Based Manipulation under Uncertainty
Michael Koval mkoval@cs.cmu.edu
February 9, 2016 Hands: Design and Control for
Dexterous Manipulation
1
motion planning problem
2
qs
qg
Q
Qobs
Qfree
motion planning problem
3
motion planning problem
4
motion planning problem
5
6
(4×)
9
10
HERB Personal Robotics Lab
Andy CMU/NREC
Robonaut 2 NASA/JSC
ADA Personal Robotics Lab
12
What caused these failures?
motion planning problem
13
14
15
object pose uncertainty
object pose uncertainty
16
motion planning problem
17
motion planning problem
18
proprioceptive uncertainty
19
proprioceptive uncertainty
20
model uncertainty
21K. Hauser. “Robust contact generation for robot simulation with unstructured meshes.” ISRR, 2013.
22
Source: https://youtu.be/S8qkaTsr2_o - ESSEMTEC pick and place machine23
Source: https://youtu.be/yygM-MSxvew - ELCON part feeder machine24
Source: https://youtu.be/nkLd45Ftfhc - ABB bottle packing robot25
26
How can we manipulate under uncertainty?
27
Closed-loop or open-loop?28
Non-deterministic or probabilistic uncertainty?
?
?
?
“worst case” “average case”
29
Closed-form or sample-based representation?
X1
X2
X3
30
“Kalman filter” “particle filter”
Estimate, react, or plan?
31
Visual Feedback
32
Tactile Feedback
No Feedback
33
Image-space visual servoing
put a figure or video here
S. Hutchinson, G.D. Hager, and P. Corke. "A tutorial on visual servo control." T-RA, 1996.
34
Markerless real-time articulated tracking
put a figure or video here
M. Klingensmith et al. ”Closed-loop servoing using real-time markerless arm tracking." ICRA, 2013. (Workshop)
T. Schmidt, R.A. Newcombe, and D. Fox. “DART: Dense Articulated Real-Time Tracking.” RSS, 2014. T. Schmidt, K. Hertkorn, R.A. Newcombe, Z. Marton, M. Suppa, and D. Fox. "Depth-based tracking with physical constraints for robotic manipulation.” ICRA, 2015.
35
Visual Feedback
36
Tactile Feedback
No Feedback
Visual Feedback
37
Tactile Feedback
No Feedback
38
Use “guarded moves” to reduce uncertainty
put a figure or video here
39
Plan a sequence that maximizes information gain
put a figure or video here
S. Javdani, M. Klingsmith, J.A. Bagnell, N.S. Pollard, S.S. Srinivasa. "Efficient touch based localization through submodularity." ICRA, 2013.
Estimate the pose of the object using tactile sensing
put a figure or video here
M.C. Koval, M.R. Dogar, N.S. Pollard, and S.S. Srinivasa “Pose estimation for contact manipulation using manifold particle filters." IROS, 2013. M.C. Koval, N.S. Pollard, and S.S. Srinivasa. “Manifold representations for state estimation in contact manipulation.” ISRR, 2013. M.C. Koval, N.S. Pollard, and S.S. Srinivasa. “Pose estimation for planar contact manipulation with manifold particle filters.” IJRR, 2015.
Closed-loop grasping using contact sensing
put a figure or video here
M.C. Koval, N.S. Pollard, S.S. Srinivasa. “Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty." RSS, 2014. M.C. Koval, N.S. Pollard, S.S. Srinivasa. “Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty.” IJRR, 2015.
Learn feedback policies that use sensor feedback
put a figure or video here
P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal. "Online movement adaptation based on previous sensor measurements.” IROS, 2011.
Learn feedback policies that use sensor feedback
put a figure or video here
J. Fu, S. Levine, P. Abbeel. “One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors.” arXiv, 2015.
Visual Feedback
44
Tactile Feedback
No Feedback
Visual Feedback
45
Tactile Feedback
No Feedback
Open-loop robotic part alignment
M. Erdmann and M. Mason. “An exploration of sensorless manipulation.” IEEE Journal of Robotics and Automation, 1988.
46
M. Dogar and S. Srinivasa. “Push-grasping with dexterous hands: Mechanics and a method.” IROS, 2010. M. Dogar and S. Srinivasa. “A framework for push-grasping in clutter.” RSS, 2011. M. Dogar, K. Hsiao, M. Ciocarlie, and S. Srinivasa “Physics-based grasp planning through clutter.” RSS, 2012.
Push Grasping47
Rearrangement Planning
M. Dogar and S. Srinivasa. “A Planning Framework for Non-Prehensile Manipulation under Clutter and Uncertainty.” AuRo, 2012.
48
Robust Trajectory Selection
M.C. Koval, J.E. King, N.S. Pollard, and S.S. Srinivasa. “Robust trajectory selection for rearrangement planning as a multi-armed bandit problem.” IROS, 2015.
49
Convergent Planning
A.M. Johnson, J.E. King, and S.S. Srinivasa. “Convergent Planning.” RAL, 2016. In press.
50
51
A brief introduction to POMDPs.
a = (q,∆t)
actionstates = (q, x)
observationo = (oq, oc)
T = p(s′|s, a) Ω = p(o|s, a)
52
53
S
state space
dim(S) = n
Planning in Belief Space
54
∆
belief space
dim(∆) = ∞
S
state space
dim(S) = n
Planning in Belief Space
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
55
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
56
57
Point-based solvers
∆
∆
Vπ
b0b0
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
58
Point-based solvers
∆
∆
Vπ
b0b0b1
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
59
Point-based solvers
∆
∆
Vπ
b0b0b1 b2
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
60
Point-based solvers
∆
∆
Vπ
b0b0b1 b2b3
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
61
Point-based solvers
∆
∆
Vπ
b0b0b1 b2b3 b4
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
62
∆
∆
Vπ
b0
V∗
Point-based solvers
π∗ = argmax
πV
π!
b(s0)"
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
63
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
64
b0
a1 a2 a3
65D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
b0
a1 a2 a3
b0
s′ ∼ T (s, a, s′)
a ∼ πexplore(b0)
66D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
b0
a1 a2 a3
b0
b0
o ∼ p(o|s, a)
s′ ∼ T (s, a, s′)
a ∼ πexplore(b0)
b1
o1
67D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
68D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
69
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
70
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a2 a3
b0
b0b1
o1
V (b1) V (b2)
a1
71
a∗ = argmaxiQ(b0, ai)
Q(b 0, a
1)
Q(b 0, a
2)
Q(b 0, a
3)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
72
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
73
Heuristics / BoundsCombine online and offline planning.
74
The post-contact belief space is small
∆
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
75
∆
The post-contact belief space is smallM. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
76
∆
The post-contact belief space is small
∆o
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
77
∆
The post-contact belief space is small
∆o
R(∆o)
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
78
∆
Decompose into pre- and post-contact policies
πc
post-contact policy computed offline
closed-loop once per object
pre-contact policy computed online move-until-touch
once per problem
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
79
80
Physics-Based Manipulation under Uncertainty
Michael Koval mkoval@cs.cmu.edu
February 9, 2016 Hands: Design and Control for
Dexterous Manipulation
81