Physics-Based Manipulation under...
Transcript of Physics-Based Manipulation under...
![Page 1: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/1.jpg)
Physics-Based Manipulation under Uncertainty
Michael Koval [email protected]
February 9, 2016 Hands: Design and Control for
Dexterous Manipulation
1
![Page 2: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/2.jpg)
motion planning problem
2
qs
qg
Q
Qobs
Qfree
![Page 3: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/3.jpg)
motion planning problem
3
![Page 4: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/4.jpg)
motion planning problem
4
![Page 5: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/5.jpg)
motion planning problem
5
![Page 6: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/6.jpg)
6
![Page 7: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/7.jpg)
(4×)
![Page 8: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/8.jpg)
![Page 9: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/9.jpg)
9
![Page 10: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/10.jpg)
10
![Page 11: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/11.jpg)
HERB Personal Robotics Lab
Andy CMU/NREC
Robonaut 2 NASA/JSC
ADA Personal Robotics Lab
![Page 12: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/12.jpg)
12
What caused these failures?
![Page 13: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/13.jpg)
motion planning problem
13
![Page 14: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/14.jpg)
14
![Page 15: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/15.jpg)
15
object pose uncertainty
![Page 16: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/16.jpg)
object pose uncertainty
16
![Page 17: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/17.jpg)
motion planning problem
17
![Page 18: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/18.jpg)
motion planning problem
18
![Page 19: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/19.jpg)
proprioceptive uncertainty
19
![Page 20: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/20.jpg)
proprioceptive uncertainty
20
![Page 21: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/21.jpg)
model uncertainty
21K. Hauser. “Robust contact generation for robot simulation with unstructured meshes.” ISRR, 2013.
![Page 22: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/22.jpg)
22
![Page 23: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/23.jpg)
Source: https://youtu.be/S8qkaTsr2_o - ESSEMTEC pick and place machine23
![Page 24: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/24.jpg)
Source: https://youtu.be/yygM-MSxvew - ELCON part feeder machine24
![Page 25: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/25.jpg)
Source: https://youtu.be/nkLd45Ftfhc - ABB bottle packing robot25
![Page 26: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/26.jpg)
26
How can we manipulate under uncertainty?
![Page 27: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/27.jpg)
27
![Page 28: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/28.jpg)
Closed-loop or open-loop?28
![Page 29: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/29.jpg)
Non-deterministic or probabilistic uncertainty?
?
?
?
“worst case” “average case”
29
![Page 30: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/30.jpg)
Closed-form or sample-based representation?
X1
X2
X3
30
“Kalman filter” “particle filter”
![Page 31: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/31.jpg)
Estimate, react, or plan?
31
![Page 32: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/32.jpg)
Visual Feedback
32
Tactile Feedback
No Feedback
![Page 33: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/33.jpg)
33
Image-space visual servoing
put a figure or video here
S. Hutchinson, G.D. Hager, and P. Corke. "A tutorial on visual servo control." T-RA, 1996.
![Page 34: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/34.jpg)
34
Markerless real-time articulated tracking
put a figure or video here
M. Klingensmith et al. ”Closed-loop servoing using real-time markerless arm tracking." ICRA, 2013. (Workshop)
![Page 35: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/35.jpg)
T. Schmidt, R.A. Newcombe, and D. Fox. “DART: Dense Articulated Real-Time Tracking.” RSS, 2014. T. Schmidt, K. Hertkorn, R.A. Newcombe, Z. Marton, M. Suppa, and D. Fox. "Depth-based tracking with physical constraints for robotic manipulation.” ICRA, 2015.
35
![Page 36: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/36.jpg)
Visual Feedback
36
Tactile Feedback
No Feedback
![Page 37: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/37.jpg)
Visual Feedback
37
Tactile Feedback
No Feedback
![Page 38: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/38.jpg)
38
Use “guarded moves” to reduce uncertainty
put a figure or video here
![Page 39: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/39.jpg)
39
Plan a sequence that maximizes information gain
put a figure or video here
S. Javdani, M. Klingsmith, J.A. Bagnell, N.S. Pollard, S.S. Srinivasa. "Efficient touch based localization through submodularity." ICRA, 2013.
![Page 40: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/40.jpg)
Estimate the pose of the object using tactile sensing
put a figure or video here
M.C. Koval, M.R. Dogar, N.S. Pollard, and S.S. Srinivasa “Pose estimation for contact manipulation using manifold particle filters." IROS, 2013. M.C. Koval, N.S. Pollard, and S.S. Srinivasa. “Manifold representations for state estimation in contact manipulation.” ISRR, 2013. M.C. Koval, N.S. Pollard, and S.S. Srinivasa. “Pose estimation for planar contact manipulation with manifold particle filters.” IJRR, 2015.
![Page 41: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/41.jpg)
Closed-loop grasping using contact sensing
put a figure or video here
M.C. Koval, N.S. Pollard, S.S. Srinivasa. “Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty." RSS, 2014. M.C. Koval, N.S. Pollard, S.S. Srinivasa. “Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty.” IJRR, 2015.
![Page 42: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/42.jpg)
Learn feedback policies that use sensor feedback
put a figure or video here
P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal. "Online movement adaptation based on previous sensor measurements.” IROS, 2011.
![Page 43: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/43.jpg)
Learn feedback policies that use sensor feedback
put a figure or video here
J. Fu, S. Levine, P. Abbeel. “One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors.” arXiv, 2015.
![Page 44: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/44.jpg)
Visual Feedback
44
Tactile Feedback
No Feedback
![Page 45: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/45.jpg)
Visual Feedback
45
Tactile Feedback
No Feedback
![Page 46: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/46.jpg)
Open-loop robotic part alignment
M. Erdmann and M. Mason. “An exploration of sensorless manipulation.” IEEE Journal of Robotics and Automation, 1988.
46
![Page 47: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/47.jpg)
M. Dogar and S. Srinivasa. “Push-grasping with dexterous hands: Mechanics and a method.” IROS, 2010. M. Dogar and S. Srinivasa. “A framework for push-grasping in clutter.” RSS, 2011. M. Dogar, K. Hsiao, M. Ciocarlie, and S. Srinivasa “Physics-based grasp planning through clutter.” RSS, 2012.
Push Grasping47
![Page 48: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/48.jpg)
Rearrangement Planning
M. Dogar and S. Srinivasa. “A Planning Framework for Non-Prehensile Manipulation under Clutter and Uncertainty.” AuRo, 2012.
48
![Page 49: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/49.jpg)
Robust Trajectory Selection
M.C. Koval, J.E. King, N.S. Pollard, and S.S. Srinivasa. “Robust trajectory selection for rearrangement planning as a multi-armed bandit problem.” IROS, 2015.
49
![Page 50: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/50.jpg)
Convergent Planning
A.M. Johnson, J.E. King, and S.S. Srinivasa. “Convergent Planning.” RAL, 2016. In press.
50
![Page 51: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/51.jpg)
51
A brief introduction to POMDPs.
![Page 52: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/52.jpg)
a = (q,∆t)
actionstates = (q, x)
observationo = (oq, oc)
T = p(s′|s, a) Ω = p(o|s, a)
52
![Page 53: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/53.jpg)
53
S
state space
dim(S) = n
Planning in Belief Space
![Page 54: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/54.jpg)
54
∆
belief space
dim(∆) = ∞
S
state space
dim(S) = n
Planning in Belief Space
![Page 55: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/55.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
55
![Page 56: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/56.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
56
![Page 57: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/57.jpg)
57
Point-based solvers
∆
∆
Vπ
b0b0
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
![Page 58: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/58.jpg)
58
Point-based solvers
∆
∆
Vπ
b0b0b1
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
![Page 59: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/59.jpg)
59
Point-based solvers
∆
∆
Vπ
b0b0b1 b2
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
![Page 60: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/60.jpg)
60
Point-based solvers
∆
∆
Vπ
b0b0b1 b2b3
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
![Page 61: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/61.jpg)
61
Point-based solvers
∆
∆
Vπ
b0b0b1 b2b3 b4
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
V π =∞!
t=1
γtR(st, at)
![Page 62: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/62.jpg)
62
∆
∆
Vπ
b0
V∗
Point-based solvers
π∗ = argmax
πV
π!
b(s0)"
J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. IJCAI, 2003. H. Kurniawati, D. Hsu, and W.S. Lee. SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. RSS, 2008.
![Page 63: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/63.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
63
![Page 64: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/64.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
64
![Page 65: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/65.jpg)
b0
a1 a2 a3
65D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
![Page 66: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/66.jpg)
b0
a1 a2 a3
b0
s′ ∼ T (s, a, s′)
a ∼ πexplore(b0)
66D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
![Page 67: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/67.jpg)
b0
a1 a2 a3
b0
b0
o ∼ p(o|s, a)
s′ ∼ T (s, a, s′)
a ∼ πexplore(b0)
b1
o1
67D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
![Page 68: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/68.jpg)
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
68D. Silver and J. Veness. "Monte-Carlo planning in large POMDPs." NIPS, 2010. A. Somani, N. Yi, D. Hsu, and W.S. Lee. "DESPOT: Online POMDP planning with regularization." NIPS, 2013.
![Page 69: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/69.jpg)
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
69
![Page 70: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/70.jpg)
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a1 a2 a3
b0
b0b1
o1
70
![Page 71: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/71.jpg)
b0
o2
b3
a1 a2 a3
b0b2
o2
b0
a2 a3
b0
b0b1
o1
V (b1) V (b2)
a1
71
a∗ = argmaxiQ(b0, ai)
Q(b 0, a
1)
Q(b 0, a
2)
Q(b 0, a
3)
![Page 72: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/72.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
72
![Page 73: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/73.jpg)
∆
Vπ V
∗
Offline Planning Point-Based Methods
Online Planning
b0
a1 a2 a3
b1 b2 b3 b4 b5
o1 o2 o1 o3o2
b
73
Heuristics / BoundsCombine online and offline planning.
![Page 74: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/74.jpg)
74
The post-contact belief space is small
∆
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
![Page 75: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/75.jpg)
75
∆
The post-contact belief space is smallM. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
![Page 76: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/76.jpg)
76
∆
The post-contact belief space is small
∆o
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
![Page 77: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/77.jpg)
77
∆
The post-contact belief space is small
∆o
R(∆o)
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
![Page 78: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/78.jpg)
78
∆
Decompose into pre- and post-contact policies
πc
post-contact policy computed offline
closed-loop once per object
pre-contact policy computed online move-until-touch
once per problem
M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. IJRR, 2016. M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. RSS, 2014.
![Page 79: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/79.jpg)
79
![Page 80: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/80.jpg)
80
![Page 81: Physics-Based Manipulation under Uncertaintygraphics.cs.cmu.edu/nsp/course/16899-s16/lectures/mkoval.pdf · M. Koval, N. Pollard, and S. Srinivasa. Pre- and post-contact policy decomposition](https://reader033.fdocuments.us/reader033/viewer/2022050201/5f54e452cd44c60e267f9f66/html5/thumbnails/81.jpg)
Physics-Based Manipulation under Uncertainty
Michael Koval [email protected]
February 9, 2016 Hands: Design and Control for
Dexterous Manipulation
81