HAND TRACKING, FINGER IDENTIFICATION, AND CHORDIC MANIPULATION ON A MULTI-TOUCH SURFACE
Haptic Surface Manipulation
description
Transcript of Haptic Surface Manipulation
![Page 1: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/1.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Haptic Surface Manipulation
ICDA Group Manufacturing and Systems
Ford Research LaboratoryP.Stewart, P.Buttolo, Y.Chen, A.Marsan
![Page 2: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/2.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Overview• Review
– Applications of Surface Manipulation• Simulation versus Design• Level of Control
– Existing Methodologies and Implementations• Issues With Novel Interfaces
– Data Structure Implications– User Interface and Physical Interface
• A Novel Touch-Enabled Interaction Method
![Page 3: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/3.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Surface Manipulation• What for ?
– Medical Applications– Entertainment
• Movies• Videogames
– Design and Manufacturing• Fashion• Electronics, Appliances• Automotive
![Page 4: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/4.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Simulation Versus Design
Simulation Design
Medical Applications
Entertainment Manufacturing
![Page 5: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/5.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Simulation: How Realistic ?
Believable Accurate reproduction
Entertainment Medical Applications
![Page 6: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/6.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Design: How Accurate ?
Accuracy,Level of control
ManufacturingEntertainment
![Page 7: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/7.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Level Of Control
• Function of application – Critical for Manufacturing
• Exactly the designer intent– Not just a good shape, but the desired shape
• Quality– Accurate, fair, smooth
![Page 8: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/8.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Design And Manufacturing
• Data structure/User interface• Physical/Digital• Issues with force-feedback interfaces• Overview of existing methods/packages
– providing force-feedback interaction
![Page 9: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/9.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Different Data Representations• Traditionally:
– Voxmaps (CFD)– Tessellated meshes(CAE, CFD)– Free-form surfaces (CAD)
• Boundaries are getting fuzzier!• User-Interface as the New Differentiator
![Page 10: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/10.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Clay as a Data Structure• Clay is not entirely “free-form”!
– Clay is just a medium– Different interfaces and levels of control:
• CNC machines• Tools to trace arcs and straight lines• Free-hand
• Main issues with clay are lack of integration and poor reusability
![Page 11: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/11.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Force-feedback Interfaces
• What is unique about using force-feedback?• Does it add “value” ?• What if it is turned-off ?
– Passive manipulation using 3D mouse• MagellanTM, MicroScribeTM
![Page 12: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/12.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Force-feedback Interfaces
• Better sense of space• Better hand-eye coordination• Clay-like “feeling”
• Poorer control ?– Lack of data on human perception
• Lederman, Klatzky, Tan, Srinivasan, …
![Page 13: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/13.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Force-feedback Interfaces• Quality of interaction
– function of computational refresh rate– haptic device structure/bandwidth
• Clay Buck: multi-fingered, two handed, dexterous manipulation
![Page 14: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/14.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Overview of Methods/Applications
• FreeForm– SensAble Technologies
• inTouch– Gregory, Ehmann, Lin– University of North Carolina
• D-NURBS Haptic Sculpting– Dachille, Qin, Kaufman, El-Sana– State University of New York at Stony Brook
![Page 15: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/15.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
FreeFormTM
• Voxmap based• Clay like paradigma• Math modeling tools• Surfacing tools• Commercial package (SensAble)
![Page 16: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/16.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
inTouch
• Polygonal-mesh based• Multi-resolution• Painting & Sculpting• Mesh collision detection (H-Collide)• Slide-mode / Stick-mode• Research platform (U. North Carolina)
![Page 17: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/17.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
D-NURBS Haptic Sculpting
• D-NURBS Based• Physical modeling• Mass-points• Rope tool
![Page 18: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/18.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Surface Sculpting
• Physical Approach– Push / Pull– Add / Remove
• Abstract Approach– Rope tool– Slide to / Stick to
![Page 19: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/19.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Push / Pull Implementation1.Free motion
2.Push
3.Free motion, other side4.Pull
Device
![Page 20: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/20.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Stick-to-surface/Stick-to-pen
• The user interface is constrained to follow the surface contour (Browsing) and at the same time the surface is manipulated to follow the user's motion (Sculpting).
• This method was initially developed to sculpt rational B-splines.
![Page 21: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/21.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
NURBS Surfaces
• A non-rational, non-periodic, B-spline surface is defined by the following:
n
i
m
jjiqjpi vNuNvu
0 0,,, )()(,S P
• where the basis functions Ni,p(u), Nj,q(u) are piecewise polynomials of order p and q respectively, recursively defined over two sets of non-decreasing knot sequences in the parametric domain, respectively, u0,…un, and v0,…,vm.
![Page 22: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/22.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
NURBS Surfaces
• Knots, basis functions and basis-maximum-points for a 3rd order B-spline.
N0,2
N1,2N2,2 N3,2 N4,2 N5,2 N6,2 N7,2
N8,2
umax,1umax,0
u3 u4 u5 u6 u7 u8u0=u1=u2 u9=u10=u11
umax,5umax,8umax,2 umax,6 umax,7umax,4umax,3
![Page 23: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/23.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Browsing and Sculpting
1. Motion in the Browsing Space determines the new interaction/tracking point.
2. Motion in the Manipulation Space determines the new shape of the surface.
3. The trade-off between "who’s tracking who" is achieved by decomposing the Cartesian space into the manipulation and browsing orthogonal subspaces.
![Page 24: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/24.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Browsing
• Goal: find point on surface in contact with haptic device– Closest Point, Interaction Point, Constrained Position– GOD point – Salisbury, Zilles– Proxy – Ruspini, Kolarov– Tracking Point – Thompson, Cohen
Device
Tracking Point
![Page 25: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/25.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Browsing
• FRL: Combination of Newton-Raphson, & others• Linear Approximation, Thompson, U. of Utah
Dt=Device(t)
Tt=Tracking(t)
Tt+1=Tracking(t+1)
B = Browsing
![Page 26: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/26.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Sculpting
• Direct manipulation of Control Points
Device
NURBS Control PointsM
Active Control Point
![Page 27: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/27.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Sculpting
• Manipulation at the Maximum-influence Points
Maximum-influence Points
Device M
Active Control Point
![Page 28: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/28.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Sculpting
• Manipulation of multiple control points– More details in P.Buttolo, P.Stewart, Y.Chen, “Force-enabled
sculpting of CAD models,” ASME DCS, Orlando, Nov 2000
Maximum-influence Points
DeviceM
Active Control Points
![Page 29: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/29.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
PD Manipulation Loop
• Update the surface:St+1(u,v) = St(u,v) + fm(M)
• One-Step manipulation:fm(M)= M
• PD Controlfm(M)= KPM+KVM
![Page 30: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/30.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Space Decomposition
• Example:Browsing on X,Sculpting on Z
D = B + M
St(u,v)
D1
Su
SuB
D
Tt=St(ut,vt)
Tt+1=St(ut+1,vt+1)
Z
X
![Page 31: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/31.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
• One way of finding browsing and editing motion is to use projection matrices.
• For example, these are for Editing Space: Z and Browsing Space: X
1000
MM
TzM DM 0DM
0001
BM
TxB DMB 0 D
![Page 32: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/32.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Projecting Motion on Browsing Subspace
St(u,v)
D1
Su
SuB
D
Tt=St(ut,vt)
Tt+1=St(ut+1,vt+1)
Z
X
uB
uu
SS
BB
lBu
![Page 33: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/33.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Sculpting surface at the tracking point
• Convergence speed for three difference control gains
Dt
Tt+1=St+1(ut+1,vt+1)KP = 0.4KP = 0.3KP = 0.25
M
![Page 34: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/34.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
A Few Examples of Manipulation
• Movies:1. Direct Manipulation of Control Points2. Editing Space: Normal, Browsing Space: Tangent3. Editing Space: Z, Browsing Space: X,Y4. Editing Space: Z, Browsing Space: X5. Editing Space: Y, Browsing Space: X6. Editing Space: Y,Z, Browsing Space: X7. Averaging by Repeated Motion8. An Example of More Complex Manipulation
![Page 35: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/35.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Direct Manipulation of Control Points
![Page 36: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/36.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Editing Space: Normal, Browsing Space: Tangent
![Page 37: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/37.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Editing Space: Z, Browsing Space: X,Y
![Page 38: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/38.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Editing Space: Z, Browsing Space: X
![Page 39: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/39.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Editing Space: Y, Browsing Space: X
![Page 40: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/40.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Editing Space: Y,Z, Browsing Space: X
![Page 41: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/41.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Averaging by Repeated Motion
![Page 42: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/42.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
An Example of More Complex Manipulation
![Page 43: Haptic Surface Manipulation](https://reader036.fdocuments.us/reader036/viewer/2022062521/568155a4550346895dc382eb/html5/thumbnails/43.jpg)
ACM-2001 Pietro Buttolo Ford Research Laboratory
Other Research at FRL: The Haptic Buck
More details in P.Stewart, P.Buttolo, “Putting People Power Into Virtual Reality,” Design 2000, Special Issue of ASME Mechanical Engineer Design, http://www.memagazine.org/medesign/putting/putting.html