Transparency Analysis and Haptic Synchronization Scheme for Force-reflecting Teleoperation
description
Transcript of Transparency Analysis and Haptic Synchronization Scheme for Force-reflecting Teleoperation
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Networked Media LaboratoryDept. of Information & Communications
Gwangju Institute of Science & Technology (GIST)[email protected]
http://nm.gist.ac.kr/~shlee
1
Transparency Analysis and Haptic Synchronization Scheme for Force-reflecting Teleoperation
Seokhee Lee
Lab. Seminar 2009.08.15
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Contents
Introduction Requirements of Force-reflecting teleoperation EBA-based Teleoperation Delay Jitter Problem and Related Work
Transparency Analysis-based Ap-proach Transparency Analysis Haptic Synchronization Simulation and Experiment Results
Conclusions and Future Work
2
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Requirements of Force-reflecting Teleoperation
Stability The primary requisite for safe system If the output response are bounded for all bounded inputs,
the system is said to be stable. Instability
Uncontrollable oscillations and chaotic behavior Sometimes, serious damages to the user
Transparency Transparency ≈ haptic realism Mathematically more difficult to analyze since the ultimate
goal is to make the user experience a “good feeling” Optimal transparency
The user cannot distinguish between direct and tele-interaction with a remote environment.
3
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
EBA-based Teleoperation
EBA (energy bounding algorithm) Stability algorithm of a haptic simulation system (Kim & Ryu, 2004)
EBA passifies virtual environment and restricts the energy generated in the ZOH (zero order hold) within a consumable energy limit in the haptic device.
Can be applied to teleoperation to ensure robust stability regardless of the amount of time delays and packet losses (Seo et al. 2008).
4
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
EBA-based Teleoperation Master EBA
5
Slave EBA
,max ,max
,min ,min
,max 1 ,max
,min ,min
,01 1
2
0
,max
with the following bounding laws:( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
where( ) min( , ( )),
( ) ( ),
( ),
( 1)
(
m m m m
m m m m
m m m
m m
MD MDm n
mk
m
if n n then n n
if n n then n n
n c n
n n
P nc
X k
n
2
22 2
2
2,min 2 2
2
( 1) ( 1)) ,
( ) ( )
( 1) ( 1)( ) ,
( ) ( )
where is a positive constant.
m mm m
m m
m mm m m
m m
m
F n F nc c
X n X n
F n F nn c c
X n X n
c
Control law Control law
Bounding law Bounding law
( ) ( 1) ( ) ( )where
( ) ( 1)( ) ( ) 0( )
( ) ( ) ( 1)
mEBA mEBA m m
md mEBAm m
m
m m m
F n F n n X n
F n F nn for X n
X nX n X n X n
,max ,max
,min
( ) ( 1) ( ) ( )where
( ) ( 1)( ) ( ) 0
( )( ) ( ) ( 1),
( ) ( ) ( ),with the following bounding laws:
( ) ( ) ( ) ( )
( ) (
sEBA sEBA s s
s sEBAs s
s
s s s
s sd s
s s s s
s s
F n F n n e n
F n F nn for e n
e ne n e n e n
e n x n x n
if n n then n n
if n n
,min
,max 1 ,max
,min ,min
,01 1
2
0
2
2,max 2 2
2,min 2 2
) ( ) ( )
where( ) min( , ( )),
( ) ( ),
( ),
( 1)
( 1) ( 1)( ) ,( ) ( )
( 1) ( 1( )
( )
s s
s s s
s s
SD SDs n
sk
s ss s s
s s
s ss s s
s
then n n
n c n
n n
P nc
e k
F n F nn c ce n e n
F n F nn c c
x n
2
2
),
( )
where is a positive constant.s
s
x n
c
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Delay Jitter Problem Delay jitter effect of haptic event
Delayed data transmission, out-of-order arrivals, and empty sampling instances
Teleoperation with delay jitter Instability Transparency deterioration
EBA with delay jitter EBA guarantees stable teleoperation over net-
work delay and packet losses Limitation
It cannot overcome transparency deterioration according to time-varying network situation.
6
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
7
Related Work Adaptive buffering control using moving-average smooth-ing technique (Wongwirat & Ohara, 2006) Buffering time is adjusted to twice the moving average delay. Problem: the authors mentioned the importance of an optimum buffer
size for the transparency but it remained further study. Adaptive buffering control with interpolation scheme (Berestesky et al. 2004) Stability is guaranteed by compressing and expanding buffered data. Problem: although this scheme improved performance of position
tracking and stability, it did not focus on transparency of force. VTR (virtual time rendering) (Ishibashi et al. 2004)
Dynamically adapting the buffering time to improve the interactivity of haptic events in haptic-based NVEs
Problem: little attention has been given to the transparency in the force-reflecting teleoperation.
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
8
Transparency Analysis-based Approach Transparency analysis
Quantifies the force feedback distortions caused by network delay and packet loss.
Predicts the maximum allowable delay and loss for the predefined transparency requirements.
Haptic synchronization based trans-parency analysis Improves transparency over time-varying delay.
By controlling the playout time of the transmitted haptic event with the transparency-related parameters
By synchronizing the local haptic event with the trans-mitted event according to the transparency analysis
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Transparency Analysis Definition of Transparency
Similarity between the force feedback for a slave robot (F-
sEBA) and that for a user in master side (FmEBA) Force feedback increase according to the
user's input Robot keeps in contact with a wall and a user feels the force
feedback by moving haptic device facing the wall with con-stant velocity (vm).
From the control law in master EBA
9
/
1
/
,max ,max1
/
1 ,max1
( ),
( )
min( , ( ))
inter
inter
inter
T
in m mn
T
in m mn
T
m m mn
F v n
F v n
v c n
,
: update time period ( ): interaction time ( )
: increase of ( ): maximum value of ( )
inter
in mEBA
in max in
secT secF F NF F N
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Transparency Analysis Approximation of force
feedback increase Assumption: c1m≥2c2m
γm,max(n) in bounding laws con-verges into c2m when Fm(n-1) increases monotonically.
Force feedback decrease caused by delay Network delay reduces Tinter as
much as the delayed time If the network delay in-
creases, the force feedback decreases in proportional to c2m∙vm.
10
,max 2in m m interF c v T
, 2de delay m m delayF c v T
, : decrease according to delay
: network delayde delay mEBA
delay
F F
T
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Transparency Analysis Force feedback decrease
(Fde,loss) caused by loss Transparent loss time (Ttr,loss): time
period when the force feedback continuously increases even though there exist the packet losses
Maximum allowable delay (Tal,delay) and loss (Tal,loss) Predefined transparency require-
ments Maximum allowable force feedback
(Fal) Maximum allowable force feedback
decrease by packet loss (Fal,loss)
11
,,
2
loss mEBA losstr loss
m m
F FT
c v
, 2 ,( )de loss m m loss tr lossF c v T T
max: maximum force feedback without delay and lossF
,, ,
2
al lossal loss tr loss
m m
FT T
c v
max ,,
2
al loss alal delay
m m
F F FT
c v
,
: the latest received force feedback before the packet losses
: the latest updated FmEBA
before the packet losses: loss time (time period when
the pack
loss
mEBA loss
loss
F
F
Tet losses happen)
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Haptic Synchronization Transparency im-
provement Output time control of
delayed force Controls playout time of
transmitted event. With transparency-re-
lated parameters Output time control of
local position Synchronizes local event
with transmitted event. To minimize the de-
crease of interaction time caused by delay
12
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Haptic SynchronizationOutput time control of delayed force Ideal target output time xn
f
The time at which the event should be output in the case where network de-lay jitter is always smaller than an estimated maxi-mum network delay jitter Jmax
Target output time tnf
The time when the haptic event should be output in the case where network de-lay jitters exists
13
1 1 max 1 1 ,1
1 ,
1 1
( ), if , otherwise
( ) ( 2)
f f fal delayf
fal delay
f f f fn n
D A J D T Tx
T T
x x T T n
1 1
1
*
( 2)
( 1)
f f
f fn n n
f fn n n
t x
t x S n
t t S n
: output time of force, : arrival time
: generation time of force
fn n
fn
D A
T
0 1
*
: slide time: total slide time
0, ( 1)
: modified target output time of force
n
n
n n n
fn
SS
S S S S n
t
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Haptic SynchronizationOutput time Dn
f
By comparing the arrival time An and the target output time tn
f Virtual time expansion
Delays the target output time Total delay increase, loss rate decrease
To minimize the transparency degradation caused by delay
Only when packet loss time is larger than allowable packet loss time
Virtual time contraction Advances the target output time Total delay decrease, loss rate increase
To minimize the transparency degradation caused by loss
Only when the haptic interactions do not happen
14
,if
loss al loss
fn n n
T T
S A t
Virtual time expan-sion
Virtual time contrac-tion
,
1
if [( or 0) and
( )]
( , , )else 0
m mEBA
f fn n al delay
fn n n n
n
X F
t T T
S min r S t A
S
*
, ,
ff n n nn f
n
A if A tD
t otherwise
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Haptic Synchronization Output time control of local position
In order to minimize the decrease of the interaction time Tin-
ter caused by delay, the playout time of the local position Xm is synchronized with the transmitted haptic event Fds.
Ideal target output time xnP
Output time DnP
If the virtual-time expansion or contraction is executed for transmitted event, the target output time and output time of Xm are also changed.
15
1 1 1 1
1 1
( )
( ) ( 2)
p p f f
p p p pn n
x T t T
x x T T n
1 1
( 2)
p p
p pn n n
p pn n
t x
t x S n
D t
: generation time of positionpnT
: target output time of positionpnt
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Simulation Results MATLAB/SIMULINK sim-
ulation Verification of the the trans-
parency analysis Wall contact motion
Slave robot keeps in con-tact with the wall.
Haptic device movement vm=0.05 m/s
16
1 2
0.5 , 20000PD controller: _ 200 /EBA: 1000 , 200
s e
s s
M Kg K N mproportional gain N m
c N m c N m
1 2EBA: 1000 , 200m mc N m c N m
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Simulation Results Delay effect
Transparency analysis Fde,delay=10∙Tdelay
Delay increase of 100ms -> force decrease of 1N
Simulation results With delay 0~600ms
Loss effect Transparency analysis
Ttr,loss=0.002 s Fde,loss=10 ∙(Tloss-0.002)
Packet losses for 50 ms -> force decrease of 0.48N
Simulation results With loss time 2~300ms
17
Simulated results closely follow predicted values -> the transparency analysis is valid
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Experimental Results
Verification of the pro-posed scheme Transparency improvement
over network delay jitter Assumptions
Maximum allowable force feedback decrease by packet loss (Fal,loss)=1N
Time-varying network delay
18
1 2
PD controller: _ 50 / _ 0.1 /EBA: 1000 , 50s s
proportional gain N mderivative gain Ns m
c N m c N m
1 2EBA: 1000 , 50m mc N m c N m
300 , 50Avg ms Std ms
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Experimental Results Wall contact motion
vm=0.2 m/s Transparency com-
parison Each force feedback
with different schemes is compared with the transparent force feed-back (FsEBA).
RMS force feedback er-rors Moving-average adap-
tive buffering (MAB) = 3.7 N
VTR = 3 N Skipping = 2.6 N Proposed scheme = 1 N
19
Transpar-ent force feedback
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Remote Calligraphy System
20
Master
Haptic data
transport
Haptic synchronization
FmdMaster EBA
Xm
FmEBA
Slave
Haptic synchronization
Haptic data
transport
Xsd
Fs
Xm
Slave EBA
Xs
FsEBA
PD control
+-
es
Fs
Fds
Xdm
HIP position
Virtual brush position
HIP
0
0.5
1
1.5
2
0 10 20 30 40 50 60 70 80 90
Forc
e (N
)
Time (s)
FmEBAFsEBA
No network de-lay Standard shape
and force Transparent force
feedback
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Remote Calligraphy System The user writes the character as feeling the similar
force feedback to the transparent force feedback Force feedback error is less than 0.2 N
The force errors between the reaction forces and the standard force 0.6 N (MAB), 0.4 N (VTR), 0.3 N (skipping), and 0.1 N (proposed scheme)
21
0
0.5
1
1.5
2
2.5
0 10 20 30 40 50 60 70 80 90
Forc
e (N
)
Time (s)
FmEBA without delay FmEBA with MABFmEBA with VTR FmEBA with skippingFmEBA with the proposed scheme
0
0.5
1
1.5
2
2.5
0 10 20 30 40 50 60 70 80 90
Forc
e (N
)
Time (s)
FsEBA without delay FsEBA with MABFsEBA with VTR FsEBA with skippingFsEBA with the proposed scheme
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
Remote Calligraphy System User can write the character most similarly to
the standard shape by using the proposed scheme.
With the other schemes, although the user thinks that he or she is writing the character well, unintended tick lines are drawn.
22
with MAB with VTR with skippingwith the proposed scheme
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
23
Conclusions Transparency analysis and haptic synchroniza-
tion scheme for EBA-based force-reflecting teleoperation Transparency analysis
The force feedback distortions according to network variations are quantified.
Haptic synchronization scheme based on transparency analysis
The optimization of the adaptation parameter (buffering time) of the scheme for realistic haptic interactions (transparency)
Simulation and experimental results Transparency analysis provides an acceptable quantification
method The scheme guarantees more transparent haptic interactions
over time-varying network delays
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
24
Future Work Accuracy and generality improvement
More simulations and experiments with various haptic in-teraction scenarios
Paper work Taylor & Francis, Cybernetics and Systems (regular issue) Taylor & Francis, International Journal of Human-Computer
Interaction (regular issue) Springer’s Multimedia Tools and Applications (special issue:
Multimodal Interaction and Multimodal Content Manage-ment)
Manuscript submission deadline: 1 October 2009 Notification of acceptance: 1 December 2009
IEEE Transaction on Systems, Man, and Cybernetics (regular issue)
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
25
Questions & Comments
END
DEPT. OF INFO. & COMM., GISTNetworked Media Lab.
ReferencesJ.-P. Kim and J. Ryu, “Stable haptic interaction control using energy bound-
ing algorithm,” in Proc. IEEE/RSJ IROS, 2004.
C. Seo, J. Kim, J. Kim, J. Yoon, and J. Ryu, “Stable bilateral teleop-eration using the energy-bounding algorithm: Basic idea and feasibility tests,” in Proc. IEEE/ASME AIM, 2008.
O. Wongwirat and S. Ohara, “Haptic media synchronization for re-mote surgery through simulation,” IEEE Multimedia, 2006.
P. Berestesky, N. Chopra, and M. W. Spong, “Discrete time passiv-ity in bilateral teleoperation over the internet,” in Proc. IEEE ICRA, 2004.
Y. Ishibashi, H. Kasugai, and M. Fujimoto, “An intra-stream syn-chronization algorithm for haptic media in networked virtual environments,” in Proc. ACM SIGCHI ACE, 2004.
26