Post on 13-Jan-2016
description
Consistent Data Replication: Is it feasible in WANs?
Yi LinBettina Kemme
Marta Patiño-Martínez Ricardo Jiménez-Peris
Sep 2, 2005
Data Replication: What,Why,How?
… …
Montreal MontrealToronto Ottawa
TorontoToronto MontrealMontreal OttawaOttawa
Without Replication With Replication
Benefits: Fault Tolerance, Performance Challenge: keep data consistent
WAN
Data Replication: challenge
w(x) w(x)
xx xx xx
Replica control
• Keep data consistent
Motivations
• Most replication protocols have been proved to perform well in LANs.
• Little work has been done in WANs– GlobData [DMBS02], Tech Report [JHU02]
• Are these protocols also feasible in WANs?– Protocols working well in LANs may not work well in
WANs.
• Why? What are the bottlenecks?• Any solutions?
Intro to Group Communication Systems
• GCS provides – multicast primitives to all members in the group– Group maintenance (removal of failed members, etc.)
• Ordering– Unordered– Total order (messages delivered in all members in the
same order)
• Reliability– Different degrees of delivery guarantees in case of
site failures– Analyzed in paper;
Data Replication: Using Group Communication Systems
xxx x
w(x)w(x)
w(x) w(x)
x x
Total Order
• Read-Only requests:• Executed in the local site
• Update requests: • Multicast in total order
firstly.• executed according to total
order delivery.• Num of msgs for an update
• 1 total order
w(x)w(x)
Symmetric
Data Replication: Using Group Communication Systems
xxx x
w(x)w(x)
w(x) w(x)
x x
Total Order
• Read-Only requests:• Executed in the local site
• Update requests: • Request totally ordered firstly.• executed only in the primary
site• Multicast the changes in
unordered msg.• Apply change in other sites
• Num of msgs for an update• 1 total order + 1 unordered• Local write (w(x))
•1 total order within response time
• Remote write (w(x))•1 total order + 1 unordered within response time
w(x)w(x) primary
Primary Copy
unordered
Data Replication: Using Group Communication Systems
xxx x
w(x)w(x)
w(x) w(x)
x x
Total Order
• Read-Only requests:• Executed in the local site
• Update requests: • Request totally ordered firstly.• executed locally• Multicast the changes in
unordered msg.• Apply change in other sites
• Num of msgs for an update• 1 total order + 1 unordered• No concurrent conflicting req
•1 total order within response time
• Has concurrent conflicting req•1 total order + 1 unordered within response time
w(x)w(x)
Local Copy
unordered
Num of messages summary
Symmetric Primary Copy Local Copy
Total num of msgs
1 total order 1 total order
1 unordered
1 total order
1 unordered
Num of msgs within respone time
1 total order Local write
1 total order
No concurrent conflicting request
1 total order
Remote write
1 total order
1 unordered
Has concurrent conflicting request
1 total order
1 unordered
Experiment (I)
0
20
40
60
80
100
120
140
40 60 80 100 120 140 160Load (txn/s)
Res
p Ti
me
(ms)
PCSymLC
100
300
500
700
900
1100
1300
1500
5 15 25 35 45Load (txn/s)
Res
p Ti
me
(ms)
SymPC-LWLCPC-RW
LAN WAN
(5 sites, 100% update)
Experiment (I): Response time analysis
0
200
400
600
800
1000
1200
1400
1600
Sym10 PC-LW10
PC-RW10
LC10 Sym20 PC-LW20
PC-RW20
LC20
Tim
e (
ms)
Resp TimeExe TimeMsg Delivery
Experiment (II): Scalability in WAN
0
10
20
30
40
50
60
20 40 60 80 100
Load (txn/s)
Re
sp
Tim
e (
ms
)
1 site2 sites4 sites6 sites8 sites
0
200
400
600
800
1000
20 40 60 80 100
Load (txn/s)R
es
p T
ime
(m
s)
1 site2 sites4 sites6 sites8 sites
50% update, Symmetric
Read-only requests Update requests
Different Total Order Algorithms
SEQUENCER
mA (seq)
B
C
TOKEN
mA
B
C
token
LAMPORT
mA
B
C
<1,0,0>
<1,0,0>
<1,0,0>
Round Robin (ATOP)
m1A
B
C
m2
Seq #
m2m1
Experiment (III): Different Total Order Alg
0
200
400
600
800
1000
20 30 40 50 60 70Load (txn/s)
Res
p T
ime
(ms)
SEQRRLAMPTOKEN
5 sites in WAN, with replication100% update, Symmetric,
0
200
400
600
800
1000
50 100 150 200 250 300 350Load (txn/s)
Res
p Ti
me
(ms)
SEQ
RR
LAMP
TOKEN
5 sites in WAN, without replication
Conclusions
• Consistent database replication is feasible in WANs;
• In WANs, – For deterministic applications, Symmetric approach is
preferable. – For non-deterministic applications, Local Copy is
preferable;
• In WAN, total order multicast is crucial to response time. Round Robin total order has better performance over others;
• We have some other interesting optimizations. Please refer to our paper.
References• [C-JDBC] E. Ceccet, J.Marguerite, and W. Zwaenepoel. C-JDBC:
Flexible database clustering middleware. In USENIX conference 2004
• [Ganymed] C. Plattner and G. Alonso. Ganymed: Scalable replication for transactional web applications. In Middleware, 2004.
• [GlobData] L. Rodrigues, H. Miranda, R. Almeida, J. Martins, and P. Vicente. Strong Replication in the GlobData Middleware. In Workshop on Dependable Middleware-Based Systems, 2002.
• [Middle-R] R. Jimenez-Peris, M. Patiòno-Martnez, B. Kemme, and G. Alonso. Improving Scalability of Fault Tolerant Database Clusters. In ICDCS'02.
• [Conflict-Aware] C. Amza, A. L. Cox, and W. Zwaenepoel. Conict-Aware Scheduling for Dynamic Content Applications. In USENIX Symp. on Internet Tech. and Sys., 2003.
• [State Machine] F. Pedone, R. Guerraoui, and A. Schiper. The Database State Machine Approach. Distributed and Parallel Databases, 14:71-98, 2003.
• [Spread] http://www.spread.org• [JGroups] http://www.jgroups.org