Motivation Large-scale distributed application require different forms of coordination:...
-
Upload
scott-bennett -
Category
Documents
-
view
222 -
download
0
description
Transcript of Motivation Large-scale distributed application require different forms of coordination:...
![Page 1: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/1.jpg)
ZooKeeper: A Distributed Coordi-nation Service for Distributed Ap-
plications
![Page 2: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/2.jpg)
Motivation• Large-scale distributed application
require different forms of coordina-tion:– Configuration– Group membership and leader election– Synchronization
![Page 3: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/3.jpg)
Related Works• Amazon Simple Queue Service-queu-
ing• [25]-leader election• [27]-configuration• Chubby[6]-locking service with
strong synchronization
![Page 4: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/4.jpg)
Zookeeper• Zookeeper is a distributed service for
distributed applications. It support:– Synchronization– Configuration management– Naming service
![Page 5: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/5.jpg)
Why Do We Need Zookeeper• Zookeeper is simple• Zookeeper is replicated• Zookeeper is ordered• Zookeeper is fast
![Page 6: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/6.jpg)
Data Model
• Regular Znode• Ephemeral Znode• Sequential flag
![Page 7: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/7.jpg)
Watches
Zookeeper
Client1
Client4
crea
te/ex
ist/
(WAT
CH)
setDataNOTI
FICA
TION
![Page 8: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/8.jpg)
Client API• create(path, data, flags)• delete(path, version)• exists(path, watch)• getData(path, watch)• setData(path, data, version)• getChildren(path, watch)• sync(path)
![Page 9: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/9.jpg)
Zookeeper Service Architec-ture
• Read request is handled by local server• Write request is sent to the leader, the leader
broadcasts the change to the Zookeeper through Zab an atomic broadcast protocol.
![Page 10: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/10.jpg)
Setup Zookeeper• Download:
http://www.apache.org/dyn/closer.cgi/zookeeper/
• Configure Zookeeper:– Standalone Mode– Replicated Mode
![Page 11: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/11.jpg)
Standalone Mode• create file zoo.cfg with the content:
• Start server: bin/zkServer.sh start• Test with zookeeper client: bin/zk-
Cli.sh -server 127.0.0.1:2181
tickTime=2000 dataDir=/var/lib/zookeeper clientPort=2181
![Page 12: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/12.jpg)
Standalone Mode (2)• ls /• get• Set
• …
![Page 13: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/13.jpg)
Setup Zookeeper: Replicated mode
• Every server has the same configura-tion file.
• Create file named myid In the datadir directory. The content of myid file is an unique number. tickTime=2000
dataDir=/home/sdn/zookeeperclientPort=2181initLimit=5syncLimit=2server.1=192.168.0.94:2888:3888server.2=192.168.0.59:2888:3888…server.n=192.168.0.59:2888:3888
![Page 14: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/14.jpg)
Use cases• Naming service• Configuration management• Synchronization• Message Queue• Notification system
![Page 15: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/15.jpg)
Synchronization: Simple Lock• Client has smallest number have permission to access locked object• When the client finishes work with object, child node is deleted.
Another client has smallest number have permission to access locked object
App-Root
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
Client1 Client
2Client
3
Client4
N=crea
te(P,
EPHEM
ERAL
|SEQ
UEN-
TIAL)
N=
crea
te(P
, EP
HEM
ERAL
|SEQ
UEN
-TI
AL)
N=
create(P,
EPHEM
ERAL|SEQUEN
-
TIAL)
N=create(P,
EPHEMERAL|SEQUEN-
TIAL)
![Page 16: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/16.jpg)
Synchronization: Simple LockCheck existing of approot and create it.
Create child node with Sequential and Ephemeral flag And receive a number.
When receive the notification, check whether the number is smallest of child nodes number
![Page 17: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/17.jpg)
Synchronization: Barrierpublic class SimpleLock implements Watcher
ZooKeeper zooKeeper = new ZooKeeper("192.168.0.94:2181", 3000, this);Stat res = zooKeeper.exists(root, true);if(res==null) String abc = zooKeeper.create(root, null, Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
String createRes = zooKeeper.create(root+"/", null, Ids.OPEN_ACL_UNSAFE, CreateMod-e.EPHEMERAL_SEQUENTIAL);int number = Integer.parseInt(createRes.substring(root.length()+1));
while (true) {synchronized (root) {
//TODO check whether the number is smallest if(smallest) {dost();}else{root.wait();} }}
public void process(WatchedEvent event) { //Watcher eventsynchronized (root) {root.notify();}
}
private void dost(){System.out.println("Access System.out at "+System.currentTimeMillis());Thread.sleep(5000);
}
![Page 18: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/18.jpg)
Synchronization: Simple Lock
public static void main(String[] args){new SimpleLock();new SimpleLock();new SimpleLock();new SimpleLock();
}
/simpleLock/0000000009:9Access System.out at 1416028020742/simpleLock/0000000010:10Access System.out at 1416028025765/simpleLock/0000000011:11Access System.out at 1416028030779/simpleLock/0000000012:12Access System.out at 1416028035806
![Page 19: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/19.jpg)
Synchronization: Barrier• Every client creates child node of approot.• Whenever the number of child nodes is
enough, client will start work. AppRoot
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
Client1 Client
2Client
3
Client4
N=crea
te(P,
EPHEM
ERAL
|SEQ
UEN-
TIAL)
N=
crea
te(P
, EP
HEM
ERAL
|SEQ
UEN
-TI
AL)
N=
create(P,
EPHEM
ERAL|SEQUEN
-
TIAL)
N=create(P,
EPHEMERAL|SEQUEN-
TIAL)
![Page 20: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/20.jpg)
Synchronization: BarrierCheck existing of approot and create it.
Create child node with Sequential and Ephemeral flag And receive a number.
When receive the notification, check whether the number is smallest of child nodes number
![Page 21: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/21.jpg)
Synchronization: Barrierpublic static void main(String[] args){
new Barrier(3).run();}
Run 3 instances:
/barrier/0000000022Starting at 1416048008944
/barrier/0000000023Starting at 1416048008947
/barrier/0000000024Starting at 1416048008948
![Page 22: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/22.jpg)
Message Queue• Every client creates child node of approot.• Whenever the number of child nodes is
enough, client will start work.AppRoot
P(n)
P(n+1)
P(n+2)
P(n+3)
Zookeeper
sender1
sender2
re-ceiver
N=create(P, SEQUEN-TIAL)
N=create(P, SEQUEN-
TIAL)
get(smallestP)
![Page 23: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/23.jpg)
Message Queuepublic class MessageQueue implements Watcherpublic static class Sender extends MessageQueue implements Runnablepublic static class Receiver extends MessageQueue implements Runnable
public void sendMessage() { //Watcher eventString sendMessage = "sendMessage at "+System.currentTimeMillis();zooKeeper.create(root+"/", sendMessage.getBytes(), Ids.OPEN_ACL_UNSAFE, Create-
Mode.PERSISTENT_SEQUENTIAL);}
public void process(WatchedEvent event) { //Watcher eventsynchronized (root) {root.notify();}
}
while (true) {synchronized (root) { List<String> childs = zooKeeper.getChildren(root, true); if(childs.size()==0){root.wait();}else{ //TODO sort the list
for (String child : childs) { byte[] data = zooKeeper.getData(root+"/"+child, false, new Stat()); System.out.println("readMessage:"+new String(data)); zooKeeper.delete(root+"/"+child, 0);} } }}
![Page 24: Motivation Large-scale distributed application require different forms of coordination: Configuration Group membership and leader election Synchronization.](https://reader033.fdocuments.us/reader033/viewer/2022051102/5a4d1b307f8b9ab05999af8a/html5/thumbnails/24.jpg)
Message Queue:
Run 2 sender instances and 1 receiver instance:
readMessage:sendMessage at 1416066118908readMessage:sendMessage at 1416066118916readMessage:sendMessage at 1416066118926readMessage:sendMessage at 1416066118929readMessage:sendMessage at 1416066118939readMessage:sendMessage at 1416066118942readMessage:sendMessage at 1416066118952readMessage:sendMessage at 1416066118954readMessage:sendMessage at 1416066118964readMessage:sendMessage at 1416066118967readMessage:sendMessage at 1416066118976readMessage:sendMessage at 1416066118984readMessage:sendMessage at 1416066118989readMessage:sendMessage at 1416066118997readMessage:sendMessage at 1416066119001readMessage:sendMessage at 1416066119009readMessage:sendMessage at 1416066119016