Zeromq - Pycon India 2013
-
Upload
srinivasan-r -
Category
Technology
-
view
3.080 -
download
4
description
Transcript of Zeromq - Pycon India 2013
![Page 1: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/1.jpg)
Build your next project - Super fast,
distributed and infinitely scalable.
Using ZeroMQ
![Page 2: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/2.jpg)
Web Framework Tutorial
Application
Client (Browser)
![Page 3: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/3.jpg)
Real World
![Page 4: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/4.jpg)
Queues• Apache ActiveMQ• RabbitMQ• StormMQ• Redis…
![Page 5: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/5.jpg)
ZeroMQ
0MQ
ØMQ
![Page 6: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/6.jpg)
Basic Patterns• Request/Reply• Publish/Subscribe• Pipelining• Paired Sockets
![Page 7: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/7.jpg)
Request - Reply
![Page 8: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/8.jpg)
Hello World Serverimport zmq
context = zmq.Context()
print "Starting hello world server..."socket = context.socket(zmq.REP)socket.bind("tcp://*:5555")
while True: message = socket.recv() print "Got: ", message
# Send the reply. socket.send ("World")
![Page 9: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/9.jpg)
Hello World Clientimport zmq
context = zmq.Context()
# Socket to talk to serverprint "Connecting to hello world server..."socket = context.socket(zmq.REQ)socket.connect ("tcp://localhost:5555")
# Do 10 requests, waiting each time for a responsefor req_no in range (10): socket.send ("Hello")
# Get the reply. message = socket.recv() print "Received reply ", req_no, "[", message, "]"
![Page 10: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/10.jpg)
Hello World Client 2import sysimport timeimport zmq
context = zmq.Context()print "Connecting to hello world server..."socket = context.socket(zmq.REQ)socket.connect ("tcp://localhost:5555")
while True: socket.send(sys.argv[2])
# Get the reply. message = socket.recv() print "Received reply ", "[", message, "]"
time.sleep(float(sys.argv[1]))
![Page 11: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/11.jpg)
Publish - Subscribe
![Page 12: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/12.jpg)
Stock Ticker Serverimport zmqimport timeimport random
context = zmq.Context()socket = context.socket(zmq.PUB)socket.bind("tcp://*:5556")scrips = ['AAPL', 'GOOG', 'MSFT', 'AMZN']
while True: scrip = random.choice(scrips) price = random.randrange(20,700) msg = "%s: %d" % (scrip, price) print msg socket.send(msg) time.sleep(0.5)
![Page 13: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/13.jpg)
Stock Ticker Clientimport sysimport zmq
context = zmq.Context()socket = context.socket(zmq.SUB)
print "Collecting updates from stock server..."socket.connect ("tcp://localhost:5556”)
scrip_filter = sys.argv[1:] if len(sys.argv) > 1 else ["AAPL"]for scrip in scrip_filter: socket.setsockopt(zmq.SUBSCRIBE, scrip)
while True: string = socket.recv() stock, price = string.split() print "%s: %s" %(stock, price)
![Page 14: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/14.jpg)
Parallel Pipeline
![Page 15: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/15.jpg)
import antigravity
![Page 16: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/16.jpg)
XKCD Task Ventilatorimport sysimport zmq
context = zmq.Context()
sender = context.socket(zmq.PUSH)sender.bind("tcp://*:5557")
print "Press Enter when the workers are ready: "_ = raw_input()print "Sending tasks to workers..."
base_url = 'http://xkcd.com/'urls = [base_url + str(i) for i in xrange(1, int(sys.argv[1]))]
for url in urls: sender.send(url)
![Page 17: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/17.jpg)
XKCD Task Workerimport jsonimport zmq
import requestsfrom lxml import etree
def get_title(url): """Given a XKCD url, retrieve the title of the image.""" r = requests.get(url) tree = etree.HTML(r.content) title = tree.xpath('//div[@id="comic"]/img/@title') if title: return title[0] else: return None
![Page 18: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/18.jpg)
XKCD Task Workercontext = zmq.Context()
# Socket to receive messages onreceiver = context.socket(zmq.PULL)receiver.connect("tcp://localhost:5557")
# Socket to send messages tosender = context.socket(zmq.PUSH)sender.connect("tcp://localhost:5558")
# Process tasks foreverwhile True: s = receiver.recv() title = get_title(s) print s, title
# Send a JSON payload to sink sender.send(json.dumps({"url":s, "title":title}))
![Page 19: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/19.jpg)
XKCD Sinkimport zmq
context = zmq.Context()
# Socket to receive messages onreceiver = context.socket(zmq.PULL)receiver.bind("tcp://*:5558")
while True: s = receiver.recv() print s
![Page 20: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/20.jpg)
Transport Types• inproc://name• ipc:///tmp/filename• tcp://hostname:port• pgm://interface:address:port &
epgm://interface:address:port
![Page 21: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/21.jpg)
Languages• C/C++• C#• Java• Erlang• Go• Ruby• Node.js• Objective-C…
![Page 22: Zeromq - Pycon India 2013](https://reader033.fdocuments.us/reader033/viewer/2022061103/5404ecb38d7f729e768b498f/html5/thumbnails/22.jpg)
http://zero.mq/
http://zguide.zeromq.org/py:all
https://github.com/cnu/zeromq-talk
Links