Docker + GCE + etcd + ray tracing
-
Upload
syoyo-fujita -
Category
Engineering
-
view
1.286 -
download
2
description
Transcript of Docker + GCE + etcd + ray tracing
Docker + GCE + etcd +
ray tracing@syoyo
Sunday, April 27, 14
ray tracing
Sunday, April 27, 14
Problem
• A lot of computing resource required
• Imagine 60fps photorealistic ray tracing for Oculus VR
• ~ 10K nodes
• Efficient deploy, operation for the renderer(ray tracer)
Sunday, April 27, 14
Why docker
• Hoping...
• Fine resource control
• CPU limit, memory limit, etc
• Easy versioning management of the renderer
• Application sandboxing
Sunday, April 27, 14
My contribution (1/2)
Sunday, April 27, 14
Why etcd?
• Hoping...
• scalable management of cluster information
• node IP addr, redis addr, etc.
• up to 10K nodes possible(?)
Sunday, April 27, 14
Why GCE
• CPU-intensive instance available
• fast VM boot, minute-level charge
• CoreOS image avaiable
• 282.0.0(Docker 0.10)
Sunday, April 27, 14
Render A Render B Render C Render D
Render E
etcd
Architecture
Sunday, April 27, 14
Container
• Busybox(8MB)
• Our renderer(single binary. 72MB)
• node.js(11 MB)
• ~ 90 MBBusybox
Renderer(Ray tracer)
node.js(application frontend)
Sunday, April 27, 14
DockerfileFROM syoyo/aobenchMAINTAINER Syoyo Fujita([email protected])
ADD libdl.so.2 /lib64/ADD librt.so.1 /lib64/
ADD lte /bin/lte
ADD worker.js /home/default/worker.js
ADD node /bin/nodeADD node_modules /home/default/node_modules
Sunday, April 27, 14
Render A Render B Render C
etcd
etcd: 172.17.42.1:4001
web
fron
tend
redis
Sunday, April 27, 14
Get redis info from etcd
http.get(etcdHost + '/v2/keys/redis-server', function(res) {
if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); }
res.on('data', function(chunk) { var j = JSON.parse(chunk);
var redisURL = url.parse(j['node']['value']);
var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr);
Sunday, April 27, 14
Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) {
// kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); }
Sunday, April 27, 14
• http://lighttransportengine.com/shader/
Sunday, April 27, 14
Future work
• ~10K nodes
• Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container.
• Confirmed InfiniBand working.
• Automation using Docker API
Sunday, April 27, 14