Download - Docker + GCE + etcd + ray tracing

Transcript
Page 1: Docker + GCE + etcd + ray tracing

Docker + GCE + etcd +

ray tracing@syoyo

Sunday, April 27, 14

Page 2: Docker + GCE + etcd + ray tracing

ray tracing

Sunday, April 27, 14

Page 3: Docker + GCE + etcd + ray tracing

Problem

• A lot of computing resource required

• Imagine 60fps photorealistic ray tracing for Oculus VR

• ~ 10K nodes

• Efficient deploy, operation for the renderer(ray tracer)

Sunday, April 27, 14

Page 4: Docker + GCE + etcd + ray tracing

Why docker

• Hoping...

• Fine resource control

• CPU limit, memory limit, etc

• Easy versioning management of the renderer

• Application sandboxing

Sunday, April 27, 14

Page 5: Docker + GCE + etcd + ray tracing

My contribution (1/2)

Sunday, April 27, 14

Page 6: Docker + GCE + etcd + ray tracing

Why etcd?

• Hoping...

• scalable management of cluster information

• node IP addr, redis addr, etc.

• up to 10K nodes possible(?)

Sunday, April 27, 14

Page 7: Docker + GCE + etcd + ray tracing

Why GCE

• CPU-intensive instance available

• fast VM boot, minute-level charge

• CoreOS image avaiable

• 282.0.0(Docker 0.10)

Sunday, April 27, 14

Page 8: Docker + GCE + etcd + ray tracing

Render A Render B Render C Render D

Render E

etcd

Architecture

Sunday, April 27, 14

Page 9: Docker + GCE + etcd + ray tracing

Container

• Busybox(8MB)

• Our renderer(single binary. 72MB)

• node.js(11 MB)

• ~ 90 MBBusybox

Renderer(Ray tracer)

node.js(application frontend)

Sunday, April 27, 14

Page 10: Docker + GCE + etcd + ray tracing

DockerfileFROM syoyo/aobenchMAINTAINER Syoyo Fujita([email protected])

ADD libdl.so.2 /lib64/ADD librt.so.1 /lib64/

ADD lte /bin/lte

ADD worker.js /home/default/worker.js

ADD node /bin/nodeADD node_modules /home/default/node_modules

Sunday, April 27, 14

Page 11: Docker + GCE + etcd + ray tracing

Render A Render B Render C

etcd

etcd: 172.17.42.1:4001

web

fron

tend

redis

Sunday, April 27, 14

Page 12: Docker + GCE + etcd + ray tracing

Get redis info from etcd

http.get(etcdHost + '/v2/keys/redis-server', function(res) {

if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); }

res.on('data', function(chunk) { var j = JSON.parse(chunk);

var redisURL = url.parse(j['node']['value']);

var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr);

Sunday, April 27, 14

Page 13: Docker + GCE + etcd + ray tracing

Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) {

// kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); }

Sunday, April 27, 14

Page 14: Docker + GCE + etcd + ray tracing

• http://lighttransportengine.com/shader/

Sunday, April 27, 14

Page 15: Docker + GCE + etcd + ray tracing

Future work

• ~10K nodes

• Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container.

• Confirmed InfiniBand working.

• Automation using Docker API

Sunday, April 27, 14