Docker + GCE + etcd +

ray tracing@syoyo

Sunday, April 27, 14

ray tracing

Sunday, April 27, 14

Problem

• A lot of computing resource required

• Imagine 60fps photorealistic ray tracing for Oculus VR

• ~ 10K nodes

• Efficient deploy, operation for the renderer(ray tracer)

Sunday, April 27, 14

Why docker

• Hoping...

• Fine resource control

• CPU limit, memory limit, etc

• Easy versioning management of the renderer

• Application sandboxing

Sunday, April 27, 14

My contribution (1/2)

Sunday, April 27, 14

Why etcd?

• Hoping...

• scalable management of cluster information

• node IP addr, redis addr, etc.

• up to 10K nodes possible(?)

Sunday, April 27, 14

Why GCE

• CPU-intensive instance available

• fast VM boot, minute-level charge

• CoreOS image avaiable

• 282.0.0(Docker 0.10)

Sunday, April 27, 14

Render A Render B Render C Render D

Render E

etcd

Architecture

Sunday, April 27, 14

Container

• Busybox(8MB)

• Our renderer(single binary. 72MB)

• node.js(11 MB)

• ~ 90 MBBusybox

Renderer(Ray tracer)

node.js(application frontend)

Sunday, April 27, 14

DockerfileFROM syoyo/aobenchMAINTAINER Syoyo Fujita(syoyo@lighttransport.com)

ADD libdl.so.2 /lib64/ADD librt.so.1 /lib64/

ADD lte /bin/lte

ADD worker.js /home/default/worker.js

ADD node /bin/nodeADD node_modules /home/default/node_modules

Sunday, April 27, 14

Render A Render B Render C

etcd

etcd: 172.17.42.1:4001

web

fron

tend

redis

Sunday, April 27, 14

Get redis info from etcd

http.get(etcdHost + '/v2/keys/redis-server', function(res) {

if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); }

res.on('data', function(chunk) { var j = JSON.parse(chunk);

var redisURL = url.parse(j['node']['value']);

var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr);

Sunday, April 27, 14

Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) {

// kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); }

Sunday, April 27, 14

• http://lighttransportengine.com/shader/

Sunday, April 27, 14

Future work

• ~10K nodes

• Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container.

• Confirmed InfiniBand working.

• Automation using Docker API

Sunday, April 27, 14