⇨ Asy Ronen (A.K.A Async)⇨ Eran Harel (@eran_ha)

Before OB1K...

● Multitenancy on a single Tomcat JVM● Our API hosts actually executed 8 modules

on a single JVM

Before OB1K...

● Applicative logs clutter● Monitoring clutter

Before OB1K...

● Hard to troubleshoot performance issues● Hard to fine tune JVMs● Shared configuration

Before OB1K...

● Deployments means start / stop for several modules

● Sub optimal for CD environment...

Before OB1K...

● The container (Tomcat) was installed by Chef, and had a different life cycle then the web-modules

● Container upgrades were a nightmare

What is OB1K?

Asynchronous IO based on Netty

Synchronous Modules based on Jetty 9

Drop-in replacement for Tomcat apps

Kick A$$ Performance :)

Self Contained

Isolated Services

● Own JVM● Own JVM settings● Own configuration● Own port● Own logs directory● Own log configuration

Full control over your module

● JVM options● Configuration properties● Logging● Ports● Compression● Threads● SSL● Etc

Seamless RPC

● Strongly Typed RPC Client

Seamless RPC

● RPC complexity is transparent to the client

Seamless RPC

● Server implementation only worries about the business logic

Seamless RPC

● Configurable client behavior (e.g. retries)

Seamless RPC

● Built-In serialization supporto MsgPacko JSONo Compression

HTTP Streams

Configuration as code

Free Standard API entry points● /selftest● /version● /help (/)● /properties● /ping● /presence● /jmx● /tailLog● /endPoints

Free Service Monitoring

Service Discovery Integration

Easier upgrades, easier deployment

Same execution path for Dev / Prod / JUnit

Maven Archetype Scaffolding$ mvn archetype:generate -DarchetypeGroupId=com.outbrain -DarchetypeArtifactId=outbrain-ob1k-archetype -DarchetypeVersion=1.0.18 -DinteractiveMode=false -Dversion=trunk -DgroupId=com.outbrain -Dpackage=com.outbrain.YOUR_PACKAGE_NAME -DartifactId=YOUR_SERVICE_NAME

A set of asynchronous modules

● Async memcached client● Async RPC client● Async HTTP client● Async DB query execution● Async Cassandra client● Async Event Stream support● Composable Futures infrastructure

General Flow

The main class

Jetty Serverpublic class GruffaloServer { private static final Logger logger = LoggerFactory.getLogger(GruffaloServer.class);

public static void main(String[] args) { new GruffaloServer().build().start(); logger.info("******** Gruffalo started ********"); }

public Server build() { return new JettyServerBuilder().useConfigurationPorts().setContextPath("/Gruffalo"); }}

Netty Server public static Server buildServer() { final String contextPath = "/Ob1kTemplate";

Context ctx = new SpringContextBuilder(contextPath). addInitParam("self-tests", "properties,zookeeper,url,valid-context"). setMainContext("main", "classpath:applicationContext-Ob1kTemplate-all.xml"). addSubContext("ops", "classpath:WEB-INF/spring/ops-services.xml"). addSubContext("service", "classpath:WEB-INF/spring/Ob1kTemplate-service.xml"). build();

ServerBuilder serverBuilder = new ServerBuilder(). setContextPath(contextPath). setContext(ctx). addStaticPath("/html"). addStaticPath("/css"). addBaseServices("ops"). addServiceFromContext("service", Ob1kService.class, "/api"). createServiceFromContext("service", Ob1kNamedService.class, "/names"). addEndpoint("handleFirstRequest", "/first/{id}"). addEndpoint("handleSecondRequest", "/second/{name}"). addEndpoint("handleThirdRequest", "/third/{state}/{city}"). addService().useConfigurationPorts();

return serverBuilder.build(); }

OB1K Servicepublic class Ob1kService implements Service { private static final Logger log = LoggerFactory.getLogger(Ob1kService.class); private final String greetingMessage;

public Ob1kService(final String greetingMessage) { this.greetingMessage = greetingMessage; }

public ComposableFuture<String> echo(final String name) { return ComposableFutures.fromValue("hello " + name + ". " + greetingMessage); }}

OB1K client

new ClientBuilder<>(IOb1kService.class). setProtocol(ContentType.JSON). setRequestTimeout(requestTimeout). setRetries(retries). addEndpoint("http://somehost:8080/Ob1kApp/Ob1kService"). build();

ComposableFuturepublic interface ComposableFuture<T> extends Future<T> { <R> ComposableFuture<R> continueWith(final ResultHandler<T, R> handler); <R> ComposableFuture<R> continueOnSuccess(final SuccessHandler<? super T, ? extends R> handler); ComposableFuture<T> continueOnError(final ErrorHandler<? extends T> handler);

void onResult(OnResultHandler<T> handler); void onSuccess(OnSuccessHandler<? super T> handler); void onError(OnErrorHandler handler);

ComposableFuture<T> withTimeout(long duration, final TimeUnit unit);}

ComposableFuture ExampleComposableFuture<Response> f1 = client.httpGet("http://www.google.co.il/search?q=term1");ComposableFuture<Response> f2 = client.httpGet("http://www.google.co.il/search?q=term2");ComposableFuture<Response> f3 = client.httpGet("http://www.google.co.il/search?q=term3");

ComposableFuture<List<Response>> combined = all(f1, f2, f3);ComposableFuture<String> res = combined.continueWith((result) -> { // prepare combined result.});

schedule(() -> { return trySomething().continueOnError((error) -> { return getFallback(); });}, 30, TimeUnit.SECONDS);

How does it all works ?

● All IO is non blocking and done via Netty● Basically one thread per core, all working

unless there is nothing to do● Context switch is minimized● Transport is always HTTP, payload may vary● Adjusted Executor for blocking actions(if you

must)

Why Streams ?

● Time to serve is limited, better to serve results as they come

● A stream of results is, well… a stream● Request/response model is not a good fit for all

scenarios, thing twitter streaming API● Streams can be combined in all sorts of wonderful

ways● Implementation uses RxJava● Ask netflix :)

What is the catch?

● Blocking code is (almost)forbidden in the async mode

● Writing asynchronous code has a learning curve

● Libraries have to be adapted

Alternatives

FinagleDrop WizardVert.xPlay FrameworkRESTEasy Etc

Future Features

● Client Side Load Balancing● Client Side Discovery (Consul based)● Aggregation● Scatter-Gather● Throttling● Session Tracing (a-la Zipkin)

* W

e’re

recr

uitin

g ;)