Manage streams with iteratee and play framework

Streams on top of Scala By @waxzce Quentin ADAMLambdaCon 2015 Bologna

Quentin ADAM from the Clever Cloud@waxzce on twitter github- soundcloud instagram .Who am I ?

My day to day work : Clever Cloud, make your app run all the time

And learn a lot of things about your code, apps, and good/bad designKeep your apps online. made with node.js, scala, java, ruby, php, python, go or dockerAnd learn a lot of things about your code, apps, and good/bad design

Why are streams so trendy these days?

2 main reasonsData is becoming bigger and bigger

We need to act while data transferring is occurring

&We are now using lots of permanent connections

Reality for many developerswebsocketshttp/2Example What is inside an http request ?In many cases the request is manipulated all from memory

Uploading a 20 GB file on a post request and storing it example

Its a bad idea to put the body part in memoryCreate a temp file to store the data

I hate file systemsI hate file systemsDo not use the file system as a datastoreFile systems are POSIX compliantPOSIX is ACIDPOSIX is powerful but is a bottleneck File System is the nightmare of ops File System creates coupling (host provider/OS/language)SPOF-free multi tenant File System is a unicorn

The file is transferring two times

Not so good

Lets act on streams!Classic java stream management

Classic java stream management

BuffersBuffer managementBuffer exceptionBuffer overflowClassic java stream management Low performances if not bufferedNot modularThread blockingCode is uglyNo back pressure Error handling is badi/o management and business code are mixed

I cant sleep at night

We have to find a better wayData structures to express data stream managementPlay-IterateesFirst option

Like a recipeConsume the data

Iteratee : how to manage a streamProduce the dataEnumerator : data stream

Set of tools to do cool things with Iteratee and EnumeratorEnumeratee

Simple example to startWhats our goalGet Reactive Manifesto wordsStream it to an iteratee to group it by pair of wordPrint itAn Enumerator

An iteratee

An iteratee

An iteratee to manage the stream

An iteratee

Iteratee composition

Purely functional

Type safe

composition

Whats trendy?But iteratees are going to die

Scalaz streamsWhats in the box ?Based on scalaz basics

Process is key

Simply use the collection api

demo

Bindings with http with (netty or http4s)scodecBut we need to talk with othersLike our friends on top of JVMAnd they have less expressive type system+ politicsReactive streamsenterAkka streamsCurrent 1.0-M5 (milestone) (M5 out yesteday)Basic example

Basic example

Demo again

More possibilitiesAre you convince to use stream ?

Go nowData streaming programming is now trending

Thx to @clementdclement delafargue

Im @waxzce on twitter

Im the CEO of

A PaaS provider, give it a try ;-) Thx for listening & questions time

Coupon for Clever Cloud trial :

lambdacon