Post on 11-Apr-2017
akka streamsReactive Integrations with
that just work™
Johan Andrén Konrad Malawski
Johan AndrénAkka Team Stockholm Scala User Group
Konrad `ktoso` MalawskiAkka Team, Reactive Streams TCK, Persistence, HTTP
Make building powerful concurrent & distributed applications simple.Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community
Complete Java & Scala APIs for all features
What’s in the toolkit?
“Stream”has many meanings
akka streamsAsynchronous back pressured stream processing
Source Sink
Flow
akka streamsAsynchronous back pressured stream processing
Source Sink
(possible) asynchronous
boundaries
Flow
akka streamsAsynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
OutOfMemoryError!!
Flow
akka streamsAsynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
hand me 3 morehand me 3 more
1 msg/s Flow
akka streamsNot only linear streams
Source
SinkFlow
SourceSink
FlowFlow
Reactive StreamsReactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments as well as network protocols
http://www.reactive-streams.org
Part of JDK 9java.util.concurrent.Flow
Reactive Streams
RS Library A RS library B
async boundary
Reactive Streams
RS Library A RS library B
async boundary
Make building powerful concurrent & distributed applications simple.
The APIAkka Streams
Complete and awesome Java and Scala APIs (Just like everything in Akka)
Akka Streams in 20 seconds:
Source<Integer, NotUsed> source = null; Flow<Integer, String, NotUsed> flow = Flow.<Integer>create().map((Integer n) -> n.toString()); Sink<String, CompletionStage<Done>> sink = Sink.foreach(str -> System.out.println(str)); RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);runnable.run(materializer);
Akka Streams in 20 seconds:
CompletionStage<String> firstString = Source.single(1) .map(n -> n.toString()) .runWith(Sink.head(), materializer);
Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:
Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
Check out the “Implementing an akka-streams materializer for big data”
talk later today.
AlpakkaA community for Streams connectors
http://blog.akka.io/integrations/2016/08/23/intro-alpakka
Alpakka – a community for Stream connectors
Threading & Concurrency in Akka Streams Explained (part I)
Mastering GraphStages (part I, Introduction)
Akka Streams Integration, codename Alpakka
A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II)
Writing Akka Streams Connectors for existing APIs
Flow control at the boundary of Akka Streams and a data provider
Akka Streams Kafka 0.11
Alpakka – a community for Stream connectors
Existing examples:MQTT AMQP
Streaming HTTPStreaming TCP
Streaming FileIOCassandra Queries
“Reactive Kafka” (akka-stream-kafka)S3, SQS & other Amazon APIs
Streaming JSONStreaming XML
…
Alpakka – a community for Stream connectors
Demo
Alpakka – a community for Stream connectors
Demo
Akka Streams & HTTP
streams& HTTP
A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time! TCP is a ______ protocol?
A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time! TCP is a STREAMING protocol!
Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a: Flow[HttpRequest, HttpResponse]
Streaming in Akka HTTP
HttpServer as a: Flow[HttpRequest, HttpResponse]
HTTP Entity as a: Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
Streaming in Akka HTTP
HttpServer as a: Flow[HttpRequest, HttpResponse]
HTTP Entity as a: Source[ByteString, _]
Websocket connection as a: Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
Streaming from Akka HTTP
Streaming from Akka HTTP
Streaming from Akka HTTP
No demand from TCP=
No demand upstream=
Source won’t generate tweets
=>
Bounded memory stream processing!
Demo
Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) );
final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080");
}
Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) );
final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080");
}
Streaming from Akka HTTP (Scala)object Example extends App with SprayJsonSupport with DefaultJsonProtocol { import akka.http.scaladsl.server.Directives._ implicit val system = ActorSystem() implicit val mat = ActorMaterializer()
implicit val jsonRenderingMode = EntityStreamingSupport.json() implicit val TweetFormat = jsonFormat1(Tweet)
def tweetsStreamRoutes = path("tweets") { complete { Source.repeat(Tweet("")) } } Http().bindAndHandle(tweetsStreamRoutes, "127.0.0.1", 8080) System.out.println("Running at http://localhost:8080");}
Next steps for Akka
Completely new Akka Remoting (goal: 700.000+ msg/s (!)), (it is built using Akka Streams, Aeron).
More integrations for Akka Streams stages, project Alpakka.
Reactive Kafka polishing with SoftwareMill, Krzysiek Ciesielski
Akka Typed progressing again, likely towards 3.0.
Akka HTTP 2.0 Proof of Concept in progress.
Collaboration with Reactive Sockets
Ready to adopt on prod?
Totally, go for it.
Akka <3 contributionsEasy to contribute tickets:
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
Akka Stream Contrib https://github.com/akka/akka-stream-contrib
Mailing list: https://groups.google.com/group/akka-user
Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
Reactive PlatformReactive Platform
Reactive Platform
Further reading:Reactive Streams: reactive-streams.org Akka documentation: akka.io/docs Free O’Reilly report – very out soon.
Example Sources: ktoso/akka-streams-alpakka-talk-demos-2016
Get involved: sources: github.com/akka/akka mailing list: akka-user @ google groups gitter channel: https://gitter.im/akka/akka
Contact: Konrad ktoso@lightbend.com Malawski http://kto.so / @ktosopl
Thanks!Questions?
@apnylle johan.andren@lightbend.com
@ktosopl konrad.malawski@lightbend.com